AWS Blog - Manage IoT device state anywhere using AWS IoT Device Shadow service and AWS IoT Greengrass

, , , , ,

Discover my latest blog post on AWS official blog channel, where I delve into managing IoT devices from anywhere! Whether you’re interested in a humble Raspberry Pi application or eager to explore broader applications like home automation or industrial IoT solutions, this post has got you started.

Happy reading!

Blog address: https://aws.amazon.com/blogs/iot/manage-iot-device-state-anywhere/
Source code: https://github.com/aws-samples/manage-IoT-device-using-device-shadow-blog

Read More

Share Comments

AWS Step Functions with ECS Anywhere on NanoPi Sample

, , , ,

This is a demo solution that is using AWS Step Functions and ECS Anywhere to complete a simple data processing task by using cloud orchestration (Step Functions) and local computing resources (a NanoPi).

Data flow

  1. User upload a file to a s3 bucket
  2. S3 triggers step functions via cloudtrail and event bridge
  3. Event bridge triggers a step function state machine
  4. State machine triggers a ECS Anywhere task to download the file from s3 to local (to do some processing), if file name matches condition

Architecture

NanoPi that runs ECS Anywhere

NanoPi Neo2 with LED hat in my home office, running AWS ECS Anywhere.

Read More

Share Comments

Using SSM to access EC2 instances

, , , , , , ,

1. Benefits of using for connecting EC2 instances

AWS Systems Manager (SSM) is an AWS service that you can use to view and control your infrastructure on AWS. It can securely connect to a managed node. The SSM Agent is installed in EC2 OS. It is pre-installed on many amazon Machine Images (AMIs).

With SSM:

  • No need to open SSH port in security group for EC2
  • No need to create and manage SSH keys

And SSM works regardless if the EC2 instance is in public or private (NAT or Endpoint) subnet.

Requirements for SSM working:

  • AWS instances:
    • SSM agent installed in instance (pre-installed in many AMIs already)
    • Connectivity to the AWS public zone endpoint of SSM (IGW, NAT or VPCE)
    • IAM role providing permissions
  • On-Prem instances:
    • SSM agent installed in instance
    • Connectivity to the AWS public zone endpoint of SSM (Access to public internet)
    • Activation (Activation Code and Actuation ID)
    • IAM role providing permissions

2. EC2 Instance in public subnet

  • 2.1. Make sure the EC2 instance has a public IP. It could be the public IP assigned during creation, or an Elastic IP.
  • 2.2. EC2 instance should have Internet access (for calling SSM endpoint). In public subnet it is done via Internet Gateways. See details from Session Manager prerequisites, in “Connectivity to endpoints” section.
  • 2.3. You can use VPC Reachability Analyzer to troubleshoot the connectivity between your EC2 and Internet gateway.
  • 2.4. Create an EC2 Instance profile has IAM policy AmazonSSMManagedInstanceCore. Read the details from Step 4: Create an IAM instance profile for Systems Manager
  • 2.5 Attach the EC2 Instance profile to your instance.
  • 2.6 Reboot the EC2 instances.

3. EC2 instance in private subnet, with NAT connectivity

In this case, EC2 instances have no public IP, but they can still talk to internet via NAT.

  • 3.1. Make sure EC2 instances in private subnet can access internet, via a NAT Gateway or NAT instance.
  • 3.2. The rest will be the same as EC2 instances in public subnet, starting from 2.2

4. EC2 instance in private subnet, without NAT connectivity but VPC endpoints

In this case, the EC2 instance (no public IP) won´t have access internet via NAT but VPC endpoints, some extra works are required

  • 4.1 Create VPC endpoints for System Manager. Remember to allow HTTPS (port 443) outbound traffic in security group for your endpoint (ssm, ssmmessages and ec2messages)
  • 4.2. Create an IAM Role as EC2 profile that contains at least the following 2 policies
    • aws managed policy AmazonSSMManagedInstanceCore
    • a custom policy for accessing an AWS owned S3 buckets.
  • 4.3 Attach this instance profile to your EC2 instance
  • 4.4 Make sure enable “DNS resolution” and “DNS hostnames” for you VPC
  • 4.5 In addition, if your EC2 instance need to access other AWS services such as S3, remember to create needed endpoints for them as well. (For S3 you can choose either Gateway or Endpoint. At this moment Gateway is free.) Note that you need to add the endpoint into the private subnet route table. The following screenshot shows the route table entity of a S3 Gateway endpoint, which is using prefix lists.

5. Verification

Once the SSM is fully up-and-running, the EC2 instance (either in public/private subnet) will appear in Fleet Manager in SSM web console.

Share Comments

Building a Very Slow Movie Player

, , , , ,

Inspired by Bryan Boyer and Tom Whitwell, I am building a Very Slow Movie Player (VSMP).

With VSMP,

  • Kiki’s Delivery Service (running time 1h42m): takes 7 days to play (with 1 frame per 20 seconds, as in above demo)
  • Laputa: Castle in the Sky (running time 2h4m): takes 2 months to play (with 1 frame per 120 seconds, as default setting)

Read More

Share Comments

Okta and AWS Control Tower - a happy path demo

, , , ,

This is a happy path demo of setting up Okta as the Idp for AWS Control Tower (via AWS SSO).
Goal: To utilize users and groups in Okta to manage AWS control tower.

1. Create a brand new Control Tower instance

In this demo, we create the AWS Control Tower instance in a brand new AWS account. During this process, control tower creates several services/components, such as AWS Organizations, AWS SSO, default organizations unit (OU) “Security” and 2 AWS accounts “Log Archive” and “Audit”.

In the AWS SSO, some default SSO user groups are created for managing Control Tower:

The default admin user for organization management account is “AWS Control Tower Admin”.

Detailed user info

And it belongs to 2 groups: AWSAccountFactory and AWSControlTowerAdmins

Read More

Share Comments

How to build an IoT connected car - Part 2: Data Analytics in the Cloud

, , , , , , , , , , ,

In Part 1, we have talked about the hardware/software running on the edge (the car) for collecting data.

Now we have the data, and how to gain some insights by doing data analytics? I have been using the following products, and would like to share my quick thoughts

  • Azure Time Series Insight (TSI)
  • Azure Databricks
  • Azure Data Explorer (ADX)
  • PowerBI
  • Grafana

Read More

Share Comments

How to build an IoT connected car - Part 1: On the Edge

, , , , , , ,

Previously I wrote a blog about how to measure hamster via IoT wheel. This reminds me another personal project I did back to the winter of 2018/2019, for measuring car performance.

Read More

Share Comments

How to measure your Hamster's running with wireless IoT

, , , , , ,

We recently welcomed our new family member Qiuqiu (球球) (a girl Syrian/Golden hamster) home. She seems to enjoy the new environment fairly well, but she is a quiet girl - does not show much activities during the day time.

Of course we understand hamsters are nocturnal animals, which means they are sleeping in day time and become more active at night. But I started wondering how she was doing during the nights, especially how much she ran on the hamster wheel.

Let’s do something about it.

Picture: Qiuqiu with her wheel

Read More

Share Comments

Encoding issue when calling API via powershell

, ,

Recently we need to fetch a big dataset from an API via powershell, then import to Azure Data Explorer (ADX).

Problem

1
2
#Used Measure-Command for measuring performance
Measure-Command {curl 'THE_API_END_POINT' | select -expand Content > data.json}

The data.json file looks perfectly fine, but during import to ADX, it reported error “invalid json format”.

Troubleshooting

  1. Using online validation tool such as https://jsonlint.com/, copy & paste the content from data.json. The json objects are valid.

  2. Using local tool jsonlint, reports error. It shows the data.json file has encoding issue.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    PS C:\Users\lufeng\Desktop> jsonlint .\data.json
    Error: Parse error on line 1:
    ��[ { " _ i d " : {
    ^
    Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 'undefined'
        at Object.parseError (C:\Users\lufeng\AppData\Roaming\npm\node_modules\jsonlint\lib\jsonlint.js:55:11)  
        at Object.parse (C:\Users\lufeng\AppData\Roaming\npm\node_modules\jsonlint\lib\jsonlint.js:132:22)      
        at parse (C:\Users\lufeng\AppData\Roaming\npm\node_modules\jsonlint\lib\cli.js:82:14)
        at main (C:\Users\lufeng\AppData\Roaming\npm\node_modules\jsonlint\lib\cli.js:135:14)
        at Object.<anonymous> (C:\Users\lufeng\AppData\Roaming\npm\node_modules\jsonlint\lib\cli.js:179:1)      
        at Module._compile (internal/modules/cjs/loader.js:955:30)
        at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
        at Module.load (internal/modules/cjs/loader.js:811:32)
        at Function.Module._load (internal/modules/cjs/loader.js:723:14)
        at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)

    Solution

    Switch to a different powershell command solved the problem

    1
    2
    Invoke-WebRequest -Uri 'THE_API_END_POINT' -OutFile data.json

EOF

Share Comments

How to Decrypt Native App's HTTPS Traffic (and Debug for In-app Browser)

, , , ,

Problem with in-app browser of LinkedIn and Facebook iOS apps

Recently our QA reported an interesting issue regarding the native app and our website: When the webpage was shared on Linkedin iOS App and/or Facebook iOS App, the built-in browsers cannot show it correctly but a blank page.

  • This issue only happens on some of the iOS apps (see the list below).
  • Other iOS native apps have no problem.
  • Safari and Chrome for iOS have no problem.
  • All Android-based native apps have no problem.
  • All desktop browsers have no problem.

Read More

Share Comments