Lambda SQS Consumer
Lambda SQS Consumer is a template for a NodeJS AWS Lamda consumer for an AWS SQS queue.
Quick Start
## replace `lambda-sqs-consumer` arg with your repo name
npx @meetbit/lambda-sqs-consumer lambda-sqs-consumer
Requirements
Before getting started, make sure you have the following installed on your system:
- Node 16.14.x (use nvm to install)
- Yarn 1.2x.x
- AWS CLI 2.3.x
- Zip 3.0 (UNIX systems)
- 7-Zip (Windows)
Environment Variables
Lambda SQS Consumer requires a few environment variables to run. Create a .env
file in the project's root with the following values:
Key | Value |
---|---|
QUEUE_URL | your_queue_url |
Getting Started
To get started, you will have to setup your SQS Queue and Lambda Function.
SQS Queue
-
Create SQS queue. Take note of your queue url.
## replace `lambda-sqs-consumer-queue` with your Queue Name ## replace `us-east-1` with your region aws sqs create-queue --queue-name lambda-sqs-consumer-queue --attributes file://aws-setup/queue-attributes.json --region us-east-1
-
Provide permissions to message senders. The easiest way to do this is to use the AWS Console online. Go to SQS > Your Queue > Access Policy > Access policy (Permissions) > Edit > Access Policy. It should look something like the one below. In the object under the
Statement
array with anSid
of"__sender_statement"
, add the ARNs of IAM Users or Roles that would need to send messages to this queue under thePrincipal.AWS
array.{ "Version": "2008-10-17", "Id": "__default_policy_ID", "Statement": [ { "Sid": "__owner_statement", ... }, { "Sid": "__sender_statement", "Effect": "Allow", "Principal": { "AWS": [ "iam-role-or-user-arn" ] }, "Action": "SQS:SendMessage", "Resource": "sqs-arn" } ] }
Lambda Function
-
Package the Lambda Function
## linux / mac yarn run package ## windows yarn run package-win
-
Create IAM role for the lamda function. The easiest way to do this is to use the AWS Console online. Go to IAM > Roles > Create Role. Provide the IAM role with the following AWS Managed Policy:
- AWSLambdaSQSQueueExecutionRole
-
Create the Lambda Function
## replace `lambda-sqs-consumer` with your function name ## replace `role-arn` with your created role arn ## replace `us-east-1` with your region aws lambda create-function --function-name lambda-sqs-consumer --zip-file fileb://function.zip --runtime nodejs16.x --role role-arn --region us-east-1 --handler index.handler
- Configure Environment Vairables. The easiest way to do this is through the AWS Console online. Go to Lambda > Functions > Your Function > Configuration > Environment Variables > Edit. Add the following environment variables along with any other variables your function requires.
-
Add Lambda Trigger to your queue. The easiest way to do this is through the AWS Console online. Go to SQS > Your Queue > Lambda triggers > Configure Lambda trigger function. Find and choose your previously created Lambda function.
Secret Value QUEUE_URL your_queue_url
Customization
You can customize your SQS queue and Lambda function by editing /aws-setup/queue-attributes.json
and /aws-setup/lambda-role.json
. You can also edit the CLI commands when creating the SQS queue, IAM role and lambda function to reflect your choices.
Deployment
Whenever you make changes to your Lambda function, you can deploy your code by packaging your function first and then deploying. If you wish to use CI/CD, this template includes provisions for using Github Actions for CI/CD which is explained in the next section.
-
Package the Lambda Function
## linux / mac yarn run package ## windows yarn run package-win
-
Update the Lambda Function
## replace `lambda-sqs-consumer` with your function name aws lambda update-function-code --function-name lambda-sqs-consumer --zip-file fileb://function.zip
CI/CD
Included in this template are provisions to use Github Actions for CI/CD. By default, your lambda function is updated when a pull request to the master
branch is closed.
Setup
-
Configure Action. In
.github/workflows/ci-cd.yml
, changelambda-sqs-consumer
in line 48 to the name of your Lambda function. You may also configure what can trigger the Github Action in lines 4-8. -
Create IAM User for the Github Action. The easiest way is to do this through the AWS Console online. Create a new policy for the user with the permissions below. Take note of the the Access Key and Secret.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "lambda:UpdateFunctionCode", "lambda:UpdateFunctionConfiguration" ], "Resource": [ "*", ] } ] }
-
Setup Github Repository Secrets. To do this, go to Your Repositoy > Settings > Secrets & variables > Actions > New repository secret and add the following:
Secret Value ACTION_AWS_ACCESS_KEY_ID iam_access_key ACTION_AWS_SECRET_ACCESS_KEY iam_secret AWS_REGION lambda_region
Resources
This template uses different dependencies/frameworks/services. Here are links to the most important packages you will need as you use this template.
License
Lambda SQS Consumer is under the MIT License.