Overview
This article demonstrates the development of a Serverless API utilizing AWS services and establishes a Continuous Integration/Continuous Deployment (CICD) pipeline within the AWS environment.
Part 1: Explores the creation of a Lambda function to handle requests from the API Gateway and persist the data in DynamoDB using the Serverless Application Model.
Part 2: Details the necessary steps to configure a CodeCommit repository in AWS and set up a CICD pipeline that automatically initiates a build process upon the submission of new changes to the repository.
Prerequisites
For this project, you'll need an AWS account (Free Tier is sufficient). The following AWS components will be utilized:
- AWS API Gateway
- AWS Lambda
- AWS DynamoDB
- AWS CodeCommit
- AWS CodeBuild
- AWS CodePipeline
- AWS S3
- Miscellaneous - CloudWatch, IAM, etc
Ensure you have your local development environment set up as follows:
-
Install AWS CLI: Follow the guide here to install the AWS Command Line Interface.
-
Install AWS SAM (Serverless Application Model): Install SAM CLI by following the instructions here.
-
Choose an IDE: Use IntelliJ or a similar IDE for development. I prefer IntelliJ
-
Maven for Packaging: Make sure you have Maven installed for packaging your application.
-
Optional: Docker (if you need to test Lambda functions locally): Install Docker if you plan to test Lambda functions locally.
These tools and components will form the foundation of the project.
Part 1: Development
In this section, we will see the process of creating a starter project using AWS SAM, including completing the handler class, building, deploying to AWS, and conducting testing using Postman
Environment Setup
-
AWS Setup:
Go to the AWS console at https://aws.amazon.com/console/, Log in using your admin user credentials.
- Create a User in IAM:
- In IAM, create a user dedicated for local development and CLI/SAM usage.
- Create a DynamoDB Table:
- Name: "users", Primary Key: "id" (Type: String)
- Create a User in IAM:
-
Configure AWS CLI on Local Machine:
- Open a terminal and run
$ aws configure
- Provide the access keys for the IAM user created earlier and other default values
- Open a terminal and run
-
Initialize a Project using AWS Serverless Application Model (SAM):
- Open your terminal and run
$ sam init
- Choose the AWS quick start template.
- Opt for a "Hello World" example.
- Select Java 11 or 17, package type as zip, and use Maven as the dependency manager.
- Enable logging and monitoring with CloudWatch and XRay.
- Open your terminal and run
-
Rename the Project: Rename the project to your preferred name.
-
Open the Project in IntelliJ: Launch IntelliJ, and open the project.
-
Add Dependencies to pom.xml:
-
Add the necessary dependencies to the
pom.xml
file. You only need to add DynamoDB, as other dependencies will be automatically included by SAM.<dependencies> <dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-lambda-java-core</artifactId> <version>1.2.2</version> </dependency> <dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-lambda-java-events</artifactId> <version>3.11.0</version> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.13.2</version> <scope>test</scope> </dependency> <dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk-dynamodb</artifactId> <version>1.12.573</version> </dependency> </dependencies>
-
Write the Handler Class
For the lambda function, edit the handler class generated automatically by sam, and add the following code; this is a simple code, and for real projects, you may want to use more modular code.
public class UserRequestHandler implements RequestHandler<Map<String,String>, Map<String,String>> {
private AmazonDynamoDB amazonDynamoDB;
private String DYNAMODB_TABLE_NAME = "users";
private Regions REGION = Regions.US_EAST_1;
@Override
public Map<String,String> handleRequest(Map<String,String> input, Context context) {
this.initDynamoDbClient();
LambdaLogger logger = context.getLogger();
logger.log("Input payload:" + input.toString());
String userId = UUID.randomUUID().toString();
String firstName= input.get("firstName");
String lastName= input.get("lastName");
Map<String, AttributeValue> attributesMap = new HashMap<>();
attributesMap.put("id", new AttributeValue(userId));
attributesMap.put("firstName", new AttributeValue(firstName));
attributesMap.put("lastName", new AttributeValue(lastName));
logger.log(attributesMap.toString());
amazonDynamoDB.putItem(DYNAMODB_TABLE_NAME, attributesMap);
Map<String, String> response = new HashMap<>();
response.put("id", userId);
response.put("firstName", firstName);
response.put("lastName", lastName);
return response;
}
private void initDynamoDbClient() {
this.amazonDynamoDB = AmazonDynamoDBClientBuilder.standard()
.withRegion(REGION)
.build();
}
}
Update SAM Template file
The SAM template file plays a pivotal role in building and deploying changes to AWS. Update the file for the project. The key elements to focus on within this file are the Lambda function names and the API Gateway endpoints. They are central to the functionality of your serverless application.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
pc-aws-user-api
Sample SAM Template for pc-aws-user-api
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 20
MemorySize: 128
Tracing: Active
Api:
TracingEnabled: true
Resources:
UserRequestHandlerLambdaFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: PcAwsUsersApi
Handler: com.pc.aws.users.UserRequestHandler::handleRequest
Runtime: java11
Architectures:
- x86_64
MemorySize: 512
Environment: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
Variables:
PARAM1: VALUE
JAVA_TOOL_OPTIONS: -XX:+TieredCompilation -XX:TieredStopAtLevel=1 # More info about tiered compilation https://aws.amazon.com/blogs/compute/optimizing-aws-lambda-function-performance-for-java/
Events:
PcUsers:
Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /users
Method: post
ApplicationResourceGroup:
Type: AWS::ResourceGroups::Group
Properties:
Name:
Fn::Sub: ApplicationInsights-SAM-${AWS::StackName}
ResourceQuery:
Type: CLOUDFORMATION_STACK_1_0
ApplicationInsightsMonitoring:
Type: AWS::ApplicationInsights::Application
Properties:
ResourceGroupName:
Ref: ApplicationResourceGroup
AutoConfigurationEnabled: 'true'
Outputs:
# ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
# Find out more about other implicit resources you can reference within SAM
# https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
PcAwsUsersApi:
Description: API Gateway endpoint URL for Prod stage
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/users/"
UserRequestHandlerLambdaFunction:
Description: Lambda Function ARN
Value: !GetAtt UserRequestHandlerLambdaFunction.Arn
UserRequestHandlerLambdaFunctionIamRole:
Description: Implicit IAM Role created
Value: !GetAtt UserRequestHandlerLambdaFunctionRole.Arn
Build and Deploy Code using SAM
In IntelliJ, open the terminal, and execute the following commands:
$ sam build
$ sam deploy –guided
When prompted, provide the Stack name as "PcAwsUsersApi", and choose the default options.
The output will showcase the CloudFormation stack that has been created and will also provide the API Gateway endpoint.
Test the API
Before testing the API, grant DynamoDB access to the Lambda role created by SAM.
- Open the Lambda function in the AWS Console.
- Navigate to "Permissions", and locate the role name.
- Add the "DynamoDBFullAccess" permission to this role.
This step ensures that the Lambda function has the necessary permissions to interact with DynamoDB.
Navigate to API Gateway, then follow these steps to obtain the URL for your service:
-
Go to API Gateway in the AWS Console.
-
Select your API.
-
In the left sidebar, click on "Stages."
-
Under "Stages", select the "Prod" stage.
-
In the Stage Editor section, you'll find the "Invoke URL". Copy this URL.
- Launch the Postman application.
- Create a POST Request: Set up a new request with the following details:
- Method: POST
- URL: Paste the API Gateway endpoint URL obtained earlier.
- Set Headers: Add any necessary headers (e.g.,
Content-Type: application/json
).
- Add Request Body: Create a JSON object representing the user data you want to send in the request. This would be the data you expect to be stored in DynamoDB.
Part II Implement CICD Pipeline
In this section, we'll demonstrate how to build a CICD pipeline using AWS CodeCommit, CodeBuild, and CodePipeline. The pipeline will initiate a build process that fetches code from the repository, builds it using the build file, and triggers CloudFormation to create the stack.
Create CodeCommit Repository
Login to AWS, and search for CodeCommit service.
Create a new repository in AWS CodeCommit to store your project's source code.
Commit Code to Repository
In IntelliJ, open the terminal and enter the following commands to commit the code created in Step I
$ git init → This initialize local git
$ git add . → This will stage files
$ git commit -m "commit to CodeCommit"
Push Changes to the remote Repo
Copy the CodeCommit repo URL from aws console.
$ git remote add origin <repo URL>
$ git push --set-upstream origin master --> This will prompt for user/password
Create AWS CodeBuild project
Set up a CodeBuild project that specifies how to build your application. This includes defining build environments, build commands, and dependencies.
- Create a S3 bucket to store the build files.
- Search for AWS Code Build, create a new Project, and provide CodeCommit as a source for code and S3 for artifacts storage
To set up CodeBuild, you'll need to create a build specification file named buildspec.yml
in your project directory. This file will contain the build commands and instructions for CodeBuild.
You can refer to the official documentation here for detailed instructions on creating a buildspec.yml
file.
version: 0.2
phases:
install:
runtime-versions:
java: corretto11
pre_build:
commands:
- echo Nothing to do in the pre_build phase...
build:
commands:
- echo Build started on `date`
- sam build
- sam package --output-template-file pcoutputtemplate.yaml --s3-bucket com-appsdev-pc-001
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- pcoutputtemplate.yaml
Push the new file to the repository from local.
After this, you can build the project to see if the Code is Pulled from Repo and artifacts are built.
Create a CodePipeline
- Create a new pipeline in AWS CodePipeline.
- Connect the pipeline to your CodeCommit repository as the source.
- Configure the pipeline to use CodeBuild as the build stage.
- Add a deploy stage to trigger CloudFormation using the template you created.
The pipeline will automatically build and deploy code based on the git commit.
Test the Pipeline
- Open your project in IntelliJ.
- Make a small change in one of your files, for example, add another logger line.
- Commit the change to your local Git repository.
- Push the commit to the CodeCommit repository.
$ git branch CR01
$ git checkout CR01
$ git add .
$ git commit -m “CR01”
$ git push --set-upstream origin CR01
You cand also create a pull request in aws code commit, just for simplicity I am merging from local
$ git checkout master
$ git merge --ff-only CR01
$ git push
- Go to the AWS CodePipeline console.
- You should see the pipeline start automatically. It will pull the latest code from the repository, build the project using CodeBuild, and deploy it using CloudFormation.
- Monitor the progress in the CodePipeline dashboard.
This will simulate the process of making changes to your code, pushing them to the repository, and having the CICD pipeline automatically trigger and deploy the updated code.
By harnessing AWS API Gateway, Lambda, DynamoDB, CodeCommit, CodeBuild, and CodePipeline, the article demonstrates how to architect a resilient and automated deployment process.
Thank you for reading. May your serverless endeavors be met with success and innovation!