Cyber attacks against APIs (Application Programming Interfaces) are on the increase. These attacks arise from issues with proper authentication, authorization, unnecessary data exposure, lack of request limits, resource consumption, and use of vulnerable third-party APIs.
Gaps in APIs can occur before requests reach the APIs, within the code housing the APIs, and even along the path of the APIs’ communication with downstream services, dependencies, or other microservices.
Attackers leverage flaws in APIs to gain access to confidential data, harvest or manipulate data, or even make your service unavailable through distributed denial of service attacks.
In this article, you’ll learn to deploy your APIs in Lambda and apply some security measures pre-function, within the function, and post-function.
Table of Contents
What is an API?
The focus of this article is the security of Application Programming Interfaces (APIs). An API is an interface that connects two programms or applications, allowing them to exchange data and communicate.
An API can be internal to an organization or it can belong to a third-party that allows other users to consume their data through the API.
Requirements/Prerequisites
While this tutorial is beginner-friendly, you’ll need the following prerequisites to follow along seamlessly:
A basic knowledge of the AWS Cloud.
An AWS account with administrator access.
AWS CLI. You can find the installation guide here. Follow the instructions for your operating system.
Python. You can visit Python’s official documentation site for a guide on how to download and install Python for your specific operating system.
Pipenv or any Python virtual environment creation tool. You can find the Pipenv installation guide here.
A basic knowledge of Git.
An API client, like Postman or Thunderclient.
Project Goal
By the end of this project, you should be able to deploy APIs in Lambda securely, leveraging AWS cloud-native security services.
Project Overall Architecture
Below is the architecture of the project workflow:
As shown in the architectural diagram, when a user sends a request (a JSON object consisting of the user’s name) to an API hosted in Lambda, the user first gets authenticated by an authentication service called Amazon Cognito.
The request passes through a Web Application Firewall, then an API Gateway. API Gateway will perform a check to see if the user is authorized to access the API using the token that the user sends with the request after authentication. API Gateway then allows the traffic to pass through to the API if the user is authorized.
The user’s request will first get to an External Lambda function, which will then save the user’s name as a message to a Simple Notification Service (SNS) topic. This will then invoke an Internal Lambda to run and log the output in Amazon CloudWatch logs. The SNS topic will be accessed by External Lambda using the SNS’s unique identifier stored in Amazon Secrets Manager.
AWS Set Up
You’ll need to set up an AWS environment to get started. This requires creating an account if you don’t already have one.
Following account creation, a root user is automatically created, with all privileges attached to the user. Security best practice is to create another user with administrator privileges and use this user for subsequent tasks.
Then, create an access key for this user, which usually consists of two parts (Access Key ID and Secret Access Key) by navigating to the following:
IAM —→ Users —→ Create Access key
Follow the prompts and choose the Command Line Interface option. Check the Confirmation box, and go on to create the key. Download the CSV file provided, or manually copy the Access Key ID and Secret Access Key. Save them securely.
Open up your terminal and run the following commands using the AWS CLI:
aws configure
The above command will give some prompts to provide the components of the Access Key created earlier and your default region (the AWS region hosting the service you intend to interact with).
Clone Project
In the next step, you’ll clone the GitHub repository containing the assets and resources used in the project implementation.
Visit the project URL and clone the repository locally.
git clone <repository_clone_url>
Set Up Simple Notification Service
Amazon Simple Notification Service (SNS) connects system components, enabling asynchronous communication and messaging among them.
Find SNS on the console, click on it, and create a topic that your APIs will send messages to. After successfully creating a topic, navigate to the topic, and in the topic details, you’ll find the topic’s ARN. An ARN is an Amazon Resource Name, and it’s a unique string attached to a resource you’ve created on AWS to help identify the resource. Copy the ARN of the topic.
Set Up Secrets Manager
Amazon Secrets Manager is used to store, manage, and retrieve sensitive information such as keys, credentials, tokens, and so on. You’ll store the Topic ARN created earlier. With this approach, you’ll demonstrate how your API can securely access the data and information it needs for its performance.
Go to Secrets Manager on the AWS console and create a secret. Provide the secret’s details, and add a new secret named TOPIC_ARN as the key and the actual SNS Topic ARN as the value.
Next, you’ll create some Lambda functions to serve your APIs and consume the output of the APIs. There’re three Lambda functions to set up. Two of the functions will host APIs, each of which can only be accessed by specific users. These will be referred to as ExternalLambda. The third Lambda will consume the output of the External Lambda functions through SNS.
Set Up Internal Lambda
AWS Lambda is a serverless service on AWS that users can leverage to run application functions or code when needed. You’re billed for your Lambda function based on the number of invocations of the function, the duration each invocation lasted, and the amount of memory allocated to the function. Lambda can be provisioned to use any runtime, such as Python or NodeJS. In this demonstration, you’ll focus on the NodeJS runtime.
Now that you know what Lambda is and does, you can create one. Let’s call the first Lambda function InternalLambda. On the AWS console, search for Lambda, and on the Lambda dashboard, click Create a function and provide the details. We’ll be using Node.js – JavaScript at the backend as the runtime of choice.
For the Permissions details, let Lambda create a default IAM Role. This default role is named according to your function, and the permissions attached to the role allow your Lambda function to send logs to CloudWatch, another AWS service used for monitoring and observability.
As you can see in the last image above, the Lambda function you’ve created needs a trigger and sometimes, a destination. For your InternalLambda, the trigger is the SNS topic we configured earlier. This Lambda will read the messages that’ve been published to it, and then you can access the message from your client or even CloudWatch logs.
To achieve this, click the Add trigger button and provide the details.
Next, you’ll provide the code you want to invoke through Lambda. Find the code in the GitHub repository that you cloned earlier. Paste the code in the Lambda function code space and click on Deploy to deploy the function.
secure-lambda/InternalLambda/index.js
export const handler = async (event) => {
try {
console.log('Request successfully received from SNS');
let name = event['Records'][0]['Sns']['Message'];
let response = {
statusCode: 200,
body: JSON.stringify(`Hello ${name}. Greetings from InternalLambda!`),
};
console.log('Response: ', response);
return response;
} catch (err) {
let response = {
statusCode: 500,
body: JSON.stringify('An error occurred while processing your request.'),
};
console.error('Error processing event', err);
return response;
}
};
The function defined in the index.js file above is simply taking the event object sent to it from SNS and extracting the Message attribute within it. We’re using console.log here to view outputs from the function and ensure it behaves as expected. Just don’t use this in a production-ready application.
Set Up External Lambda
You’ll be creating two external Lambda functions: 1 and 2. These two functions will receive user requests, process them, and publish messages to your SNS topic.
On the Lambda console, create another function and name it ExternalLambda1. Allow Lambda to create a default IAM Role, as previously.
Paste the code snippet below in the ExternalLambda1 code space:
secure-lambda/ExternalLambda1/insex.js
import {
GetSecretValueCommand,
SecretsManagerClient,
} from "@aws-sdk/client-secrets-manager";
import { SNSClient,
PublishCommand
} from "@aws-sdk/client-sns";
const secretsManagerClient = new SecretsManagerClient();
const snsClient = new SNSClient({});
// Fetch topicArn from AWS Secrets Manager
async function getSecretValue(secretName) {
try {
const data = await secretsManagerClient.send(
new GetSecretValueCommand({
SecretId: secretName,
}),
);
if (data.SecretString) {
return JSON.parse(data.SecretString);
} else {
let buff = Buffer.from(data.SecretBinary, 'base64');
return JSON.parse(buff.toString("utf-8"));
}
} catch (err) {
console.error('Error retrieving secret', err); // added for debugging
throw err;
}
}
export const handler = async (event) => {
let name = event['name'];
console.log(`Request successfully received from ${name}`);
// Retrieve SNS Topic ARN from Secrets Manager
let topicArn;
let response;
try {
const secret = await getSecretValue('LambdaSNSTopicARN');
topicArn = secret.TOPIC_ARN;
} catch (err) {
response = {
statusCode: 500,
body: JSON.stringify('An error occured, try again later.'),
};
console.error('Failed to load SNS Topic ARN from Secrets Manager', err);
return response;
}
// Publish to SNS topic
try {
const snsResponse = await snsClient.send(
new PublishCommand({
Message: name,
TopicArn: topicArn,
})
);
console.log("Message published successfully:", snsResponse.MessageId);
response = {
statusCode: 200,
body: JSON.stringify(`Hello ${name}. Greetings from ExternalLambda1! Message forwarded to InternalLambda.`),
};
return response;
} catch (err) {
response = {
statusCode: 500,
body: JSON.stringify(`Sorry ${name}.An error occurred while processing your request.`),
};
console.error("Failed to publish message:", err);
return response;
}
};
The code above leverages the AWS SDK to fetch the ARN of the SNS topic created earlier from Secrets Manager. It then publishes a message to the topic.
The SDK already comes installed in the Lambda function. Outside of Lambda, the SDK has to be explicitly installed. The function receives its event from the client via API Gateway, which we’ll configure later.
The SNS topic you created earlier will be the destination for this function. For Lambda to publish a topic to SNS, it needs the necessary permission attached to its IAM Role. AWS can automatically take care of that during your configuration, as shown below.
For the trigger, you’ll use another service known as API Gateway. More on that later.
Follow the same steps to provision another Lambda known as ExternalLambda2.
The outcome of the External Lambda setup is as shown below:
Paste the code below into ExternalLambda2. It performs the same function as ExternalLambda1, but their output differ. Each of the two Lambda functions will be receiving traffic to a specific user that’s authorized to access the function.
secure-lambda/ExternalLambda2/index.js
import {
GetSecretValueCommand,
SecretsManagerClient,
} from "@aws-sdk/client-secrets-manager";
import { SNSClient,
PublishCommand
} from "@aws-sdk/client-sns";
const secretsManagerClient = new SecretsManagerClient();
const snsClient = new SNSClient({});
// Fetch topicArn from AWS Secrets Manager
async function getSecretValue(secretName) {
try {
const data = await secretsManagerClient.send(
new GetSecretValueCommand({
SecretId: secretName,
}),
);
if (data.SecretString) {
return JSON.parse(data.SecretString);
} else {
let buff = Buffer.from(data.SecretBinary, 'base64');
return JSON.parse(buff.toString("utf-8"));
}
} catch (err) {
console.error('Error retrieving secret', err);
throw err;
}
}
export const handler = async (event) => {
let name = event['name'];
console.log(`Request successfully received from ${name}`);
// Retrieve SNS Topic ARN from Secrets Manager
let topicArn;
let response;
try {
const secret = await getSecretValue('LambdaSNSTopicARN');
topicArn = secret.TOPIC_ARN;
} catch (err) {
response = {
statusCode: 500,
body: JSON.stringify('An error occured, try again later.'),
};
console.error('Failed to load SNS Topic ARN from Secrets Manager', err);
return response;
}
// Publish to SNS topic
try {
const snsResponse = await snsClient.send(
new PublishCommand({
Message: name,
TopicArn: topicArn,
})
);
console.log("Message published successfully:", snsResponse.MessageId);
response = {
statusCode: 200,
body: JSON.stringify(`Hello ${name}. Greetings from ExternalLambda2! Message forwarded to InternalLambda.`),
};
return response;
} catch (err) {
response = {
statusCode: 500,
body: JSON.stringify(`Sorry ${name}.An error occurred while processing your request.`),
};
console.error("Failed to publish message:", err);
return response;
}
};
Before moving on, you need to modify the External Lambda’s IAM Roles. Currently, IAM Roles only have permissions to write to CloudWatch and SNS (automatically added). External Lambda also needs permission to fetch the ARN of the SNS topic that was created earlier.
The point here is to show how to leverage a secrets manager, such as AWS Secrets Manager, to store sensitive information or data, and still access these securely. This approach is more secure than storing the ARN as an environment variable within Lambda.
Navigate to IAM, and click on Policies tab on the left. This brings you to a list of policies. Next, click on Create policy.
Search for secrets manager in the Policy editor.
Select the permissions Lambda needs to access Secrets Manager. In this case, that would be Read —> GetSecretValue.
Select Specific for Resources, and click on Add ARNs. On the next tab, add the details of the Secrets Manager Secret created earlier.
The Secret’s ARN will be populated here.
Next, give the policy a name and create it.
Next, navigate to Roles, and search for the IAM Roles assigned to the External Lambda functions. These are named according to the Lambda.
Click Add permissions to add a new permission to the IAM Role selected.
Configure Web Application Firewall
A firewall is a system placed in front of an application, workload, APIs, and so on to inspect traffic, filter it, and either allow or block the traffic based on some preconfigured rules.
For this project, you’ll use the AWS Web Application Firewall (WAF) service to inspect user requests before routing the traffic to your APIs running in Lambda.
Head over to the AWS console and search for WAF.
Click on the IP sets tab on the left. This will enable you to create a list of IP addresses that you want to allow (as in this case) or deny.
The IP addresses should include a CIDR block. For instance, if adding a single IP address, it should be X.X.X.X/32. The same applies to IP address ranges such as X.X.X.X/24.
Next, click on the Web ACLs tab, then Create web ACL.
Choose Regional resources as the Resource type, and enter your region. It’s best to keep all resources you’re creating in this project within the same region. Give your Web ACL a name, then click next.
Add rules to the Web ACL.
Choose a rule type. In this case, you’ll use IP set, and give the rule a name. Choose the IP set created earlier.
Select Source IP address, and Count as the Action. For this project, you’ll focus on counting the requests sent to your APIs. But as shown in the image below, you can perform other actions, such as allow, block, and so on.
Your final rule configuration will appear this way.
Scroll down, then click on Create web ACL.
Configure Cognito User Pools
Amazon Cognito is an identity management service used for creating and managing users. You can leverage it to authenticate and authorize users to applications, APIs, or other workloads.
You’ll create User Pools within Cognito and add a user to each pool. You’ll configure how these users can be authenticated and authorized to access the External Lambda functions already created.
Search for Cognito on AWS.
Click on Get started for free, then Create user pool.
Select Single-page application (SPA), give the User pool the name MyUserPool1, and select Email as an option for sign-in. This means the main attribute users will provide at signup and sign-in will be their email address. Leave everything else as the default. We’ll keep things as simple as possible.
After creating the User pool, you’ll find the page shown below. You can view the login and signup page for the pool you’ve just created by clicking on the View login page button.
You can add App clients to your User Pool. By default, a client named MyUserPool1 will be added to the pool. Navigate to your User pool, and click on App clients to see details of this client. Note the Client ID. You’ll also make some edits to the App client by clicking on the Edit button.
You’ll edit the Authentication flows field by ticking the Sign in with username and password… and Sign in with server-side administrative credentials… boxes. These changes will enable you to authenticate the user who will be added to this client programmatically, rather than through a UI. With this approach, we can fetch the token assigned to the user by Cognito and use this token to authorize access to Lambda.
Now, add a user to this pool. The user needs a valid email address. You’ll need the login page URL to create the user.
You need access to the email used to create the user. Fetch the code sent to the email address and submit to confirm the account.
Follow the same steps and create another User pool named MyUserPool2. Add a user with a different email to this pool.
Configure API Gateway
API Gateway is a service used to manage access and route traffic to API backend services such as APIs. It serves as a reverse proxy and provides an extra layer of security for backend services.
You’ll configure API Gateway to direct traffic to your Lambda functions.
Navigate to API Gateway and click on Create an API.
Select the REST API option —→ Build.
Select New API, provide a name, and choose Regional as the API endpoint type. IP address type can be IPv4 or Dualstack. We’ll select IPv4 here. Then create.
An important part of API Gateway configuration for this project is the Authorizer. API Gateway uses Authorizer to allow traffic from clients to backend services.
You’ll create two Authorizers. Each will be connected to one of the User pools you configured earlier. On the left-hand side of the API Gateway you configured, click on Authorizers —→ Create authorizer.
Provide the name AGAuthorizer1, and select Cognito as the Authorizer type. Add the User pool for MyUserPool1 created earlier. For the Token source, use Authorization. When you send a request from your API client, a token will be added to the request header for authorization. The token’s key will be named Authorization, while the value will be the token itself.
Create another Authorization for MyUserPool2 the same way.
Both Authorizers will appear this way.
Next, you’ll create resources and endpoints within the API Gateway that you’ve defined.
A resource in API Gateway is used to group certain endpoints within a specific path. You’ll define two resources within the API Gateway you’ve created. This will create two different paths, <BASE_URL>/<RESOURCE1> and <BASE_URL/RESOURCE2>.
On the API Gateway dashboard, navigate to your Gateway, click on Create resource, define your root path (‘/’ in your case), and provide the resource name (lambda1).
Create another resource named lambda2.
Now, click on /lambda1, then Create method to define an endpoint within this resource. You’ll use the POST method to send requests to the backend service via this endpoint.
For the backend service or Integration type, select Lambda function, and provide the ARN of ExternalLambda1.
For Authorization, select AWS IAM —→ Cognito user pool authorizers —→ AGAuthorizer1. Leave other configurations, then create the endpoint.
Repeat the same step to create a POST method for /lambda2 resource. The method should be attached to ExternalLambda2, and AGAuthorizer2.
The API Gateway you’ve created needs to be deployed to become accessible. Deployment is usually done to a Stage.
Click on Deploy API, select New stage and name the stage development. Then, deploy.
After deployment to a stage, an invoke URL will be provided. This will serve as the base URL for the endpoints you’ve defined.
The stage you’ve created needs some modifications for enhanced security. Firstly, you need to attach the WAF that you created earlier. Secondly, the default rate limit for the API deployed to this stage is 10000. Rate limit restricts excessive resource consumption and protects your API from abuse. For this project, you can reduce the limit to 50.
To test the API Gateway set up, click on the endpoint you want to test, then the Test button. This initial test doesn’t need any authorization, since the test is done directly within the Gateway.
Add JSON data as the Request body. The key will be name, and the value will be any string.
The response sent back from ExternalLambda1 shows a status code of 200, and a response body containing exactly the message expected from the Lambda function.
If you head over to CloudWatch Log groups, you should also find the Log groups that were automatically created for the Lambda functions. Click on the Log group for ExternalLambda1 and navigate to the latest Log stream. You should find the logs for the request you’ve just made from API Gateway.
Test Setup End-to-End
To test our setup properly, and from the internet, send the same request from your API client with no additional information in the request header. This should return a 401 error – Unauthorized. This is expected.
API Gateway expects an authorization token from each request it receives before routing traffic to the appropriate backend service. It validates this token through Cognito.
You’ll mimic a user login for each user added to Coginito User pools to get a token for the user. This token will then be sent alongside any request. To achieve this, you’ll use the two Python scripts I’ve provided below:
secure-lambda/auth-scripts/user1.py
import boto3
client = boto3.client("cognito-idp")
response = client.initiate_auth(
AuthFlow="USER_PASSWORD_AUTH", # or ADMIN_USER_PASSWORD_AUTH if using admin creds
AuthParameters={
"USERNAME": "", # user1 email
"PASSWORD": "" # user1 password
},
ClientId="" # Cognito App Client ID
)
id_token = response["AuthenticationResult"]["IdToken"]
access_token = response["AuthenticationResult"]["AccessToken"]
refresh_token = response["AuthenticationResult"]["RefreshToken"]
print("ID Token:", id_token)
Using the Python boto3 library, you’ll initiate an authentication request to Cognito. Provide the email address and password of the user in MyUserPool1. Also, add the Client ID of the App client.
To run the script, create an isolated environment using Pipenv, uv, or a similar library. Install the dependency used in the project as defined in the Pipfile, and run the script with the Pipenv shell.
pipenv install
pipenv shell
Python secure-lambda/auth-scripts/user1.py
The Python command will return with a token assigned to the user. Next, you use this token to authorize a user to access ExternalLambda1.
Ensure that the URL for the POST request is in the format: <BASE_URL/lambda1>. You should receive a response from API Gateway indicating success.
Now try accessing ExternalLambda2 using User1 token. You should get an Unauthorized message. Note that user1 will always receive an unauthorized message when it tries accessing ExternalLambda1 without an Authorization token in the header, a wrong token, or when it tries accessing ExternalLambda2, which it is not authorized to access.
Repeat the process with User2 using the token generated for the user in MyUserPool2. First, test access to ExternalLambda2 without a token in the request header.
Then test access with the token.
Next, try accessing ExternalLambda1 using User2.
You can also view the outcome of some of the requests made by your client on CloudWatch Logs.
Also, since WAF has been configured previously to count requests (although, in a real scenario, you want to achieve much more with WAF, such as allow or block certain traffic), you can view activities captured by WAF by navigating to the service on AWS, then searching for the WAF you configured, and navigating to Traffic overview.
You can find other details, such as the client device types and where requests originated.
Clean Up
It’s important to clean up the resources created so far after the hands-on exercise. Due to the dependencies among the resources, trying to delete a resource that another resource depends on may lead to an error. So, you should delete them in this order:
Secrets Manager
Cognito – Users, App Client, then User Pool
API Gateway – Endpoints/ Methods, Resources, API, Stage
Web Application Firewall – IP Set, Web ACL
All Lambda Functions
Lambda IAM Roles and the policies attached to them
CloudWatch Log Group for all the Lambda functions
SNS Topic
Also, you can deactivate or delete the credentials created for your IAM Admin user if not in use.
Improvements
Consider the following areas to improve, apply best practices to, and enhance the security posture of your systems further.
Use of API keys
Third-party API consumption
API inventory management/ documentation
Resource provisioning using Infrastructure as Code
Conclusion
Security at every layer of an IT system is not negotiable. In this project, we’ve demonstrated how to leverage cloud-native solutions to secure APIs hosted in a serverless service, allowing only authorized users access to the APIs.
I’m Agnes Olorundare, and you can find out more about me on LinkedIn.