In this article, I’ll teach you how you can deploy a single Docker container using a serverless service on AWS called Lambda.

Table of Contents

• Prerequisite/ Requirements

• Serverless with AWS Lambda

• How to Build, Run, and Test a Container Locally

• How to Push Your Image to Amazon Elastic Container Registry (ECR)

• How to Deploy Your Docker Image to Lambda

• Cleanup

• Conclusion

Prerequisite/ Requirements

The following tools and skills are necessary for following along with this tutorial:

• Knowledge of Docker, and have Docker installed locally.

• An AWS account with credentials with administrative privilege for making API calls via the CLI. Best practice would be to limit the privilege to exactly what needs to be done.

• AWS CLI installed locally

• Python virtual environment managers such as uv (optional)

Serverless with AWS Lambda

Containers provide a lightweight, consistent, and resource-friendly way of running applications. Serverless takes away the overhead of managing the underlying infrastructures on which the container runs. So as you can probably start to see, combining these tools helps you deploy applications in a way that lets you focus on business logic, performance, and what gives your product a competitive edge/ advantage.

One AWS tool that enables you to go serverless is Lambda. With Lambda, you’re only billed for the number of times the code in the function runs, the memory you selected at the time of provisioning the service, and the duration of each invocation of the function.

In addition to removing operational overhead, Lambda can also help you save money since you won’t have to deal with idle resources. The function only comes alive when triggered by a request sent to it.

How to Build, Run, and Test a Container Locally

Docker is a tool that helps you package applications or software into portable, standardized and shareable units that have everything the applications need such as libraries, runtime, system tools, application code, in order to run. These units are called containers.

In this section, I’ll walk you through building the Docker image, running the container, and testing it after it’s running.

You can find the project that you’ll be using here in this GitHub repository.

Build the Docker Image

To run a Docker container, you first need to build an image. The image becomes the template or class from which you create the container or instance of the class.

You can find the code to build an image in lambda_function.py.

# lambda_function.py

def lambda_handler(event, context):
    name = event["name"]
    message = f"Hello, {name}!"

    try:
        return {
            "statusCode": 200,
            "body": message
        }
    except Exception as e:
        return {
            "statusCode": 400,
            "body": {"error": str(e)}
        }

As you can see from the code above, this is a very basic Python application that expects a POST HTTP request, with a JSON payload that contains the key – name – and a corresponding value. The code then returns a greeting containing the name it has received. The application has just a single function, which also serves as the entry point to it.

To build a Docker image, you’ll need a Dockerfile to provide the blueprint for the image. For this specific case, the Dockerfile you’ll use is also very basic. Each line in a Dockerfile is called a Directive, and this provides the instruction Docker should follow when creating an image. So building a Docker image means creating a template for a container by following the instructions or directives in the Dockerfile.

# Dockerfile

FROM public.ecr.aws/lambda/python:3.12

# Copy function code... LAMBDA_TASK_ROOT is /var/task, the working directory set in the base image
COPY lambda_function.py ${LAMBDA_TASK_ROOT}    

# Set the CMD to your handler - lambda_handler
CMD ["lambda_function.lambda_handler"]

A Dockerfile usually starts with a base image. To deploy an application as a Docker container in AWS Lambda, the base image has to be of a specific kind, depending on the application run-time. For this case, you’ll need the Python run-time, so the base image is public.ecr.aws/lambda/python:3.12. It’s okay to use a different Python version.

The next directive in the Dockerfile is copying the lambda_function.py file to a specific path in the base image. That path is referenced using an environment variable that has already been defined in the base image and points to /var/task. This is the directory your code will be running from.

The last directive is simply a command to start the application when the container runs.

Now, you can run the build command from the project’s root directory:

docker build -t <IMAGE_NAME>:<iIMAGE_TAG> .

Run the Docker Container

Next, let’s create a running container from this image.

docker run -it --rm -p 8080:8080  lambda_docker:1.0.0

The command above will create a container and run it in interactive mode just so you can see the logs generated by the application in the container. Port 8080 is also exposed on the host where the container is running and mapped to the container port, which is also 8080 (defined by AWS). The container gets automatically removed once you kill the running process with CTRL + C.

Test the Running Container

Now confirm that the application running within the container can receive and process requests. To do this, use the code in the test.py file:

# test.py

import requests

url = "http://localhost:8080/2015-03-31/functions/function/invocations"

data = {
    "name": "Janet"
}

response = requests.post(url, json=data)

print("Status Code:", response.status_code)
print("Response Body:", response.json())

You can use the Python requests library to make this call. Install the library by using a virtual environment to isolate the application from your overall system. This helps prevent issues with conflicts in the versions of libraries you install for an application to use.

If you’re using uv to manage your virtual environment, simply run the command:

uv add requests

Then run the code in test.py from within the virtual environment:

uv run python3 test.py

You should see the desired response on the terminal.

How to Push Your Image to Amazon Elastic Container Registry (ECR)

Now that you have a working Docker image to deploy to Lambda, the next step is to push the image to a Docker registry. For this use case, your image has to be pushed to Amazon ECR, a container registry for storing Docker images.

To push your Docker image, you first need to tag the image, which simply means naming the image in a specific way.

Currently, this image tag is lambda-docker:1.0.0. To tag it the AWS way, first create an ECR repository. Let’s use the AWS CLI for this (this requires you to configure the AWS credentials locally by running the aws configure command and providing your credentials).

Setup Environment Variables

# Set AWS profile
export AWS_PROFILE=<PROFILE_NAME>

# Set other variables

AWS_REGION=<AWS_REGION>
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
REPO_NAME=lambda-docker
TAG=1.0.0

The above commands set the AWS_PROFILE for the CLI to target the right AWS account for API calls. The other variables specify the region, account ID, and the ECR repository name and tag.

Create ECR Repository and Authenticate

Now, create the ECR repository:

aws ecr create-repository \
  --repository-name "$REPO_NAME" \
  --region "$AWS_REGION"

Authenticate to Amazon ECR:

aws ecr get-login-password --region "$AWS_REGION" \
  | docker login \
  --username AWS \
  --password-stdin "$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com"

Tag and Push the Docker Image

Now, tag the Docker image:

docker tag $REPO_NAME:$TAG \
  $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$REPO_NAME:$TAG

Push the image to the ECR repository you created:

docker push $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$REPO_NAME:$TAG

And that’s it! Your image is now in ECR.

How to Deploy Your Docker Image to Lambda

With your image now in ECR, you can create a Lambda function. Navigate to the Lambda console, and click Create a Function.

Create Lambda Function

Select Container Image and go ahead to search for the ECR repository you created.

Next, select the image:

Leave other configurations as default and click create.

Navigate to the function after creating.

Test Deployment

Now, let’s test the deployment. For this, simply use the existing Lambda Test tab. Provide all the details needed, including the payload for your POST request.

And that’s it. You’ve successfully deployed a Docker container on AWS by leveraging ECR and Lambda. You can go a step forward by integrating API Gateway and making the function accessible from the internet.

Cleanup

Remember to delete the services you’ve created on your AWS ECR repository and Lambda to avoid extra charges.

Conclusion

Deploying your Docker container on AWS Lambda is an efficient way to get your application running quickly without being bothered by managing servers or platforms.

Thanks for reading!

They take parameters with which you can modify how they perform the sorting. And one of those parameters could be a function or even a lambda function.

In this article, you’ll learn how to sort a list with the lambda function.

What We'll Cover

• How to Sort a List in Python
• What is a Lambda Function?
• How to Sort a List with the Lambda Function
  • How to Lambdasort with the sort() Method
  • How to Lambdasort with the sorted() Function
• Conclusion

How to Sort a List in Python

You can sort a list with the sort() method and sorted() function.

The sort() method takes two parameters – key and reverse. You can use a function as the key, but the reverse parameter can only take a Boolean.

If you specify the value of the reverse parameter as True, the sort() method will perform the sorting in descending order. And if you specify false as the value of the reverse, the sorting will be in ascending order. You don’t even need to specify false as the value because it’s the default.

But both parameters are optional, so the method still works fine without them:

name_list = ['Zen Jack', 'Luigi Austin', 'Ben Benson', 'John Ann']
print("Original names:", name_list) # Original names: ['Zen Jack', 'Luigi Austin', 'Ben Benson', 'John Ann']


name_list.sort()
print("Sorted names:", name_list) # Sorted names: ['Ben Benson', 'John Ann', 'Luigi Austin', 'Zen Jack']


num_list = [34, 11, 35, 89, 37]
print("Original numbers:", num_list) # Original numbers: [34, 11, 35, 89, 37]


num_list.sort()
print("Sorted numbers:", num_list) # Sorted numbers: [11, 34, 35, 37, 89]

The sorted() function, on the other hand, also works like sort(). It takes the optional key and reverse parameters too, but it takes a compulsory parameter of the iterable you want to sort – making it ideal for sorting other iterables apart from a list.

Here’s how the sorted() function works:

name_list = ['Zen Jack', 'Luigi Austin', 'Ben Benson', 'John Ann']
print("Original names:", name_list) # Original names: ['Zen Jack', 'Luigi Austin', 'Ben Benson', 'John Ann']


sorted(name_list)
print("Sorted names:", name_list) # Sorted names: ['Ben Benson', 'John Ann', 'Luigi Austin', 'Zen Jack']


num_list = [34, 11, 35, 89, 37]
print("Original numbers:", num_list) # Original numbers: [34, 11, 35, 89, 37]


sorted(num_list)
print("Sorted numbers:", num_list) # Sorted numbers: [11, 34, 35, 37, 89]

As I pointed out earlier, you can also sort other iterables with the sorted() function. This is how I sorted a tuple with the sorted() function:

name_tuple = ('Zen Jack', 'Luigi Austin', 'Ben Benson', 'John Ann')
print("Original tuple:", name_tuple) # Original tuple: ('Zen Jack', 'Luigi Austin', 'Ben Benson', 'John Ann')


sorted(name_tuple)
print("Sorted tuple:", name_tuple) # Sorted tuple: ('Zen Jack', 'Luigi Austin', 'Ben Benson', 'John Ann')

Remember I also pointed out you can pass a function as the value for the key parameter of the sort() method and sorted() function. This function can help you be more decisive with the way you want to sort the iterable list or tuple.

For example, the sorted() function and sort() method would only sort by the first part of the string or number. But you can also sort by the second part by passing in a function as the key parameter. It is with this function that you will decide how you want to sort the list or other iterables.

A lambda function would be ideal to do this because it takes one expression. But before we dive into sorting with a lambda function, let me remind you what the lambda function is.

What is a Lambda Function?

A lambda function is an anonymous function – a function you don’t write with the def keyword. A lambda function can take many arguments, but it can only have one expression.

Since you don’t define a lambda function with the def keyword, how do you call it? You can assign a lambda function to a variable, then call it by the name of that variable.

In the example below, the addNum lambda function has 3 arguments and adds them together:

addNums = lambda a, b, c : a + b + c


res = addNums(4, 12, 4)
print(res) #20

How to Sort a List with the Lambda Function

You can “lambda sort” a list with both the sort() method and the sorted() function. Let’s look at how to lambda sort with the sort() method first.

How to Lambdasort with the sort() Method

Let’s take the names we sorted before and sort them by the second name. This lambda function would be ideal in sorting by the second name:

lambda name: name.split()[1]

The lambda function splits a name and takes the second part of the name – the second name. The first part is the first name and it would be [0].

You can pass in this lambda function as the key parameter – sorting the names by the second names:

name_list = ['Zen Jack', 'Luigi Austin', 'Ben Benson', 'John Ann']
print("Original list", name_list) # Original list ['Zen Jack', 'Luigi Austin', 'Ben Benson', 'John Ann']


name_list.sort(key=lambda name: name.split()[1])
print("Sorted name list", name_list) # Sorted name list ['John Ann', 'Luigi Austin', 'Ben Benson', 'Zen Jack']

You can see the names got sorted by the alphabetical order of the second names. John Ann came first, and Zen Jack came last. Ann starts with A and Jack starts with J.

If you want, you can even pass in a function directly. What you need to do is to not call the function. You have to pass it in as an object.

name_list = ['Zen Jack', 'Luigi Austin', 'Ben Benson', 'John Ann']
print("Original list", name_list) # Original list ['Zen Jack', 'Luigi Austin', 'Ben Benson', 'John Ann']


def sort_by_second_name(name):
    return name.split()[1]


name_list.sort(key=sort_by_second_name)
print("Sorted name list", name_list) # Sorted name list ['John Ann', 'Luigi Austin', 'Ben Benson', 'Zen Jack']

Now, let’s sort the numbers based on their last digits. In case you don’t know, if you use the remainder (%) operator on two numbers, it divides the two numbers and returns the remainder:

print(4 % 2) # returns 0
print(44 % 11) # returns 0
print(104 % 60) # returns 44
print(21 % 2) # returns 1

But if you “mod” a number with multiple digits with 10, it returns the last digit of the number:

print (44 % 10) # returns 4
print (402 % 10) # returns 2
print (152 % 10) # returns 2
print (1596 % 10) # returns 6

That’s how you can get the second number and sort the numbers based on it.

Here’s how I sorted the numbers from the previous examples with a lambda function:

num_list = [22, 34, 11, 35, 89, 37, 93, 56, 108]
print('Original Number:', num_list) # Original Number: [22, 34, 11, 35, 89, 37, 93, 56, 108]


num_list.sort(key=lambda num: num % 10)
print('Lambda sorted number:', num_list) # Lambda sorted number: [11, 22, 93, 34, 35, 56, 37, 108, 89]

As you can see, this lambda function, lambda num: num % 10 is responsible for sorting the numbers based on each of the second digits. I passed in the number to the lambda function and got the last digit with % 10. This lambda function runs through each of the numbers and gets their last digits.

If you want, you can even pass in a function directly as the key:

num_list = [22, 34, 11, 35, 89, 37, 93, 56, 108]
print('Original Number:', num_list) # Original Number: [22, 34, 11, 35, 89, 37, 93, 56, 108]


def sort_by_second_num(num):
    return num % 10


num_list.sort(key=sort_by_second_num)
print('Lambda sorted number:', num_list) # Lambda sorted number: [11, 22, 93, 34, 35, 56, 37, 108, 89]

How to Lambdasort with the sorted() Function

In this example, we are going to sort a list of tuples using the jersey numbers of some footballers.

The only difference between the sort() and sorted() method is that sorted() takes a compulsory iterable and sort() does not.

So, to lambda sort with the sorted() function, all you need to do is pass in the list as the iterable and your lambda function as the key:

 footballers_and_nums = [("Fabregas", 4),("Beckham" ,10),("Yak", 9), ("Lampard", 8), ("Ronaldo", 7), ("Terry", 26), ("Van der Saar", 1), ("Yobo", 2)]


sorted_footballers_and_nums = sorted(footballers_and_nums, key=lambda index : index[1])


print("Original footballers and jersey numbers", footballers_and_nums) # Original footballers and jersey numbers [('Fabregas', 4), ('Beckham', 10), ('Yak', 9), ('Lampard', 8), ('Ronaldo', 7), ('Terry', 26), ('Van der Saar', 1), ('Yobo', 2)]


print("Sorted footballers by jersey numbers:", sorted_footballers_and_nums) # Sorted footballers by jersey numbers: [('Van der Saar', 1), ('Yobo', 2), ('Fabregas', 4), ('Ronaldo', 7), ('Lampard', 8), ('Yak', 9), ('Beckham', 10), ('Terry', 26)]

The lambda function that performed the sorting is this lambda index : index[1]). The lambda went through all the tuples in the list, took the second index ([1]), and used those to do the sorting.

Conclusion

This article showed you how to sort a list with the sort() method and sorted() function using a lambda function.

But that was not all. We looked at how both the sort() method and sorted() function work on their own without a lambda function. I also reminded you of what the lambda function is, so you could understand how I did the sorting with a lambda function.

Thank you for reading!

You can define them using the lambda keyword followed by the function's inputs, a colon, and the function's expression. The output of a lambda function is returned as the result of the expression, rather than a return statement.

The main purpose of lambda functions is to allow for the creation of small, throw-away functions that you can use in other parts of a program. This can be useful when you need to pass a simple function as an argument to another function, for example.

Syntax for creating lambda functions

Here is the syntax for creating a lambda function in Python:

lambda arguments: expression

Lambda Functions vs Named Functions Defined Using the def Keyword

When compared to named functions defined using the def keyword, lambda functions have some key differences.

Named functions can have multiple expressions and they use return statements. Lambda functions can only have one expression and the value of that expression is returned automatically.

Named functions can also be reused throughout a program, while lambda functions are used only once.

def greet(name):
    return "Hello " + name
print(greet("John")) 

# Output: "Hello John"

As you can see, the named function is defined using the def keyword, followed by the function name and its inputs in parentheses. The function body is indented, and the return statement is used to return the result of the expression.

Named functions can be called multiple times, making them more flexible than lambda functions for complex operations.

Here's the equivalent code using a named function defined with the lambda keyword:

greet = lambda name: "Hello " + name
print(greet("John")) 

# Output: "Hello John"

How to Use Lambda Functions

This section covers the basics of how to create and use lambda functions, including their syntax and how to use them as arguments in higher-order functions such as map, filter, and reduce.

Additionally, this section explores how you can use lambda functions in sorting and in list comprehensions. By the end, you should have a foundational understanding of the basic uses of lambda functions.

How to use lambda functions as arguments in higher-order functions (map, filter, reduce)

Lambda functions are often used as arguments in higher-order functions such as map, filter, and reduce. These functions allow you to apply a given operation to every element of a list or other iterable.

Here's an example of using a lambda function with the map function:

numbers = [1, 2, 3, 4, 5]
squared_numbers = list(map(lambda x: x**2, numbers))
print(squared_numbers) 

# Output: [1, 4, 9, 16, 25]

In this example, the lambda function takes one input x and returns the square of that value. The map function applies this operation to every element of the numbers list and returns a new list with the results.

Here's another example using the filter function:

numbers = [1, 2, 3, 4, 5]
even_numbers = list(filter(lambda x: x % 2 == 0, numbers))
print(even_numbers) 

# Output: [2, 4]

In this example, the lambda function takes one input x and returns True if x is even, and False otherwise. The filter function applies this operation to every element of the numbers list and returns a new list with only the elements that returned True.

How to use lambda functions to return functions as values

You can also use lambda functions to return functions as values. For example:

def make_adder(x):
    return lambda y: x + y
add5 = make_adder(5)
print(add5(3)) 

# Output: 8

In this example, the make_adder function takes one input x and returns a lambda function that takes one input y and returns the sum of x and y. The add5 variable is assigned the result of calling make_adder(5), meaning it now references a lambda function that adds 5 to its input.

How to use lambda functions in sorting

You can also use lambda functions in sorting operations to specify custom sort orders. For example:

numbers = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5]
sorted_numbers = sorted(numbers, key=lambda x: -x)
print(sorted_numbers) 

# Output: [9, 6, 5, 5, 5, 4, 3, 3, 2, 1, 1]

In this example, the lambda function takes one input x and returns -x, meaning that the sort order will be in descending order. The sorted function sorts the numbers list based on the values returned by the lambda function.

Limitations of Lambda Functions

While lambda functions are a convenient way to write short and simple functions, they have some limitations.

One of the main limitations is that lambda functions are limited to a single expression, meaning that they cannot contain multiple statements or complex control flow.

In addition, lambda functions cannot be referenced by name and can only be invoked when they are defined, which makes them less flexible than named functions.

Another limitation is that lambda functions do not have a name, which can make debugging more difficult and make it harder to understand the code.

In general, it's a good idea to use named functions for complex operations and only use lambda functions for short and simple operations.

Advanced Uses of Lambda Functions

This section covers how to use lambda functions with advanced functions such as reduce, filter, sorted, and key arguments. Additionally, this section provides information on using lambda functions to create anonymous functions for event handlers.

How to use lambda functions with reduce

The reduce function is a higher-order function that takes a binary function (a function that takes two arguments) and a list. It returns a single value that is the result of applying the binary function to the elements of the list in a cumulative way.

For example, to calculate the product of all elements in a list, you could use the following code:

from functools import reduce
numbers = [1, 2, 3, 4, 5]
product = reduce(lambda x, y: x*y, numbers)
print(product) 

# Output: 120

In this example, the lambda function lambda x, y: x*y is used as the binary function in the reduce function. The reduce function starts by applying the binary function to the first two elements of the list, and then applies the result to the next element, and so on until it has processed all elements of the list.

How to use lambda functions with filter

The filter function is another higher-order function that takes a function and a list, and returns a new list that contains only the elements of the original list for which the function returns True.

For example, to filter out even numbers from a list, you could use the following code:

numbers = [1, 2, 3, 4, 5]
even_numbers = list(filter(lambda x: x % 2 == 0, numbers))
print(even_numbers) 

# Output: [2, 4]

In this example, the lambda function lambda x: x % 2 == 0 is used as the function argument in the filter function. The filter function invokes this lambda function for each element in the numbers list and includes the element in the result list only if the lambda function returns True.

How to use lambda functions with the sorted function

The sorted function is a built-in function that sorts a list of elements. The sorted function can take an optional key argument, which is a function that takes an element of the list and returns a value that is used as the sort key.

For example, to sort a list of dictionaries by a specific key, you could use the following code:

employees = [{"name": "John", "age": 32},              {"name": "Jane", "age": 27},              {"name": "Jim", "age": 40}]
sorted_employees = sorted(employees, key=lambda x: x["age"])
print(sorted_employees)

# Output: [{"name": "Jane", "age": 27}, 
#          {"name": "John", "age": 32}, 
#          {"name": "Jim", "age": 40}]

In this example, the lambda function lambda x: x["age"] is used as the key argument in the sorted function. The sorted function uses this lambda function to extract the "age" value for each dictionary in the employees list and uses these values as the sort keys.

How to use lambda functions in the key argument of various functions

In addition to the sorted function, many other functions in Python can take a key argument, including the max, min, and sorted functions.

The key argument is a function that takes an element of the list and returns a value that is used as the sort key, as well as for comparison purposes in the case of the max and min functions.

For example, to find the employee with the highest salary in a list of employees, you could use the following code:

employees = [{"name": "John", "salary": 50000}, {"name": "Jane", "salary": 55000}, {"name": "Jim", "salary": 60000}]
highest_salary_employee = max(employees, key=lambda x: x["salary"])
print(highest_salary_employee) 

# Output: {"name": "Jim", "salary": 60000}

In this example, the lambda function lambda x: x["salary"] is used as the key argument in the max function. The max function uses this lambda function to extract the "salary" value for each employee in the employees list and uses these values to compare the employees and find the one with the highest salary.

How to use lambda functions to create anonymous functions for event handlers

You can also use lambda functions to create anonymous functions for event handlers in GUI programming or for other similar purposes.

For example, in the following code, a button click event is handled using a lambda function in Tkinter (a GUI programming toolkit for Python):

import tkinter as tk

def on_button_click():
    print("Button clicked!")

root = tk.Tk()
button = tk.Button(root, text="Click Me!", command=lambda: print("Button clicked!"))
button.pack()
root.mainloop()

In this example, we use the lambda function lambda: print("Button clicked!") as the command argument of the Button widget in Tkinter. When the button is clicked, the lambda function is executed and the message "Button clicked!" is printed to the console.

This demonstrates the versatility and flexibility of lambda functions. You can use them in a variety of contexts where anonymous functions are required.

Best Practices for Using Lambda Functions

This section covers a range of best practices, including keeping lambda functions simple and easy to understand, avoiding complex expressions and statements, choosing the appropriate type of function for the task, and documenting lambda functions for better code readability.

We'll also highlight the importance of using descriptive variable names in lambda functions to improve the readability of your code.

Keep lambda functions simple and easy to understand

One of the best practices for using lambda functions is to keep them simple and easy to understand.

Lambda functions are intended to be small, anonymous, single-expression functions, and complex or multi-statement functions are better suited to be defined using the def keyword.

For example, the following lambda function is simple, easy to understand, and does exactly what it is intended to do:

square = lambda x: x * x
print(square(5)) 

# Output: 25

Avoid complex expressions and statements in lambda functions

In addition to keeping lambda functions simple, it is also important to avoid complex expressions and statements in lambda functions.

Complex expressions and statements make the code harder to understand and maintain, and can lead to bugs.

For example, the following lambda function is too complex and difficult to understand:

calculate = lambda x, y: x + y if x > y else x - y
print(calculate(5, 10)) 

# Output: -5

In such cases, it is better to define a named function using the def keyword and provide a meaningful name for the function. This makes the code more readable and easier to maintain:

def calculate(x, y):
    if x > y:
        return x + y
    else:
        return x - y

print(calculate(5, 10)) 

# Output: -5

When to use lambda functions and when to use named functions

Lambda functions are best used in situations where you need a small, anonymous, single-expression function. They are not well suited for complex functions with multiple expressions and statements.

For example, a good use case for a lambda function is as an argument to a higher-order function such as map, filter, or reduce. A bad use case for a lambda function is a complex function with multiple expressions and statements.

In general, it is better to use named functions defined using the def keyword for functions that are complex, multi-statement, or are used multiple times in your code.

Document lambda functions for better code readability

Another best practice for using lambda functions is to document them for better code readability.

While lambda functions are often intended to be simple and easy to understand, it can still be helpful to provide a brief explanation of what the function does in the form of a docstring or a comment.

For example, the following lambda function is documented for better code readability:

# This lambda function returns the square of its input
square = lambda x: x * x
print(square(5)) 

# Output: 25

Use descriptive variable names in lambda functions

Finally, it is important to use descriptive variable names in lambda functions, just as you would in any other function. Descriptive variable names make the code easier to understand and maintain.

For example, the following lambda function uses descriptive variable names:

# This lambda function returns the sum of its inputs
sum = lambda x, y: x + y
print(sum(5, 10)) 

# Output: 15

By following these best practices, you can ensure that your lambda functions are clear, concise, and easy to understand, making your code more readable, maintainable, and error-free.

Conclusion

In this guide, we covered the basics of lambda functions in Python, including their definition and purpose, syntax, and basic and advanced usage in various applications.

We also discussed some best practices for using lambda functions, including keeping them simple and easy to understand, avoiding complex expressions and statements, choosing the appropriate type of function for the task, and documenting them for better code readability.

Lambda functions can be a powerful tool for writing concise, readable, and efficient code. But they have some limitations, such as being restricted to a single expression and having limited functionality compared to named functions. It's important to consider these limitations and choose the appropriate type of function for the task at hand.

In conclusion, this guide has provided an overview of lambda functions and their uses in Python, and I hope that it has been helpful in your journey to learn more about this topic. For further learning, you may want to explore the official Python documentation and practice using lambda functions in your own projects.

Visit my blog to read more articles like this. You may also choose to follow me on Twitter.

In this article, you'll learn how to create and use anonymous functions in Python. They are also called lambda functions.

We'll begin with a quick overview of how regular functions are created in Python. Then you'll learn the syntax and practical applications of anonymous functions in Python.

You'll also see some of the differences between lambda and regular functions in Python, and when to use lambda functions.

How to Use Functions in Python

Functions prevent you from reinventing the wheel when certain logic is required multiple times.

Consider the code below:

first_addition = 2+3 
print(first_addition) # 5 

second_addition = 3+5 
print(second_addition) # 8

We had to recreate the logic for addition multiple times in different variables. Imagine if you had to do this a hundred times.

With a function, you can create the logic once and reuse it as much as you want to. Here's an example in Python:

def add_numbers(a,b): return a + b 

print(add_numbers(2,3)) # 5 
print(add_numbers(3,5)) # 8 
print(add_numbers(5,7)) # 12

Using the def keyword, we created a function called add_numbers(a,b). It takes two parameters – a and b. The function returns the sum of a and b.

So to use the logic multiple times, we just had to call the function and pass in different parameters for different operations:

print(add_numbers(2,3)) # 5 
print(add_numbers(3,5)) # 8 
print(add_numbers(5,7)) # 12

Now let's take a look at anonymous/lambda functions in Python.

How to Use Lambda Functions in Python

An anonymous function in Python is a function without a name. It can be immediately invoked or stored in a variable.

Anonymous functions in Python are also known as lambda functions.

Here's the syntax for creating a lambda function in Python:

lambda parameter(s) : expression

There are three values that define a lambda function as seen in the syntax above:

• A lambda function is created using the lambda keyword.
• The keyword is followed by one or many parameters.
• Lastly, an expression is provided for the function. This is the part of the code that gets executed/returned.

The parameter(s) and expression are separated by a colon.

Here's an example:

add_numbers = lambda a,b : a + b 

print(add_numbers(2,3)) # 5

In the code above, we created a lambda function with two parameters – a and b. The function returns the sum of the parameters.

That is: lambda a,b : a + b

Note that the function has no name. We assigned the lambda function to a variable called add_numbers so that we can easily invoke the function through the variable.

Without assigning a lambda function to a variable, you'd have something like this:

print(lambda a,b : a + b)
# <function <lambda> at 0x7f757922fb00>

The code above simply returns a lambda object in the console.

You can immediately call a lambda function using parenthesis:

(lambda a,b : a + b)(2,3)

When the code above is printed, you'll get 5 returned.

What Is the Difference Between Lambda and Regular Functions in Python?

Here are some differences between lambda functions and regular functions in Python:

When to Use a Lambda Function in Python

Although you can use both regular functions and lambda functions to achieve the same results, here are some of the reasons why you might pick a lambda function:

First of all, you can use a lambda function when you need a function that'll be used just once. This is especially useful when working with functions like map, reduce, filter. Consider the code below:

def double_number(n):
    return n + n

numbers = [1, 3, 5, 7, 9]

double_result = map(double_number, numbers)

print(list(double_result))
# [2, 6, 10, 14, 18]

In the code above, we created a regular function called double_number which doubles the value of a number.

Although the function was passed in as a parameter to the map function — map(double_number, numbers), we had to write the logic before using it.

With lambda functions, you can do this:

numbers = [1, 3, 5, 7, 9]

double_result = map(lambda x : x+x, numbers)

print(list(double_result))
# [2, 6, 10, 14, 18]

As you can see above, we just passed a lambda function — lambda x : x+x — as a parameter to the map function: map(lambda x : x+x, numbers).

We were able to achieve the same result with fewer lines of code. There was no need for defining a function first before use.

You can also use a lambda function when you need a function that should be invoked immediately. As you can see in the example in the previous point, we first defined the regular function before using it. Lambda functions can be invoked immediately after creating them.

Finally, lambdas are useful when you want to use a function inside a function. Or when you want to create a function that returns a function.

Summary

In this article, we talked about lambda functions in Python. There are functions without a name and can be executed with a single line of code.

We saw how to use regular functions in Python with some examples.

We then saw the syntax and a practical example of using lambda functions.

Lastly, we talked about the differences between lambda functions and regular functions in Python, and when to use lambda functions.

Happy coding!

You can use Netlify to host websites, and it helps you update and release new changes. It also provides additional features such as security, user authentication and authorization services, and more.

This guide focuses on showing you how to set up a Netlify serverless function to hide Application Programming Interface (API) keys in a client-side application.

For this lesson, you will create a stock photo search engine web application, deploy it on Netlify, and make an API call to the Pixabay API using Netlify serverless functions.

This is the same process for front-end applications built with ReactJS, NextJS, VueJS, Angular, or other JavaScript frameworks.

Prerequisites

To follow along with this tutorial, you should have the following:

• Netlify account (you can sign up here)

• Basic understanding of RESTful APIs, Lambda functions, and async/await concepts.

The final demo app lives in the main branch on GitHub: https://netlify-func-demo.netlify.app

What is a Netlify function?

Netlify functions are serverless or lambda functions provided by Netlify. You use them to deploy server-side code or backend logic without the need for a dedicated server.

The purpose of this Netlify function is to manage serverless event-driven code and send HTTP requests that return a JSON response.

"Serverless functions, branded as Netlify Functions when running on Netlify, are a way to deploy server-side code as API endpoints" - Netlify Docs

It securely accesses environment variables behind the scenes via the Amazon Web Services (AWS) lambda function.

Secret credentials such as access tokens or API keys, hidden solely using environment variables, are less secure. This is because they can be easily retrieved from the Developer Tools through the API fetch request in the browser.

The API keys, if hijacked, can be misused by malicious actors, which might affect your app build threshold or cost you more if it's a paid API service.

Other serverless functions used for running code without having to manage servers include AWS Lambda functions, Azure functions, and Google cloud functions.

How to Set Up a Client-side Application

How to Clone the Demo App

To get started with this tutorial, you can clone the stock photo search engine app GitHub repository. See the live preview on Netlify at https://netlify-func-demo.netlify.app.

The first step is to clone the repository:

git clone https://github.com/frankiefab100/netlify-serverless-functions-demo.git

Next, change to the netlify-serverless-functions-demo directory.

 cd netlify-serverless-functions-demo

Then you'll need to install dependencies.

npm install
#OR 
yarn add

Now run the development server. Run the following command to start the app on the server:

netlify dev

The app will be ready on https://localhost:8888.

Alternatively, you can skip the above steps if you wish to follow along by building the app from scratch. In the next step, you will build a stock photo search engine JavaScript application.

How to Build the Demo App using JavaScript

The first step is to set up a front-end app. Open your favorite code editor, such as VS Code.

Then, create a dist directory and inside it create an index.html file. Populate it with the following code:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <link rel="stylesheet" href="style.css" />
    <title>Stock Photos Search Engine</title>
  </head>
  <body>
    <div class="container">
      <header>
        <h1>Search For Stock Photos</h1>
        <div class="search-section">
          <input
            type="text"
            name="search"
            class="search"
            placeholder="Enter a keyword"
          />
          <input
            id="searchBtn"
            class="search-btn"
            type="submit"
            value="Search"
          />
        </div>
      </header>

      <div class="photo-wrapper">
        <img src="" alt="" class="photo" />
      </div>
    </div>

    <script src="./script.js" type="module"></script>
  </body>
</html>

In the same dist directory, add the following styling to style.css:

/* dist/style.css */
* {
  box-sizing: border-box;
  margin: 0;
  padding: 0;
}

body {
  color: #222;
  font-family: "Roboto", sans-serif;
  font-size: 1rem;
  margin: 0 auto;
  width: 100vw;
  height: 100vh;
}

.container {
  display: flex;
  flex-direction: column;
  justify-content: center;
  align-items: center;
  text-align: center;
  min-height: 100vh;
  min-width: 100vw;
  width: 100%;
  height: 100%;
}

h1 {
  padding-bottom: 20px;
}

.search-section {
  display: inline;
  text-align: center;
  min-width: 310px;
}

.search,
.search-btn {
  border: none;
  border-radius: 5px;
  font-size: 1rem;
  padding: 15px;
  height: 50px;
}

.search {
  background-color: #d1f3bf;
  color: #222;
  min-width: 225px;
}

.search-btn {
  background-color: #04ab04;
  color: #f6f6f6;
  cursor: pointer;
  margin-left: 5px;
  min-width: 80px;
  transition: all 0.3s ease-in-out;
}

.search-btn:hover {
  background-color: #2dc22d;
}

.photo-wrapper {
  display: flex;
  gap: 15px;
  margin: 30px;
}

.photo-wrapper img {
  width: 200px;
}

Sign Up for a Free Account on Pixabay

The first step to using the Pixabay API is to sign up for an account with your email.

Pixabay API key section

As shown in the image above, your API key can be found below the parameters section in the Pixabay API Docs website.

Create an environment variable

Environment variables (commonly known as "env") are combinations of key/value pairs that can affect the behavior and processes of an operating system or application.

Using environment variables is recommended to configure third-party services and their credentials during development.

Install dotenv

Once you complete the account creation on Pixabay, open your terminal and install dotenv as a package. This will enable your app to read environment variables saved in the .env file.

npm install dotenv
#OR
yarn add -D dotenv

In the next step, you will save the API key in a .env file.

Create the .env file

In the root directory of your app, create a .env file and store the API keys copied from your Pixabay Profile.

PIXABAY_API_KEY=123456-7890

Where PIXABAY_API_KEY=123456-7890 represents the API key value.

Note: Replace this key/value pair with the appropriate value.

Create a .gitignore file

To avoid committing sensitive files and values such as node_modules and secret keys to a public repository, create a .gitignore file in the same project root directory and add the following to it:

node_modules
.env
.netlify

The .netlify folder which contains compiled serverless functions together with other files listed will be excluded when the project is pushed to GitHub or any other version control system.

Create a get request function

Now, you should add the fetch request logic in the script.js. You will adjust the API logic later using Netlify functions.

/* dist/script.js */
const dotenv = require("dotenv").config();

const searchbar = document.querySelector(".search");
const submitBtn = document.querySelector(".search-btn");
const photoWrapper = document.querySelector(".photo-wrapper");

submitBtn.addEventListener("click", () => {
  getPhoto(searchbar.value);
  searchbar.value = "";
});

window.addEventListener("keydown", (e) => {
  if (e.keyCode === 13) {
    getPhoto(searchbar.value);
    searchbar.value = "";
  }
});

async function getPhoto(keyword) {
  const apiKey = PIXABAY_API_KEY;
  let apiURL = `https://pixabay.com/api/?key=${apiKey}&q=${keyword}&image_type=photo&safesearch=true&per_page=3`;

  try {
    const response = await fetch(apiURL, {
      method: "GET",
      headers: { accept: "application/json" },
    });
    const data = await response.json();

    let imageURL = data.hits;

    imageURL.forEach((result) => {
      let imageElement = document.createElement("img");
      imageElement.setAttribute("src", `${result.webformatURL}`);
      photoWrapper.appendChild(imageElement);
    });
  } catch (error) {
    alert(error);
  }
}

Note: As mentioned earlier, if the codebase of this app is published on GitHub. The API key will still be accessible from the client side on a browser, although the .env file that contains the secret key was excluded.

To illustrate this, select the testing branch of this app repository. The Live site preview will display the following Reference Errors in your browser console:

Uncaught ReferenceError: require is not defined
Uncaught ReferenceError: require is not defined at getPhotos
Uncaught ReferenceError: process is not defined at getPhotos

This is because there is no way to reference the environment variables specified in the .env file, since they weren't committed to the public repository on GitHub.

In the next step, select and clone the [testing](https://github.com/frankiefab100/netlify-serverless-functions-demo/tree/testing) branch on your local machine with the following commands:

git clone https://github.com/frankiefab100/netlify-serverless-functions-demo.git
cd netlify-serveless-functions-demo
npm install
netlify dev

The app should launch on your browser via localhost:8888.

Now, go to the Developer tools, right-click and select Inspect. Alternatively, press the F12 key. Then, navigate to Network tab and tab on the getPhotos.js request URL.

API key displayed in the Network tab in Developer tools

You should see the API key publicly exposed in the Network tab's Headers section and return as a response data in your browser.

This is a security issue since the Network tab in the Developer tools is typically responsible to display informations such as the request URL, response status, and response data.

API key returned as a response data and exposed in the URL of the request

In the next section, you will find a way to secure the API key using Netlify functions.

How to Get Started with Netlify Functions

First, you'll need to go into your terminal and install Netlify CLI and Lambda as Devdependencies. You can do that by running this command:

npm install -g netlify-cli netlify-lambda
#OR 
yarn add -D netlify-cli netlify-lambda --save-dev

Add custom build and development commands in package.json

These commands build and start the app in the server and also launch the app on your web browser. Here's an example of how you might add these script commands in the package.json file:

"scripts": {
   "build": "npm run-script",
   "dev": "netlify dev"
 }

Install Axios

You will use the axios.get method, because it is a node function unlike the fetch method that is intended for browser runtime.

To install Axios, open your terminal and enter the command:

npm install axios
#OR
yarn add -D axios

In this case, you are working with a vanilla JavaScript app, so you should import Axios in the getPhotos.js file as:

const axios = require("axios");

For JavaScript libraries, like React, import it as follows:

import axios from "axios";

Create a serverless function

In the root of the project, create a folder named netlify,and inside it create another folder functions. In this functions directory, create a file named getPhotos.js.

You will create a serverless function in the getPhotos. This will completely hide the API keys while fetching images from the Pixabay API.

//netlify/functions/getPhotos.js 
require("dotenv").config();

const axios = require("axios");

exports.handler = async (event, context) => {
  try {
    const { keyword } = event.queryStringParameters;
    let response = await axios.get(
      `https://pixabay.com/api/?key=${process.env.PIXABAY_API_KEY}&q=${keyword}&image_type=photo&safesearch=true&per_page=3`,
      {
        headers: { Accept: "application/json", "Accept-Encoding": "identity" },
        params: { trophies: true },
      }
    );

    let imageURL = response.data.hits;

    return {
      statusCode: 200,
      body: JSON.stringify({ imageURL }),
    };
  } catch (error) {
    return {
      statusCode: 500,
      body: JSON.stringify({ error }),
    };
  }
};

Here, the process.env.PIXABAY_API_KEY references API key environment configuration specified in the .env file for development mode.

The keyword parameter accepts a string accessible in the queryStringParameters property and returns a response data stored in the variable imageURL. This will get passed to the script.js as request response (we'll discuss this later).

If the GET request is successful, it returns a response of statusCode 200 with the corresponding response as a JSON object. For errors, we will get an alert with the error message and status code.

Due to changes in version, Axios might return Buffer as a response in your terminal window, that looks like this:

Buffer response from Axios in Terminal

To prevent this, you should attach the following to the GET request:

 let response = await axios.get(
      `https://pixabay.com/api/?key=${process.env.PIXABAY_API_KEY}&q=${keyword}&image_type=photo&safesearch=true&per_page=3`,
      {
        headers: { Accept: "application/json", "Accept-Encoding": "identity" },
        params: { trophies: true },
      }
    );

Create a Netlify configuration file

In the project root directory, create a netlify.toml file. This file specifies how Netlify builds and deploys your app.

Now, add the following build configurations in netlify.toml:

[build]
  command = "npm run build"
  functions = "netlify/functions"
  publish = "dist"

Note:

• command = "npm run build" triggers the Netlify CLI to build the app from the functions.

• functions = "netlify/functions" indicates that the getPhotos functions exist in the netlify/functions directory.

• publish = "dist" identifies dist as the directory where the file will be served from.

Update the script.js file with the Netlify functions request URL

Next, update the apiURL from this:

let apiURL = `https://pixabay.com/api/?key=${apiKey}&q=${keyword}&image_type=photo&safesearch=true&per_page=3`;

to the functions HTTP request endpoint:

  let apiURL = `/.netlify/functions/getPhotos?keyword=${keyword}`;

This serverless function will get queried to the client-side of your app through the endpoint: /.netlify/functions/getPhotos. Once a fetch request is sent, the getphotos function will be invoked and accessed in the script.js.

The getPhotos Netlify functions' response imageURL will be passed and the data accessed as the value of the keyword parameter in the query string of function. It will get loop through to return three images from the Pixabay API to the client-side.

The script.js file should look like this:

/* dist/script.js */
const searchbar = document.querySelector(".search");
const submitBtn = document.querySelector(".search-btn");
const photoWrapper = document.querySelector(".photo-wrapper");

submitBtn.addEventListener("click", () => {
  getPhoto(searchbar.value);
  searchbar.value = "";
  photoWrapper.innerHTML = "";
});

window.addEventListener("keydown", (e) => {
  if (e.keyCode === 13) {
    getPhoto(searchbar.value);
    searchbar.value = "";
    photoWrapper.innerHTML = "";
  }
});

async function getPhoto(keyword) {
  let apiURL = `/.netlify/functions/getPhotos?keyword=${keyword}`;

  try {
    const response = await fetch(apiURL, {
      method: "GET",
      headers: { accept: "application/json" },
    });
    const data = await response.json();

    for (let i = 0; i < data.imageURL.length; i++) {
      let imageElement = document.createElement("img");
      imageElement.setAttribute("src", `${data.imageURL[i].webformatURL}`);
      photoWrapper.appendChild(imageElement);
    }
  } catch (error) {
    alert(error);
  }
}

Note: From the codebase above, your environment variable is secure since it is accessed from the serverless function.

Run the development server

Now, execute your app by running the command:

netlify dev

This will load the getPhotos function via https://localhost:8888/.netlify/functions/getPhotos.

netlify build terminal output

Then, start the development server and launch the application in your default web browser on localhost:8888.

netlify function ready on https://localhost:8888

At this point, the Netlify function is fully setup, and you can now proceed to make GET HTTP requests.

How to Send Fetch Requests

Now that you already have the web app served, go ahead and send a fetch request. Enter some text in the search input field and hit Enter or click the button to fetch images from the Pixabay API.

flower images fetched from Pixabay API

For more information about the Pixabay API, see the Pixabay documentation.

The preceding command will send a request to the serverless function and then return six responses. Here's what the response looks like in your terminal windows:

Request from ::1: GET /.netlify/functions/getPhotos?keyword=flower
Response with status 200 in 4895 ms.

You can also use Developer tools through the Network tab to validate the request.

serverless function API response data

The fetch request returns our app URL, the getPhotos Netlify function, script.js and the images from Pixabay.

Netlify function fetch response from Network tab's header section in the browser

How to Host the App on the Remote Repository

Now, you should push your project to GitHub version control.

git add .
git commit -m"initial commit"
git push origin -u main

How to Deploy the App and Serverless Function on Netlify

Once you have published the project on GitHub, log in to Netlify and connect your GitHub account by granting authorization.

Import project for deployment on Netlify

As shown in the image above, click the Add A New Project and then search for the app repository from the list. Next, click on Build Your Site. This will take a few minutes to complete.

You just deployed the app from the Netlify UI. Your app is now ready at: https://netlify-func-demo.netlify.app..

The fetch request URL should look like this: https://netlify-func-demo.netlify.app/.netlify/functions/getPhotos.

Optional - How to Configure the Netlify App from the Dashboard

Alternatively, you can configure the Netlify command from the Netlify Dashboard. If these settings are already specified in netlify.toml, it will override any corresponding configuration.

First, select the project directory's Site settings.

Site settings for project directory on Netlify

Set the Build command and Publish directory

Navigate to Site settings > Deploy > Build & deploy and edit the following commands and then save the changes:

• Build command: npm run build

• Publish directory: dist

Build and deploy section on Netlify

Set the Functions directory

By default, Netlify uses netlify/functions as the directory to find the functions to deploy them. In our case, our serverless function getPhotos can be found in the netlify/functions directory.

Now we'll set a custom functions directory so Netlify can find your compiled functions. Go to Site settings > Functions and enter the directory path where the functions are stored in your repository.

Functions directory section on Netlify

How to Set Environment Variables for Production

In the Netlify Dashboard, click on Site settings > Build & deploy > Environment > Environment variables and then configure KEY/VALUE pairs as follows:

PIXABAY_API_KEY=your-api-key-here

Environment variables section on Netlify

Click on Save, and then redeploy your app so the changes can be added.

To redeploy the app on Netlify, navigate to Deploys > Trigger deploy. Next, select Clear cache and redeploy site.

Trigger redeployment tab on Netlify

Note: The environment variable (PIXABAY_API_KEY) name should match the one mentioned in the getPhotos codebase.

For a React application, prepend the API environment variable with the prefix REACT_APP so they can be read from the .env file.

 REACT_APP_PIXABAY_API_KEY=your-api-key-here

How to Initialize the Serverless Function to the Remote App

To connect your project directory to the existing web app deployed on Netlify, login to your Netlify account from the terminal:

netlify login

Next, initialize the app on Netlify by running the command in your terminal:

netlify init

Your app is now configured for continuous deployment via Netlify.

How to Build the Netlify Serverless Function

You need to link your app to site ID on Netlify before running the build command on your terminal. To connect your local project folder to its site ID on Netlify, enter the command in your terminal:

netlify link

This will prompt you to link the folder to a site through any of the following options:

• Search by full or partial site name

• Choose from a list of your recently updated sites

• Enter a site ID

Once you select your preferred option, it will connect the project folder to the hosted site on Netlify. This allows you run Netlify CLI commands and also automatically deploy the project repository once there are any changes in the codebase.

In the next step, you will build a serverless function while it is running on the server. To activate the build command defined in netlify.toml file, run the following command:

netlify build

This runs the npm run-script command under the hood as specified in the package.json. Now, your serverless function in netlify/functions directory is packaged and bundled successfully!

How to Test the Application

To test and confirm that the Netlify function is working fine, run this command on your terminal:

netlify functions:serve

This injects your project environment variables from the .env file and runs the serverless function.

How to Confirm API Keys Security

To inspect your app and confirm that the API keys are hidden from the public, follow the steps below:

Click on your hosted app URL, and navigate to Developer tools by pressing the F12 key or right-clicking and selecting Inspect. Navigate to Network tab, where you should see the fetched data from the Pixabay API.

API key hidden from the public and unauthorized parties after inspection through the Developer tools

Now you've confirmed that you've successfully configured a serverless function and have it deployed on Netlify.

Conclusion

This tutorial introduced serverless functions, asynchronous JavaScript, and Restful API concepts.

Hopefully you now know how to create a serverless/lambda function and secure any sensitive value such as API keys in your frontend JavaScript application.

If you got stuck with anything, you can access the complete source code in this GitHub repository.

Thank you for reading! If you have any questions, feel free to reach out to me via Twitter.

Relevant Links

• Netlify Functions

• Netlify Intro to Serverless Functions

• Netlify Functions Example

Following tutorials when you start learning serverless is a good first step. But to really get better, you need to build your own projects.

The problem is that coming up with ideas that are realistic but help you grow is hard.

To help you out, I’ve come up with 7 awesome project ideas to progressively help you become a better serverless dev.

If you’re new to working with Serverless, then start with the first projects. If you’ve built APIs and Dynamo with Serverless before, then pick the projects that cover the topics you want to learn most.

I've also made this into a video if you want to watch and follow along:

Combination API Project

This project is designed to get you familiar with deploying an API with Lambda. You'll also practice calling other APIs and merging that data.

There are lots of logic you could use for this, but here are two examples:

• Get Steam game deals, converted into your local currency
• Get news translated into your local language

The architecture for this one is simple but that is exactly what you want as your first project. The idea is that the API you create will take some parameters, so look something like this

https://apiurl.amazonaws.com/dev/steamdeals?currency=EUR
https://apiurl.amazonaws.com/dev/news/fr

The logic is all going to be written into the Lambda. I'm not going to write the full code but here's some pseudo-code:

const handler = (event: APIGatewayProxyEvent) => {
    // get the path or query string parameter value

    // hit the first API to get the first data

    // get the data you need to translate / convert

    // pass the data from API1 into API2

    // combine the data 

    // return data in API Gateway format
}

If you need some free and public APIs to use, then here's a massive doc of them to choose from https://github.com/public-apis/public-apis.

Then building this try using the Serverless Framework or AWS CDK. These tools will make you far more valuable as a developer.

To get started with the Serverless Framework check out this video.

URL Shortener Project

This project will get you deploying your first DynamoDB table, then writing and reading data from it.

There are going to be two API endpoints. One to add a new URL to the shortener, one to give a shortened URL and get the original.

Here's the pseudo-code for the two endpoints:

// Adding a new URL
const handler = (event: APIGatewayProxyEvent) => {
    // get their original url (body of the event)

    // generate a random 5 character code

    // write to Dyanamo - {id: code, url: originalUrl }

    // return the url (https://{apiurl}.amazonaws.com/dev/get/{code})
}

To get the apiURL we can pass it in as an environment variable. If you are using the Serverless Framework you can do this by adding this to the environment section of your config:

apiUrl: {
  "Fn::Join": [
    "",
    [
      "https://",
      { Ref: "HttpApi" },
      ".execute-api.${self:provider.region}.amazonaws.com",
    ],
  ],
},

// Getting a URL by code
const handler = (event: APIGatewayProxyEvent) => {
    // get code from the url path

    // get from dynamo by ID

    // return the original url
}

You can take this one step further by changing the status code of the getUrl code to return a 301 - permanent redirect. You'll need to add some extra headers but this will automatically redirect the user to the desired page.

Reminder App Project

This project will teach you about Secondary Indexes in Dynamo as well as Dynamo Time-To-Live. You will also get to try either email automation with Amazon Simple Email Service (SES) or text messaging with Simple Notification Service (SNS).

You could also build a simple front end app and learn about hosting it in S3 and using CloudFront to distribute it.

The idea for this app is that you can post a new reminder to the first API endpoint. This will write a new record in DynamoDB, but you will have added a global secondary index (GSI) to the table. This means that you can get a reminder by id, or you can query based on the user.

It will also have a Time-To-Live (TTL) which will allow you to trigger a Lambda at the time of the reminder. The code for set reminders will looks pretty similar to the previous project.

The table will look something like this:

The Time-To-Live (TTL) tells dynamo that once the time reaches this date, delete the record from Dynamo.

Two things to note with TTL:

• Make sure that this is the Unix timestamp for the deletion date – but in seconds. new Date('october 20 2022').getTime() will be in milliseconds, so just divide by 1000.
• The record will be deleted within a 15-minute window after your TTL so don't panic if it's been 5 minutes and the record hasn't been deleted yet.

You can then set up a second Lambda which is triggered any time a record is deleted from Dynamo. This then sends the message to either their email or phone.

// Send reminder
const handler = async (event: DynamoDBStreamEvent) => {
    // get the list of deleted records

    // map over each record
       // Call SES or SNS to send the reminder
}

The second API endpoint will be to get all of the reminders for a user. Make sure you've set up a GSI with a partition key of userId and a sort key of TTL.

// Get My Reminders
const handler = (event: APIGatewayProxyEvent) => {
    // get the userId from request

    // params = KeyConditionExpression: "'userId' = userId",

    // Query Dynamo

    // format the reminders 

    // return to the user
}

For the front end you can write a simple web app with two sections: create new reminder, and list my reminders. This can be in React, Vue, Angular, or even raw HTML, CSS, and JavaScript.

Once you have the app you can use serverless-s3-sync to automatically push the code into S3 (which you create in Serverless).

Live Chat App Project

This project will teach you how to build WebSockets. A user can either create a new 'room' or join an existing room. Any messages sent are sent to everyone connected to the room.

WebSockets work slightly differently than regular APIs. Instead of having multiple endpoints, you have one endpoint and different message types.

There are some default message types and some custom ones:

• connectToRoom – Custom
• createRoom – Custom
• onMessage – Custom
• disconnect – Default

When a user connects to the WebSocket, they can either send a connectToRoom or createRoom message. Both of these will create a record in Dynamo with the user's WebSocket connectionId and the roomId. As we did before, we'll have a GSI in Dynamo so we can later query to get all users for a roomId.

The code for connectToRoom and createRoom will look very similar to previous "writing data to Dynamo" lambdas.

You may want to check in connectToRoom that the room does exist first. You can do that by querying for all users by roomId. If there are no users in the room then that means they're trying to connect to a room that doesn't exist anymore.

Now a user is in a room they can send a message. Here's the pseudo-code for that Lambda:

// onMessage
const handler = (event: WebsocketMessageEvent) => {
    // get the user's connectionId

    // get the user by connectionId from Dynamo to get the roomId

    // query for all users in that roomId

    // send the message to all users

    // return
}

Finally, there is the onDisconnect which will be a simple Lambda that just deletes the user's record from Dynamo.

Idea Voting App Project

This project will teach you to design and build more advanced Dynamo tables and also work with Cognito for authentication. You should also build a simple front end for this to learn how to integrate Cognito into web apps.

The tool allows you to ask your community for ideas and find out which of those are most popular.

This starts with all users needing to sign up to your application. We use Cognito for this and do it so that we can ensure that each person can only vote once.

You'll need to then create a few endpoints:

• create a board
• add an idea to a board
• vote for an idea
• get board details

Create a board

The first API endpoint that we need is one to create a new board for ideas. Anyone can create a board and the record in Dynamo is really simple. Just a boardId and then some info about the board owner, maybe a title and description.

Add ideas to a board

Next we need to be able to add ideas to a board. This will be another API endpoint and Lambda. With the idea database record we need to be a bit smarter as we need to be able to directly reference an idea, but also get all ideas for a board.

To do this we have a schema like this. pk is a partition key and sk is a sort key, which are both needed for querying on DynamoDB.

With the id we can directly reference the idea, but we can also query on the record. We can get all ideas for board 1234 by querying for pk = 1234.

Add vote to the idea

Now that we have ideas on the board, we need our users to vote for them. This will be a new API endpoint and Lambda. This Lambda has a bit more work to do than the other two. First, we'll look at the schema for this record.

This may look strange at first but I'll explain why we want to structure it like this.

For a given idea, we want to know how many votes it has. We can find that out by querying on pk = ideaId which will return all votes for the idea.

When adding a vote we want to check that the user hasn't already voted for the idea. We can do that by querying pk = ideaId && sk = userId. If we find a record that matches, we know they've already voted for this idea. If not then we can add the vote record for this user and idea.

Get Board Details

We can now write a lambda that will query the data we have:

export const handler = async (event: APIGatewayProxyEvent) => {
    // get boardId from the request
    // query all ideas on the board
    // map over each idea
        // query for all votes on this idea
    // format it all into a nice format
    // return to the user
}

Get votes for a user

Lastly, we want to be able to show all votes for a given user. We can't query where sk = userId as you always need a partition key when querying DynamoDB. Therefore we created a second partition key (pk2) which contains the userId. Now we can query pk2 = userId to get all ideas that a user has voted for.

This pattern of having a second GSI where the PK and SK are switched is very common. It allows you to have a many-to-many relationship. You can query for all of B that are connected to a specific A and all of A that are connected to a specific B. This is often called a junction table.

Messaging App Project

This project will teach you to about Dynamo compound keys and give you more practice with websockets and Cognito.

This app allows users to sign up, request to join a room, and then see all messages that have been sent in that room. The owner of a room decides whether to let someone join the room.

The architecture will start by looking a lot like the live chat application – with a WebSocket connection with Lambdas for onConnect, onDisconect, createGroup, joinGroup, listMyGroups handleJoinGroupRequest and sendMessage.

Before we get onto the Lambda code, we want to add Cognito to the application so we can have users signup. This will be used later when a user requests access to a group.

The createGroup Lambda will be almost identical to the live chat solution. It just creates a simple group record.

Websocket Connect / Disconnect

Now when a user logs in and connects to the WebSocket, we can create an onConnection Lambda. This will get the userId and verify their Cognito user token. This token will easily give us the userId and userName.

If they don't pass a token or the token is invalid (or has expired) then we can kill the websocket, stopping them from doing anything else without a valid token.

If the token is valid then we can store a simple connection record. This will allow us to send messages back down the websocket later.

Just remember to delete this record when the user disconnects from the WebSocket. We can use the default $disconnect action to trigger an onDisconnect lambda.

Create a group

The first thing that someone needs to be able to do is to create a group that they can then invite their friends to.

This will involve creating two records in Dynamo. The first is just a record of the group. This will include the group ID, group name, and who is the group owner.

The second is going to be a group membership record. This will be saying that this user is part of this group.

This first part will allow us to query on pk = groupId and sk startsWith('user') to get all users for the group. That will be used in the sendMessage Lambda to get all the users to send the WebSocket message to.

The second parts (PK2, SK2) are added to allow us to get all groups that a user is authorised for by querying where PK2 = userId. This is used when we need to get a list of all of the user's groups.

Joining a group

The join Group request will now create a record in Dynamo for an access request. This will not give the user access to the group but will allow the owner of the group to review the access requests.

These records will be the first time that we're going to use compound keys. This is taking the sort key to the next level and the record will look like this:

This enables us to query where pk = groupId and sk startsWith('joinRequest'). This will return all of the access requests for that group. When someone makes this request we can first check that they are the owner of the group.

The group owner then has two options – accept or reject. If the owner rejects the user we can just delete the access request record. If they accept the user then we need to add an authorised user. This will require a new record to Dynamo.

Send Message

Now we're onto the core of the messaging app – sending and storing messages.

In the sendMessage WebSocket Lambda, we can query for all users in the group and send the message to all current connections. We also need to store the message in Dynamo.

Get Previous Messages

When a user logs in they need to be able to get the messages they missed whilst offline. With the message data structured this way, we can query where:

pk = groupId and sk > 'message#{ timestamp for yesterday }'

This will return all messages created since yesterday.

We could also allow them to get older messages by passing up the last message they have. This will allow us to get the next chuck of messages. This will allow us to create an infinite up scroll on the message history.

Event Driven E-Commerce System Project

This project will teach you about Event Bridge, plus give you some extra practice with DynamoDB Table design and services like SES and SNS for email and text.

This system will have products and filtering, carts and orders as you would expect. The key here is that order placement, order status changes, and delivery updates will all be handled through Event Bridge.

Storing Products

To start we need to store products. We can structure our sort keys in a way to allow a level of hierarchy for our products.

This allows us to query for pk = clothing and sk beginsWith mens to get all men's clothes or for pk = clothing and sk beginsWith womens#trousers to get all women's trousers.

Placing an Order

The next part is being able to place an order. We're not going to try and take payment, just add an order record to the dynamoDB table.

Event Bridge

The big difference here is that we're going to start using Event Bridge. This is a tool which allows us to have events which can trigger multiple lambdas. This is great as we can add a new listener without having to change the upfront code.

Order Created

We're going to have two lambdas that listen for the orderCreated event. The first is going to take the order data and send it to a warehouse API to get packed. The second is going to send the user an order confirmation email

Order Packed

We're going to pretend that there is a real warehouse and after getting the order packing list, they packed up and got the order ready for shipping. They have a system that calls our API and changes the order status to packed.

This change in the Dynamo record will trigger another Event Bridge event for orderPacked. This also has two listeners: one to email an update to the user, and another to email a delivery service to collect the parcel from the warehouse and deliver it to the customer.

Order Shipped

Similarly we're going to pretend that a delivery company took the parcel and delivered it. They call another Order Shipped API endpoint and that changes the status of the order in the database again.

That triggers another event for orderDelivered and that has two listeners:

One for sending a "thank you" message to the customer.

And another one that will do something a bit different. It will take the order, remove any personal data, and store it into another DynamoDB table.

This is something becoming increasingly common as a preparation step for a data scientist. We remove the personal data to reduce the legal issues but still allow a data scientist to do things like train a model to give you "People also bought ...." kinds of recommendations.

What's Next

If you like these project ideas but don't know where to start, then I have a full video course that will teach you how to build all of these projects.

Check out the course here.

You can also download a Free PDF of these projects so you can visualise your progress in your learning journey.

Lambda has gained massive popularity over the past few years. It has various use cases, from running simple scripts to gluing flows and processes within a serverless architecture or when running microservices.

Still, if you've just started working in a new environment or organisation, you might feel cautious about using it because it's complex to configure.

This is fair with any new technology you're getting ready to start using, and being careful is a sign of good business sense.

An argument you might often hear is that lambda might slow down your operation because of cold starts.

This is also a common topic in technical interviews, so it's worth digging a bit deeper if you're keeping an eye on that next dream cloud engineering position.

Photo by Myriams-Fotos on pixabay

In this article, we'll talk about one way that can help enhance the performance of your Lambda functions.

But let's start first with an overview of what happens when you execute a Lambda function, touching on cold starts.

How is a Lambda Function Executed?

When your Lambda function is triggered, the Lambda service runs the code in an "Execution Environment" with three phases: Init, Invoke and Shutdown.

• In the Init phase, Lambda creates the execution environment, downloads the code, sets up the configuration (for example, memory), and runs the code that lives outside of the handler function. (You might already have the gist of the article at this point – but read on for further explanation and a demo.)
• In the Invoke phase, Lambda runs the code inside the handler function
• In the Shutdown phase, Lambda terminates the environment

There could be multiple Invocations between an Init phase and a Shutdown phase if the Lambda got triggered a bunch of times in a row in a fairly short period of time (this time is not precisely determined in the Lambda docs).

This means that if multiple consecutive executions of the Lambda happened in a short timeframe, the Lambda service would leverage the first created execution environment to run the following invokation. That is, it would only call the handler function with every new call to the Lambda without having to create a new execution environment and without having to run the code that's outside the handler function.

The Init phase happens only once upon the first execution, and we can visualise it as two parts (both of them run once per execution):

• The part where a new execution environment is created and the code is downloaded is the cold start.
• The part where the code outside the Lambda handler is run is the initialisation code.

From the AWS docs, this picture provides a proper visualisation of the 3 phases:

The Function INIT (the last block of the INIT phase) will run only once, along with the first two blocks that form the cold start.

Ok enough talk – show us the code!

Lambda Function Code

In the previous section, we discussed that Lambda would run the initialisation code (code outside the handler function) once upon first execution. And it would run the code inside the handler using the same execution environment if the following invocations did not happen too far from each other.

That's a perfect way to leverage the initialisation code to run the reusable code and save time re-running this code every time with every invocation.

Assume we have a Lambda function that's interacting with a DynamoDB to retrieve a set of items (please keep in mind that this code is for demo purposes only, so it's not the cleanest you'll see).

Create an index.js file:

const { DynamoDB } = require("@aws-sdk/client-dynamodb");

exports.handler = async (event, context) => {
  const dynamodb = new DynamoDB({ region: "ap-southeast-2" }); // creating a new instance of DynamoDB
  const params = {
    TableName: "company",
  };

  const results = await dynamodb.scan(params); // avoid using scan in production - it's expensive and not performant

  return {
    statusCode: 200,
    body: JSON.stringify(results),
  };
};

In the previous snippet, the initialisation code is the following:

const { DynamoDB } = require('@aws-sdk/client-dynamodb');

It will run once upon first execution and won't run in the following executions if they happen in a short time – that is, the subsequent executions happened before the Shutdown phase was reached.

Lambda Demo

Awesome, now we're ready for the demo. In this section, we will build the infrastructure and test the performance of the Lambda.

Build the Infrastructure

As this tutorial aims to demonstrate how to enhance the performance of our Lambda, we won't be putting much focus on the infrastructure as code.

Follow these steps to build and deploy your app:

• Install the dependencies: npm install
• Zip your Lambda File: zip -r ./get-companies-lambda.zip index.js node_modules
• Download the following Terraform file: https://gist.github.com/AHaydar/bfc173db2078b2eeb884da8632248c5d
• Apply the Terraform changes: terraform apply

Open the AWS console and validate that both DynamoDB and Lambda got created correctly (I am using the ap-southeast-2 region in this example).

Test the performance

First, navigate to your Lambda function in the AWS console and click the Test button to run your code. You will get a result similar to the following:

Notice the following in the last line:

Duration: 569.85 ms

and

Init Duration: 429.13 ms

The Init Duration is the duration to run the code outside the Lambda handler – that's the code requiring the DynamoDB SDK.

The Duration is the duration of running the code inside your Lambda handler – instantiating DynamoDB passing the "ap-southeast-2" region and scanning the table.

Click the Test button again. You should get a response like this:

Notice the last line – it does NOT contain an Init Duration. If your result included an Init Duration, it means the execution environment was shutdown before you clicked the Test button again. In this case, click it another time.

So in the second execution, we saved the Init Duration. It's a hint to optimise our function and add more common code outside the Lambda handler. How can we do that?

const { DynamoDB } = require("@aws-sdk/client-dynamodb");

const dynamodb = new DynamoDB({ region: "ap-southeast-2" }); // creating a new instance of DynamoDB
const params = {
  TableName: "company",
};

exports.handler = async (event, context) => {
  const results = await dynamodb.scan(params); // avoid using scan in production - it's expensive and not performant

  return {
    statusCode: 200,
    body: JSON.stringify(results),
  };
};

In the above snippet, I moved the instantiation of DynamoDB outside of the handler, along with the params object.

So this would increase the execution time of the Init Code (outside the lambda handler), which runs only during the first execution of the Execution environment lifecycle. And it would decrease the execution time of the Lambda Handler, which happens upon every invocation.

Keep in mind that you should never write business logic that relies on having a running execution environment. Your code should always assume it needs to init the execution environment upon every invocation.

What other tips do you have on optimising your Lambda performance? Share them with me on Twitter or LinkedIn

In this article we're going to learn how to make an application using AWS Serverless that lets us identify images of celebrities. We'll use AWS Rekognition for AI-based identification.

We are going to attach an event to the S3 Bucket so that whenever a file gets uploaded to the bucket, it'll invoke a Lambda function that will process the information from the image and save it to the DynamoDB table.

Before you use DynamoDB, just make sure that you check out this guide on AWS DynamoDB Pricing so you only spend what you want to spend.

Tech Spec

We are going to use Lambda functions to code our project logic and AWS Rekognition for AI-based image identification of the celebrity.

If we get valid data from the AWS Rekognition API then we are going to store that data in a DynamoDB Table.

All these resources except from the S3 bucket will be created inside the serverless.yml file.

Project Setup

We are going set up all the things we need in this project step by step. First we'll go through the serverless.yml file. To learn more about this file, check out this post. Let’s get started with the first step.

Our project folder structure should look like this at the end of this tutorial.

How to set up the serverless.yml file**

We will break down the serverless.yml file into different parts to make it easier to understand.

How to set permissions and configure the project**

service: meta-data-serverless

provider:
  name: aws
  runtime: nodejs12.x
  environment:
    DYNAMO_TABLE_NAME: MetaData
    BUCKET_NAME: new-bucket-caps2
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:PutItem
        - rekognition:RecognizeCelebrities
        - s3:Get*
        - s3:List*
      Resource: "*"

In this code block, we are setting different environment variables and AWS IAM permissions which will be given to our lambda functions. So for our use, we need to write an item to the DynamoDB table, use AWS Rekognition's API to do image identification on the image, and get the file from S3 (all of which we've done in the above code).

Note that you will need to create a new public S3 bucket and set the name of that bucket here in place of “new-bucket-caps2” as the BUCKET_NAME property. To read more about IAM roles check out the official AWS documentation.

How to add a lambda function**

functions:
  processImg:
    handler: src/index.handler
    events:
      - s3:
          bucket: ${self:provider.environment.BUCKET_NAME}
          event: s3:ObjectCreated:*
          existing: true

In the functions block, we are defining a single lambda function which will be invoked when any file is uploaded to the S3 bucket.

As you can see, we are attaching an event to this lambda function on which it will be invoked. s3:ObjectCreated is the event when any file gets uploaded into the S3 bucket.

We are also declaring that this bucket already exists by setting the existing option to true. So make sure that you create this bucket before you deploy the project.

We are also referencing the environment variable for the bucket name which we created in the above section.

How to add DynamoDB table configuration**

resources:
  Resources:
    UsersDynamoDbTable:
      Type: AWS::DynamoDB::Table
      DeletionPolicy: Retain
      Properties:
        AttributeDefinitions:
          - AttributeName: id
            AttributeType: S
        KeySchema:
          - AttributeName: id
            KeyType: HASH
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        TableName: ${self:provider.environment.DYNAMO_TABLE_NAME}

In this block, we are defining our DynamoDB table and its configuration. Any resource we want to create on our AWS account is defined under the resources block in the serverless.yml file. Here we are defining things like table attributes, key schema, and how much provisioned throughput capacity we want to give our table.

For the table attributes, all other attributes will be dynamically added to the table except the id. We will generate the id in the code using a module called UUID.

How to set up the lambda function**

After creating the serverless.yml file, it is now time to create our lambda function which we defined inside the yml file. So let’s get started on this.

We will again see different parts of the lambda function so you can understand it better.

Imports

const AWS = require("aws-sdk");
const {
    v4: uuidv4
} = require('uuid');
const rekognition = new AWS.Rekognition();
const dynamoDb = new AWS.DynamoDB.DocumentClient();

We are importing two packages, aws-sdk and UUID, to call the APIs for DynamoDB and AWS Rekognition. We are also initializing instances of them.

Define the parameters

const Bucket = event.Records[0].s3.bucket.name;
const Name = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, " "));

const params = {
    Image: {
        S3Object: {
            Bucket,
            Name
        }
    }
};

When our lambda gets called by an S3 event, it receives data about the object which was uploaded to the S3 bucket. Were we are just getting that object data like the name of the Bucket it was uploaded to and the name of the file as well.

After that we are passing this data into the parameter object we will be passing to the AWS Rekognition API call.

Call the AWS Rekognition API

const celebrityData = await rekognition.recognizeCelebrities(params).promise();
if (celebrityData.CelebrityFaces && celebrityData.CelebrityFaces.length) {

    const {
        Name,
        Urls,
        KnownGender,
        Face
    } = celebrityData.CelebrityFaces[0];
    const closelyMatchedEmotion = Face.Emotions.reduce((prev, current) => (prev.Confidence > current.Confidence) ? prev : current)

    const params = {
        TableName: process.env.DYNAMO_TABLE_NAME,
        Item: {
            id: uuidv4(),
            Name,
            readMore: Urls,
            KnownGender,
            closelyMatchedEmotion
        },
        ConditionExpression: "attribute_not_exists(id)"
    };
    await dynamoDb.put(params).promise();

Finally, we are calling the AWS Rekognition API with the parameters we declared in the previous step. After we get the response from the API, we check to see if it was able to identify the celebrity or not.

If it found celebrity data, then we are fetching data like Name, Gender, Emotion in the image, and so on from the identified celebrity data.

Then we generate an id using the UUID package we imported earlier. The last thing we're doing is inserting this data into the DynamoDB table.

Note that to query this saved data with non-key attributes you will need to create an index if you don’t wish to scan the whole table. Check out this post to learn how to create a DynamoDB Global Secondary Index using AWS Serverless.

Conclusion

If you reached this point, then congratulations! You now have an application that will identify celebrity data from an image.

You can now just go to your created S3 bucket and upload any celebrity image and then wait for a couple of seconds, then check the DynamoDB table to see the results saved there.

You can enhance this application in many ways. For example you can add APIs like GET to get the data and see the data which got added to the DynamoDB table. You can also use MongoDB in place of DynamoDB. To learn more about the differences between these two check out DynamoDB Vs MongoDB.

Get the source code

Click here to get the source code for this application.

You can check out more articles like this on my site.

Serverless, the new buzzword in town has been gaining a lot of attention from the pros and the rookies in the tech industry. Partly due to the manner in which cloud vendors like AWS have hyped the architecture, from conferences to meetups to blog posts to almost everywhere. But serverless is not just about the hype, it promises the possibility of ideal business implementations which sounds quite pleasant to the ears and probably light on the budget as well.

“Focus on your application, not the infrastructure”

Sounds relieving though, knowing a lot of your daylight hours go into implementing, maintaining, debugging, and monitoring the infrastructure. With all of that infrastructure heavy lifting out of the way, we really can focus on the business goals our applications serve. A lot of productive efforts could be channeled in the right direction, where they were meant to be ideally. Maybe it sounds too good to be true, but this is the way things should have been. At least for those of you who cant afford to spend a lot of time caught up in the web of intricacies in modern complex infrastructure.

Expectations apart, Serverless is really breaking ground in its path to disrupt your server infrastructure. Serverless is already used in production by companies like Netflix, Reuters, AOL, and Telenor. Industry-wide adoption is constantly increasing. Serverless is all set to take up its own place, but don’t expect Serverless to conquer your infrastructure completely. There will be use cases where serverless might prove to be the wrong choice.

So, What is Serverless?

Serverless is a cloud computing execution model where the cloud provider dynamically manages the allocation and provisioning of servers. A serverless application runs in stateless compute containers that are event-triggered, ephemeral (may last for one invocation), and fully managed by the cloud provider. Pricing is based on the number of executions rather than pre-purchased compute capacity, isn’t it the ideal framework for that project you have been planning since a long time? Well, go ahead do it.

Serverless applications are event-driven cloud-based systems where application development rely solely on a combination of third-party services, client-side logic and cloud-hosted remote procedure calls (Functions as a Service).

Most of the cloud providers have invested heavily in serverless and thats a lot of money; with the given massive promotion and realistic offering you can safely assume serverless to be one of the most used cloud services in upcoming years. Here are some of the currently available cloud services:

Source: https://www.slideshare.net/loige/building-a-serverless-company-with-nodejs-react-and-the-serverless-framework-jsday-2017-verona

• AWS Lambda

• Google Cloud Functions

• Azure Functions

• IBM OpenWhisk

• Alibaba Function Compute

• Iron Functions

• Auth0 Webtask

• Oracle Fn Project

• Kubeless

Traditional vs. Serverless Architecture

Source: https://www.gocd.org/2017/06/26/serverless-architecture-continuous-delivery/

For years your applications have run on servers which you had to patch, update, and continuously look after late nights and early mornings due to all the unimaginable errors that broke your production. As long as you managed them, the whole responsibility of their proper functioning was on you. Serverless tends to be unlike the aforementioned, you no longer need to worry about the underlying servers. Reason being, they are not managed by you anymore and with management out of the picture the responsibility falls on the Cloud vendors. But regardless the cool features of Serverless in some cases, the traditional architecture outshines it.

Pricing

One of the major advantages of using Serverless is reduced cost, for years the cost of provisioning servers and maintaining that 24x7 server team which blew a hole in your pocket is gone. The cost model of Serverless is execution-based: you’re charged for the number of executions. You’re allotted a certain number of seconds of use that varies with the amount of memory you require. Likewise, the price per MS (millisecond) varies with the amount of memory you require. Obviously, shorter running functions are more adaptable to this model with a peak execution time of 300-second to 15-minutes for most Cloud vendors.

The winner here is Serverless Architecture.

Networking

The downside is that Serverless functions are accessed only as private APIs. To access these you must set up an API Gateway. This doesn’t have an impact on your pricing or process, but it means you cannot directly access them through the usual IP, snap!

The winner here is Traditional Architecture.

3rd Party Dependencies

Most, if not all of your projects have external dependencies, they rely on libraries that are not built into the language or framework you use. You often use libraries with functionality that includes cryptography, image processing, etc., these libraries can be pretty heavy. Without system-level access, you must package these dependencies into the application itself.

Reinventing the wheel isn’t always a good idea.

The winner here is based on the context. For simple applications with few dependencies, Serverless is the winner; for anything more complex, Traditional Architecture is the winner.

Environments

Setting up different environments for Serverless is as easy as setting up a single environment. Given that it’s pay per execution, this is a large improvement over traditional servers, you no longer need to set up dev, staging, and production machines. Eventually you’d lose count of all the environments, at some point.

The winner here is Serverless Architecture.

Timeout

With Serverless computing, there’s a hard 15-minute timeout limit for AWS Lambda. Too complex or long-running functions aren’t good for Serverless, but having a hard timeout makes it impossible to perform certain tasks. A hard limit on this time makes Serverless unusable for applications that have variable execution times, and for certain services which require information from an external source. This is likely to change in the future.

The clear winner here is Traditional Architecture.

Scale

Scaling process for Serverless is automatic and seamless, but there is a lack of control or entire absence of control. While automatic scaling is great, it’s difficult not to be able to address and mitigate errors related to new Serverless instances.

It’s a tie between Serverless and Traditional Architecture.

Functions as a Service (FaaS)

FaaS is an implementation of Serverless architectures where engineers can deploy an individual function or a piece of business logic. They start within milliseconds (~100ms for AWS Lambda) and process individual requests within a 300-second to 15-minute timeout imposed by most cloud vendors.

Principles of FaaS:

• Complete management of servers

• Invocation based billing

• Event-driven and instantaneously scalable

Key properties of FaaS:

Independent, server-side, logical functions

FaaS are similar to the functions you’re used to writing in programming languages, small, separate, units of logic that take input arguments, operate on the input and return the result.

Stateless

With Serverless, everything is stateless, you can’t save a file to disk on one execution of your function and expect it to be there at the next. Any two invocations of the same function could run on completely different containers under the hood.

Ephemeral

FaaS are designed to spin up quickly, do their work and then shut down again. They do not linger unused. As long as the task is performed the underlying containers are scrapped.

Event-triggered

Although functions can be invoked directly, yet they are usually triggered by events from other cloud services such as HTTP requests, new database entries or inbound message notifications. FaaS are often used and thought of as the glue between services in a cloud environment.

Scalable by default

With stateless functions multiple containers can be initialised, allowing as many functions to be run (in parallel, if necessary) as needed to continually service all incoming requests.

Fully managed by a Cloud vendor

AWS Lambda, Azure Functions, IBM OpenWhisk and Google Cloud Functions are most well-known FaaS solutions available. Each offering typically supports a range of languages and runtimes e.g. Node.js, Python, .NET Core, Java.

The Serverless App

A Serverless solution consists of a web server, Lambda functions (FaaS), security token service (STS), user authentication and database.

Source: http://blog.tonyfendall.com/2015/12/serverless-architectures-using-aws-lambda/

• Client Application — The UI of your application is rendered client side in Modern Frontend Javascript App which allows us to use a simple, static web server.

• Web Server — Amazon S3 provides a robust and simple web server. All of the static HTML, CSS and JS files for our application can be served from S3.

• Lambda functions (FaaS) — They are the key enablers in Serverless architecture. Some popular examples of FaaS are AWS Lambda, Google Cloud Functions and Microsoft Azure Functions. AWS Lambda is used in this framework. The application services for logging in and accessing data will be built as Lambda functions. These functions will read and write from your database and provide JSON responses.

• Security Token Service (STS) — generates temporary AWS credentials (API key and secret key) for users of the application. These temporary credentials are used by the client application to invoke the AWS API (and thus invoke Lambda).

• User Authentication — AWS Cognito is an identity service which is integrated with AWS Lambda. With Amazon Cognito, you can easily add user sign-up and sign-in to your mobile and web apps. It also has the options to authenticate users through social identity providers such as Facebook, Twitter or Amazon, with SAML identity solutions, or using your own identity system.

• Database — AWS DynamoDB provides a fully managed NoSQL database. DynamoDB is not essential for a serverless application but is used as an example here.

Benefits of Serverless Architecture

From business perspective

1. The cost incurred by a serverless application is based on the number of function executions, measured in milliseconds instead of hours.

2. Process agility: Smaller deployable units result in faster delivery of features to the market, increasing the ability to adapt to change.

3. Cost of hiring backend infrastructure engineers goes down.

4. Reduced operational costs

From developer perspective

1. Reduced liability, no backend infrastructure to be responsible for.

2. Zero system administration.

3. Easier operational management.

4. Fosters adoption of Nanoservices, Microservices, SOA Principles.

5. Faster set up.

6. Scalable, no need to worry about the number of concurrent requests.

7. Monitoring out of the box.

8. Fosters innovation.

From user perspective

1. If businesses are using that competitive edge to ship features faster, then customers are receiving new features quicker than before.

2. It is possible that users can more easily provide their own storage backend(i.e Dropbox, Google Drive).

3. It’s more likely that these kinds of apps may offer client-side caching, which provides a better offline experience.

Drawbacks of Serverless Architecture

From business perspective

1. Reduced overall control.

2. Vendor lock-in requires more trust for a third-party provider.

3. Additional exposure to risk requires more trust for a third party provider.

4. Security risk.

5. Disaster recovery risk

6. Cost is unpredictable because the number of executions is not predefined

7. All of these drawbacks can be mitigated with open-source alternatives but at the expense of cost benefits mentioned previously

From developer perspective

1. Immature technology results in component fragmentation, unclear best-practices.

2. Architectural complexity.

3. The discipline required against function sprawl.

4. Multi-tenancy means it’s technically possible that neighbour functions could hog the system resources behind the scenes.

5. Testing locally becomes tricky.

6. Significant restrictions on the local state.

7. Execution duration is capped.

8. Lack of operational tools

From user perspective

1. Unless architected correctly, an app could provide a poor user experience as a result of increased request latency.

Serverless Frameworks

Source: https//serverless.com

Serverless platforms need infrastructures where they can be executed, provider agnostic frameworks provide a platform agnostic way to define and deploy Serverless code on various cloud platforms or commercial services.

• Serverless Framework (Javascript, Python, Golang)

• Apex (Javascript)

• ClaudiaJS (Javascript)

• Sparta (Golang)

• Gordon (Javascript)

• Zappa (Python)

• Up (Javascript, Python, Golang, Crystal)

Summary

Serverless architecture is certainly very exciting, but it comes with a bunch of limitations. As the validity and success of architectures depend on the business requirements and by no means solely on technology. In the same way, Serverless can shine when used in proper place.

Take a look at the awesomeness that is Serverless, it's time to take a peek at what Serverless looks from the inside. Here are few links to get you started on your Serverless journey.

serverless/examples
_Serverless Examples - A collection of boilerplates and examples of serverless architectures built with the Serverless…_github.com

anaibol/awesome-serverless
_awesome-serverless - :cloud: A curated list of awesome services, solutions and resources for serverless / nobackend…_github.com

Building Serverless Contact Form For Static Websites
_Handling static forms with AWS serverless lambda functions_hackernoon.com

I hope this article helped in the understanding of Serverless Computing. I’d love to hear about how you use Serverless in your projects. Share the knowledge, help it reach more people.

In the first part of this tutorial, we created an API which passed requests through to a Lambda which returned the top tv show or movie for that genre. Now we’re going to use DynamoDB to allow users to vote for their favourite genre.

If you’ve not read the first part of this series, then check it out here!

DynamoDB

DynamoDB is a non-relational database created by Amazon that we can use to store the users’ votes. It’s also great, because we can access it easily using the aws-sdk which Lambdas have preinstalled.

The first thing we need to do is to create a table to store the movie votes. Navigate to DynamoDB in AWS and click “Create table”.

On the next page, we will name our table and provide a primary key. The primary key needs to be unique so we don’t have two records with the same key. We can call the table “movie-api” and set the primary key to “movie-genre”, as each movie should only appear once in each genre.

We’ve now set up all that we need to set up in DynamoDB so we can go back into our code.

Adding a Dynamo handler

Getting and putting data into a Dynamo table is done using the documentClient on aws-sdk, but the structure of the requests is very specific. To make our lives easier, we can make a Dynamo handler to do all of the formatting.

Start by creating a new file called “dynamo.js” in the “movieAPI” Lambda. In this file we start by requiring aws-sdk and creating our documentClient.

const AWS = require('aws-sdk');

let documentClient = new AWS.DynamoDB.DocumentClient({    'region': 'eu-west-1'});

We now want to create and export a class that has three methods on it: a get, a write, and an update.

module.exports = class DB {    get(key, value, table) {}    write(ID, data, table) {}    async increment(ID, table) {}}

We’ll start by creating our get method. The first thing we need to do is to check that we have a valid key, value, and table.

if (!table) throw 'table needed';if (typeof key !== 'string') throw `key was not string and was ${JSON.stringify(key)} on table ${table}`;if (typeof value !== 'string') throw `value was not string and was ${JSON.stringify(value)} on table ${table}`;

Because we want this method to be promise-based, we need to return a new Promise.

return new Promise((resolve, reject) => {})

To get data from Dynamo, we need to pass a set of parameters to the document client. These params need to include TableName and Key.

let params = {    TableName: table,    Key: {[key]: value}};

We pass these params to documentClient and then reject if there is an error or resolve if there isn’t.

documentClient.get(params, function(err, data) {    if (err) {        console.log(`There was an error fetching the data for ${key} ${value} on table ${table}`, err);        return reject(err);    }    return resolve(data.Item);});

A similar process is done for the write method. We check that the parameters are valid, create the parameters, and pass them to documentClient.

return new Promise((resolve, reject) => {    if (typeof ID !== 'string') throw `the id must be a string and not ${ID}`;    if (!data) throw "data is needed";    if (!table) throw 'table name is needed';

    let params = {        TableName: table,        Item: { ...data, ID: ID }    };

    documentClient.put(params, function(err, result) {        if (err) {            console.log("Err in writeForCall writing messages to dynamo:", err);            console.log(params);            return reject(err);        }        console.log('wrote data to table ', table)        return resolve({ ...result.Attributes, ...params.Item });    });});

The increment method is a lot easier. To increment, we try getting the data for that key, increase the count by one, and then write it back to the database. If we can’t get the data, or if there is no count on the data, then we assume that we need to set the count to 0.

async increment(ID, table) {    if (!table) throw 'table needed';    if (!ID) throw 'ID needed';    let data;    try {        data = await this.get('movie-genre', ID, table);        if (!data.count) throw 'no count in data'    } catch (err) {            data = { "movie-genre": ID, count: 0 };    };    let newData = { ...data, count: data.count + 1 };    return this.write(ID, newData, table);}

Modifying our Lambda

Now we have an easy way to get, write, and update our Dynamo table. We can use this to allow our users to vote. Inside “index.js”, we first need to import our new Dynamo class and create an instance of it.

const DB = require('./dynamo');const Dynamo = new DB();

Now, inside of our putMovie we can add the logic to allow users to vote. The two things we need to get are movie from the body and genre from the path parameters. We then combine these to create our movie-genre ID. This then gets passed into Dynamo.increment with a table name of movie-api and our putMovie is complete.

const putMovie = async event => {    let { movie } = JSON.parse(event.body);    let genre = event.pathParameters.genre;    let ID = `${movie}-${genre}`;    return Dynamo.increment(ID, 'movie-api')}

To get this to work when we receive the Put request, we need to slightly modify our base handler function.

if (event.httpMethod === 'PUT') {    let response = await putMovie(event)    return done(response);}

Because we’ve added AWS to our Lambda, we need to run npm init and then npm install — save aws-sdk while in the Lambda folder. This can be done locally and uploaded, or done using Cloud9.

Adding the API Gateway method

With the new function, we can add a new method to our API. In API Gateway, we can select our “movieAPI” and then select “/movies/{genre}”. Click “Actions” -> “Create Meth_o_d” and choose to add a “PUT” method.

This “PUT” can be directed at our “movieAPI”, and tick “Use Lambda Proxy Integration”. Once saved we can test it out. On the method we can click “TEST” and enter a genre and a body containing a movie. When we click “TEST” we get a response containing the movie and new count. As this is the first vote, the count will be 1.

Running the test a second time will now increment the votes for this movie by one.

Modifying the GET method

Now that we have a new voting system, we can update our “GET” to use this new data. We need to get all of the movies that are in the requested genre and list them in order of votes.

We first need to make a new dynamo method. This method will scan each of the entries and pick out the ones that match our criteria.

scan(key, value, table) {    return new Promise((resolve, reject) => {        let params = {             TableName: table,             FilterExpression: `${key} = :value`,             ExpressionAttributeValues: { ':value': value }         };         documentClient.scan(params, function(err, data) {             if (err) reject(err);             resolve(data);         });    });}

We can now modify our getMovie function to use this new Dynamo method. We need to pass the genre, selected movie, and current count.

const getMovie = async event => {    let genre = event.pathParameters.genre;    let data = await Dynamo.scan('genre', genre, 'movie-api');    let result = data.Items.sort((a,b) => b.count - a.count);    result = result.map(({count, ID, genre})=> { return {count, ID, genre}});    return data;}

The last thing to do is to add an await before out getMovie function so that it handles the async database scan.

let response = await getMovie(event);

Testing

When we hit this new “GET” endpoint we receive an ordered list of all of the movies in the database.

[  {    "count": 2,    "ID": "Desperado (1995)-action",    "genre": "action"  },  {    "count": 1,    "ID": "Team America (2004)-action",    "genre": "action"  }]

Summary

We’ve now built an API that can handle “GET” and “PUT” requests, storing and retrieving data from a Dynamo database. You can also reuse a lot of the Dynamo class code for other APIs that work with Dynamo.

Want some practice?

If you’ve enjoyed this, then why not try to implement a similar setup for tv shows? If you do, then let me know how it goes!

You can also improve this API by making sure that Desperado (1995) and desperado (1995) both count towards the same movie, or only allow a certain format of movie title.

If you’ve liked this, then make sure to give it a clap and subscribe for more Amazon tutorials and guides. See you in the next article and Keep Coding!

Build a real-world serverless application on AWS with Serverless framework and ready-to-use functions from StackStorm Exchange open-source catalog.

Episode One | Episode Two | Episode Three | Episode Four

In this second episode, I’ll add two more actions: create a contact in ActiveCampaign, and record a user in Dynamo DB. The full code for this episode is on Github at 2-add-more-actions branch.

Add more actions

To continue to the next step, you’ll need an ActiveCampaign account. Fear not: it takes 2 minutes — I just tried — they didn’t ask me for a credit card or anything stupid. Just email. Once in, go to “My Settings”, and select “Developer”.

Note the URL and Key fields in the "API access" section. Add them into env.yml like this:

# Copy to env.yml and replace with your values.# Don't commit to Github :)slack:  admin_token: "xoxs-111111111111-111111111111-..."  organization: "your-team"active_campaign:  url: "https://YOUR-COMPANY.api-us1.com"  api_key: "1234567a9012a12345z12aa2..."

PRO TIP: If you’re not in the mood for Active Campaign, or any sign-ups, just mock up the API endpoint with something like Mocky or mockable.ioand adjust the examples accordingly. For bonus points, create another serverless function with API Gateway endpoint in your serverless.yml and use it to mock the ActiveCampaign API call.

Now I am ready to add an Active Campaign lambda function. Same routine as in Episode One: start with finding a new action in a pack with sls stackstorm info --pack activecampaign (hint: the action you're looking is contact_sync which adds and updates a contact). Inspect the action with sls stackstorm info --action activecampaign.contact_sync:

Bunch of parameters, but besides the config, only email is required. I also want to use first_name and last_name. The function definition in serverless.yml will look like this:

RecordAC:    timeout: 10    memorySize: 128    stackstorm:      action: activecampaign.contact_add      input:        email: "{{ input.body.email }}"        first_name: "{{ input.body.first_name }}"        last_name: "{{ input.body.last_name }}"      config: ${file(env.yml):activecampaign}

I had to bump up the timeout as the ActiveCampaign API is taking longer than Lambda's default 6 seconds during Christmas season. But I can reclaim invocation cost by lowering memory use from default 512Mb. This time I don't bother to expose it to AWS API Gateway - we can conveniently test it with sls.

The function is ready to fly to AWS. We could do it at once with sls deploybut let's take it slow again and repeat the deployment steps just like in episode 1 of this tutorial to engrave the workflow in your mind:

1. Build and package the bundle:

sls package

2. Test locally (note I’m using --passthrough to only test transformations, remove it to make an actual call):

sls  stackstorm docker run --function RecordAC \--passthrough \--verbose --data '{"body":{"email":"santa@mad.russian.xmas.com", "first_name":"Santa", "last_name": "Claus"}}'

3. Deploy to AWS:

sls deploy

4. Test on AWS with sls invoke:

sls invoke --function RecordAC --logs \ --data '{"body":{"email":"santa@mad.russian.xmas.com", "first_name":"Santa", "last_name": "Claus"}}'

5. Check the logs:

sls logs --function  RecordAC

and I checked, it works: Santa Claus appeared in ActiveCampaign’s contact list. How do I know it’s our Lambda? Because I no longer believe Santa is real enough to subscribe to community without our little Lambda function.

PRO TIP: If you hit a bug or want a feature in a pack you are using from StackStorm Exchange, you can fix it on the spot. Packs are cloned under ./~st2/packs. Find your action there, modify the code, and use a local run to debug and validate.
Big bonus for pushing fixes back to the Exchange: each pack is already a git clone that makes it naturally easy to contribute back to the community.

Let’s add the final action, RecordDB, that writes contact info into a DynamoDB table. I could have used the aws.dynamodb_put_item action from the AWS pack on StackStorm Exchange - the pack is heavily used and well maintained. But I decided to make it a native Lambda: it's just 30 lines of code, with no extra Python dependencies, since the Boto library is already in the AWS Lambda environment.

The function’s code goes into ./record_db/handler.py:

The final serverless.yml with all three actions now looks like this:

Wow! The file doubled. The function itself is just 2 lines (35:36), but quite a few interesting additions took place. Let’s review them:

1. Line 9 — Table name generated to avoid collisions between regions and environments.
2. Lines 10:13 — IAM Role created to give the function access to the DynamoDB table.
3. Line 39:56 — AWS CloudFormation template defining the table.

The last point brings up an important observation: Serverless framework simplifies mainly the FaaS part of serverless. But because servereless is more than FaaS, you’ll find yourself writing quite a lot of CloudFormation to provision other services. Serverless framework leaves space for provider specific resources. To be seriously serverless on AWS, master CloudFormation. Looking for “provider-agnostic serverless”? Double-check your assumptions.

Also, I moved memorySize and timeout to apply to all functions (lines 6:7). And the API Gateway endpoint is gone from the InviteSlack function: it served the demonstration role in the first episode, but now we've learned to test Lambda functions directly. We'll return to API Gateway later to invoke the final StepFunction end-to-end workflow.

Let’s get the RecordDB function up to the sky and running.
Deploy, invoke, logs. Rinse repeat.

# Deploysls deploy ...

# Invokesls invoke --function RecordDB --logs --data '{"body":{"email":"santa@mad.russian.xmas.com", "first_name":"Santa", "last_name": "Claus"}}'...

# Check logs.sls logs --function RecordDB.

Now all three actions are here, waiting to be wired together into a final workflow with a StepFunction. But I’m just reminded not to be Mr. Scrooge: it’s the holiday season, let’s take it easy and put it off to the next episode. Until then, keep the spirits high!

The complete code example for this episode is on Github at 2-add-more-actions branch.

Episode 3: Wiring the actions with a StepFunction

Hope this helped you learn something new, find something interesting, or provoked some good thoughts. Please share your thoughts in the comments here, or tweet me @dzimine.

When you’re building Lambda functions, it’s easy to get trapped in an “Invalid ELF header” nightmare. The problem is that your binaries are built for the wrong platform. Here’s what’s going on, and how you can fix it easily using Docker.

When most people get started with Lambda functions, they’ll use the online editor in the console to input their code. This is fine for your first example or two, but pretty soon you’ll want to do something wild and crazy like, y’know, import a library.

Over the past few weekends I’ve been working on my first Alexa skill for an Amazon Echo and I’ve reached that point where I want to use some third party code to add some additional functionality.

Your options for supplying the code for your Lambda function, as seen in the AWS Console

The online editor simply lets you edit a single file. If you want to refer to other files — including imported libraries — you can upload them in a ZIP file (Amazon call this a deployment package). But if you‘re working on a Mac or a Windows computer, there’s a catch.

When you use pip to install Python libraries on your laptop, it gives you binaries (.so files) that are built to run on your machine. But when the Lambda function runs in the AWS cloud it is going to be running on Linux — and binaries built for Mac (these are often called ‘darwin’ builds) or Windows won’t run on Linux (and vice versa).

If you upload the Mac version, you’ll see “invalid ELF header” logs when you try to test your Lambda function.

Invalid ELF header indicates it’s the wrong binary format for the platform

So you’re going to need Linux versions of those library files. But what if you don’t have a Linux box to hand?

You could grab yourself an EC2 instance from Amazon (or a droplet on Digital Ocean, or any Linux VM of your preference) but to my mind that’s quite a performance, and could even cost you a little bit of money (especially if you forget to take the EC2 box down again when you don’t need it).

I think the easiest solution is to use Docker.

The Docker approach

With Docker you can very easily can run a Linux container locally on your Mac, install the Python libraries within the container so they are automatically in the right Linux format, and zip up the files ready to upload to AWS. You’ll need Docker for Mac (or Windows) installed first.

Spin up an Ubuntu container that can see the Lambda code you want to upload.

docker run -v <directory with your code>:/working -it --rm ubuntu

• The -vflag makes your code directory available inside the container in a directory called “working”.
• The -itflag means you get to interact with this container.
• The --rm flag means Docker will remove the container when you’re finished.
• ubuntu is the name of an official container image containing, you guessed it, Ubuntu. If this container image isn’t already on your machine, Docker will download it for you.

You should now be inside the container at a shell prompt looking something like this:

root@c1996f32a397:/#

Install pip and zip:

$ apt-get update$ apt-get install python-pip$ apt-get install zip

Move into the working directory (you should be able to see your Lambda function code here):

$ cd working

Use pip to get the library/ies you’re interested in. You can use the -t flag to tell pip to put the libraries here in the current directory, which will be more convenient later as it’s where the AWS deployment package wants them to be:

$ pip install -t . <library>

If you’re very curious, you can take a look to see what this installs. In my own case I installed the editdistance library, which gave me the following additional directories and files.

editdistance:__init__.py __init__.pyc _editdistance.h bycython.so def.h

editdistance-0.3.1.dist-info:DESCRIPTION.rst INSTALLER METADATA RECORD WHEEL metadata.json top_level.txt

You can see that bycython.so file? This is the correct, Linux version of the binary that AWS was objecting to when I hit the Invalid ELF header (shown in the error log screenshot above).

Create the ZIP file with your Lambda code (in my case, a single file called lambda_function.py) and the libraries (for me, the two editdistance directories and their contents.

$ zip linux-lambda.zip lambda_function.py

$ zip -r linux-lambda.zip edit*

The -r flag on zip tells it to recursively add the contents of directories.

Now you have an archive file called linux-lambda.zip which is ready to upload to AWS. And because the directory is mounted from the host (your Mac) into the container, you can simply upload the file into the console.

Back in the terminal type exit to quit the container, and it will be as if it never existed, except for the existence of the linux-lambda.zip file, which is still available on the host.

Upload the ZIP file in the console, save it and try running a test. Invalid ELF header error no more!

If this article helps you out, please hit the ? button to make it easier for other people to find it. If you really like it, why not go bananas and share it too?

I’ve written a few other posts about what I’m learning as I write my first Alexa skill, like this one where I add database storage capabilities to my Lambda function. If you find them useful, you might be interested in a book I’m writing called Adventures with Alexa. Pick your own price!