Running AWS lambdas locally

Running AWS lambdas locally

While AWS lambdas provide a great way to build and maintain apis, they do increase complexity of running them locally for debugging or testing during development.

Fortunately, there are tools that allow you to do that. I will use AWS SAM for this demo.

How it works

AWS Serverless Application Model (AWS SAM) is an open-source framework for building serverless applications using infrastructure as code (IaC).

SAM can be used for different things, but in this demo I will only show how to use it to run lambdas locally.

Doing that is simple:

  1. create an AWS SAM template to define resources you need
  2. run AWS SAM

Installing AWS SAM

Installation docs can be found here.

If you are using brew, you can run brew install aws-sam-cli.

To validate your installation, run sam --version

Setup

I will use two simple lambdas that read and write to Redis. First, lets run redis as a docker container:

# docker-compose.yaml
services:
  redis:
    image: redis:latest
    container_name: redis
    ports:
      - "6379:6379"

Now, let's create two simple lambdas that will read and write to Redis.

Lambda that sets a value in Redis:

// cmd/writer/main.go
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"os"
	"time"

	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-lambda-go/lambda"
	"github.com/go-redis/redis/v8"
)

var rdb *redis.Client
var ctx = context.Background()

func init() {
	rdb = redis.NewClient(&redis.Options{
		Addr:     os.Getenv("REDIS_ADDR"),
		Password: os.Getenv("REDIS_PASSWORD"),
		DB:       0,
	})
}

func handler(req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	var input struct {
		Key   string `json:"key"`
		Value string `json:"value"`
		TTL   int    `json:"ttl_seconds,omitempty"`
	}
	err := json.Unmarshal([]byte(req.Body), &input)
	if err != nil {
		body, _ := json.Marshal(map[string]string{"error": "invalid request body"})
		return events.APIGatewayProxyResponse{StatusCode: 400, Body: string(body)}, nil
	}

	ttl := time.Duration(input.TTL) * time.Second
	err = rdb.Set(ctx, input.Key, input.Value, ttl).Err()
	if err != nil {
		body, _ := json.Marshal(map[string]string{"error": err.Error()})
		return events.APIGatewayProxyResponse{StatusCode: 500, Body: string(body)}, nil
	}

	body, _ := json.Marshal(map[string]string{"message": fmt.Sprintf("Key %s set successfully", input.Key)})
	return events.APIGatewayProxyResponse{
		StatusCode: 200,
		Body:       string(body),
	}, nil
}

func main() {
	lambda.Start(handler)
}

Lambda that reads a value from Redis:

// cmd/reader/main.go
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"net/http"
	"os"

	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-lambda-go/lambda"
	"github.com/go-redis/redis/v8"
)

var rdb *redis.Client

func init() {
	rdb = redis.NewClient(&redis.Options{
		Addr:     os.Getenv("REDIS_ADDR"),
		Password: "",
		DB:       0,
	})
}

func handler(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	key := request.QueryStringParameters["key"]
	if key == "" {
		body, _ := json.Marshal(map[string]string{"error": "Missing 'key' query parameter"})
		return events.APIGatewayProxyResponse{
			StatusCode: http.StatusBadRequest,
			Body:       string(body),
		}, nil
	}

	val, err := rdb.Get(ctx, key).Result()
	if err == redis.Nil {
		body, _ := json.Marshal(map[string]string{"error": fmt.Sprintf("Key '%s' not found", key)})
		return events.APIGatewayProxyResponse{
			StatusCode: http.StatusNotFound,
			Body:       string(body),
		}, nil
	} else if err != nil {
		body, _ := json.Marshal(map[string]string{"error": fmt.Sprintf("Redis error: %v", err)})
		return events.APIGatewayProxyResponse{
			StatusCode: http.StatusInternalServerError,
			Body:       string(body),
		}, nil
	}

	body, _ := json.Marshal(map[string]string{
		"key":   key,
		"value": val,
	})

	return events.APIGatewayProxyResponse{
		StatusCode: http.StatusOK,
		Body:       string(body),
	}, nil
}

func main() {
	lambda.Start(handler)
}

Now lets dockerize these lambdas, since we will run SAM lambdas as docker containers.

Writer:

// cmd/writer/Dockerfile
FROM --platform=linux/amd64 golang:1.23.5-alpine AS build

WORKDIR /app

COPY go.mod go.sum ./
RUN go mod download

COPY cmd/writer/*.go ./
RUN GOOS=linux GOARCH=amd64 go build -o writer main.go

FROM public.ecr.aws/lambda/go:1
COPY --from=build /app/writer ${LAMBDA_TASK_ROOT}/writer
CMD ["writer"]

Reader:

// cmd/reader/Dockerfile
FROM --platform=linux/amd64 golang:1.23.5-alpine AS build

WORKDIR /app

COPY go.mod go.sum ./
RUN go mod download

COPY cmd/reader/*.go ./
RUN GOOS=linux GOARCH=amd64 go build -o reader main.go

FROM public.ecr.aws/lambda/go:1
COPY --from=build /app/reader ${LAMBDA_TASK_ROOT}/reader
CMD ["reader"]

That completes our setup from the api's perspective. We need to create a template for AWS SAM to run.

We will define each lambda as a resource, and set some globals that apply to every lambda. Since lambda is running in it's own network, we are using host.docker.internal to route back to the docker host. This will allow our lambdas to communicate with redis. By default AWS SAM will look for ./template.yaml, although you can provide a custom path.

# template.yaml
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Globals:
  Function:
    Timeout: 10 # seconds
    MemorySize: 128
    Environment:
      Variables:
        REDIS_ADDR: host.docker.internal:6379
        REDIS_PASSWORD: ""

Resources:
  Reader:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      Events:
        Read:
          Type: Api
          Properties:
            Path: /read
            Method: get
    Metadata:
      Dockerfile: cmd/reader/Dockerfile
      DockerContext: .

  Writer:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      Events:
        Write:
          Type: Api
          Properties:
            Path: /write
            Method: post
    Metadata:
      Dockerfile: cmd/writer/Dockerfile
      DockerContext: .

How to run

Now that we have all the parts set up, we can build our project by running sam build. AWS SAM will create docker images and all configuration needed in it's build directory. Which you can later deploy to production if needed. I have used AWS SAM only for local testing though.

Once the project is built, we can run and serve our lambdas. AWS SAM will use a default gateway unless a gateway is specified manually.

One other important note is that AWS SAM uses your default AWS configuration unless specified otherwise. I have created a local profile which I use during development.

// .aws/config
[profile local]
aws_access_key_id = test
aws_secret_access_key = test
region = us-east-1
output = json

Now when running AWS SAM, we can provide the local profile.

  1. docker compose up - run redis
  2. sam build - build our project
  3. sam local start-api --profile local - serve our lambdas (default port is 3000)

You can test the lambdas by running:

  1. curl -X POST "http://127.0.0.1:3000/write" -H "Content-Type: application/json" -d '{"key":"hello","value":"world"}'
  2. curl -X GET "http://127.0.0.1:3000/read?key=hello"

The first time you hit the gateway, it might take a bit longer as AWS SAM wraps your docker image, but afterwards is behaves like a normal lambda invocation.

This wraps up AWS SAM demo for running lambdas locally. While it provides many more features, the most valuable for me was just running my lambdas locally. It is easily configurable and flexible.

If you need to add an authorizer to your gateway, you have that option as well.