Running AWS services locally

Whenever you want to test your cloud applications that use some specific cloud provider services, you need to have a way to run those services locally. Otherwise, you will pollute your cloud resources and potentially increase cloud costs or slow down your development by testing in cloud environments.

Testing your codebase is of course necessary and in general speeds up your development, however in some cases, having the option to test manually is necessary.

Localstack provides a very simple way to run your aws services.

Setup

Installation docs can be found here.

If you are using macos, you can just run brew install localstack/tap/localstack-cli.

You can also run localstack as a docker container.

services:
  localstack:
    container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}"
    image: localstack/localstack
    ports:
      - "127.0.0.1:4566:4566" # localstack gateway
      - "127.0.0.1:4510-4559:4510-4559" # external services port range
    environment:
      - DEBUG=${DEBUG:-0}
    volumes:
      - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

Usage

Using localstack is the same as using aws. To use the cli, you can create an aws profile:

# .aws/config
[profile local]
region = us-east-1
output = json
endpoint_url = http://localhost:4566


# .aws/credentials
[local]
aws_access_key_id = test
aws_secret_access_key = test

With this in place, you can use the aws cli with localstack. For example:

1. aws s3 mb s3://test-bucket --profile local
2. aws sns create-topic --name test-topic --profile local
3. aws secretsmanager create-secret --name test-secret --secret-string '{"key": "value"}' --profile local

Also, you can configure your IaC tools like terraform or terragrunt to work with localstack. To do that, setup your provider:

provider "aws" {
  region                      = "us-east-1"
  access_key                  = "test"
  secret_key                  = "test"
  s3_use_path_style           = true
  skip_credentials_validation = true
  skip_requesting_account_id  = true
  endpoints {
    lambda                       = "http://localhost:4566"
    apigateway                   = "http://localhost:4566"
    s3                           = "http://localhost:4566"
    dynamodb                     = "http://localhost:4566"
    cognitoidentity              = "http://localhost:4566"
    cognitoidp                   = "http://localhost:4566"
    cloudformation               = "http://localhost:4566"
  }
}

Unfortunately, some of the services and features are not free.

Persistence between restarts is a pro feature. To get around it, check this repo.

cognito-idp is also not free. In one of the next blogs I will show how to get around it.