As a softaware developer, one of the crucial tasks you'll encounter is implementing and testing a repository layer that interacts with databases. In this blog post, we'll explore how to create integration tests for a DynamoDB repository using LocalStack and Testcontainers, allowing you to test database interactions locally without relying on a live AWS environment.
This is part of an ongoing project in which I am developing a sample application showcasing hexagonal architecture. You can access the complete code for this project atgithub.com/Desgue/hexagonal-architecture-go..
What you going to need:
Docker installed
Testcontainers library
LocalStack Docker image
Go 1.20 or higher
Understanding LocalStack
LocalStack is a cloud service emulator that runs in a single container on your laptop or in your CI environment. With LocalStack, you can run your AWS applications or Lambdas entirely on your local machine without connecting to a remote cloud provider!
Simply, LocalStack provides a docker container that emulates a few Amazon Web Services so a team can test its application in a local environment without worrying about IAM policies and configuration details for each member as it would in a live environment.
The Role of Testcontainers
Testcontainers is an open source framework for providing throwaway, lightweight instances of databases, message brokers, web browsers, or just about anything that can run in a Docker container.
The Testcontainers API gives us a nice way of interacting with docker containers programmatically, without the need to run bash scripts or elaborate OS calls to run and stop our containers.
You simply call the RunContainer function from the localstack package and you have a LocalStackContainer instance to work with. Further, we will go deeper into what is happening, but for now, I just want to show how easy it is to initialize a docker container using this API.
container, err := localstack.RunContainer(
ctx,
testcontainers.WithImage("localstack/localstack:latest")
)
This code snippet will create and run the docker container from the latest image, easy-peasy right? Now you can call the Terminate method and that will stop and remove the container.
if err := container.Terminate(ctx); err != nil {
panic(err)
}
Setup
Before diving into the integration tests, we'll set up the structure of our DynamoDB repository and the User model.
We will structure our code in the following way:
├── repository
│ ├── dynamorepo.go
│ │── dynamorepo_test.go
├── domain
│ ├── user.go
Now we need to get the packages we will use in our application:
go get github.com/aws/aws-sdk-go
go get github.com/testcontainers/testcontainers-go
go get github.com/testcontainers/testcontainers-go/modules/localstack
In the domain/user.go file, we define a User struct with two attributes: ID and Name. This simplified model will serve as our database entries.
// domain/user.go
type User struct {
Id string `json:"id"`
Name string `json:"name" `
}
The better way to define the ID is to use something as UUID, but for simplicity, we will pass the ID when constructing the user.
In the repository/dynamorepo.go file, we create our DynamoDB repository. It holds the DynamoDB client and table name required for our database interactions. We'll focus on one method: Insert and FindById.
//repository/dynamorepo.go
type dynamoRepo struct {
client *dynamodb.DynamoDB
table string
}
func NewDynamoRepo(endpoint string) *dynamoRepo {
sess := session.Must(session.NewSessionWithOptions(session.Options{
Config: aws.Config{
Region: aws.String("us-east-1"), // As we are using the local instance of dynamodb it doesnt matter the region we choose
Endpoint: aws.String(endpoint),
},
Profile: "default",
}))
return &dynamo{
client: dynamodb.New(sess),
table: "test-table",
}
}
We will hardcode some values here and there but for this tutorial that doesn't matter.
//repository/dynamorepo.go
func (d *dynamoRepo) Insert(user domain.User) (domain.User, error) {
entityParsed, err := dynamodbattribute.MarshalMap(user)
if err != nil {
return domain.User{}, err
}
input := &dynamodb.PutItemInput{
Item: entityParsed,
TableName: aws.String(d.tableName),
}
_, err = d.client.PutItem(input)
if err != nil {
return domain.User{}, err
}
return user, nil
}
func (d *dynamoRepo) FindById(id string) (domain.User, error) {
result, err := d.client.GetItem(&dynamodb.GetItemInput{
TableName: aws.String(d.tableName),
Key: map[string]*dynamodb.AttributeValue{
"id": {
S: aws.String(id),
},
},
})
if err != nil {
return domain.User{}, err
}
if result.Item == nil {
return domain.User{}, errors.New("No user found with given ID")
}
foundUser := domain.User{}
err = dynamodbattribute.UnmarshalMap(result.Item, &foundUser)
if err != nil {
return domain.User{}, err
}
return foundUser, nil
}
Insert: Accepts a User struct, inserts it into the DynamoDB table, and returns the inserted User.FindById: Takes an ID and searches for a User with the given ID.
The testing can begin (kinda)
Now that we have our functionality we need to make sure it works, we will use the standard lib testing package for that. So basically the test logic is the following:
Start the Localstack container
Instantiate the db with the container endpoint
Call our tested function
Assert test cases
Terminate container
We still need one more setup function to make the code less repetitive, the createTable() function, feel free to skim through this implementation as it only regards the DynamoDB API.
In our dynamorepo.go:
// repository/dynamorepo.go
func (d *dynamoRepo) createTable() error {
input := &dynamodb.CreateTableInput{
AttributeDefinitions: []*dynamodb.AttributeDefinition{
{
AttributeName: aws.String("id"),
AttributeType: aws.String("S"),
},
},
KeySchema: []*dynamodb.KeySchemaElement{
{
AttributeName: aws.String("id"),
KeyType: aws.String("HASH"),
},
},
TableName: aws.String(d.table),
}
_, err := d.client.CreateTable(input)
if err != nil {
return err
}
return nil
}
Connecting to Localstack
Now that we have our basic functionality ready to be tested and our helper functions set up we can start writing our tests.
Our container variable will hold the LocalStackContainer object, we then Defer an anonymous function to terminate the container when the test is done.
// dynamorepo_test.go
ctx := context.Background()
container, err := localstack.RunContainer(ctx, testcontainers.WithImage("localstack/localstack:latest"))
if err != nil {
t.Fatal(err)
}
defer func() {
if err := container.Terminate(ctx); err != nil {
panic(err)
}
}()
Now we need to get the hostname and port to format our endpoint string and pass it to our repository constructor.
// dynamorepo_test.go
provider, err := testcontainers.NewDockerProvider()
if err != nil {
t.Fatal("Error in getting docker provider")
}
host, err := provider.DaemonHost(ctx)
if err != nil {
t.Fatal("Error in getting provider host")
}
With a little bit of research, I found out that testcontainers run the container default at internal port 4566/tcp and map it to a random port in the Docker host.
The equivalent docker command to localstack.RunContainer is :
docker run -p 4566/tcp localstack/localstack:latest
The container.MapperPort method returns the corresponding port mapped to the port value we pass it.
// dynamorepo_test.go
mappedPort, err := container.MappedPort(ctx, nat.Port("4566/tcp"))
if err != nil {
t.Fatal("Error in getting the external mapped port")
}
We can now format our endpoint string pass it to the repository constructor and build our DynamoDB client to start performing our database calls and test our logic.
// dynamorepo_test.go
endpoint := fmt.Sprintf("http://%s:%d", host, mappedPort.Int())
repo := NewDynamoRepository(endpoint)
Testing the repository
With our LocalStack containers up and running, we can now focus on creating the tests that validate the functionality of our DynamoDB repository. The process is simple, we create a table and a new user struct. We then insert this user into the table and verify that the operation was executed without errors by comparing the attributes of the returned user with those of the new user we defined earlier. The assertions are made using reflect.DeepEqual, making sure each of the attributes are equal one to another.
This way it guarantees that our basic functionality is tested.
// dynamorepo_test.go
func TestInsert(t *testing.T) {
ctx := context.Background()
// Run localstack container
container, err := localstack.RunContainer(ctx, testcontainers.WithImage("localstack/localstack:latest"))
if err != nil {
t.Fatal(err)
}
// stop and remove localstack container
defer func() {
if err := container.Terminate(ctx); err != nil {
panic(err)
}
}()
// Testcontainer NewDockerProvider is used to get the provider of the docker daemon
provider, err := testcontainers.NewDockerProvider()
if err != nil {
t.Fatal("Error in getting docker provider")
}
// From the provider we can find the docker host so we can compose our endpoint string
host, err := provider.DaemonHost(ctx)
if err != nil {
t.Fatal("Error in getting provider host")
}
// Gett external mapped port for the container port
mappedPort, err := container.MappedPort(ctx, nat.Port("4566/tcp"))
if err != nil {
t.Fatal("Error in getting the external mapped port")
}
endpoint := fmt.Sprintf("http://%s:%d", host, mappedPort.Int())
repo := NewDynamoRepository(endpoint)
if err := repo.createTable(); err != nil {
t.Fatal(err)
}
newUser := domain.User{
Id: "1",
Name: "Tester",
}
gotUser, err := repo.Insert(newUser)
if err != nil {
t.Errorf("Got error inserting user: %s", err)
}
if !reflect.DeepEqual(gotUser.Id, "1") {
t.Errorf("Got %v want %v", gotUser.Id, "1")
}
if !reflect.DeepEqual(gotUser.Name, "Tester") {
t.Errorf("Got %v want %v", gotUser.Name, "Tester")
}
func TestFindById (t *testing.T) {
/*
Run the container and format the endpoint as in TestSave
That way we can make sure no same container is being used to test different
functions, that way one test can't interfer in the other
*/
repo := NewDynamoRepository(endpoint)
if err := repo.createTable(); err != nil {
t.Fatal(err)
}
newUser := domain.User{
Id: "1",
Name: "Tester",
}
_, err = repo.Insert(newUser)
if err != nil {
t.Errorf("Got error inserting user: %s", err)
}
gotUser, err := repo.FindById(newUser.Id)
if err != nil {
t.Errorf("Got error searching user: %s", err)
}
if !reflect.DeepEqual(gotUser.Id, "1"){
t.Errorf("Got %v want %v", gotUser.Id, "1")
}
if !reflect.DeepEqual(gotUser.Name, "Tester"){
t.Errorf("Got %v want %v", gotUser.Name, "Tester")
}
}
Conclusion
In conclusion, this blog post has walked you through the essential steps of setting up integration tests for a DynamoDB repository using LocalStack and Testcontainers. By leveraging these tools, you can effectively test your database interactions locally without the need for a live AWS environment. Through our testing process, we verified the repository's functionality, ensuring that the methods worked as expected. While this blog provides a solid foundation for DynamoDB integration testing, it's important to remember that this is just the beginning, there's room for further improvement, including error handling and expanding test coverage for a more robust testing suite.