Suppose we are developing a SpringBoot application offering two REST endpoints, one for storing records into and one to fetch records from an AWS DynamoDB table. AWS DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability1. To be able to integration test our application we usually would have to create a “test” DynamoDB table within the AWS Cloud and configure our application so that it can connect to the DynamoDB table and execute the desired create and read operations. Or we could use Testcontainers in combination with LocalStack for local integration testing our application backed by a DynamoDB.
What is Testcontainers?
Testcontainers is an open source framework for providing throwaway, lightweight instances of databases, message brokers, web browsers, or just anything which is able to run within a Docker container2.
Testcontainers may replace ordinary mocks or complicated environment configurations. Test dependencies are defined as code, containers will be created before test execution and deleted afterwards automatically. The only requirement is Docker. Testcontainers supports languages such as Java, Go, .NET, Node.js, Python, Rust, Haskell, Ruby, Clojure, or Elixir.
Why to use Testcontainers?
Testcontainers may help executing data access layer tests without requiring a complex setup on developer machines. Furthermore, automated UI tests can be executed using containerized web browsers that are compatible with Selenium. Testcontainers provide a predefined set of modules, which are pre-configured implementations of various dependencies, simplifying writing integration tests3, for example:
Benefits10 of Testcontainers
- On-demand isolated infrastructure provisioning
- No need of a pre-provisioned integration testing infrastructure
- Testcontainers will provide the required services before running your tests
- Consistent experience on both local and CI environments
- Run integration tests right from your IDE
- No need to push changes and wait for CI/CD pipeline to complete
- Reliable test setup using wait strategies
- Out-of-the-box strategy implementations to ensure containers are fully initialized before test execution
- Testcontainers modules already implement wait strategies for given technology
- Advanced network capabilities
- Map container’s ports onto ports available on host machine
- Create volumes and networks
- Connect multiple containers
- Automatic clean up
- Created resources (containers, volumes, networks, etc.) are removed automatically after test execution
- Reliable cleanup, even if the test exits abnormally
How to use Testcontainers?
Using Java and assuming Docker is already installed on the host, the simplest way to use a distinct test container module (e.g. LocalStack) is to create an instance and call additional configuration methods:
LocalStackContainer localStack = new LocalStackContainer(DockerImageName.parse("localstack/localstack"))
.withCopyFileToContainer(
MountableFile.forHostPath("src/test/resources/init-resources.sh"),
"/etc/localstack/init/ready.d/init-resources.sh"
);This example uses the configuration method withCopyFileToContainer to mount an initialization script from the host into the container. Testcontainers instances provide additional useful configuration methods11, such as:
| waitingFor(WaitStrategy) | Specify the WaitStrategy12 to use to determine if the container is ready |
| withCommand(String) | Set the command that should be run in the container |
| withCopyFileToContainer(MountableFile, String) | Set the file to be copied before starting a created container |
| withEnv(String, String) | Add an environment variable to be passed to the container |
| withNetwork(Network) | Set the network for this container, similar to the –network option on the Docker CLI |
| withStartupAttempts(int) | Allow container startup to be attempted more than once if an error occurs |
| withExposedPorts(int…) | Set the ports that this container listens on |
Generic Testcontainers
Of course, Testcontainers also provides the ability to start arbitrary containers:
GenericContainer genericContainer = new GenericContainer("myimage:latest")
.withExposedPorts(8883, 8884)
.withEnv("LOGLEVEL", "DEBUG")
.withStartupAttempts(2)
.waitingFor(Wait.forLogMessage(".*Ready to accept connections.*\\n", 1));What is LocalStack?
LocalStack offers the ability to deploy and test applications locally to reduce development time. Furthermore, cost generating AWS cloud usage and test complexity is reduced13, as no cloud resources have to be provisioned.
The LocalStack container provides four life cycle phases / stages:
- BOOT – the container is running but the LocalStack runtime has not been started
- START – the Python process is running and the LocalStack runtime is starting
- READY – LocalStack is ready to serve requests
- SHUTDOWN – LocalStack is shutting down
It is possible to hook into each of these life cycle phases using custom shell or Python scripts. Each life cycle phase has its own directory in /etc/localstack/init . Individual files, stage directories or the entire init directory can be mounted from the host into the container:
/etc
└── localstack
└── init
├── boot.d
├── ready.d
├── shutdown.d
└── start.dScripts located in:
boot.d/are executed before LocalStack startsstart.d/are executed when LocalStack starts upready.d/are executed when LocalStack becomes readyshutdown.d/are executed when LocalStack shuts down
For both test cases, the following init-resources.sh script is used and mounted to /etc/localstack/init/ready.d to create the DynamoDB test table:
#!/bin/bash
awslocal dynamodb create-table \
--region eu-central-1 \
--table-name test \
--attribute-definitions AttributeName=id,AttributeType=S \
--key-schema AttributeName=id,KeyType=HASH \
--billing-mode PAY_PER_REQUEST
The command shown above creates a DynamoDB table named “test”. DynamoDB is a schemaless database, therefore it is only necessary to define an attribute (type S = String) acting as a primary key (partition key)14. Furthermore, billing-mode is set to “PAY_PER_REQUEST” (instead of using the default value “PROVISIONED”) to keep the command as simple as possible15. Of course nothing is billed as all resources are running locally, though the DynamoDB API requires billing information.
Now the goal is to test create and read operations provided by endpoints within a SpringBoot application using the combination of Testcontainers and LocalStack.
The test case
This example is going to test two simple create and read operations: First, a POST request is executed adding a new record together with a randomly created UUID used as primary key (partition key) to the DynamoDB test table. Second, the same record will be queried by executing a GET request passing the corresponding UUID as @PathVariable.
Our application is based on Java17 and SpringBoot v3.2.4. Spring Boot makes it easy to create stand-alone, production-grade Spring based applications that “just run”16. As build tool Maven v3.9.3 is used17. Furthermore, a Docker runtime18 is needed on the host.
The complete source code is available19. The test can be executed by running mvn clean install within the project root folder. The test class executing the above mentioned workflow is defined as follows:
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
@Testcontainers
class CrudTest {
static LocalStackContainer localStack =
new LocalStackContainer(DockerImageName.parse("localstack/localstack"))
.withCopyFileToContainer(
MountableFile.forHostPath("src/test/resources/init-resources.sh"),
"/etc/localstack/init/ready.d/init-resources.sh"
);
@BeforeAll
static void init() {
localStack.start();
}
@AfterAll
static void stop() {
localStack.stop();
}
@DynamicPropertySource
static void configureProperties(final DynamicPropertyRegistry dynamicPropertyRegistry) {
dynamicPropertyRegistry.add("aws.accessKeyId", () -> localStack.getAccessKey());
dynamicPropertyRegistry.add("aws.secretKeyId", () -> localStack.getSecretKey());
dynamicPropertyRegistry.add("dynamodb.endpoint", () -> localStack.getEndpoint());
dynamicPropertyRegistry.add("dynamodb.tablename", () -> "test");
}
@Test
void postAndGet() {
final PersonTO personIn = new PersonTO("John", "Doe", 30);
final String id = given()
.contentType(ContentType.JSON)
.body(personIn)
.when()
.post("/person")
.then()
.statusCode(200)
.extract()
.path("id");
final PersonTO personOut = given()
.when()
.get("/person/" + id)
.then()
.statusCode(200)
.extract()
.as(PersonTO.class);
assert personIn.equals(personOut);
}
}The test class annotations @SpringBootTestand @Testcontainerssetup the test context to be able to use Testcontainers. First, a new instance of a LocalStack container is created and configured to copy the init-resources.shscript to the distinct life cycle configuration directory (below ready.d/). The test classinit() function is executed @BeforeAll tests and starts the LocalStack container. The stop() function is executed after all tests and stops the LocalStack container. The function configureProperties() extends the SpringBoot context setting the dynamic properties aws.accessKeyId, aws.secretKeyId and dynamodb.endpoint which are used to create a DynamoDB client bean inside the Spring context.
The method postAndGet() executes the test case described above. A POST request to the /person endpoint with sample data provided as HTTP message body is executed. The POST requests response contains a JSON with a UUID which is extracted and used as @PathVariable of the following GET request. The response of the GET request is expected to contain the entity which was stored to the DynamoDB table before. @AfterAll the test container is stopped.
As you can see, no mocks are involved here: The Spring context is started and data is stored into and fetched from the DynamoDB table backed by the LocalStack test container whose lifecycle is reliably managed by Testcontainers!
Conclusion
Testcontainers in combination with LocalStack is a powerful tool to test software making use of AWS Cloud resources such as DynamoDB, SQS, SNS, Kinesis, Lamda and a lot more. It provides a reliable and efficient testing environment which can easily be managed and used in various frameworks. Testcontainers in combination with LocalStack will help you to ensure application robustness and readyness for production in the AWS Cloud!
- https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Introduction.html ↩︎
- https://testcontainers.com/ ↩︎
- https://testcontainers.com/modules/ ↩︎
- https://testcontainers.com/modules/localstack ↩︎
- https://testcontainers.com/modules/google-cloud ↩︎
- https://testcontainers.com/modules/postgresql ↩︎
- https://testcontainers.com/modules/mongodb ↩︎
- https://testcontainers.com/modules/kafka ↩︎
- https://testcontainers.com/modules/hivemq ↩︎
- https://testcontainers.com/getting-started/ ↩︎
- https://javadoc.io/doc/org.testcontainers/testcontainers/latest/index.html ↩︎
- https://java.testcontainers.org/features/startup_and_waits/ ↩︎
- https://www.localstack.cloud/ ↩︎
- https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html ↩︎
- https://docs.aws.amazon.com/cli/latest/reference/dynamodb/create-table.html ↩︎
- https://spring.io/projects/spring-boot ↩︎
- https://maven.apache.org/ ↩︎
- https://www.docker.com/ ↩︎
- https://github.com/ConSol/TestContainersDemoSB ↩︎
