Introduction to Testcontainers

In this post we’ll look at Testcontainers and how it can be used to spin up throwaway test databases for your integration tests. You’ll see that Testcontainers offers an excellent alternative to in memory databases, such as H2.

In memory databases have benefits in terms of speed and simplicity, but there’s one fundamental drawback. Your integration tests use a database technology that is substantially different to the one your application will use in production. Testcontainers allows you to spin up the database of your choice, so that your persistence tests run against the same database technology that you’ll use in production. This makes your tests more authentic and reduces risk.

Note: As well as databases, Testcontainers supports lots of other tech, such as Rabbit MQ, Kafka and Elasticsearch. This article will focus on database support.

Spring Boot Demo App

To demonstrate how Testcontainers can be used in a typical Java project, I’ve created a simple Spring Boot REST API for creating and retrieving Customers. The REST API uses a Hibernate persistence tier to talk to a MySQL database. I’ll show you how Testcontainers can be used to support both persistence tier integration tests and end to end tests.

The full source code is available on Github, so feel free to pull it before you go any further.

Rest API

The REST API uses a CustomerDao to create and read the Customer entity.

@Slf4j
@RestController
@AllArgsConstructor
public class CustomerController {

    private CustomerDao customerDao;

    @PostMapping(path = "/api/customer")
    public Customer createCustomer(@RequestBody Customer customer){

        log.info("saving [{}]", customer);
        Long persistedCustomerId = customerDao.save(customer);
        log.info("returning [{}]", persistedCustomerId);

        return customerDao.findById(persistedCustomerId).get();
    }

    @GetMapping(path = "/api/customer/{id}")
    public Customer getCustomer(@PathVariable("id") Integer customerId){

        log.info("retrieving customer Id [{}]", customerId);
        Optional<Customer> customer = customerDao.findById(Long.valueOf(customerId));
        log.info("returning [{}]", customer);

        return customer.orElseThrow(() -> new RuntimeException("customer not found for Id " + customerId));
    }

}

Customer Dao

The CustomerDao provides methods for querying by customer Id, querying by first name and saving a Customer. Only the DAO is shown below, but if you want to see the accompanying Spring Hibernate configuration, you can find it here.

@Repository
@AllArgsConstructor
public class CustomerDao {

    private SessionFactory sessionFactory;

    @Transactional
    public List<Customer> findByLastName(final String lastName) {

        Query<Customer> query = sessionFactory.getCurrentSession()
                                               .createQuery("From Customer Where lastName= :lastName")
                                               .setParameter("lastName", lastName);

        return query.list();
    }

    @Transactional
    public Optional<Customer> findById(Long customerId){

        return Optional.ofNullable(sessionFactory.getCurrentSession().get(Customer.class, customerId));
    }

    @Transactional
    public Long save(Customer customer){

        return (Long)sessionFactory.getCurrentSession().save(customer);
    }

}

Testing the Demo App

Before writing any of the integration or end to end tests, we’re going to create a custom test container so that we can spin up a MySQL database.

Defining a Custom Test Container

For simple use cases Testcontainers comes with a bunch of predefined database containers you can use out of the box. For example, the default MySQLContainer container could be added to your tests as follows.

private static MySQLContainer sqlContainer = new MySQLContainer<>("mysql:8.0");

If you require more control, like access to the startup and shutdown hooks, you can define your own test container by extending one the standard containers. That’s exactly what I’ve done with CustomMySqlContainer below.

@Slf4j
public class CustomMySqlContainer extends MySQLContainer<CustomMySqlContainer> {

    private static final String DB_IMAGE = "mysql:5.7.34";
    private static CustomMySqlContainer mysqlContainer;

    private CustomMySqlContainer() {
        super(DB_IMAGE);
    }

    public static synchronized CustomMySqlContainer getInstance() {

        if (mysqlContainer == null) {
            mysqlContainer = new CustomMySqlContainer();
        }
        return mysqlContainer;
    }

    @Override
    public void start() {

        super.start();
        System.setProperty("TEST_DB_URL", mysqlContainer.getJdbcUrl());
        System.setProperty("TEST_DB_USERNAME", mysqlContainer.getUsername());
        System.setProperty("TEST_DB_PASSWORD", mysqlContainer.getPassword());
    
        log.info("started MySql container TEST_DB_URL [{}] TEST_DB_USERNAME [{}] TEST_DB_PASSWORD [{}]");
    }

    @Override
    public void stop() {
        super.stop();
    }
}
  • Line 4 – defines the Docker image I want to run in the test container. Here I’ve specified mysql:5.7.34, which will be pulled from the public Docker Hub repo. In an enterprise environment you could specify an image from a private enterprise container repository.
  • Line 5 – defines CustomMysqlContainer as static.  This is so that we can create CustomMysqlContainer as a singleton.
  • Lines 7 to 9 – private constructor creates an instance of MysqlContainer using the specified image.
  • Lines 11 to 17 – synchronized getInstance method creates a CustomMysqlContainer singleton.
  • Lines 20 to 28 – is a container startup event hook that you can use to get access to the container after it’s started. This is a great way to get hold of container metadata that you might need for your tests. On lines 23 to 25 I get the URL, username and password from the container and set them as system properties so that they can be referenced later in the Spring configuration file application.yaml. When tests are executed, the container is created before the Spring test application context is loaded. This allows us to get important info about the test container and expose it as environment variables before Spring bootstraps.
  • Lines 31 to 33 – calls stop on the parent container class.

Note that CustomMySqlContainer is a generic component and so it wouldn’t make sense to add this class to every project. Instead it could be added to a utils Jar and reused across projects.

We now have a custom test container that we can use to spin up a MySQL instance whenever we need it. Next, we’ll look at how CustomMySqlContainer can be used as part of the DAO integration tests.

CustomerDao Integration Tests with Testcontainers

CustomerDaoTests is a standard Spring Boot integration test that uses JUnit 5 and the custom CustomMysqlContainer we created above.

@SpringBootTest
@Testcontainers
class CustomerDaoTests {

  @Autowired
  private CustomerDao customerDao;

  @Container
  public static CustomMySqlContainer mySqlContainer =  CustomMySqlContainer.getInstance()
                              .withInitScript("database/customer-schema.sql");

  @Test
  public void should_returnCustomers_with_LastNameBloggs() {

    List<Customer> customers = customerDao.findByLastName("Bloggs");

    assertThat(customers.get(0).getFirstName(), is("Joe"));
    assertThat(customers.get(0).getLastName(), is("Bloggs"));
    assertThat(customers.get(0).getDateOfBirth(), is(LocalDate.of(1983, 5, 17)));
    assertThat(customers.get(0).getGender(), is("male"));
  }

  @Test
  public void should_returnSavedCustomer() {

    Customer customer = new Customer();
    customer.setFirstName("Jane");
    customer.setLastName("Doe");
    customer.setDateOfBirth(LocalDate.of(1981, 5, 9));
    customer.setGender("female");

    customerDao.save(customer);

    List<Customer> customers = customerDao.findByLastName("Doe");

    assertThat(customers.get(0).getFirstName(), is("Jane"));
    assertThat(customers.get(0).getLastName(), is("Doe"));
    assertThat(customers.get(0).getDateOfBirth(), is(LocalDate.of(1981, 5, 9)));
    assertThat(customers.get(0).getGender(), is("female"));
  }

}
  • Line 1 – @SpringBootTest is the standard Boot annotation for loading the Spring test application context. The test application context includes all Spring managed components in the application and allows us to inject the @CustomerDao test subject.
  • Line 2 – @Testcontainers is a JUnit 5 Jupiter extension. It scans the test for fields annotated with @Container and calls the lifecycle methods of the annotated container reference.
  • Lines 8 to 10 – @Container is used to mark the CustomMySqlContainer that we created earlier. This ensures that Testcontainers will call the lifecycle methods on CustomMySqlContainer to start and stop it part of the tests. CustomMySqlContainer is defined as a static field, so that it’s shared across all test methods in the class. If we defined CustomMySqlContainer as an instance field, a new container instance would be started and stopped for each test method.  .withInitScript("database/customer-schema.sql") passes an initialization script to the container, which is run against the database as soon as it starts. This is an ideal hook for creating a schema and setting up test data.
  • Lines 12 to 21 – A simple DAO test that retrieves a customer by last name, Bloggs. In order for this data to exist in the database, we seeded it on startup, using the customer-schema.sql script.
  • Lines 23 to 40 – Tests the save methods by creating a customer and then retrieving it.

End to End Tests with Testcontainers

Testcontainers is also useful for writing authentic end to end tests. The pattern is almost identical to that described in the CustomerDaoTest above. This time however, we’re testing our API end to end, including the dta acces tier.

@Testcontainers
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class E2ECustomerAPITest {

  @Autowired
  private TestRestTemplate restTemplate;

  @Container
  public static CustomMySqlContainer mySqlContainer =  CustomMySqlContainer.getInstance()
                              .withInitScript("database/customer-schema.sql");

  @Test
  public void should_returnCustomer_forCustomerId_1() {

    ResponseEntity<Customer> customerResponse = restTemplate.getForEntity("/api/customer/1", Customer.class);

    assertThat(customerResponse.getStatusCode(), is(HttpStatus.OK));

    assertThat(customerResponse.getBody().getFirstName(), is("Joe"));
    assertThat(customerResponse.getBody().getLastName(), is("Bloggs"));
    assertThat(customerResponse.getBody().getDateOfBirth(), is(LocalDate.of(1983, 5, 17)));
    assertThat(customerResponse.getBody().getGender(), is("male"));
  }

  @Test
  public void should_saveCustomer_andReturnNewEntity() {

    Customer customer = new Customer();
    customer.setFirstName("Jane");
    customer.setLastName("Doe");
    customer.setDateOfBirth(LocalDate.of(1981, 5, 9));
    customer.setGender("female");

    HttpEntity customerEntity = new HttpEntity<>(customer);

    ResponseEntity<Customer> savedCustomerResponse = restTemplate.postForEntity("/api/customer", customerEntity, Customer.class);

    assertThat(savedCustomerResponse.getBody().getId(), is(notNullValue()));
    assertThat(savedCustomerResponse.getBody().getFirstName(), is("Jane"));
    assertThat(savedCustomerResponse.getBody().getLastName(), is("Doe"));
    assertThat(savedCustomerResponse.getBody().getDateOfBirth(), is(LocalDate.of(1981, 5, 9)));
    assertThat(savedCustomerResponse.getBody().getGender(), is("female"));
  }

}

Test Execution Times

Testcontainers is a great addition to your test toolbox and an excellent alternative to in memory databases. That said, there is a performance trade off you should be aware of. Testcontainer integration tests are slower than in memory tests. This shouldn’t come as a great surprise, as Testcontainers needs to spin up a container, start your chosen database and expose it from the container, before you can run your tests. An in memory database in comparison starts very quickly and results in faster test execution and developer feedback.

You can speed things up by reusing the test database container for multiple tests. From experience I’d say this is almost essential if you want reasonable execution times. If you really do need the isolation of a new database instance per test, you’ll have to tolerate significantly slower tests.

Its also important to remember that you should use TestContainers sparingly and mocks liberally. For example if you’re unit testing the service layer of you application, you should mock out the persistence tier using a mocking framework like Mockito, rather than using Testcontainers.

Testcontainers is great when you want to focus on persistence tier testing or you want some realistic end to end tests, but for everything else you’re much better off mocking out your persistence tier to keep tests as fast as possible.

Wrapping Up

All in all, Testcontainers is a great alternative to in memory database testing. Although there’s a trade off in terms of test execution time, I think it’s a fair price to pay for authentic tests that more closely aligned to your production setup.

Don’t forget that you can grab the full source code from Github and if you have any questions please leave a comment below.