Testcontainers: Testing with Real Dependencies

Software evolves over time and automated testing is an essential prerequisite for Continuous Integration and Continuous Delivery. Developers write various types of tests, such as unit tests, integration tests, performance tests, and E2E tests for measuring different aspects of the software.

Usually, unit testing is done to verify only business logic. And depending on the part of the system that is tested, external dependencies tend to be mocked or stubbed.

But the unit tests alone don’t give much confidence because the actual end-to-end functionality depends on various external service integrations. So, integration tests are used to verify the overall behavior of the system by using real dependencies.

Traditionally integration testing is a complex process that can involve:

• Installing and configuring the required dependent services such as databases, message brokers, etc.
• Setting up the web or application server
• Building and deploying the artifact (jar, war, native executable, etc) on the server
• Running integration tests

With Testcontainers, you can have the lightweight experience and simplicity of unit tests, combined with the reliability of integration tests running against real dependencies.

Why is testing with real dependencies important?

Tests should enable the developers to verify application behavior with quick feedback cycles during the actual development activity.

Testing with mocks or in-memory services not only gives the wrong impression that the system is working fine but can also to significantly delay the feedback cycle. Tests using real dependencies exercise the actual code and give more confidence.

Consider a common scenario of using in-memory databases like H2 for testing while using Postgres or SQL Server in production. There are a couple reasons why this is a bad practice.

1. Compatibility Issues

Any non-trivial application will leverage some of the database-specific features that might not be supported by in-memory databases. For example, a common way to apply pagination is using LIMIT and OFFSET.

SELECT id, name FROM employee ORDER BY name LIMIT 25 OFFSET 50

Imagine using the H2 database for testing and MS SQL Server for production. When you test with H2, the tests will pass, giving a wrong impression that your code is working fine. But it will fail in production because MS SQL Server doesn’t support LIMIT … OFFSET syntax.

2. In-memory databases may not support all the features of your production database

Sometimes applications use database vendor-specific advanced features which may not be fully supported by in-memory databases. Examples can include XML/JSON transformation functions, WINDOW Functions, and Common Table Expressions (CTE). In these cases, it’s impossible to test using in-memory databases.

These frequently grow into even larger problems when you’re mocking services in your own code. While mocks can help test scenarios where you can successfully extract the mock definition to use as a contract for services, this verification of compatibility oftentimes only adds complexity to the test setup.

The typical use of mocks won’t allow you to reliably verify that the your system behavior will work in the production environment. It also won’t give you confidence in the test suite’s ability to catch issues caused by code incompatibilities and third-party integrations. 

So, it’s strongly recommended to write tests using real dependencies as much as possible and use mocks only when needed.

Testing with real dependencies using Testcontainers

Testcontainers is a testing library that enables you to write tests using real dependencies with disposable Docker containers. It provides a programmable API to spin up required dependent services as Docker containers. This way, you can write tests using real services instead of mocks. So, regardless of whether you’re writing unit, API, or end-to-end tests, you can write tests using real dependencies with the same programming model.

Testcontainers libraries are available for the following languages and integrate well with most of the frameworks and testing libraries:

• Java
• Go
• Node.js 
• .NET
• Python
• Rust

Case study

Let’s see how Testcontainers can be used to test various slices of an application and how all of them look like “Unit tests with real dependencies”.

We’ll use example code from a SpringBoot application implementing a typical API service that’s consumed via a web app and uses Postgres for storing data. But since Testcontainers provides you with an idiomatic API for your favorite language, a similar setup can be achieved in all of them.

Treat these examples as illustrations to get a feel of what’s possible. And if you’re in the Java ecosystem, then you’ll recognize the tests you’ve written in the past or take inspiration on how you can do it.

Testing data repositories

Let’s say we have the following Spring Data JPA repository with one custom method.

public interface TodoRepository extends PagingAndSortingRepository<Todo, String> {
   @Query("select t from Todo t where t.completed is false")
   Iterable<Todo> getPendingTodos();
}

As we mentioned above, using an in-memory database for testing while using a different type of database for production isn’t recommended and can cause issues. A feature or query syntax supported by your production database type might not be supported by an in-memory database.

For example, the following query (which you might have in your data migration scripts) would work fine in Postgresql but will break in the case of H2.

INSERT INTO todos (id, title)
VALUES ('1', 'Learn Modern Integration Testing with Testcontainers')
ON CONFLICT do nothing;

So, it’s always recommended to test with the same type of database that’s used for production.

We can write unit tests for TodoRepository using SpringBoot’s slice test annotation @DataJpaTest. We’ll do this by provisioning a Postgres container using Testcontainers as follows:

@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@Testcontainers
class TodoRepositoryTest {
    @Container
    static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:14-alpine");

    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", postgres::getJdbcUrl);
        registry.add("spring.datasource.username", postgres::getUsername);
        registry.add("spring.datasource.password", postgres::getPassword);
    }

    @Autowired
    TodoRepository repository;

    @BeforeEach
    void setUp() {
        repository.deleteAll();
        repository.save(new Todo(null, "Todo Item 1", true, 1));
        repository.save(new Todo(null, "Todo Item 2", false, 2));
        repository.save(new Todo(null, "Todo Item 3", false, 3));
    }

    @Test
    void shouldGetPendingTodos() {
        assertThat(repository.getPendingTodos()).hasSize(2);
    }
}

The Postgres database dependency is provisioned by using Testcontainers JUnit5 Extension, and the test talks to the real Postgres database. For more information on using container lifecycle management see Testcontainers and JUnit integration.

By testing with the same type of database that’s used for production, instead of using an in-memory database, the chance of database compatibility issues is avoided altogether and increases the confidence in our tests.

For database testing, Testcontainers provides special JDBC URL support which makes it easier to work with SQL databases.

Testing REST API endpoints

We can test API endpoints by bootstrapping the application along with the required dependencies such as the database provisioned via Testcontainers. The programming model for testing REST API endpoints is the same as the Repository unit test.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@Testcontainers
public class TodoControllerTests {
    @LocalServerPort
    private Integer port;
    
    @Container
    static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:14-alpine");

    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", postgres::getJdbcUrl);
        registry.add("spring.datasource.username", postgres::getUsername);
        registry.add("spring.datasource.password", postgres::getPassword);
    }

    @Autowired
    TodoRepository todoRepository;

    @BeforeEach
    void setUp() {
        todoRepository.deleteAll();
        RestAssured.baseURI = "http://localhost:" + port;
    }

    @Test
    void shouldGetAllTodos() {
        List<Todo> todos = List.of(
                new Todo(null, "Todo Item 1", false, 1),
                new Todo(null, "Todo Item 2", false, 2)
        );
        todoRepository.saveAll(todos);

        given()
                .contentType(ContentType.JSON)
                .when()
                .get("/todos")
                .then()
                .statusCode(200)
                .body(".", hasSize(2));
    }
}

We’ve bootstrapped the application using the @SpringBootTest annotation and used RestAssured for making API calls and verifying the response. This will give us more confidence in our tests as there are no mocks involved, and it enables developers to do any kind of internal code refactoring without breaking API contact.

End-to-end testing using Selenium and Testcontainers

Selenium is a popular browser automation tool for performing end-to-end testing. Testcontainers provides a Selenium module that simplifies the execution of selenium-based tests in a Docker container.

@Testcontainers
public class SeleniumE2ETests {
   @Container
   static BrowserWebDriverContainer<?> chrome = new BrowserWebDriverContainer<>().withCapabilities(new ChromeOptions());
 
   static RemoteWebDriver driver;
   
   @BeforeAll
   static void beforeAll() {
       driver = new RemoteWebDriver(chrome.getSeleniumAddress(), new ChromeOptions());
   }
 
   @AfterAll
   static void afterAll() {
       driver.quit();
   }
 
   @Test
   void testViewHomePage() {
      String baseUrl = "https://myapp.com";
      driver.get(baseUrl);
      assertThat(driver.getTitle()).isEqualTo("App Title");
   }
}

We’re able to run Selenium tests using the same programming model with the WebDriver provided by Testcontainers. Testcontainers even makes it easy to record videos of the test execution without having to go through a complex configuration setup.

You can take a look at the Testcontainers Java SpringBoot QuickStart project for reference.

Conclusion

We looked at various types of tests that developers use for their applications: data access layer, API tests, and even end-to-end tests. We also discovered how using Testcontainers libraries simplifies the setup to run these with the real dependencies like the actual version of the database you’ll use in production. 

Testcontainers is available in multiple popular programming languages for example Java, Go, .NET, and Python. It also offers an idiomatic approach to transforming your tests with real dependencies into unit tests that developers know and love.

Testcontainers-based tests run the same way in your CI pipeline and locally, whether you choose to run an individual test via your IDE, a class of tests, or even the whole suite from the command line. This gives you unparalleled reproducibility of issues and developer experience.

Finally, Testcontainers enables writing tests using real dependencies without having to use mocks which brings more confidence to your test suite. So, if you’re a fan of a practical approach, check out the Testcontainers Java SpringBoot QuickStart, which has all the test types we looked at in this article available to run from the get-go.

Learn more

• Sign up for a Testcontainers Cloud account.
• Connect on the Testcontainers Slack.
• Learn about Testcontainers best practices.
• Get started with the Testcontainers guide.
• Subscribe to the Docker Newsletter.
• Have questions? The Docker community is here to help.
• New to Docker? Get started.