Skip to main content

Software evolves over time and automated testing is an essential prerequisite for Continuous Integration and Continuous Delivery. Developers write various types of tests such as unit tests, integration tests, performance tests, and E2E tests for measuring different aspects of the software.

Usually, unit testing is done to verify only business logic, and depending on the part of the system that is tested, external dependencies tend to be mocked or stubbed.

But the unit tests alone don’t give much confidence because the actual end-to-end functionality depends on various external service integrations. So, integration tests are used to verify the overall behavior of the system by using real dependencies.

Traditionally integration testing is a complex process that might involve the following steps:

  • Install and configure the required dependent services such as databases, message brokers, etc
  • Setup web or application server
  • Build and deploy the artifact (jar, war, native executable, etc) on the server
  • Finally, run integration tests

However, by using Testcontainers you can have both the lightweight experience and simplicity of unit tests, and the reliability of integration tests running against real dependencies.

1. Why is Testing with Real Dependencies Important?

Tests should enable the developers to verify application behavior with quick feedback cycles during the actual development activity.

Testing with mocks or in-memory services not only gives the wrong impression that the system is working fine but also has the potential to delay the feedback cycle significantly. Tests using real dependencies exercise the actual code and give more confidence.

Consider a common scenario of using in-memory databases like H2 for testing while using Postgres or SQL Server in production. There are a few reasons why this is a bad practice.

Compatibility Issues:

Any non-trivial application will leverage some of the database-specific features which might not be supported by in-memory databases. For example, a common way to apply pagination is using LIMIT and OFFSET.

SELECT id, name FROM employee ORDER BY name LIMIT 25 OFFSET 50

Imagine using the H2 database for testing and MS SQL Server for production. When you test with H2 the tests will pass giving a wrong impression that your code is working fine but will fail in production because MS SQL Server doesn’t support LIMIT … OFFSET syntax.

In-memory databases may not support all the features of your production database:

Your application may use database vendor-specific advanced features like XML/JSON transformation functions, WINDOW Functions, and Common Table Expressions (CTE) which may not be fully supported by in-memory databases. In such cases, it is not even possible to test using in-memory databases.

Very often these grow into an even larger problem when you’re mocking services in your own code.  While mocks can help to test scenarios where you can successfully extract the mock definition to use as a contract for services, very often such verification of compatibility only adds complexity to the test setup.

And the typical use of mocks neither allows you to reliably verify that the behavior of your system would work in the production environment nor gives you confidence that the test suite will catch the issues when incompatibilities with your code and third-party integrations arise. 

So, it is strongly recommended to write tests using real dependencies as much as possible and use mocks sparingly only when needed.

2. Testing with Real Dependencies using Testcontainers

Testcontainers is a testing library that enables you to write tests using real dependencies by using disposable Docker containers. It provides a programmable API to spin up required dependent services as Docker containers so that you can write tests using real services instead of mocks. So, irrespective of whether you are writing unit tests, API tests, or end-to-end tests, you can write tests using real dependencies with the same programming model.

Testcontainers libraries are available for the following languages and integrate well with most of the frameworks and testing libraries:

  • Java
  • Go
  • Node.js 
  • .NET
  • Python
  • Rust

3. Case Study

Let us see how Testcontainers can be used to test various slices of the application and all of them look like “Unit tests with real dependencies”. In this article we’ll use example code from a SpringBoot application implementing a typical API service, that’s consumed via a web app and uses Postgres for storing data. But since Testcontainers provides you with an idiomatic API for your favorite language, a similar setup can be achieved in all of them. So treat the examples as illustrations to get a feel of what’s possible. And if you’re in the Java ecosystem then you’ll recognize the tests you’ve written in the past or take inspiration on how you can do it.

3.1. Testing Data Repositories

Let’s say we have the following Spring Data JPA repository with one custom method.

public interface TodoRepository extends PagingAndSortingRepository<Todo, String> {
   @Query("select t from Todo t where t.completed is false")
   Iterable<Todo> getPendingTodos();
}

As we mentioned above, using an in-memory database for testing while using a different type of database for production is not at all recommended practice and can cause many issues. A feature or query syntax supported by your production database type might not be supported by an in-memory database.

For example, the following query which you might have in your data migration scripts would work fine in Postgresql but break in the case of H2.

INSERT INTO todos (id, title)
VALUES ('1', 'Learn Modern Integration Testing with Testcontainers')
ON CONFLICT do nothing;

So, it is always recommended to test with the same type of database that is used for production.

We can write unit tests for TodoRepository using SpringBoot’s slice test annotation @DataJpaTest by provisioning a Postgres container using Testcontainers as follows:

@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@Testcontainers
class TodoRepositoryTest {
    @Container
    static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:14-alpine");

    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", postgres::getJdbcUrl);
        registry.add("spring.datasource.username", postgres::getUsername);
        registry.add("spring.datasource.password", postgres::getPassword);
    }

    @Autowired
    TodoRepository repository;

    @BeforeEach
    void setUp() {
        repository.deleteAll();
        repository.save(new Todo(null, "Todo Item 1", true, 1));
        repository.save(new Todo(null, "Todo Item 2", false, 2));
        repository.save(new Todo(null, "Todo Item 3", false, 3));
    }

    @Test
    void shouldGetPendingTodos() {
        assertThat(repository.getPendingTodos()).hasSize(2);
    }
}

The Postgres database dependency is provisioned by using Testcontainers JUnit5 Extension and the test talks to the real Postgres database. For more information on using container lifecycle management see Testcontainers and JUnit integration.

By testing with the same type of database that is used for production, instead of using an in-memory database, the chance of database compatibility issues is avoided altogether and increases the confidence in our tests.
For database testing, Testcontainers provide special JDBC URL support which makes it easier to work with SQL databases.

3.2. Testing REST API Endpoints

We can test API endpoints by bootstrapping the application along with the required dependencies such as the database provisioned via Testcontainers. The programming model for testing REST API endpoints is the same as the Repository unit test.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@Testcontainers
public class TodoControllerTests {
    @LocalServerPort
    private Integer port;
    
    @Container
    static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:14-alpine");

    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", postgres::getJdbcUrl);
        registry.add("spring.datasource.username", postgres::getUsername);
        registry.add("spring.datasource.password", postgres::getPassword);
    }

    @Autowired
    TodoRepository todoRepository;

    @BeforeEach
    void setUp() {
        todoRepository.deleteAll();
        RestAssured.baseURI = "http://localhost:" + port;
    }

    @Test
    void shouldGetAllTodos() {
        List<Todo> todos = List.of(
                new Todo(null, "Todo Item 1", false, 1),
                new Todo(null, "Todo Item 2", false, 2)
        );
        todoRepository.saveAll(todos);

        given()
                .contentType(ContentType.JSON)
                .when()
                .get("/todos")
                .then()
                .statusCode(200)
                .body(".", hasSize(2));
    }
}

We have bootstrapped the application using the @SpringBootTest annotation and used RestAssured for making API calls and verifying the response. This will give us more confidence in our tests as there are no mocks involved and it enables developers to do any kind of internal code refactorings without breaking API contact.

3.3. End-To-End Testing using Selenium and Testcontainers

Selenium is a popular browser automation tool for performing end-to-end testing. Testcontainers provides a Selenium module that simplifies the execution of selenium-based tests in a docker container.

@Testcontainers
public class SeleniumE2ETests {
   @Container
   static BrowserWebDriverContainer<?> chrome = new BrowserWebDriverContainer<>().withCapabilities(new ChromeOptions());
 
   static RemoteWebDriver driver;
   
   @BeforeAll
   static void beforeAll() {
       driver = new RemoteWebDriver(chrome.getSeleniumAddress(), new ChromeOptions());
   }
 
   @AfterAll
   static void afterAll() {
       driver.quit();
   }
 
   @Test
   void testViewHomePage() {
      String baseUrl = "https://myapp.com";
      driver.get(baseUrl);
      assertThat(driver.getTitle()).isEqualTo("App Title");
   }
}

We are able to run Selenium tests using the same programming model by using the WebDriver provided by Testcontainers. Testcontainers even make it easy to record videos of the test execution without having to go through a complex configuration setup.

You can take a look at the Testcontainers Java SpringBoot QuickStart project for reference.

4. Conclusion

We looked at various types of tests that developers use for their applications: data access layer, API tests, and even end-to-end tests, and how using Testcontainers libraries simplifies the setup to run these with the real dependencies like the actual version of the database you’ll use in production. 

Testcontainers is available in multiple popular programming languages for example Java, Go, .NET, and Python, and gives you an idiomatic approach to transforming your tests with real dependencies into unit tests that developers know and love.

Testcontainers based tests run the same way in your CI pipeline, and locally, whether you choose to run an individual test via your IDE, a class of tests, or even the whole suite from the command line alike which gives you unparallel reproducibility of issues and developer experience.

Even more, Testcontainers enable writing tests using real dependencies without having to use mocks which brings more confidence to your test suite. So, if you’re a fan of a practical approach, check out the Testcontainers Java SpringBoot QuickStart which has all the test types we looked at in this article available to run from the get-go!

Siva Katamreddy

Siva Katamreddy is a Developer Advocate at AtomicJar sharing the awesomeness of Testcontainers and helping the developers to build and release their software with confidence. He is a published author, and has written multiple books on Java technologies.