Running Testcontainers Tests on GitLab CI

Testcontainers is a testing library that enables you to run your tests with dependencies like databases, message queues, search engines etc., using ephemeral Docker containers. Testcontainers manage the lifecycle of the docker containers using programmable API, which gives finer control over the required application dependencies setup.

GitLab is a popular Git-based Source Code management (SCM) platform. In addition to being an SCM platform, GitLab provides CI/CD, Issue Management, and Code Review capabilities.

This article will look into how to run Testcontainers-based tests on the GitLab CI Platform. We’ll resolve the problem of lacking a Docker environment in the default containerized runners by using the Docker-in-Docker pattern, and then both simplify the setup and speed up the execution by configuring tests to run with Testcontainers Cloud.

We’re going to use an example Java/SpringBoot application, which you can find on GitHub if you want to follow along: testcontainers-showcase.

GitLab CI Pipeline Setup

• Create a new repository in GitLab, or you can import an existing project into GitLab using the Import Project feature.
• Create a .gitlab-ci.yml file at the root of the repository in which we are going to define a pipeline including a job test as follows:

variables:
 MAVEN_OPTS: >-
   -Dhttps.protocols=TLSv1.2
   -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository
   -Dorg.slf4j.simpleLogger.showDateTime=true
   -Djava.awt.headless=true
 MAVEN_CLI_OPTS: >-
   --batch-mode
   --errors
   --fail-at-end
   --show-version
   --no-transfer-progress

image: maven:3-eclipse-temurin-19

cache:
 paths:
   - .m2/repository

test:
 stage: test
 script:
   - 'mvn $MAVEN_CLI_OPTS verify'

We have used a Docker container-based executor and defined a test job within the test stage in which we run tests using Maven. Now, commit the changes and push them to the repository. The pipeline automatically gets triggered and will fail with the following error: Could not find a valid Docker environment.

06:26:06.194 [testcontainers-lifecycle-0] INFO  org.testcontainers.dockerclient.DockerMachineClientProviderStrategy - docker-machine executable was not found on PATH ([/opt/java/openjdk/bin, /usr/local/sbin, /usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin])
06:26:06.198 [testcontainers-lifecycle-0] ERROR org.testcontainers.dockerclient.DockerClientProviderStrategy - Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
	UnixSocketClientProviderStrategy: failed with exception InvalidConfigurationException (Could not find unix domain socket). Root cause NoSuchFileException (/var/run/docker.sock)As no valid configuration was found, execution cannot continue.
See https://www.testcontainers.org/on_failure.html for more details.

Testcontainers-based tests failed because the Docker environment is not available in our executor. To fix the issue, we can use the Docker-In-Docker approach described in https://www.testcontainers.org/supported_docker_environment/continuous_integration/gitlab_ci/  to provide a docker environment inside our executor.

Using Docker-in-Docker (DinD)

Edit the .gitlab-ci.yml file to include the Docker-In-Docker service (docker:dind) and set the DOCKER_HOST variable to tcp://docker:2375 and DOCKER_TLS_CERTDIR to an empty string.

services:
 - name: docker:dind
   command: ["--tls=false"]

variables:
 DOCKER_HOST: "tcp://docker:2375"
 DOCKER_TLS_CERTDIR: ""
 DOCKER_DRIVER: overlay2
 MAVEN_OPTS: >-
   -Dhttps.protocols=TLSv1.2
   -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository
   -Dorg.slf4j.simpleLogger.showDateTime=true
   -Djava.awt.headless=true
 MAVEN_CLI_OPTS: >-
   --batch-mode
   --errors
   --fail-at-end
   --show-version
   --no-transfer-progress

image: maven:3-eclipse-temurin-19

cache:
 paths:
   - .m2/repository

test:
 stage: test
 script:
   - 'mvn $MAVEN_CLI_OPTS verify'

The pipeline will run, and the tests will now run successfully.

However, running Docker-in-Docker is not always a good practice: the setup is complex and easy to get wrong which often leading to performance problems especially if running the containers is a significant part of the load.

This is where Testcontainers Cloud comes into the picture to make it easy to run Testcontainers-based tests simpler and more reliably. 

By using Testcontainers Cloud, you don’t have to install Docker daemon on the runner, and containers will be running in the on-demand cloud environments so that you don’t need to use powerful CI workers with high CPU/Memory for your builds.

Let us see how to use Testcontainers Cloud with minimal setup and run Testcontainers-based tests.

Testcontainers Cloud based setup

Here’s how you can set up Testcontainers Cloud for your GitLab pipeline. In general, the CI configuration for Testcontainers Cloud requires two things. 

The first is an agent app, which runs in user space, doesn’t require special privileges, and is small enough to download as a part of the actual “running tests” step. 

And second, configure the access token so the agent uses your Testcontainers Cloud environment.

1. Sign Up for a Testcontainers Cloud account at https://app.testcontainers.cloud/signup
2. Once logged in, create an organisation
3. Navigate to the Testcontainers Cloud dashboard and generate a Service account:

Next, we need to set the TC_CLOUD_TOKEN as an environment variable.

• Go to project’s Settings > CI/CD and expand the Variables section.
• Select Add variable and fill in the details:
  • Key: TC_CLOUD_TOKEN
  • Value: Enter the Service Account Access Token
  • Type: Variable
  • Select Mask Variable checkbox
  • Click on Add Variable

Now, update .gitlab-ci.yml to remove the DinD configuration and install the Testcontainers Cloud agent as follows:

variables:
 MAVEN_OPTS: >-
   -Dhttps.protocols=TLSv1.2
   -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository
   -Dorg.slf4j.simpleLogger.showDateTime=true
   -Djava.awt.headless=true
 MAVEN_CLI_OPTS: >-
   --batch-mode
   --errors
   --fail-at-end
   --show-version
   --no-transfer-progress

image: maven:3-eclipse-temurin-19

cache:
 paths:
   - .m2/repository

test:
 stage: test
 script:
   - curl -fsSL https://app.testcontainers.cloud/bash | bash
   - 'mvn $MAVEN_CLI_OPTS verify'

With Testcontainers Cloud tests should run quicker as compared to the  Docker-in-Docker setup.

Just by enabling Testcontainers Cloud we managed to speed up the build by almost 40%, 8 minutes compared to the initial 13! 

We can also leverage Testcontainers Cloud’s TurboMode in conjunction with build tools that feature parallel run capabilities to run our tests even faster.

In the case of Maven, we can use the -DforkCount=N system property to specify the degree of parallelisation. For Gradle, we can specify the degree of parallelisation using the maxParallelForks property.

We tried running the tests in parallel with the Docker-in-Docker setup but couldn’t get it to work reliably, as the tests were failing due to the limited CPU/Memory resources. This is not surprising, given that without Testcontainers Cloud we are forced to run all the containers for all the forks and the tests themselves on the same worker machine exhausting its capacity.

While Testcontainers Cloud allows moving containers off the runner node, the tests themself will consume significant resources while running in parallel. To maximise the effect of the Turbo Mode, let’s use a machine type with sufficient CPU/memory specs.

GitLab.com provides different machine types which you can choose based on your needs. Following are available machine types for Linux SaaS Runner.

If you are running on a Linux runner, the default machine type is small. To use a different runner, you can specify tags as follows:

test:
 tags: [ saas-linux-medium-amd64 ]
 stage: test
 script:
   - curl -fsSL https://app.testcontainers.cloud/bash | bash
   - 'mvn $MAVEN_CLI_OPTS verify -DforkCount=4'

By using saas-linux-medium-amd64 runner with Testcontainers Cloud TurboMode the tests will run faster.

We can see that the tests ran much quicker by combining the build tool’s parallelisation capabilities with Testcontainers Cloud Turbo Mode on a machine type with decent CPU/Memory specs. All in all, parallelising the tests allowed us to get from 13:03 minutes to run the build to a mere 4:24, which is 3 times faster!

Conclusion

In this article, we have explored how to run Testcontainers-based tests on GitLab.com using the Docker-in-Docker service. Then we learned how to simplify the setup using Testcontainers Cloud and run tests faster. We also explored leveraging Testcontainers Cloud TurboMode combined with your build tool’s parallel execution capabilities to run tests faster. This made the build 3 times faster than the original solution, and allowed us to avoid the complexity of the Docker-in-Docker configuration.

Though we have demonstrated this setup using a Java project as an example, Testcontainers libraries exist for other popular languages too, and you can follow the same pattern of configuration to run your Testcontainers-based tests on GitLab CI in Golang, .NET, Python, and NodeJS.