Buildkite is a powerful CI/CD platform addressing the modern complex software development needs with its rich ecosystem of plugins. Buildkite provides a highly scalable CI/CD platform supporting customizable runners, dynamic pipelines, realt-time test analysis, and many more features.
Testcontainers is an open source framework for provisioning throwaway, on-demand containers for development and testing use cases. Testcontainers make it easy to work with databases, message brokers, web browsers, or just about anything that can run in a Docker container.
This article will explore how to run Testcontainers-based tests on the Buildkite platform.
We’re going to use an example Java/Spring Boot application, which you can find on GitHub:
Getting Started with Buildkite
Buildkite Pipelines uses a hybrid architecture consisting of the following:
- Buildkite Dashboard: A software-as-a-service (SaaS) control panel for managing CI/CD pipelines. This coordinates work with Agents and displays results.
- Agents: Small, reliable, and cross-platform build runners. These are hosted by you, either on-premise or in the cloud. They execute the work they receive from the Buildkite dashboard.
The following diagram shows the split in Buildkite between the SaaS platform and the agents running on your infrastructure.
Source: https://buildkite.com/docs/pipelines/architecture
To use Buildkite, first we need to signup for an account. You can sign up using Email and Password or Sign up with GitHub. Once the account is created, create an organization which is a central place for your pipelines, test suites, teams and dashboards. Our next step is to set up an agent.
Agent Setup
The Buildkite agent is a small, reliable and cross-platform build runner that makes it easy to run automated builds on your own infrastructure. Its main responsibilities are polling buildkite.com for work, running build jobs, reporting back the status code and output log of the job, and uploading the job’s artifacts.
A buildkite agent can be installed on a wide range of operating systems and Cloud platforms. Once you go to your organization’s Dashboard (https://buildkite.com/<organization>), click on the Agents link in the top navigation bar. You can see Agent Quick Start Guides for different platforms.
In this tutorial, we are going to use a Ubuntu VM as an agent. You can create a Ubuntu VM locally or create a VM based on Ubuntu OS on any cloud platform like AWS, Azure, GCP, etc.
NOTE: The actual builds will be run on the agents. So make sure you allocated enough RAM and storage space to be able to run your workloads.
In the Agent Quick Start Guides, click on Ubuntu and follow the instructions to install the buildkite agent.
If you successfully installed the agent, then the agent will be listed in the Agents page similar to the following:
The actual builds will be executed on the agents, so we should have the necessary software installed to run the build pipelines. As our application is a Java/Spring Boot 3 application which uses Testcontainers for testing, we should have Java 17+ and Docker installed on the agent.
On our Ubuntu VM, let’s install Java 17 using the following command:
$ sudo apt install openjdk-17-jdk $ java -version openjdk version "17.0.8.1" 2023-08-24 OpenJDK Runtime Environment (build 17.0.8.1+1-Ubuntu-0ubuntu123.04) OpenJDK 64-Bit Server VM (build 17.0.8.1+1-Ubuntu-0ubuntu123.04, mixed mode, sharing)
Next, install Docker on our Ubuntu VM by following the official documentation at https://docs.docker.com/engine/install/ubuntu/. The buildkite agent will be running the agent with the user buildkite-agent. So, let’s add the user buildkite-agent to the docker group using the following command:
sudo usermod -aG docker buildkite-agent
To take the group changes into effect, restart the Ubuntu VM and we should be good to run our build pipelines on our agent.
Project Setup
In this article, we are going to create a project on GitHub and configure it to use Buildkite Pipelines as our Continuous Integration(CI) platform. If you already have an application you can use it or create a new project.
We are going to use the testcontainers-showcase repository which is a Spring Boot application using Testcontainers for testing.
Create .buildkite/pipelines.yml file with the following content in the root directory of the project, commit and push the changes.
steps:
- label: "Test"
commands:
- ./mvnw test
# if you are using Gradle
# - ./gradlew test
Buildkite Pipeline Setup
Now go to Buildkite Dashboard and click on the New pipeline button and configure the pipeline for your project by following the steps below:
- Enter the Git Repository url as https://github.com/testcontainers/testcontainers-showcase.git
- Enter Name as testcontainers-showcase
- Under the Steps section, click on Add button and select Read steps from repository option. This will read the pipeline definition from the .buildkite/pipeline.yml file in the repository.
- Now click on the Create Pipeline button.
- Optionally we can configure Webhooks as mentioned in the next step so that whenever you push any changes into the repository, the build will be triggered.
Next, click on the New Build button to manually trigger the pipeline, provide a Message and click on Create Build button.
The Buildkite Pipeline should be executed successfully as follows:
We have created a Buildkite agent using a Ubuntu VM, but there are other options like using Docker containers as agents as well. Setting up Docker inside Docker containers is tricky in certain environments.
This is where Testcontainers Cloud comes into the picture to make it easy to run Testcontainers-based tests simpler and more reliably.
By using Testcontainers Cloud, you don’t even need to have Docker daemon running on the agent, and containers will be running in the on-demand cloud environments so that you don’t need to use powerful CI agents with high CPU/Memory for your builds.
Let us see how to use Testcontainers Cloud with minimal setup and run Testcontainers-based tests.
Testcontainers Cloud based Setup
Testcontainers Cloud helps you to run Testcontainers based tests at scale by spinning up the dependent services as Docker containers on cloud and running your tests connecting to those services.
If you don’t have a Testcontainers Cloud account already, create an account as follows and get Service Account Token as follows:
- Sign Up for a Testcontainers Cloud account at https://app.testcontainers.cloud/signup
- Once logged in, create an organization
Navigate to the Testcontainers Cloud dashboard and generate a Service account:
Next, we need to set the TC_CLOUD_TOKEN as an environment variable.As mentioned in the Managing pipeline secrets, we can use various secret storage services like AWS Secrets Manager, Hashicorp Vault, etc to store your sensitive information. We can also use Buildkite agent’s environment hook to export secrets to a job.
To keep it simple for this tutorial, let’s configure TC_CLOUD_TOKEN as an environment variable using environment hooks. Create a file with name environment in /etc/buildkite-agent/hooks/ directory with the following content:
#!/bin/bash export TC_CLOUD_TOKEN=<YOUR_TOKEN_VALUE> set -e
Next, update the .buildkite/pipelines.yml file as follows:
steps:
- label: "Test"
commands:
- sh -c "$(curl -fsSL https://get.testcontainers.cloud/bash)"
- ./mvnw test
We have included a command before executing our tests to start the Testcontainers Cloud agent which takes the TC_CLOUD_TOKEN as an environment variable using environment hooks.
Now if you commit and push the updated .buildkite/pipelines.yml file then the pipeline will run the tests using Testcontainers Cloud. You should see the following logs statements indicating that the Testcontainers-based tests are using Testcontainers Cloud instead of the default Docker daemon.
Running global environment hook $ /etc/buildkite-agent/hooks/environment # TC_CLOUD_TOKEN added ... ... ... 04:19:19.600 [Test worker] INFO org.testcontainers.DockerClientFactory - Connected to docker: Server Version: 78+testcontainerscloud (via Testcontainers Desktop 1.5.3) API Version: 1.43 Operating System: Ubuntu 20.04 LTS Total Memory: 7407 MB
If you are a paid customer of Testcontainers Cloud, you can also leverage Testcontainers Cloud’s TurboMode in conjunction with build tools that feature parallel run capabilities to run our tests even faster. You can check Testcontainers Cloud pricing information at https://testcontainers.com/cloud/pricing/.
In the case of Maven, we can use the "-DforkCount=N" system property to specify the degree of parallelisation. For Gradle, we can specify the degree of parallelisation using the "maxParallelForks" property.
We can enable parallel execution of our tests using 4 forks in .buildkite/pipelines.yml as follows:
steps:
- label: "Test"
commands:
- sh -c "$(curl -fsSL https://get.testcontainers.cloud/bash)"
- ./mvnw test -DforkCount=4
For more information on using Turbo Mode, please refer to https://knowledge.testcontainers.cloud/turbo-mode.
Conclusion
In this article, we have explored how to run Testcontainers-based tests on Buildkite Pipelines using the default Docker daemon. Then we learned how to create a Testcontainers Cloud account and configure the pipeline to run tests using Testcontainers Cloud. We also explored leveraging Testcontainers Cloud TurboMode combined with your build tool’s parallel execution capabilities.
Though we have demonstrated this setup using a Java project as an example, Testcontainers libraries exist for other popular languages too, and you can follow the same pattern of configuration to run your Testcontainers-based tests on Azure Pipelines in Golang, .NET, Python, NodeJS, etc.
You can get started with Testcontainers Cloud by creating a free account at https://testcontainers.com/cloud/.
