Creating modules in Testcontainers for Go

Often your projects depend on a certain technology in a very recurrent manner, and you could see yourself in the situation of moving code to interact with that technology from one project to another. “Why don’t we package this code into a library and consume it everywhere?”

That’s when your mind recalls certain software engineering best practices you learnt while at university, such as modularity, decoupling and cohesion. 

In this blog post you’ll understand how to create a new module for Testcontainers for Go, representing that technology you use in your projects and for which you want to test your code against it.

Creating a module from scratch

First, it’s important to create the distinction between, within the scope of Testcontainers for Go, a Go module, which will expose an API to the users, and an example module, which will represent a code snippet to be copied across projects, therefore, no public API will reside in it.

Testcontainers for Go provides a command-line tool to generate the scaffolding for the code of the module you are interested in, so you can get started quickly. This tool will generate:

• a Go module for the technology, including:
  • go.mod and go.sum files, including the current version of Testcontainer for Go.
  • a Go package named after the module, in lowercase
  • a Go file for the creation of the container, using a dedicated struct in which the value of the image flag is used as Docker image.
  • a Go test file for running a simple test for your container, consuming the above struct.
  • a Makefile to run the tests in a consistent manner
  • a tools.go file including the build tools (e.g. gotestsum) used to build/run the example.
• a markdown file in the docs/modules directory including the snippets for both the creation of the container and a simple test.
• a new Nav entry for the module in the docs site, adding it to the mkdocs.yml file located at the root directory of the project.
• a GitHub workflow file in the .github/workflows directory to run the tests for the technology.
• an entry in Dependabot’s configuration file, in order to receive dependency updates.

It’s important to mention that the code generation tool is able to generate a real Go module, or an example module. What are the differences between them?

Now that you know the difference between a Testcontainers for Go module and an example, you can move on and understand how to run the code generation tool.

The code generation tool

The code generation tool is a Go program that lives in the modulegen directory as a Go module, to avoid distributing Go dependencies with Testcontainers for Go, and its only purpose is to create the scaffolding for a Go module using Go templates.

The command-line flags of the code generation tool are described in the table below:

If the module name or title does not contain alphanumeric characters, the program will exit the generation. And if the module already exists, it will exit without updating the existing files.

From the modulegen directory, please run:

go run . --name ${NAME_OF_YOUR_MODULE} --image "${REGISTRY}/${MODULE}:${TAG}" --title ${TITLE_OF_YOUR_MODULE} –as-module

or for creating an example module:

go run . --name ${NAME_OF_YOUR_MODULE} --image "${REGISTRY}/${MODULE}:${TAG}" --title ${TITLE_OF_YOUR_MODULE}

Now that we have bootstrapped the layout for the module, it’s time to add functionality for it.

Adding types and methods to the module

We are going to propose a set of steps to follow when adding types and methods to the module:

1. Make sure a public Container type exists for the module. This type has to use composition to embed the testcontainers.Container type, promoting all the methods from it.
2. Make sure a RunContainer function exists and is public. This function is the entrypoint to the module and will define the initial values for a testcontainers.GenericContainerRequest struct, including the image, the default exposed ports, wait strategies, etc. As a result, the function must initialize the container request with the default values.
3. Define container options for the module leveraging the testcontainers.ContainerCustomizer interface, that has one single method: Customize(req *GenericContainerRequest).

We consider that a best practice for the options is to define a function using the With prefix, and then return a function returning a modified testcontainers.GenericContainerRequest type. For that, the library already provides a testcontainers.CustomizeRequestOption type implementing the ContainerCustomizer interface, and we encourage you to use this type for creating your own customizer functions.

4. At the same time, you might need to create your own container customizers for your module. Make sure they implement the testcontainers.ContainerCustomizer interface. Defining your own customizer functions is useful when you need to transfer a certain state that is not present at the ContainerRequest for the container, and possibly using an intermediate Config struct. Please take a look at MyCustomizer and WithMy in the snippet below.
5. The options will be passed to the RunContainer function as variadic arguments after the Go context, and they will be processed right after defining the initial testcontainers.GenericContainerRequest struct using a for loop.
6. If needed, define public methods to extract information from the running container, using the Container type as receiver. As an example, a connection string to access a database.
7. Document the public API with Go comments.
8. Extend the docs to describe the new API of the module. The code generation tool already creates a parent Module reference section, including a Container options and a Container methods subsections; within each subsection, please define a nested subsection for each option and method, respectively.

The following snippet would represent an example of what the code should look like:

type ModuleContainer struct {
    testcontainers.Container
    cfg *Config
}

// Config type represents an intermediate struct for transferring state from the options to the container
type Config struct {
    data string
}

// RunContainer is the entrypoint to the module
func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*Container, error) {
    cfg := Config{}

    req := testcontainers.ContainerRequest{
        Image: "my-image",
        //...
    }
    genericContainerReq := testcontainers.GenericContainerRequest{
        ContainerRequest: req,
        Started:          true,
    }
    //...
    for _, opt := range opts {
        req = opt.Customize(&genericContainerReq)

        // If you need to transfer some state from the options to the container, you can do it here
        if myCustomizer, ok := opt.(MyCustomizer); ok {
            config.data = customizer.data
        }
    }
    //...
    container, err := testcontainers.GenericContainer(ctx, genericContainerReq)
    //...
    moduleContainer := &Container{Container: container}
    // use config.data here to initialize the state in the running container
    //...
    return moduleContainer, nil
}

// MyCustomizer type represents a container customizer for transferring state from the options to the container
type MyCustomizer struct {
    data string
}

// Customize method implementation
func (c MyCustomizer) Customize(req *testcontainers.GenericContainerRequest) testcontainers.ContainerRequest {
    req.ExposedPorts = append(req.ExposedPorts, "1234/tcp")
    return req.ContainerRequest
}

// WithMy function option to use the customizer
func WithMy(data string) testcontainers.ContainerCustomizer {
    return MyCustomizer{data: data}
}

// WithSomeState function option leveraging the functional option
func WithSomeState(value string) testcontainers.CustomizeRequestOption {
    return func(req *testcontainers.GenericContainerRequest) {
        req.Env["MY_ENV_VAR"] = value
    }
}

// ConnectionString returns the connection to the module container
func (c *Container) ConnectionString(ctx context.Context) (string, error) {...}

ContainerRequest options

In order to simplify the creation of the container for a given module, Testcontainers for Go provides a set of testcontainers.CustomizeRequestOption functions to customize the container request for the module. These options are:

• testcontainers.CustomizeRequest: a function that merges the default testcontainers.GenericContainerRequest with the ones provided by the user. Recommended for completely customizing the container request.
• testcontainers.WithImage: a function that sets the image for the container request.
• testcontainers.WithConfigModifier: a function that sets the config Docker type for the container request. Please see Advanced Settings for more information.
• testcontainers.WithEndpointSettingsModifier: a function that sets the endpoint settings Docker type for the container request. Please see Advanced Settings for more information.
• testcontainers.WithHostConfigModifier: a function that sets the host config Docker type for the container request. Please see Advanced Settings for more information.
• testcontainers.WithWaitStrategy: a function that sets the wait strategy for the container request, adding all the passed wait strategies to the container request, using a testcontainers.MultiStrategy with 60 seconds of deadline. Please see Wait strategies for more information.
• testcontainers.WithWaitStrategyAndDeadline: a function that sets the wait strategy for the container request, adding all the passed wait strategies to the container request, using a testcontainers.MultiStrategy with the passed deadline. Please see Wait strategies for more information.

You could use any of them to customize the container when calling the RunContainer function.

container, err := RunContainer(ctx,
    testcontainers.CustomizeRequest(testcontainers.GenericContainerRequest{
        ContainerRequest: testcontainers.ContainerRequest{
            Image:          "localstack/localstack:2.0.0",
            Env:            map[string]string{"SERVICES": "s3,sqs"},
         },
     }),
 )

Migrating existing examples into modules

It might be the case that we already have an example module for the technology you are interested in, and by definition it does not expose any public API. In this case, you could migrate it to a module.

The steps to convert an existing example, aka ${THE_EXAMPLE}, into a module are the following:

1. Rename the module path at the go.mod file for your example.
2. Move the examples/${THE_EXAMPLE} directory to modules/${THE_EXAMPLE}.
3. Move the ${THE_EXAMPLE} dependabot config from the examples section to the modules one, which is located at the bottom.
4. In the mkdocs.yml file, move the entry for ${THE_EXAMPLE} from examples to modules.
5. Move docs/examples/${THE_EXAMPLE}.md file to docs/modules/${THE_EXAMPLE}, updating the references to the source code paths.
6. Update the Github workflow for ${THE_EXAMPLE}, modifying names and paths.

After following these steps, the technology will be available as a module and you can start adding types and methods as we described above.

Conclusion

In this blog post we shared how to create a Go module for Testcontainers for Go, adding custom types, methods and customizers. Then, we also explained the differences between a Go module or an example module. And finally, we discovered how to use the code generation tool to create the scaffolding for a new module or a new example. Want to know more about modules?

• https://testcontainers.com/modules/
• https://golang.testcontainers.org/modules/