Skip to main content

When we create an application, no matter the programming language we use, we’d love to have a great experience while building the software project. And this experience includes having a great build tool to perform any task related to the development lifecycle of the project: compiling it, building the release artifacts, and running the tests that must exist in the project.

Often times our build tool doesn’t support all our local development tasks, such as starting the runtime dependencies for our application. We are then forced to manage them manually with a Makefile, a shell script or an external Docker Compose file. This might involve calling them in a separated terminal, or even maintaining code for that purpose. Thankfully, there’s a better way.

In this post, I’m going to show you how to use Testcontainers for Go to start and stop the runtime dependencies of your Go application while building it and running the tests in a very simple and consistent manner. I’m going to create a super simple app using the Fiber web framework to build a simple Go application, which will connect to a PostgreSQL database to store its users. Finally, I’m going to leverage Go’s built-in capabilities and use Testcontainers for Go to start the dependencies of the application.

You can find the source code in this repository: https://github.com/testcontainers/testcontainers-go-fiber 

If you are new to Testcontainers for Go, then watch this video to get started with Testcontainers for Go.

NOTE: I’m not going to show the code to interact with the users database, as the purpose of this post is to show how to start the dependencies of the application, not how to interact with them.

Introducing Fiber

From their Fiber website:

Fiber is a Go web framework built on top of Fasthttp, the fastest HTTP engine for Go. It’s designed to ease things up for fast development with zero memory allocation and performance in mind.

Why Fiber? There are various frameworks for working with HTTP in Go, such as gin, or gobuffalo, and many Gophers directly stay in the net/http package of the Go’s standard library. In the end, it doesn’t matter which library of framework we choose, as it’s independent of what we are going to demonstrate here.

Let’s create now the default Fiber application:

package main

import (
   "log"
   "os"

   "github.com/gofiber/fiber/v2"
)

func main() {
   app := fiber.New()

   app.Get("/", func(c *fiber.Ctx) error {
       return c.SendString("Hello, World!")
   })

   log.Fatal(app.Listen(":8000"))
}

As we said, our application will connect to a postgres database to store its users. In order to share state across the application, we are going to create a new type representing the App.  This App type will include information about the Fiber application, and the connection string for the users database.

// MyApp is the main application, including the fiber app and the postgres container
type MyApp struct {
   // The name of the app
   Name string
   // The version of the app
   Version string
   // The fiber app
   FiberApp *fiber.App
   // The database connection string for the users database. The application will need it to connect to the database,
   // reading it from the USERS_CONNECTION environment variable in production, or from the container in development.
   UsersConnection string
}

var App *MyApp = &MyApp{
   Name:            "my-app",
   Version:         "0.0.1",
   // in production, the URL will come from the environment
   UsersConnection: os.Getenv("USERS_CONNECTION"),
}

func main() {
   app := fiber.New()

   app.Get("/", func(c *fiber.Ctx) error {
      return c.SendString("Hello, World!")
   })

   // register the fiber app
   App.FiberApp = app

   log.Fatal(app.Listen(":8000"))
}

For demonstration purposes, we are going to use the main package to define the access to the users in the Postgres database. In the real-world application, this code wouldn’t be in the main package.

Running the application for local development would be this:

Testcontainers for Go

Testcontainers for Go is a Go library that allows us to start and stop Docker containers from our Go tests. It provides us with a way to define our own containers, so we can start and configure any container we want. It also provides us with a set of predefined containers in the form of Go modules that we can use to start those dependencies of our application.

Therefore, with Testcontainers we’ll be able to interact with our dependencies in an abstract manner, as we could be interacting with databases, message brokers, or any other kind dependency in a Docker container.

Starting the dependencies for development mode

Now that we have a library for it, we need to start the dependencies of our application. Remember that we are talking about the local experience of building the application, so we would like to start the dependencies only under certain build conditions, not on the production environment.

Go build tags

Go provides us with a way to define build tags that we can use to define build conditions. We can define a build tag in the form of a comment at the top of our Go files. For example, we can define a build tag called dev like this:

// +build dev
// go:build dev

Adding this build tag to a file will mean that the file will only be compiled when the dev build tag is passed to the go build command, not landing into the release artifact. The power of the go toolchain is that this build tag applies to any command that uses the go toolchain, such as go run. Therefore, we can still use this build tag when running our application with go run -tags dev ..

Go init functions

The init functions in Go are special functions that are executed before the main function. We can define an init function in a Go file like this:

func init() {
   // Do something
}

They are not executed in a deterministic order, so please consider this when defining init functions.

For our example, in which we want to improve the local development experience in our Go application, we are going to use an init function in a dev_dependencies.go file protected by a dev build tag, and from there, start the dependencies of our application, in our case the PostgreSQL database for the users.

We will use Testcontainers for Go to start this Postgres database. Let’s combine all this information in the dev_dependencies.go file:

//go:build dev
// +build dev

package main

import (
   "context"
   "log"
   "path/filepath"
   "time"

   "github.com/jackc/pgx/v5"
   "github.com/testcontainers/testcontainers-go"
   "github.com/testcontainers/testcontainers-go/modules/postgres"
   "github.com/testcontainers/testcontainers-go/wait"
)

func init() {
   ctx := context.Background()

   c, err := postgres.RunContainer(ctx,
       testcontainers.WithImage("postgres:15.3-alpine"),
       postgres.WithInitScripts(filepath.Join(".", "testdata", "dev-db.sql")),
       postgres.WithDatabase("users-db"),
       postgres.WithUsername("postgres"),
       postgres.WithPassword("postgres"),
       testcontainers.WithWaitStrategy(
           wait.ForLog("database system is ready to accept connections").
               WithOccurrence(2).WithStartupTimeout(5*time.Second)),
   )
   if err != nil {
       panic(err)
   }

   connStr, err := c.ConnectionString(ctx, "sslmode=disable")
   if err != nil {
       panic(err)
   }

   // check the connection to the database
   conn, err := pgx.Connect(ctx, connStr)
   if err != nil {
       panic(err)
   }
   defer conn.Close(ctx)

   App.UsersConnection = connStr
   log.Println("Users database started successfully")
}

The c container is defined and started using Testcontainers for Go. We are using:

  • the WithInitScripts option to copy and run a SQL script that creates the database and the tables. This script is located in the testdata folder.
  • the WithWaitStrategy option to wait for the database to be ready to accept connections, checking database logs.
  • the WithDatabase, WithUsername and WithPassword options to configure the database.
  • the ConnectionString method to get the connection string to the database directly from the started container.

The App variable will be of the type we defined earlier in this blog post, representing the application. This type included information about the Fiber application and the connection string for the users database. Therefore, after the container is started, we are filling the connection string to the database directly from the container we just started.

So far so good, we have leveraged the built-in capabilities in Go to execute the init functions defined in the dev_dependencies.go file only when the -tags dev flag is added to the go run command.

With this approach, running the application and its dependencies takes a single command!

go run -tags dev .

We will see that the Postgres database is started and the tables are created. We can also see that the App variable is filled with the information about the Fiber application, and the connection string for the users database.

Stopping the dependencies for development mode

Now that the dependencies are started, if and only if the build tags are passed to the go run command, we need to stop them when the application is stopped. We are going to reuse what we did with the build tags to register a graceful shutdown to stop the dependencies of the application before stopping the application itself, only when the dev build tag is passed to the go run command.

Our Fiber app stays untouched, and we will need to only update the dev_dependencies.go file:

//go:build dev
// +build dev

package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"os/signal"
	"path/filepath"
	"syscall"
	"time"

	"github.com/jackc/pgx/v5"
	"github.com/testcontainers/testcontainers-go"
	"github.com/testcontainers/testcontainers-go/modules/postgres"
	"github.com/testcontainers/testcontainers-go/wait"
)

func init() {
	ctx := context.Background()

	c, err := postgres.RunContainer(ctx,
		testcontainers.WithImage("postgres:15.3-alpine"),
		postgres.WithInitScripts(filepath.Join(".", "testdata", "dev-db.sql")),
		postgres.WithDatabase("users-db"),
		postgres.WithUsername("postgres"),
		postgres.WithPassword("postgres"),
		testcontainers.WithWaitStrategy(
			wait.ForLog("database system is ready to accept connections").
				WithOccurrence(2).WithStartupTimeout(5*time.Second)),
	)
	if err != nil {
		panic(err)
	}

	connStr, err := c.ConnectionString(ctx, "sslmode=disable")
	if err != nil {
		panic(err)
	}

	// check the connection to the database
	conn, err := pgx.Connect(ctx, connStr)
	if err != nil {
		panic(err)
	}
	defer conn.Close(ctx)

	App.UsersConnection = connStr
	log.Println("Users database started successfully")

	// register a graceful shutdown to stop the dependencies when the application is stopped
	// only in development mode
	var gracefulStop = make(chan os.Signal)
	signal.Notify(gracefulStop, syscall.SIGTERM)
	signal.Notify(gracefulStop, syscall.SIGINT)
	go func() {
		sig := <-gracefulStop
		fmt.Printf("caught sig: %+v\n", sig)
		err := shutdownDependencies()
		if err != nil {
			os.Exit(1)
		}
		os.Exit(0)
	}()
}

// helper function to stop the dependencies
func shutdownDependencies(containers ...testcontainers.Container) error {
	ctx := context.Background()
	for _, c := range containers {
		err := c.Terminate(ctx)
		if err != nil {
			log.Println("Error terminating the backend dependency:", err)
			return err
		}
	}

	return nil
}

In this code, at the bottom of the init function, right after setting the database connection string, we are starting a goroutine to handle the graceful shutdown, listening for the defining the SIGTERM and SIGINT signals. When a signal is put into the gracefulStop channel the shutdownDependencies helper function will be called to stop the dependencies of the application. This helper function will internally call the Testcontainers for Go’s Terminate method of the database container, resulting in the container being stopped on signals.

What’s especially great about this approach is how dynamic the created environment is. Testcontainers takes extra effort to allow parallelisation and binds containers on high level available ports. This means the dev mode won’t collide with running the tests, or you can have multiple instances of your application running without any problems!

Hey, what will happen in production?

Because our App is initialising the connection to the database from the environment:

var App *MyApp = &MyApp{
   Name:            "my-app",
   Version:         "0.0.1",
   DevDependencies: []DevDependency{},
   // in production, the URL will come from the environment
   UsersConnection: os.Getenv("USERS_CONNECTION"),
}

We do not have to worry about that value being overridden by our custom code for the local development: the UsersConnection won’t be set because everything that we showed here is protected by the dev build tag.

NOTE: Are you using Gin or net/http directly? You could directly benefit from everything that we explained here: init functions and build tags to start and graceful shutdown the runtime dependencies.

Conclusion

In this post, we have seen how to use Testcontainers for Go to start and stop the dependencies of our application while building it and running the tests, simply leveraging built-in capabilities of the Go language and the go toolchain.

The result is that we can start the dependencies of our application while building it and running the application, and we can stop them when the application is stopped. This means that our local development experience gets improved, as we don’t need to start the dependencies in a Makefile, shell script or an external Docker Compose file. And the most important thing, it only happens for development mode, passing the -tags dev flag to the go run command.

You can find the source code in this repository: https://github.com/testcontainers/testcontainers-go-fiber 

Manuel de la Peña

Manu is a software engineer at AtomicJar, focusing on Open Source development of “Testcontainers for Go”. With a diverse background in public administration, consulting, and Open Source product companies, he’s gained experience as a support engineer, trainer, and Core Engineer at Liferay. Manu also held roles such as QA Tech lead at Liferay Cloud and focused on Engineering Productivity at Elastic within the Observability team. Alongside his professional work, he organized GDG Toledo, a software community outside of Madrid, and has spoken at national and international events. Manu holds a Bachelor's degree in Computer Science and a Master's degree in Research in Software Engineering and Computer Systems, both from UNED. In his spare time, he loves teaching his sons to play role playing games.