Dynamic configuration files with Docker containers

Updated:

Note : This page may contain outdated information and/or broken links; some of the formatting may be mangled due to the many different code-bases this site has been through in over 20 years; my opinions may have changed etc. etc.

I’ve been using Docker containers on Linux systems for a while now, and have recently developed a neat solution to what I imagine is a fairly common problem.

I had a number of Docker containers that I wanted to launch from a common image, but with a slightly different configuration depending on the environment in which I was launching them. For example, a web application container might connect to a different database in a staging environment, or a MongoDB replica set name might be different. This meant my options basically looked like:

  • Maintain multiple containers / Dockerfiles.
  • Maintain the configuration in separate data volumes and use --volumes-from to pull the relevant container in.
  • Bundle the configuration files into one container, and manually specify the CMD or ENTRYPOINT values to pick this up.

None of those really appealed due to needless duplication, or the complexity of an approach that would necessitate really long docker run commands. So I knocked up a quick & simple Ruby script that I could use across all my containers, which does the following :

  • Generates configuration files from ERB templates
  • Uses values provided in YAML files, one per environment
  • Copies the generated templates to the correct location and specifies permissions
  • Executes a replacement process once it’s finished (e.g. mongod, nginx, supervisord, etc.)
  • Provides a plugin architecture to grab values from a variety of sources - environment variables, Consul clusters and so on.

This way I can keep all my configuration together in the container (or provide it at run-time), and just tell Docker which environment to use when I start it.

Note from June 2023 (nearly 10 years on from this post!): This project was called Tiller and grew significantly in scale and popularity since announced. I now consider it archived but it was a lot of fun to work on, and may still be useful. Just bear in mind there are now often much better, native solutions for the problems that Tiller was intended to solve. Back then, Docker usage and conventions weren’t as wide-spread and as standardised as they are now. A lot of software wasn’t availible in official, ready-to go images and containerising non-cloud native software was a frequent source of frustration. Nowadays, most containers have an entrypoint that performs much of the core Tiller feature-set. We’ve got Helm charts, compose files, Kubernetes configmaps/secrets and so on - but it may still solve your particular use-case. I hope it’s still of some use, and to all of the old community - thanks again for everything!

The “TL;DR” pitch for Tiller is that it’s a tool that normally runs as the CMD or EXEC inside a Docker container; it generates dynamic configuration files for your application/services and then runs the application as a replacement process. It has a pluggable architecture and comes bundled with many plugins, so you can get values for your configuration files from environment variables, a consul cluster, JSON files, and many other things. It was created to provide a simple, generic way of building flexible “parameterized containers” that can grab their configuration at run-time - think of a service that needs a different DB connection string in a production environment, or maybe you want to pass in secrets such as passwords at run-time instead of baking them in your container.

And, as the end result is a set of plain-text files, your application doesn’t need to know how to directly talk to any of these services, it still only needs to know how to load a configuration file.

For more information, see the documentation at the Github page, join the Gitter chatroom or check this blog for more updates & examples. A short selection of recent blog posts is provided below as a helpful “jumping off” point :

Comments