Using Docker

September 17th, 2015

---

Docker is a container service that encapsulates microservices automates deployment of applications. If unfamiliar with containers, think of them as virtual machines (VMs) that only provide operating system level virtualization (on top of kernel), but unlike VMs, this does not extend to hardware level. At first look, it is a light-weight version of virtual machines. This greatly supports the microservices pattern, which encourages the splitting of a large application into services that communicate through TCP, APIs or a message queue (MQ). Docker also provides a Makefile-like deployment script (called Dockerfile), and its main development is towards clustering containers and optimizing their communication between each other.

We looked towards Docker when we thought of building our own infrastructure on AWS and moved away from Heroku. To dockerize our application, we had to use Docker Compose, previously fig, a tool that as of this writing is not production-ready.

The ideal setup for Docker is to have a single command to build your entire project, and another to start the server. There should be abilities to manipulate the container to run commands, like seeding the database (which shouldn’t be ran every time you deploy!). Additionally, the commands should be the same for deploying remotely.

We will setup the obvious infrastructure by putting the database in a container and the Rails application in another. We will link them so that they can communicate to each other (and taking advantage that Postgres can be accessed over a TCP port). We will see that the host will not be localhost. The linking is where Docker Compose comes in, which implements an almost recursive building process.

To start off, we write a Dockerfile that takes a base image to start from. There are various arguments online suggesting that Ubuntu-based images are not meant to use with Docker. Because many applications are still in the process of adapting to Docker, we encourage you to find your own fit and balance between familiarity and configuration. For reference, see baseimage-docker, its rationale, controversy and alternative.

We will be using the basic Ruby base image, which is based on Ubuntu (with Debian repository). We also want Docker to cache any updates and gem installations, so we put them all in the build process (not the run process later).

# Dockerfile
# Base Image
FROM ruby:2.1.5
MAINTAINER Conjure Team <[email protected]>

# Update Environment
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev

# Build
RUN mkdir /Conjure
WORKDIR /Conjure

# Install Gem Dependencies
ADD Gemfile /Conjure/Gemfile
RUN bundle install

# Add Files
ADD . /Conjure

This file is quite straight forward: we use the Ubuntu-based image, and update it; we make our application directory from root, and make it our working directory. Making the folder our working directory means that any command which we inject inside the container will run here. We then add the Gemfile and install all the bundles. Docker caches this into a new container, and then we add the rest of the application files to the container. Note that Docker cannot cache the ADD command, so we try to leave it as late as possible.

A bug that I had before is that my local Gemfile.lock is not synchronized with the most current Gemfile dependencies. As a result, when a different Gemfile.lock is added to the container, it looks for gem versions that were not previously installed (with only the Gemfile). This causes problems and to fix them, simply ignore the lock:

# .dockerignore
Gemfile.lock

Next, we want to configure our compose file. Here, I used a common.yml file as a base, which we will reuse later when creating a different compose file for production.

# common.yml
postgres:
  image: postgres
web: 
  build: .
  command: bundle exec rails server -p 3000 -b '0.0.0.0' 
  ports: 
    - "3000:3000"

# docker-compose.yml
db:
  extends:
    file: common.yml
    service: postgres
web:
  environment:
    RAILS_ENV: development
    DATABASE_HOST: db
  extends:
    file: common.yml
    service: web
  volumes:
    - .:/Conjure
  links:
    - db

Already, we can start building the application:

docker-compose build                     # build

So what exactly does docker-compose build do? Note that we defined a build command in common.yml, which simply points to the current directory. Every other command is configured for running the container. Thus docker-compose build simply translates to docker build ., which runs the commands in the Dockerfile found in the current directory. You will see that Docker first skips the building of the db because it is simply an image (it will be built when ran). Then, Docker follows every instruction in the Dockerfile.

Before running this server, we need to change up the database configuration. Postgres’s host is no longer localhost but db, so we need to configure it that way. To make this application still runnable with traditional methods, we will put it as an environment variable (we set its value inside docker-compose.yml)

# database.yml
development: &default
  adapter: postgresql
  encoding: unicode
  pool: 5
  host: <%= ENV['DATABASE_HOST'] || "localhost" %>
  port: 5432

  database: conjure_development
  username: postgres
  password:

test:
  <<: *default
  database: conjure_test

production:
  <<: *default
  database: conjure_production
  username: postgres
  password: <%= ENV['DATABASE_PASSWORD'] %>

Now we can run (and stop) the application.

docker-compose up -d                     # start (daemon)
docker-compose logs                      # view output
docker-compose stop                      # stop daemons

Rails might throw an exception and kill itself when you attempt this, and it might be because the database is not created. Then simply create it by running commands inside the container:

docker-compose run INSTANCE COMMAND      # running commands
docker-compose run web rake db:create    # e.g. create database
docker-compose run web rake db:migrate   # e.g. migrate database
docker-compose run web rake db:seed      # e.g. seeding database

If the problem is something like “Gem not found”, then remove Gemfile.lock from the local directory. Don’t worry, after rebuilding the container, a new lock will be created.

We might want to connect to the rails console, which is as simple as running a command:

docker-compose run web rails console     # e.g. getting console

---

Docker Machine allows us to deploy to a remote server easily. Here I will simply show the commands.

docker-machine create \
--driver amazonec2 \
--amazonec2-access-key your-aws-access-key \
--amazonec2-secret-key your-aws-secret-key \
--amazonec2-vpc-id your-aws-vpc-id \
--amazonec2-subnet-id your-aws-subnet-id \
--amazonec2-region us-east-1 \
--amazonec2-zone a \
ec2box                                   # creating AWS instance with docker
docker-machine ls                        # listing all instances
eval $(docker-machine env ec2box);       # setting attachment
docker-compose build                     # directly manipulating
                                         # ... and other docker-compose commands