We have a deployed server (with no surprised) which has been up over a year. Although few services are easy to build a test environment locally (and well documented), not all of them are. As our development department is getting larger and new recruiting members are joining (including myself), we couldn’t postponed to document our server system to set up the local test system, and that’s where this “Docker” comes in.
What is Docker?
Docker is the world’s leading software container platform (what-is-docker). A container includes all the required parts to run a software such that it is isolated from one another. Therefore, a containerized software run the same regardless of the system environment.
Although it seems similar to virtual machines (VMs), unlike the VM, it virtualizes the operation system (OS) instead of hardware. In other words, containers can share the OS kernel with other containers, each running as an isolated process in an user space. So the container is more portable and efficient (in terms of space and start time) compare to the VM.
However, it does not mean that the container is superior to the VM (containers-vs-virtual-machines). First of all, it can be weaker in security. In theory, if an user has the superuser privilege within a container, the underlying operating system could be cracked. Thus it is important to treat containers the same way as any other server applications (e.g. run your services as non-root whenever possible). Also, the container might make developing an application easier by creating a lock-box to place the application on top, but it makes the developer harder to understand the lock-box. In other words, it is easy to deploy an app, yet the app might be deployed on a wrong container (e.g. you may choose an nginx image, but you are not sure if it includes the correct version of TCP Load Balancing). Lastly, it is smart to break deployments into smaller functional discrete parts, but it means you have more parts to manage.
Because of above reasons and since Docker supports the VM (Docker itself is running on the VM using xhyve or Hyper-V depending on the base OS - and virtual box for previous versions), it is possible for containers and VMs to coexists, and it is recommended to use containers and VMs depending on the situation. As a rule of a thumb, it is good to use a container to run multiple copies of a single app whereas use a VM to run multiple applications in a single system.
Docker does not introduce a new technology, but it combines existing technologies for the usability. For example, Docker uses Union Files System (UnionFS) to make the layer system. UnionFS groups directories and files in a “branch” such that branches can be stacked on top of each other. These group of branches becomes a single file system such that contents of directories with same path are merged and shown as a single directory (a policy should be existed to define a priority of overwriting a same file). Using this UnionFS, Docker creates a single system image from a layer of (read-only) images. By creating an image from a stack of intermediate images, Docker achieves version control of the image (the image have its name and tag which specifies version numbers) as well as a reusability of images (each intermediate image is itself an image which can be a base image of some containers).
Moreover, Docker accomplishes a space efficiency due to unionFS. When a container is created from an image, Docker creates a (writable) thin layer (which is discarded when the container is destroyed) on top of the underlying stack (of images). Because of this layer system, changes in a container does not affect the base image thus the image can be reused. Therefore, multiple containers can share one base image for a space efficiency.
These images can be stored in the Docker hub for free (for one private and unlimited public images), and many open sourced software images are already provided on the Docker hub and are maintained by official developers.
How to Use Docker?
Docker supports multiple platforms including Mac, Windows, Linux, AWS, and Azure. By installing Docker on a host OS, Docker becomes ready for use. Although it seems like a regular application, it is actually built upon a VM. However it is designed such that user can use it as if it is a native application (whereas it works by sending a command to a remote VM system behind the scene). All operations of Docker are based on a command line interface. All possible command can be found using “–help” option as follows:
A command to run an image as a container is as follows:
docker run [OPTIONS] IMAGE[:TAG|@DIGEST] [COMMAND] [ARG...]
and all the options can be found using “–help” or a reference page (docker-run). For example, to run a container with the ubuntu image,
docker run -it ubuntu
will create a ubuntu (latest version in Docker hub) container and start the shell (if “-it” is not given, the container exited immediately since the container is not run as an interactive mode).
Building a Docker image
From this shell, you can work as if you are remotely connected to the Ubuntu computer, such as installing packages and change settings. After the container is modified to holds all the required files and packages, this container can be committed to create an image.
docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]
Although the image can be created from the modified container, it may not be a good idea to build the image from the container in terms of a documentation (and a portability if the image should be shared without Docker hub system since an image can be gigabytes in size). So it is a good practice to use Dockerfile to build the image. Dockerfile is a file containing series of commands to build an image from a base image.
Here is an example of a Dockerfile (a Django image)
# Start from the Ubuntu 14.04 image FROM ubuntu:14.04 # Image labels LABEL com.example.vendor="Peoplefund" LABEL version="1" LABEL description="My custom docker image" LABEL maintainer="Peoplefund" # To remove debconf build warnings # ARG DEBIAN_FRONTEND=noninteractive # Change locale to fix encoding error on mail-parser install # Install needed default locale RUN echo 'en_US.UTF-8 UTF-8' >> /var/lib/locales/supported.d/local \ && echo 'ko_KR.UTF-8 UTF-8' >> /var/lib/locales/supported.d/local \ && locale-gen # Set default locale for the environment ENV LC_ALL en_US.UTF-8 ENV LANG ko_KR.UTF-8 # Change the timezone RUN mv /etc/localtime /etc/localtime.old \ && ln -s /usr/share/zoneinfo/Asia/Seoul /etc/localtime # Install required packages and clean up the apt cache RUN apt-get update \ && apt-get install --no-install-suggests -y \ sudo \ python3-pip \ libmysqlclient-dev \ libssl-dev \ && rm -rf /var/lib/apt/lists/* \ && apt-get clean # Install requirements RUN mkdir /temp COPY ["requirements/develop.txt", "requirements/production.txt", "/temp/"] RUN pip3 install -r /temp/develop.txt RUN rm -rf /temp # Set environment variable RUN mkdir /src_code ENV SRC_HOME /src_code # Copy a startup script COPY ["docker_inapi_init.sh", "/"] # Add non root user RUN adduser --disabled-password --gecos '' ubuntu # Allows user to sudo RUN echo "ubuntu ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/ubuntu \ && chmod 0440 /etc/sudoers.d/ubuntu # Set a working directory (every code below this will have a root directory as SRC_HOME) WORKDIR $SRC_HOME # Work with a non-root user for security USER ubuntu # Use a shell script as an initial process ENTRYPOINT ["/docker_inapi_init.sh"] # Below command is run if no command is given on a "docker run" else it is over-written CMD ["runserver", "0.0.0.0:8000"]
As shown in the above Dockerfile, settings and installed packages are clearly documented. After the Dockerfile is created, running “docker build” command makes an image from the Dockerfile.
docker build [OPTIONS] PATH | URL | -
Running multiple containers at once
Now we know how to create an image and run a container from an image. But it is a tedious job to build, run, and connect each applications manually, if a server requires multiple applications to run (For example, if a server is composed of multiple SQL applications). So Docker provides the docker-compose command to help this tedious job. After a file docker-compose.yml is created and a command “docker-compose up” is executed, all the setups are automatically done according to the docker-compose.yml file.
Here is an example of a docker-compose file
version: '3' services: # create a nginx container nginx: # create a container from an image image: nginx # forward localhost:8080 to a container's port 80 ports: - "8080:80" # share a host folder with a container volumes: - ./src_code/:/www/src_code/ # a network to connect this service to networks: - nginx-php-network php-fpm: # build an image from a Dockerfile in php folder build: context: ./php # share a host folder with a container volumes: - ./src_code/:/www/src_code/ # a network to connect this service to networks: - nginx-php-network networks: # create a new network for this services nginx-php-network: driver: bridge
When a server is constructed using Docker containers, each container is assigned to an ip which can be different from one case to the other. To resolve this problem, Docker allows to use a service name instead of an ip address. For example, when the above docker-compose file is used, in an nginx setting file, a php-fpm container can be specified using “php-fpm:9000”.
Docker greatly reduces a time to set up an exactly identical local test environment. Therefore, developers, if wanted, can easily reset one’s test system. Not only that, it automatically documented test environment system, thus developers can easily check the components of the test environment. Moreover, this may helps for developers to easily deploy and manage their service on a real service. Although we are using Docker as a local test environment for now, I hope in the near future that the actual Peoplefund service is deployed using Docker.