This is the first in a series of tutorials on setting up Docker Swarm using Linux hosts. Docker Swarm is a fantastic piece of technology for allowing your containerised workloads to be highly available/scalable.
As I intend to migrate at least some of this setup into the cloud, this won’t represent a production ready example of Docker Swarm at first but is mainly intended to cover the basics around networking/storage and monitoring.
Installing Docker and configuring Docker Swarm
Assuming that you have installed your Linux distro of choice already on your nodes (if not – go do that!), you can now follow the instructions at https://docs.docker.com/engine/install/ubuntu/ on each of your nodes to install Docker Engine.
Docker Swarm needs the following network access between each of your nodes, this was open by default on my nodes but if you’re running a particularly locked down network you’ll need to make sure these ports are open between each node:
– TCP port 2377 for cluster management communications.
– TCP and UDP port 7946 for communication among nodes.
– UDP port 4789 for overlay network traffic.
Once this is done you can use the instructions at https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/ to configure Docker Swarm. I’ll give a brief run through below.
Docker Swarm can be configured with two kinds of nodes:
– Manager Nodes, which schedule tasks across Worker Nodes and maintain the Swarm.
– Worker Nodes, which complete the tasks allocated to them by Manager Nodes.
In my example I will be setting up all three of my nodes as both Manager and Workers.
On your first manager node, run the below command to initiate the swarm using the IP address of the machine.
docker swarm init --advertise-addr <MANAGER-IP>You should receive an output such as below.
docker swarm init --advertise-addr 192.168.99.100 Swarm initialized: current node (dxn1zf6l61qsb1josjja83ngz) is now a manager.
Run the command ‘docker swarm join-token manager’ on your first node to give you the join command that you will need to run to join the other nodes as managers. You should get something like the below.
docker swarm join \
--token SWMTKN-1-49nj1cmql0jkz5s954yi3oex3nedyz0fb0xx14ie39trti4wxv-8vxv8rssmk743ojnwacrr2e7c \
192.168.99.100:2377
Now, you can run this command on the following nodes which should look like the below if you made sure the network prerequisites were met.
docker swarm join \ --token SWMTKN-1-49nj1cmql0jkz5s954yi3oex3nedyz0fb0xx14ie39trti4wxv-8vxv8rssmk743ojnwacrr2e7c \ 192.168.99.100:2377 This node joined a swarm as a manager.
Once all the nodes are joined, you can run the command ‘docker node ls’ on any of the Manager Nodes which on my setup gives me the below output. The * indicates the node you are currently connected to.
You now officially have a Docker Swarm set up. At this point, you can run the below code from the docker guide to create your first simple service. The benefit of running services on Docker Swarm, rather than standalone containers, is that you can scale these very easily. They will also maintain the level of ‘replicas’ by moving services around nodes, if these nodes become unavailable due to network or other issues.
docker service create --replicas 1 --name helloworld alpine ping docker.com
In this example, the –replicas flag will create and maintain a single running version of the service, if you changed the number to 3 you would get 3 concurrently running services and the Docker Swarm would automatically balance any incoming traffic between them. The –name flag is used to specify the name of the service and the ‘alpine’ specifies an alpine linux docker container that executes the command ‘ping docker.com’.
You can then use the command ‘docker service ls’ to show running services which will give you the below output.
ID NAME SCALE IMAGE COMMAND 9uk4639qpg7n helloworld 1/1 alpine ping docker.com
Now, I’ll quickly touch on storage.
Docker containers, by default, have ephemeral storage. This is lost when the container or service is removed. You can create persistent storage by mounting ‘volumes’ to your containers and services. A good way to think of volumes are like little individual blocks of storage that can be used and shared by containers.
In this example, I will show you how to mount storage from a NAS device to allow data to be used by services across all of your nodes and keep consistency. This isn’t a perfect solution (I’ll dive into why later), but is more than suitable for the purposes of this demo.
Setting up storage
In my setup, I have mounted some folders that are stored on my NAS to each of the Linux hosts using cifs. Due to all of my devices being joined to the same Active Directory domain, permissions were a breeze to set up and this scaled well across multiple hosts. When I first started using this setup, it came with one major drawback, namely that any container that utilises SQLite will almost immediately lock the database and prevent the container/service from functioning. I’ve since managed to find a solution to this by adding the “nobrl” option to the mount which I will detail below.
As I said, I’m planning to move most, if not all, of this setup to the cloud soon as services such as Azure Container Instances manage storage and high availability in a much more streamlined fashion. I’ll still show you the steps I used as they may be of some use.
To begin with, you’ll need to set up some storage on a NAS that supports cifs and has the appropriate access given. In this case, I have a service account setup called ‘Docker’ which has permissions to run the Docker service on my Linux hosts as well as to the shared storage on my NAS.
On each of the Linux hosts, I then set up some empty folders that will then be mapped to my NAS storage. In this case, I used the path /docker to store my configuration files and volumes for my containers/services.
To permantly mount these folders on your linux hosts, you will need to edit your fstab file and add a new line for each mount. This can be done by vi if you are using command line or the editor if you are using the gui with the respective commands.
sudo vi /etc/fstab sudo gedit /etc/fstab
The syntax needed will take a little getting used to if you are new to Linux but I’ll give you the example below and then we’ll break it down.
//192.168.99.90/docker /docker cifs credentials=/etc/win-credentials,uid=1001,iocharset=utf8,file_mode=0777,dir_mode=0777,nobrl,mfsymlinks,noperm 0 0
– //192.168.99.90/docker = the path to the folder that you want to map from the NAS
– /docker = the path of the folder you want to mount this data to on the local host
– cifs = the protocol used for storage, in this case cifs
– credentials=/etc/win-credentials = the path to a file containing the logon information to an account that has sufficient permissions to the folder on the NAS. Formatted as below and permissions on this file set to only allow access to the user mounting the files
username=$username password=$username domain=$domain
– uid=1001 = the uid of the user that will be used to mount the files/run docker, this can be gotten by running id -u username where ‘username’ is the username of the user you want the uid of
– iocharset=utf8 = the encoding used for the mount, utf8 ensures the maximum chance of being able to display the mounted data in a readable format
– file_mode=0777 and dir_mode=0777 = will overwrite the default is the server does not support the cifs unix extensions. 0777 will set it to R,W,X for all users but this can be changed to tighten this down as needed
– nobrl = Do not send byte range lock requests to the server. This is necessary for certain applications that break with cifs style mandatory byte range locks such as the SQLite issue above.
– mfsymlinks = enable support for Minshall+French symlinks, needed this to allow some docker containers to use this storage.
– noperm = this is needed when exposing the mounted files to other users on the local system
At this point, you can then save the /etc/fstab file and either run the command ‘mount’ or reboot the system at which point the mounts you added in /etc/fstab will be mounted at startup.
You can now reference your new storage when creating services/containers and use this as persistent storage. In the next entry in this series, we will take a dive into using Docker Compose to deploy multiple different services at the same time. We’ll also look at mounting volumes and other configurations you can set using Compose.