Managing stacks with Docker Swarm
Recently I’m interested in streamlining the deployment of applications in the most automated way possible and is not possible to do this without looking at Docker (let’s ignore Nix for now), in this post we will get to know the basics of docker swarm.
What exactly is Docker Swarm?
Perhaps it should be clear from the name, but I’m not taking my chances, Docker Swarm is a cluster of nodes in which docker is installed and each node configured in such a way that can receive tasks as sent by a manager node in the cluster, so in essence a swarm is a cluster composed of nodes, let’s review the roles available for nodes:
Is a node that has permissions to manage the swarm (cluster), you can have as many managers in your cluster as you want, or in fact your cluster could be composed entirely of manager nodes. To add a new manager node in an existent swarm we need to provide a token, such token can be generated by using the leader of the cluster (which is itself a manager node) or any other manager node:
$ docker swarm join-token manager
You’ll be presented with instructions in what you need to type in that another node to add it to the swarm as a manager. In case that the node you want to turn into a manager is already part of the cluster, you can use the
promote command specifying that node to do just that:
$ docker node promote <node name or id>
Let’s see what are worker nodes.
A worker node can’t manage the swarm, other than that is the same as a manager, so a worker node can’t list other nodes in the stack or see the services or tasks running in the whole cluster, but it can see its own containers with the usual docker commands.
A thing to be aware of is that a worker is able to just leave the swarm at any time with:
$ docker swarm leave
And of course any manager will notice the node has left the swarm but the worker node will keep appearing in the list of nodes in the swarm (
docker node ls from a manager node), this means that in order for a node to leave the swarm is not enough for the worker to just “leave the swarm” but a manager node must remove the node that just left:
$ docker node rm <worker node name/id>
Now on to leaders.
This is just a manager node that appears as the leader when we see the list of nodes that belong to the swarm, it appears as the leader because is the one that initialized the swarm in the first place with the command:
$ docker swarm init
Other than that, this node is a manager node just as any other manager in the swarm.
Yes, a swarm is an abstraction of which you can only have one instance of it, other abstractions like stacks, services, etc. you can have much of these as you please, but you can only have and manage one swarm at a time, that means there’s no such thing as
docker swarm ls to see a list of swarms, that is not possible.
What about services?
In a swarm setup, any manager node can create services, services are just like containers but they work on the whole cluster instead of only just one node, for example:
$ docker service create --name nginx -p 80:80 nginx
Will create an nginx container with said name (the name will also contain some other identifiers that the swarm appends) and expose port 80, it’s just like running a container, but the real container may be created in another node in the swarm and not just on the one we issued the command, it’s more interesting when you see all you can do with services (use
docker service create --help for that), for example we can now create replicas in the cluster:
$ docker service create --replicas 3 --name nginx -p 80:80 nginx
So now we create three replicas of nginx in the cluster, and what is interesting is that the whole swarm acts as a single node “network wise”, what I mean is that I just need the IP of a node in the cluster to put it in my browser and see the splash screen of nginx, doesn’t matter the IP is from a node that doesn’t have the nginx container running on it, a thing called the routing mesh is in charge of making that work. We can see where are the containers running with the
ps command in services:
$ docker service ps <service name or id>
And we can see all of our services with the
docker service ls command, and you’ll see these are just like containers but in relation to the whole cluster. We can also remove them, update them, inspect them, etc, the usual.
This is the last thing I want to write about in this post. Stacks manage services in a whole swarm as docker-compose manage containers in a single node, so the usual thing is that we need to rely on several images deployed through the cluster and we don’t want to manage services one by one but build a stack from a declarative definition in a file. What’s great about this is that the declarations made for the
docker stack command are already compatible with those in a
docker-compose.yml file, the only thing we need to do is use a version
3 or greater in the compose file:
# docker-compose.yml file version: "3" services: queue: image: redis:alpine ports: - 6379 networks: - public deploy: replicas: 2 restart_policy: condition: on-failure database: image: postgres:9.4 environment: - POSTGRES_PASSWORD_FILE=/run/secrets/dbpw volumes: - db-data:/var/lib/postgresql/data networks: - private deploy: placement: constraints: [node.role == manager] secrets: - dbpw app: image: example/my_app ports: - 3000:80 networks: - public depends_on: - queue deploy: replicas: 2 restart_policy: condition: on-failure admin-area: image: example/admin_panel ports: - 5001:80 environment: - ADMIN_AREA_PASSWORD=/run/secrets/adminpw networks: - private depends_on: - database deploy: replicas: 1 restart_policy: condition: on-failure secrets: adminpw worker: image: example/job_worker networks: - public - private depends_on: - database - queue deploy: mode: replicated replicas: 3 placement: constraints: [node.role == manager] visualizer: image: dockersamples/visualizer ports: - 8080:8080 stop_grace_period: 1m30s volumes: - "/var/run/docker.sock:/var/run/docker.sock" deploy: placement: constraints: [node.role == manager] networks: public: private: volumes: db-data: secrets: dbpw: external: true adminpw: external: true
Now we just have to:
docker stack deploy
And all of our services will be created, containers associated with those services will spread through the cluster to satisfy what we have in our
docker-compose.yml file, if a node were to fail the swarm will adjust itself to make up for the containers that were down due to being hosted on that failed node, so our
docker-compose.yml file becomes our single source of truth of the state of deployment strategy.
Stacks are an abstraction of which we can manage multiple instances of it, we can list all stacks:
docker stack ls
And see all services running on the stack:
docker stack services <stack id>
And see all containers running on the stack
docker stack ps <stack id>
And of course we can remove them with the
Ok, this is a very quick and simple intro to Docker Swarm, it’s not complicated at all, managing a swarm of containers is getting easier with these tools. One should see if Kubernetes might be a better fit, for what I’ve heard you could handle hundreds of containers with it 🤷♂️.