Managing stacks with Docker Swarm

Recently I’m interested in streamlining the deployment of applications in the most automated way possible and is not possible to do this without looking at Docker (let’s ignore Nix for now), in this post we will get to know the basics of docker swarm.

What exactly is Docker Swarm?

Perhaps it should be clear from the name, but I’m not taking my chances, Docker Swarm is a cluster of nodes in which docker is installed and each node configured in such a way that can receive tasks as sent by a manager node in the cluster, so in essence a swarm is a cluster composed of nodes, let’s review the roles available for nodes:

Manager node

Is a node that has permissions to manage the swarm (cluster), you can have as many managers in your cluster as you want, or in fact your cluster could be composed entirely of manager nodes. To add a new manager node in an existent swarm we need to provide a token, such token can be generated by using the leader of the cluster (which is itself a manager node) or any other manager node:

$ docker swarm join-token manager

You’ll be presented with instructions in what you need to type in that another node to add it to the swarm as a manager. In case that the node you want to turn into a manager is already part of the cluster, you can use the promote command specifying that node to do just that:

$ docker node promote <node name or id>

Let’s see what are worker nodes.

Worker node

A worker node can’t manage the swarm, other than that is the same as a manager, so a worker node can’t list other nodes in the stack or see the services or tasks running in the whole cluster, but it can see its own containers with the usual docker commands.

A thing to be aware of is that a worker is able to just leave the swarm at any time with:

$ docker swarm leave

And of course any manager will notice the node has left the swarm but the worker node will keep appearing in the list of nodes in the swarm (docker node ls from a manager node), this means that in order for a node to leave the swarm is not enough for the worker to just “leave the swarm” but a manager node must remove the node that just left:

$ docker node rm <worker node name/id>

Now on to leaders.

Leader

This is just a manager node that appears as the leader when we see the list of nodes that belong to the swarm, it appears as the leader because is the one that initialized the swarm in the first place with the command:

$ docker swarm init

Other than that, this node is a manager node just as any other manager in the swarm.

Anything else?

Yes, a swarm is an abstraction of which you can only have one instance of it, other abstractions like stacks, services, etc. you can have much of these as you please, but you can only have and manage one swarm at a time, that means there’s no such thing as docker swarm ls to see a list of swarms, that is not possible.

What about services?

In a swarm setup, any manager node can create services, services are just like containers but they work on the whole cluster instead of only just one node, for example:

$ docker service create --name nginx -p 80:80 nginx

Will create an nginx container with said name (the name will also contain some other identifiers that the swarm appends) and expose port 80, it’s just like running a container, but the real container may be created in another node in the swarm and not just on the one we issued the command, it’s more interesting when you see all you can do with services (use docker service create --help for that), for example we can now create replicas in the cluster:

$ docker service create --replicas 3 --name nginx -p 80:80 nginx

So now we create three replicas of nginx in the cluster, and what is interesting is that the whole swarm acts as a single node “network wise”, what I mean is that I just need the IP of a node in the cluster to put it in my browser and see the splash screen of nginx, doesn’t matter the IP is from a node that doesn’t have the nginx container running on it, a thing called the routing mesh is in charge of making that work. We can see where are the containers running with the ps command in services:

$ docker service ps <service name or id>

And we can see all of our services with the docker service ls command, and you’ll see these are just like containers but in relation to the whole cluster. We can also remove them, update them, inspect them, etc, the usual.

Stacks

This is the last thing I want to write about in this post. Stacks manage services in a whole swarm as docker-compose manage containers in a single node, so the usual thing is that we need to rely on several images deployed through the cluster and we don’t want to manage services one by one but build a stack from a declarative definition in a file. What’s great about this is that the declarations made for the docker stack command are already compatible with those in a docker-compose.yml file, the only thing we need to do is use a version 3 or greater in the compose file:

# docker-compose.yml file
version: "3"

services:
  queue:
    image: redis:alpine
    ports:
      - 6379
    networks:
      - public
    deploy:
      replicas: 2
      restart_policy:
        condition: on-failure

  database:
    image: postgres:9.4
    environment:
      - POSTGRES_PASSWORD_FILE=/run/secrets/dbpw
    volumes:
      - db-data:/var/lib/postgresql/data
    networks:
      - private
    deploy:
      placement:
        constraints: [node.role == manager]
    secrets:
      - dbpw

  app:
    image: example/my_app
    ports:
      - 3000:80
    networks:
      - public
    depends_on:
      - queue
    deploy:
      replicas: 2
      restart_policy:
        condition: on-failure

  admin-area:
    image: example/admin_panel
    ports:
      - 5001:80
    environment:
      - ADMIN_AREA_PASSWORD=/run/secrets/adminpw
    networks:
      - private
    depends_on:
      - database
    deploy:
      replicas: 1
      restart_policy:
        condition: on-failure
    secrets:
      adminpw

  worker:
    image: example/job_worker
    networks:
      - public
      - private
    depends_on:
      - database
      - queue
    deploy:
      mode: replicated
      replicas: 3
      placement:
        constraints: [node.role == manager]

  visualizer:
    image: dockersamples/visualizer
    ports:
      - 8080:8080
    stop_grace_period: 1m30s
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
    deploy:
      placement:
        constraints: [node.role == manager]

networks:
  public:
  private:

volumes:
  db-data:

secrets:
  dbpw:
    external: true
  adminpw:
    external: true

Now we just have to:

docker stack deploy

And all of our services will be created, containers associated with those services will spread through the cluster to satisfy what we have in our docker-compose.yml file, if a node were to fail the swarm will adjust itself to make up for the containers that were down due to being hosted on that failed node, so our docker-compose.yml file becomes our single source of truth of the state of deployment strategy.

Stacks are an abstraction of which we can manage multiple instances of it, we can list all stacks:

docker stack ls

And see all services running on the stack:

docker stack services <stack id>

And see all containers running on the stack

docker stack ps <stack id>

And of course we can remove them with the rm sub-command.

The end

Ok, this is a very quick and simple intro to Docker Swarm, it’s not complicated at all, managing a swarm of containers is getting easier with these tools. One should see if Kubernetes might be a better fit, for what I’ve heard you could handle hundreds of containers with it 🤷‍♂️.