Manager Swarms using Portainer

Once we have a docker swarm setup it can be managed via CLI or GUI.

Any kind of GUI makes the job easy. Even Kubernetes can manage docker. Docker offers Docker UCP (Universal Control Panel) to manage the docker swam. In this article to keep it simple we will be discussing only on Docker GUI tools not on any kind of orchestration tools. There are three options available for us here: Portainer, Shipyard and Rancher.

The tool which is native to windows is Portainer. For Rancher and Shipyard they have to be installed on Linux and then the windows docker can be added as a node to be managed.

The same goes with Docker UCP and Kubernetes, they also support Windows Docker but the master node should be on Linux. Initially I was excited when I googled and found UCP supports Windows.

I wanted to install UCP on my Windows docker manager node. I referred to Docker docs and was not successful.

https://docs.docker.com/datacenter/ucp/2.1/guides/admin/install/#step-5-license-your-installation

After many failed attempts I found out in Docker UCP, Windows node can join an existing cluster as worker only.  I had missed the fine print in the documents, but finally found some confirmed documentation on docker site:

https://success.docker.com/article/docker-universal-control-plane-windows-early-access

So, lets get back to the GUI tool. As I said in my first blog we will look at something which works on Windows natively, so we will go with Portainer.

Portainer image is based on Windows Nano server and it has an application called portainer.exe. Let’s start the installation and configuration here.

Step 1: We need to ensure Docker service is listening on TCP socket. By default docker allows a named pipe connection only. Dockers provide multiple options to make it work, we can use the CLI, registry or config files. But point to note is

 “The Docker service will not start if the same parameter is set in service startup and configuration file “

Check if the daemon.json file exists in C:\ProgramData\docker\config

If the file does not exist create a file using notepad and name it daemon.json .

Add the below line to the file:

{"hosts": ["tcp://0.0.0.0:2375","npipe://"]}

What does this command mean: it indicates for any IP listen on port 2375 as the primary option and then as a fall back use named pipe. Port 2375 is unencrypted and 2376 is encrypted with TLS, these are standard ports which docker uses. For production we will use encrypted port 2376, I will be writing another blog on how to configure and use it on windows, as the windows standard certificates cannot be used we need to use openssl.

Restart the docker service after we created the file.

Step 2: Allow Docker connection via firewall

We need to allow TCP connection to the Docker service on port 2375. We can do it via GUI or CLI. In CLI there is one simple command for this, open a command prompt with Administrator permission and type the below command:

netsh advfirewall firewall add rule name="Docker" dir=in action=allow protocol=TCP localport=2375 enable=yes profile=domain,private,public

As usuall lets verify if the command did the trick, by opening the Server Manager and going to Windows Firewall with Advance Security in the Tools.

Once we have the Windows Firewall with Advance Security open click on the Inbound Rules and check if Docker is created on the Inbound Rules it should have a Green tick next to it.

Now that we have the firewall setup, lets go to the next step

Step 3: Download Portainer Image

From our PowerShell console type the following command

Docker pull portainer/portainer

Once portainer is downloaded lets inspect it to find more information about the image and its layers.

Type

docker image history portainer/portainer

As you can see the base image is windows nano server 10.0.14393 and on top of it they have created a volume c:\data, they have exposed the port 9000 to run /portainer.exe application.

We can find more details if we inspect the image

For that type

Docker image inspect portainer/portainer

Let us analyze the downloaded image to see what all information we can find.

So we can see the base image is Windows OS, OS version in numbers which is nano server, and we can see the folder and port information in detail.

Step 4: Configure the VM to run the container:

As we saw in the above step, to configure or run this container we need to expose the port 9000. Alternatively we can connect directly without doing anything like mapping ports, but for the outside world to connect we need to get the port mapping established. We can expose the port 9000 of the container on any port number we wish, let’s use 9000 itself to keep it simple. How do we do that using our docker run command? We will add an parameter -p with port numbers. So it will be -p 9000:9000

Next in the image we saw the container uses folder C:\Data to store the portainer data, like the user access and other stuff, for that we need to map a drive from local server. Lets create a folder in our c:\ and name it C:\Portainer. Once we have physically created the folder we will use -v parameter to add it. So it will become -v C:\Portainer:C:\Data

So now let us see the run command

Docker run -d --restart always --name portainer -v C:\Portainer:C:\Data -p 9000:9000 portainer/portainer

-d to detach the container from console

--restart always: means if the container crashes or when docker engine restarts, start the container.

--name is the name given to the container, we set it as portainer

Portainer/portainer is the image which we are loading.

The container is started. Let us check if the container is up and running and did it mount to portainer folder created by us. To check if the container is mounted we can run the docker ps command

If we have a look at c:\portainer folder, we can see the folders and files which portainer uses to maintain manager the docker swarm.

What will happen if we don’t use -v and mount C:\Portainer, the container will still load, but every time the container is restarted the credentials and other Portainer settings, about the docker swarm would be lost from portainer and it needs to set again and again.

Step 5: Once we have the container set and running, we know that the por,t portainer would respond on is 9000 as we had set it on the docker run command. We did not mention any IP during docker run, so we dont know what IP the container is using. For that we would use the following command.

docker container inspect portainer

I did not find any help file for the portainer which mentioned about how to set an IP for the container, I tried using the –p again to see if that would help, but it didn’t. So for now we would allow the container to use random IP. If you are connecting from the PC browser to the portainer installed on the VM, in that case we will use the IP which we set in our last blog in the step F.

Scroll down to Network setting configuration , you will find the IP Address, that would be the Portainer IP and the gateway is the IP of Host server

Step 6: We will open a browser and connect with http://IPAddress:9000. Use the ip address as per our capture above. Once you are connected please enter new password for portainer admin account, and then click on create user. Ensure the password is 8 character’s long.

Once you get connected to the portainer, you need to connect to the Docker Swarm, for that enter the name of the Docker server, in our case WinDocker1 and the Endpoint URL: it will be the IP address of the Gateway, we can find it in our capture above. Enter the details and click connect

Volla !! you should be connected to the docker swarm now. We have a GUI to manage all our Windows Dockers. You have some basic information displayed, the number of nodes in the cluster, the CPU and Memory for this node etc. We will see how we can assign memory and CPU to the docker engine and for docker containers later.

Once you click on Swarm, you can will find the details about the swarm

You can click on network and get details on network, the NAT etc

I would suggest all of you to play around with the GUI and I will catch you guys in my next blog. If you want me to address any specific configurations please let me know in comments.

When I started this blog series I wanted to configure Windows Containers using Hyper-v and have GUI to manage the Containers and swarm, So far we are on track. Stay with me for my next blog and lets learn together.

If you want to use Windows server on Azure and want to try using container there, please feel free to do so. Unfortunately I have finished my free quota from Azure, so I am working on my laptop using Hyper-V. Keep learning and Enjoying I will see you in my next blog.

Docker swarm

In this blog we will be discussing the below tasks:

1) Add one more Windows servers with Docker’s on it.

2) Create a docker Swarm by promoting one node as manager and adding one node as worker/manager.

What is docker swarm? To answer it in an easy way - cluster of Docker Engines, which can be on a physical or virtual servers.

There are two type of nodes in docker swarm: Manager and Workers.

Manager node: This node handles cluster management tasks like, Scheduling service, maintaining cluster state etc.

Worker node: The sole purpose of this docker node is to execute the containers.

By default all manager nodes are worker’s also. We can promote or demote worker nodes as Managers. In production environment we try not to run containers on manager node, we set the availability for manager node as Drain. We will see later on as how to drain a node and how to run the container as service. Those are interesting topics and may require a separate blog of its own.

For step 1 we can follow the same steps as per our last post  or just to save some time we can do export and import of the VM. Lets walk through the steps on exporting and importing, this way we can save some time and also learn some new skills

Adding New Windows server:

a) Shutdown the newly installed Windows server running docker.

b) Open the hyper-v manager and then right click on the server which you want to export or clone.

Once you click on the export, you will be asked a location where to save the files. Provide a valid path to save the file. Lets assume we save it at c:\backup

Click on Export button and the export process would start. The time to export will depend on the Disk speed and IOPS.

Lets check the path and find out for our self as to what gets exported

As you can see export creates folder same as the name of the server being exported and three folders within that

Virtual Hard Disks folder will have the virtual hard drive file

Virtual Machines will have the configuration files.

Snapshots will have snapshots based on the setting, in our case the snapshot will be empty as we disabled it while creating the VM

c) Next lets import the VM created as a new server. Lets name the server as VMWD20162 indicating it to be the second server. For that on the Actions pane, click on Import Virtual Machine

Click next on the pop up screen, click on browse button and select the folder where export was stored, varify the path from the above step.

Click on next and on the following screen verify the server and click next again. Select copy the virtual machine (create new unique ID) in the import type and click next.

The three methods in the above screen are explained below:

Register: If you have a virtual machine where you have already put all of the virtual machine files exactly where you want them, and you just need Hyper-V to start using the virtual machine where it is.
Restore: If your virtual machine files are stored on a file share / removable drive / etc… and you want Hyper-V to move the files to the appropriate location for you, and then register the virtual machine.
Copy: If you have a set of virtual machine files that you want to import multiple times (e.g. you are using them as a template for new virtual machines) this is what you want to choose. This will copy the files to an appropriate location, give the virtual machine a new unique ID, and then register the virtual machine.

Now on the next screen you can set the path for the configuration files for the server where they need to be stored and similarly on the choose storage folders session mention the path for vhd.

Finally in the summary verify all the paths are okay and then click on Finish.

Wait for the new VM to be created and to be listed on the Hyper-v Manager.

Wow !! now what you will have two VM’s with same name on listed on the hyper-v manager, you have to rename the newly installed server, right click on server and click on rename and add a 2 to end of the name.

Lets start both the servers up.

d) In our previous blog we had not configured the network connection. let us set it up for the swarm to work.

Lets create a virtual switch, click on the Virtual Switch Manager on the Actions Pane

Once Launched Select Internal and click on Create Switch button

It will Create a new Virtual Switch on the Left hand pane. in the right hand pane, type in the name and description for the switch, ensure internal network is selected and then click on OK button.

Now we need to launch the network connections. To do this we can open a command prompt and type ncpa.cpl

OR

Via the GUI by clicking on control panel –> network connection.

We will have a new network adapter listed with the name we used above

Double click on that, then on properties for that adapter, select IPv4, and set the IP as shown in the capture below. You can set any IP, I just took this as an example.

e) If there are any Virtual servers running shut them down to add the network adapters.

Next right click on the VM and open settings for that VM, select Network Adapter and click on Add

Then continue adding the DocSwitch and click ok, this will add the new network adapter on the VM and also link it to the switch which we created.

f) Turn on both the servers and lets set the IP(preferred) for the newly added network cards. You would like to set the IP 192.168.10.xx, Default gateway as 192.168.10.1 as shown in the picture below

I have used the 10.11 and 10.12 as the IP’s for 2 servers.

So now we have two windows server installed with dockers, ready to join the swarm.

In production environment you will not find standalone Docker engines, they will all be in a swarm, managed either by Docker UCP or other 3rd party tools. We can have Dockers (nodes) managed using Kubernetes, at later stage I will write a blog on this as well, In that we will prepare a Linux master running the kubernetes and we will add Windows Docker server as client. For now let me come back to creating a Docker Swarm.

For this Blog we were discussing two topics, installation of Additional server and then creating a swarm.

For the second part we will follow the below steps:

a) We need to add our first node to the Docker Swarm by declaring it as the master/manager. Later we can add more nodes as masters or workers.

To add our first node we need to run the below command:

docker swarm init –advertise-addr <ip address of the server>

in our case the command would be

docker swarm init –advertise-addr 192.169.10.11

Lets verify the status of the node by running the following command:

docker node ls

This command will list all the nodes in the swarm, and we can see our first server /node is added to the swarm as leader. Next we need to add the worker.

b) To add node to the swarm we need to have the respective token. (Token for a worker and a token for a manager)

To check the token you can run the below command:

docker swarm join-token worker

or

docker swarm join-token manager

Copy one of the outputs

or

Depending on what you want the other node to be - manager or worker.

You can promote or demote nodes at later stage as well.

c) Lets connect to our other server and join it to the swarm as a worker, so i would be using the first token for that .

Done!

d) Great, now lets run the docker node ls command to find if we can see all the nodes

• What Error !!?

That is because only the Manager nodes can list all nodes. So lets connect to the master node or our first server and run the command again

We have 2 nodes, one as leader/Manager and other one as worker. Great we have our swarm ready with 2 servers/nodes in it.

Let us connect to the manager node and run the promote command just to check

docker node promote WinDocker2

and run the docker node ls again to verify the status

run the docker node ls command on the other node as well.

To demote we can run the demote command. Refer to Docker docs they have extensive help for all commands

https://docs.docker.com/engine/swarm/manage-nodes/#promote-or-demote-a-node

In the next blog we will discuss on how to setup a management portal, below is the screen shot of portainer, using it we can verify the cluster performance add registry etc.

So what are you waiting for? setup your lab and join us in learning journey. Keep learning and Enjoying I will see you in my next blog.