Docker-Swarm三节点部署 dockerswarm

学习笔记

1. 安装要求
在开始之前，Dcoker Swarm 集群部署机器需要满足以下几个条件：

虚拟了3台机器，操作系统：Linux version 3.10.0-1127.el7.x86_64
硬件配置：CPU(s)-2；3GB或更多RAM; 硬盘20G左右；
3台机器构成一个集群，机器之前网络互通；
集群可以访问外网，用于拉取镜像
禁用swap分区

2. Swarm 的工作模式
2.1 集群调度图原理

swarm 集群由管理节点（manager）和工作节点（work node）构成。

文章图片

2.2 Swarm集群调用架构图

docker 集群之间通过远程API实现通讯（Remote API）

文章图片

3. 环境准备

master节点： 192.168.11.99
node1节点：192.168.11.131
node2节点：192.168.11.130

## 1. 关闭防火墙 $ systemctl stop firewalld### 关闭防火墙自动启动 $ systemctl disable firewalld## 2. 关闭selinux### 2.1 修改配置文件永久关闭selinux【重启生效】 $ vi /etc/sysconfig/selinux SELINUX=enforcing### 2.2 临时关闭selinux $ setenforce 0## 3. 分别设置设置主机名： $ hostnamectl set-hostname swarm-master $ hostnamectl set-hostname swarm-node1 $ hostnamectl set-hostname swarm-node2### 3.1 查看hostname $ hostname## 4. 配置hosts文件 $ cat >> /etc/hosts << EOF 192.168.11.99 swarm-master 192.168.11.130 swarm-node1 192.168.11.131 swarm-node2 EOF## 5. 安裝docker $ yum -y install docker### 5.1 docker服务启动 $ systemctl enable docker && systemctl start docker

4.创建Swarm并添加节点

# Swarm-master 虚拟机操作## 1. 创建Swarm集群(记录token) [root@localhost vagrant]# docker swarm init --advertise-addr192.168.11.99 Swarm initialized: current node (6jir7f507ur05lrhtc2xq8plt) is now a manager.To add a worker to this swarm, run the following command:# 这就是添加节点的方式(要保存初始化后token，因为在节点加入时要使用token作为通讯的密钥)docker swarm join --token SWMTKN-1-30sw7z3qrh3gs0pi5fmllypw0c5dxrnpzf2uwd5nldcifzzroh-0l4pkovtgy708qs5pn5v8rzsk192.168.11.99:2377To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. ## 2. 查看集群的相关信息 [root@localhost vagrant]# docker info | grep swarm WARNING: You're not using the default seccomp profile WARNING: bridge-nf-call-ip6tables is disabled Name: swarm-master### 2.1 查看节点信息 [root@localhost vagrant]# docker node ls IDHOSTNAMESTATUSAVAILABILITYMANAGER STATUS 6jir7f507ur05lrhtc2xq8plt *swarm-masterReadyActiveLeader # Swarm-node 虚拟机## 3. 加入的集群### 3.1 Swarm-node1 节点加入到集群 [root@localhost vagrant]# docker swarm join --token SWMTKN-1-30sw7z3qrh3gs0pi5fmllypw0c5dxrnpzf2uwd5nldcifzzroh- 0l4pkovtgy708qs5pn5v8rzsk192.168.11.99:2377 This node joined a swarm as a worker.### 3.2 Swarm-node2 节点加入到集群 [root@localhost vagrant]# docker swarm join --token SWMTKN-1-30sw7z3qrh3gs0pi5fmllypw0c5dxrnpzf2uwd5nldcifzzroh- 0l4pkovtgy708qs5pn5v8rzsk192.168.11.99:2377 This node joined a swarm as a worker.## 4.查看集群节点状态 [root@localhost vagrant]# docker node ls IDHOSTNAMESTATUSAVAILABILITYMANAGER STATUS 6jir7f507ur05lrhtc2xq8plt *swarm-masterReadyActiveLeader qt9ja2qbmq4n040y60g6xbdlzswarm-node2ReadyActive scnyaecep2d1eiccvubusykwwswarm-node1ReadyActive

5. Swarm 常用操作

更改节点的availablity状态

swarm集群中node的availability状态可以为 active或者drain，其中：
active状态下，node可以接受来自manager节点的任务分派；
drain状态下，node节点会结束task，且不再接受来自manager节点的任务分派（也就是下线节点）

## 1.删除节点 [root@localhost vagrant]# docker node rm --force swarm-node1 swarm-node1 [root@localhost vagrant]# docker node ls IDHOSTNAMESTATUSAVAILABILITYMANAGER STATUS 6jir7f507ur05lrhtc2xq8plt *swarm-masterReadyActiveLeader qt9ja2qbmq4n040y60g6xbdlzswarm-node2ReadyActive ## 2. 更新状态 [root@localhost vagrant]# docker node update --availability drain swarm-node1 swarm-node1[root@localhost vagrant]# docker node ls IDHOSTNAMESTATUSAVAILABILITYMANAGER STATUS 6jir7f507ur05lrhtc2xq8plt *swarm-masterReadyActiveLeader 8nk8i2ekx8kykyzmaab475tnwswarm-node1ReadyDrain qt9ja2qbmq4n040y60g6xbdlzswarm-node2ReadyActive[root@localhost vagrant]# docker node update --availability active swarm-node1 swarm-node1[root@localhost vagrant]# docker node ls IDHOSTNAMESTATUSAVAILABILITYMANAGER STATUS 6jir7f507ur05lrhtc2xq8plt *swarm-masterReadyActiveLeader 8nk8i2ekx8kykyzmaab475tnwswarm-node1ReadyActive qt9ja2qbmq4n040y60g6xbdlzswarm-node2ReadyActive

6. Swarm 中部署服务（nginx）

Docker 1.12版本提供服务的Scaling、health check、滚动升级等功能，并提供了内置的dns、vip机制，实现service的服务发现和负载均衡能力

## 1. 部署服务### 1.1 创建网络 [root@localhost vagrant]# docker network create -d overlay nginx_net lj3drxnqik170ta6fr7f4rq5i[root@localhost vagrant]# docker network ls | grep "nginx_net" lj3drxnqik17nginx_netoverlayswarm### 1.2 部署服务 // 就创建了一个具有一个副本（--replicas 1 ）的nginx服务，使用镜像nginx [root@localhost vagrant]# docker service create --replicas 1 --network nginx_net --name my_nginx -p 80:80 ng inx ufxp6pe02z3xggd73ni2jcxay ### 1.3 查看正在运行的服务列表 [root@localhost vagrant]# docker service ls IDNAMEMODEREPLICASIMAGE ufxp6pe02z3xmy_nginxreplicated1/1nginx:latest## 2. 查询Swarm中服务的信息 [root@localhost vagrant]# docker service inspect --pretty my_nginxID:ufxp6pe02z3xggd73ni2jcxay Name:my_nginx Service Mode:Replicated Replicas:1 Placement: UpdateConfig: Parallelism:1 On failure:pause Max failure ratio: 0 ContainerSpec: Image:nginx:latest@sha256:06e4235e95299b1d6d595c5ef4c41a9b12641f6683136c18394b858967cd1506 Resources: Networks: nginx_net Endpoint Mode:vip Ports: PublishedPort 80 Protocol = tcp TargetPort = 80## 3. 查看服务在哪个节点上；（在swarm-master节点） [root@localhost vagrant]# docker service ps my_nginx IDNAMEIMAGENODEDESIRED STATECURRENT STATEERRORPORTS t7o6f7hrjd9vmy_nginx.1nginx:latestswarm-masterRunningRunning 6 minutes ago## 4. 在Swarm中动态扩展服务（scale） // 容器动态扩展到4个 [root@localhost vagrant]# docker service scale my_nginx=4 my_nginx scaled to 4 [root@localhost vagrant]# docker service ps my_nginx IDNAMEIMAGENODEDESIRED STATECURRENT STATEERRORPORTS t7o6f7hrjd9vmy_nginx.1nginx:latestswarm-masterRunningRunning 58 minutes ago ek7ckkc1g1b4my_nginx.2nginx:latestswarm-node2RunningPreparing 5 seconds ago jdl14u7a8or5my_nginx.3nginx:latestswarm-node1RunningPreparing 5 seconds ago 2vflrnohh6h8my_nginx.4nginx:latestswarm-node1RunningPreparing 5 seconds ago

7. Swarm 服务（nginx）--- 故障切換

如果一个节点宕机了（即该节点就会从swarm集群中被踢出），则Docker应该会将在该节点运行的容器，调度到其他节点，以满足指定数量的副本保持运行状态。

# 查看服务所在节点 [root@localhost vagrant]# docker service ps my_nginx IDNAMEIMAGENODEDESIRED STATECURRENT STATEERRORPORTS t7o6f7hrjd9vmy_nginx.1nginx:latestswarm-masterRunningRunning 58 minutes ago ek7ckkc1g1b4my_nginx.2nginx:latestswarm-node2RunningPreparing 5 seconds ago jdl14u7a8or5my_nginx.3nginx:latestswarm-node1RunningPreparing 5 seconds ago 2vflrnohh6h8my_nginx.4nginx:latestswarm-node1RunningPreparing 5 seconds ago

模拟其中一个节点机器关机了

# swarm-node1 节点关闭docker-service [root@localhost vagrant]# systemctl stop docker

查看主节点信息

[root@localhost vagrant]# docker node ls IDHOSTNAMESTATUSAVAILABILITYMANAGER STATUS 6jir7f507ur05lrhtc2xq8plt *swarm-masterReadyActiveLeader 8nk8i2ekx8kykyzmaab475tnwswarm-node1DownActive qt9ja2qbmq4n040y60g6xbdlzswarm-node2ReadyActive

可以看到 swarm-node1 已经关闭了

[root@localhost vagrant]# docker service ps my_nginx IDNAMEIMAGENODEDESIRED STATECURRENT STATEERRORPORTS t7o6f7hrjd9vmy_nginx.1nginx:latestswarm-masterRunningRunning about an hour ago ek7ckkc1g1b4my_nginx.2nginx:latestswarm-node2RunningRunning 19 minutes ago buaz7bbd2tjxmy_nginx.3nginx:latestswarm-node2RunningRunning 2 minutes ago jdl14u7a8or5\_ my_nginx.3nginx:latestswarm-node1ShutdownRunning 2 minutes ago 5puac9f6ya0jmy_nginx.4nginx:latestswarm-node2RunningRunning 2 minutes ago 2vflrnohh6h8\_ my_nginx.4nginx:latestswarm-node1ShutdownRunning 2 minutes ago

我们可以看到原来 swarm-node1 上的nginx 已经关闭了，而且在其他节点上再创建了等量的 nginx 服务
当我们重新恢复 swarm-node1 节点的 docker 服务（模拟机器恢复了）

[root@localhost vagrant]# docker service ps my_nginx IDNAMEIMAGENODEDESIRED STATECURRENT STATEERRORPO RTS t7o6f7hrjd9vmy_nginx.1nginx:latestswarm-masterRunningRunning about an hour ago ek7ckkc1g1b4my_nginx.2nginx:latestswarm-node2RunningRunning 24 minutes ago buaz7bbd2tjxmy_nginx.3nginx:latestswarm-node2RunningRunning 7 minutes ago jdl14u7a8or5\_ my_nginx.3nginx:latestswarm-node1ShutdownFailed 2 minutes ago"No such container: my_nginx.3…" 5puac9f6ya0jmy_nginx.4nginx:latestswarm-node2RunningRunning 7 minutes ago 2vflrnohh6h8\_ my_nginx.4nginx:latestswarm-node1ShutdownFailed 2 minutes ago"No such container: my_nginx.4…"

分配到 swarm-node1 节点的并没恢复
8. 多服务Swarm部署到集群

docker-compose 和 Swarm 结合实现对多服务进行编排

nginx服务
visualizer服务
portainer服务

docker service部署的是单个服务，我们可以使用docker stack进行多服务编排部署
创建 docker-compose.yml

version: "3" services: nginx: image: nginx ports: - 8888:80 deploy: mode: replicated replicas: 3 visualizer: image: dockersamples/visualizer ports: - "8080:8080" volumes: - "/var/run/docker.sock:/var/run/docker.sock" deploy: replicas: 1 placement: constraints: [node.role == manager] portainer: image: portainer/portainer ports: - "9000:9000" volumes: - "/var/run/docker.sock:/var/run/docker.sock" deploy: replicas: 1 placement: constraints: [node.role == manager]

通过 docker-compose.yaml 部署服务

[root@localhost testswarm]# docker stack deploy -c docker-compose.yml deploy_deamon Creating network deploy_deamon_default Creating service deploy_deamon_visualizer Creating service deploy_deamon_portainer Creating service deploy_deamon_nginx # 查看状态 [root@localhost testswarm]# docker service ls IDNAMEMODEREPLICASIMAGE il2vuomkg3bydeploy_deamon_portainerreplicated1/1portainer/portainer:latest sudtu6byntyedeploy_deamon_visualizerreplicated1/1dockersamples/visualizer:latest ywtzr8ssersfdeploy_deamon_nginxreplicated3/3nginx:latest

【Docker-Swarm三节点部署】