深度登录harbor # docker login 192.168.10.15 Username: admin Password: 12345 WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
3.4 docker swarm集群初始化
3.4.1 获取docker swarm命令帮助
获取docker swarm命令使用帮助 # docker swarm --help
Usage: docker swarm COMMAND
Manage Swarm
Commands: ca Display and rotate the root CA init Initialize a swarm 初始化 join Join a swarm as a node and/or manager 加入集群 join-token Manage join tokens 集群加入时token管理 leave Leave the swarm 离开集群 unlock Unlock swarm unlock-key Manage the unlock key update Update the swarm 更新集群
3.4.2 在管理节点初始化
本次在sm1上初始化
初始化集群 # docker swarm init --advertise-addr 192.168.10.10 --listen-addr 192.168.10.10:2377 Swarm initialized: current node (j42cwubrr70pwxdpmesn1cuo6) is now a manager.
To add a worker to this swarm, run the following command:
使用初始化过程中生成的token加入集群 [root@sw1 ~]# docker swarm join --token SWMTKN-1-297iry1n2jeh30oopsjecvsco1uuvl15t2jz6jxabdpf0xkry4-6pddlyiq5f1i35w8d7q4bl1co 192.168.10.10:2377 This node joined a swarm as a worker.
查看已加入的集群 # docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION j42cwubrr70pwxdpmesn1cuo6 * sm1 Ready Active Leader 20.10.12 4yb34kuma6i9g5hf30vkxm9yc sw1 Ready Active 20.10.12
如果使用的token已过期，可以再次生成新的加入集群的方法，如下命令所示。
重新生成用于添加工作点的token [root@sm1 ~]# docker swarm join-token worker To add a worker to this swarm, run the following command:
加入至集群 [root@sw2 ~]# docker swarm join --token SWMTKN-1-297iry1n2jeh30oopsjecvsco1uuvl15t2jz6jxabdpf0xkry4-6pddlyiq5f1i35w8d7q4bl1co 192.168.10.10:2377 This node joined a swarm as a worker.
查看node状态 # docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION j42cwubrr70pwxdpmesn1cuo6 * sm1 Ready Active Leader 20.10.12 4yb34kuma6i9g5hf30vkxm9yc sw1 Ready Active 20.10.12 mekitdu1xbpcttgupwuoiwg91 sw2 Ready Active 20.10.12
3.4.4 添加管理节点到集群
生成用于添加管理节点加入集群所使用的token [root@sm1 ~]# docker swarm join-token manager To add a manager to this swarm, run the following command:
加入集群 [root@sm2 ~]# docker swarm join --token SWMTKN-1-297iry1n2jeh30oopsjecvsco1uuvl15t2jz6jxabdpf0xkry4-7g85apo82mwz8ttmgdr7onfhu 192.168.10.10:2377 This node joined a swarm as a manager.
加入集群 [root@sm3 ~]# docker swarm join --token SWMTKN-1-297iry1n2jeh30oopsjecvsco1uuvl15t2jz6jxabdpf0xkry4-7g85apo82mwz8ttmgdr7onfhu 192.168.10.10:2377 This node joined a swarm as a manager.
查看节点状态 # docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION j42cwubrr70pwxdpmesn1cuo6 * sm1 Ready Active Leader 20.10.12 nzpmehm8n87b9a17or2el10lc sm2 Ready Active Reachable 20.10.12 xc2x9z1b33rwdfxc5sdpobf0i sm3 Ready Active Reachable 20.10.12 4yb34kuma6i9g5hf30vkxm9yc sw1 Ready Active 20.10.12 mekitdu1xbpcttgupwuoiwg91 sw2 Ready Active 20.10.12
3.4.5 模拟管理节点出现故障
3.4.5.1 停止docker服务并查看结果
停止docker服务 [root@sm1 ~]# systemctl stop docker
查看node状态，发现sm1不可达，状态为未知，并重启选择出leader [root@sm2 ~]# docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION j42cwubrr70pwxdpmesn1cuo6 sm1 Unknown Active Unreachable 20.10.12 nzpmehm8n87b9a17or2el10lc * sm2 Ready Active Leader 20.10.12 xc2x9z1b33rwdfxc5sdpobf0i sm3 Ready Active Reachable 20.10.12 4yb34kuma6i9g5hf30vkxm9yc sw1 Ready Active 20.10.12 mekitdu1xbpcttgupwuoiwg91 sw2 Ready Active 20.10.12
3.4.5.2 启动docker服务并查看结果
再次重动docker [root@sm1 ~]# systemctl start docker
观察可以得知sm1是可达状态，但并不是Leader [root@sm1 ~]# docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION j42cwubrr70pwxdpmesn1cuo6 * sm1 Ready Active Reachable 20.10.12 nzpmehm8n87b9a17or2el10lc sm2 Ready Active Leader 20.10.12 xc2x9z1b33rwdfxc5sdpobf0i sm3 Ready Active Reachable 20.10.12 4yb34kuma6i9g5hf30vkxm9yc sw1 Ready Active 20.10.12 mekitdu1xbpcttgupwuoiwg91 sw2 Ready Active 20.10.12
四、docker swarm集群应用
4.1 容器镜像准备
准备多个版本的容器镜像，以便于后期使用测试。
4.1.1 v1版本
生成网站文件v1版 [root@harbornginximg]# vim index.html [root@harbornginximg]# cat index.html v1
编写Dockerfile文件，用于构建容器镜像 [root@harbornginximg]# vim Dockerfile [root@harbornginximg]# cat Dockerfile FROM nginx:latest
[root@sm1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS ucif0ibkjqrd nginx-svc-1 replicated 1/1192.168.10.15/library/nginx:v1 *:80->80/tcp
4.2.4 查看已发布服务容器
[root@sm1 ~]# docker service ps nginx-svc-1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 47t0s0egf6xf nginx-svc-1.1192.168.10.15/library/nginx:v1 sw1 Running Running 48 minutes ago
[root@sw1 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1bdf8981f511 192.168.10.15/library/nginx:v1 "/docker-entrypoint.…"53 minutes ago Up 53 minutes 80/tcp nginx-svc-1.1.47t0s0egf6xf1n8m0c0jez3q0
[root@sm1 ~]# docker service scale nginx-svc-1=2 nginx-svc-1 scaled to 2 overall progress: 2 out of 2 tasks 1/2: running [==================================================>] 2/2: running [==================================================>] verify: Service converged
[root@sm1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS ucif0ibkjqrd nginx-svc-1 replicated 2/2192.168.10.15/library/nginx:v1 *:80->80/tcp
[root@sm1 ~]# docker service ps nginx-svc-1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 47t0s0egf6xf nginx-svc-1.1192.168.10.15/library/nginx:v1 sw1 Running Running about an hour ago oy16nuh5udn0 nginx-svc-1.2192.168.10.15/library/nginx:v1 sw2 Running Running 57 seconds ago
[root@sw1 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1bdf8981f511 192.168.10.15/library/nginx:v1 "/docker-entrypoint.…" About an hour ago Up About an hour 80/tcp nginx-svc-1.1.47t0s0egf6xf1n8m0c0jez3q0
[root@sw2 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0923c0d10223 192.168.10.15/library/nginx:v1 "/docker-entrypoint.…" About a minute ago Up About a minute 80/tcp nginx-svc-1.2.oy16nuh5udn0s1hda5bcpr9hd
问题：现在仅扩展为2个副本，如果把服务扩展到3个副本，集群会如何分配主机呢？
[root@sm1 ~]# docker service scale nginx-svc-1=3 nginx-svc-1 scaled to 3 overall progress: 3 out of 3 tasks 1/3: running [==================================================>] 2/3: running [==================================================>] 3/3: running [==================================================>] verify: Service converged
[root@sm1 ~]# docker service ps nginx-svc-1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 47t0s0egf6xf nginx-svc-1.1192.168.10.15/library/nginx:v1 sw1 Running Running about an hour ago oy16nuh5udn0 nginx-svc-1.2192.168.10.15/library/nginx:v1 sw2 Running Running 12 minutes ago mn9fwxqbc9d1 nginx-svc-1.3192.168.10.15/library/nginx:v1 sm1 Running Running 9 minutes ago
说明： 当把服务扩展到一定数量时，管理节点也会参与到负载运行中来。
4.4 服务裁减
[root@sm1 ~]# docker service scale nginx-svc-1=2 nginx-svc-1 scaled to 2 overall progress: 2 out of 2 tasks 1/2: running [==================================================>] 2/2: running [==================================================>] verify: Service converged
[root@sm1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS ucif0ibkjqrd nginx-svc-1 replicated 2/2192.168.10.15/library/nginx:v1 *:80->80/tcp
[root@sm1 ~]# docker service ps nginx-svc-1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 47t0s0egf6xf nginx-svc-1.1192.168.10.15/library/nginx:v1 sw1 Running Running 2 hours ago oy16nuh5udn0 nginx-svc-1.2192.168.10.15/library/nginx:v1 sw2 Running Running 29 minutes ago
4.5 负载均衡
服务中包含多个容器时，每次访问将以轮询的方式访问到每个容器
修改sw1主机中容器网页文件 [root@sw1 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1bdf8981f511 192.168.10.15/library/nginx:v1 "/docker-entrypoint.…" About an hour ago Up About an hour 80/tcp nginx-svc-1.1.47t0s0egf6xf1n8m0c0jez3q0 [root@sw1 ~]# docker exec -it 1bdf bash root@1bdf8981f511:/# echo "sw1 web" > /usr/share/nginx/html/index.html root@1bdf8981f511:/# exit
修改sw2主机中容器网页文件 [root@sw2 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0923c0d10223 192.168.10.15/library/nginx:v1 "/docker-entrypoint.…"42 minutes ago Up 42 minutes 80/tcp nginx-svc-1.2.oy16nuh5udn0s1hda5bcpr9hd [root@sw2 ~]# docker exec -it 0923 bash root@0923c0d10223:/# echo "sw2 web" > /usr/share/nginx/html/index.html root@0923c0d10223:/# exit
[root@sm1 ~]# curl http://192.168.10.10 sw1 web [root@sm1 ~]# curl http://192.168.10.10 sw2 web [root@sm1 ~]# curl http://192.168.10.10 sw1 web [root@sm1 ~]# curl http://192.168.10.10 sw2 web
4.6 删除服务
[root@sm1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS ucif0ibkjqrd nginx-svc-1 replicated 2/2192.168.10.15/library/nginx:v1 *:80->80/tcp
[root@sm1 ~]# docker service rm nginx-svc-1 nginx-svc-1
[root@sm1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS
4.7 服务版本更新
[root@sm1 ~]# docker service create --name nginx-svc --replicas=1 --publish 80:80 192.168.10.15/library/nginx:v1 yz3wq6f1cgf10vtq5ne4qfwjz overall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged
[root@sm1 ~]# curl http://192.168.10.10 v1
[root@sm1 ~]# docker service update nginx-svc --image 192.168.10.15/library/nginx:v2 nginx-svc overall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged
[root@sm1 ~]# curl http://192.168.10.10 v2
4.8 服务版本回退
[root@sm1 ~]# docker service update nginx-svc --image 192.168.10.15/library/nginx:v1 nginx-svc overall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged
4.9 服务版本滚动间隔更新
# docker service create --name nginx-svc --replicas 60 --publish 80:80 192.168.10.15/library/nginx:v1 pqrt561dckg2wfpect3vf9ll0 overall progress: 60 out of 60 tasks verify: Service converged
[root@sm1 ~]# docker service update --replicas 60 --image 192.168.10.15/library/nginx:v2 --update-parallelism 5 --update-delay 30s nginx-svc nginx-svc overall progress: 3 out of 3 tasks 1/3: running [==================================================>] 2/3: running [==================================================>] 3/3: running [==================================================>] verify: Service converged
docker swarm滚动更新会造成节点上有exit状态的容器,可以考虑清除 命令如下： [root@sw1 ~]# docker container prune WARNING! This will remove all stopped containers. Are you sure you want to continue? [y/N] y
4.10 副本控制器
副本控制器
[root@sm1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS yz3wq6f1cgf1 nginx-svc replicated 3/3192.168.10.15/library/nginx:v2 *:80->80/tcp
[root@sm1 ~]# docker service ps nginx-svc ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS x78l0santsbb nginx-svc.1192.168.10.15/library/nginx:v2 sw2 Running Running 3 hours ago ura9isskfxku \_ nginx-svc.1192.168.10.15/library/nginx:v1 sm1 Shutdown Shutdown 3 hours ago z738gvgazish \_ nginx-svc.1192.168.10.15/library/nginx:v2 sw1 Shutdown Shutdown 3 hours ago 3qsrkkxn32bl \_ nginx-svc.1192.168.10.15/library/nginx:v1 sm3 Shutdown Shutdown 3 hours ago psbi0mxu3amy nginx-svc.2192.168.10.15/library/nginx:v2 sw1 Running Running 3 hours ago zpjw39bwhd78 nginx-svc.3192.168.10.15/library/nginx:v2 sm1 Running Running 3 hours ago
[root@sm1 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 81fffd9132d8 192.168.10.15/library/nginx:v2 "/docker-entrypoint.…"3 hours ago Up 3 hours 80/tcp nginx-svc.3.zpjw39bwhd78pw49svpy4q8zd [root@sm1 ~]# docker stop 81fffd9132d8;docker rm 81fffd9132d8 81fffd9132d8 81fffd9132d8
[root@sm1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS yz3wq6f1cgf1 nginx-svc replicated 3/3192.168.10.15/library/nginx:v2 *:80->80/tcp [root@sm1 ~]# docker service ps nginx-svc ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS x78l0santsbb nginx-svc.1192.168.10.15/library/nginx:v2 sw2 Running Running 3 hours ago ura9isskfxku \_ nginx-svc.1192.168.10.15/library/nginx:v1 sm1 Shutdown Shutdown 3 hours ago z738gvgazish \_ nginx-svc.1192.168.10.15/library/nginx:v2 sw1 Shutdown Shutdown 3 hours ago 3qsrkkxn32bl \_ nginx-svc.1192.168.10.15/library/nginx:v1 sm3 Shutdown Shutdown 3 hours ago psbi0mxu3amy nginx-svc.2192.168.10.15/library/nginx:v2 sw1 Running Running 3 hours ago qv6ya3crz1fj nginx-svc.3192.168.10.15/library/nginx:v2 sm1 Running Running 13 seconds ago zpjw39bwhd78 \_ nginx-svc.3192.168.10.15/library/nginx:v2 sm1 Shutdown Failed 19 seconds ago "task: non-zero exit (137)"
[root@sm1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS wgqkz8vymxkr tomcat replicated 2/2 tomcat:7.0.96-jdk8-openjdk *:8080->8080/tcp
[root@sm1 ~]# docker service ps tomcat ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS fsx1fnssbmtg tomcat.1 tomcat:7.0.96-jdk8-openjdk sm3 Running Running 49 seconds ago gq0ogycj7orb tomcat.2 tomcat:7.0.96-jdk8-openjdk sm2 Running Running 58 seconds ago
[root@sm1 ~]# docker service ps tomcat ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS fsx1fnssbmtg tomcat.1 tomcat:7.0.96-jdk8-openjdk sm3 Running Running 8 minutes ago gq0ogycj7orb tomcat.2 tomcat:7.0.96-jdk8-openjdk sm2 Running Running 8 minutes ago
[root@sm2 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f650498c8e71 tomcat:7.0.96-jdk8-openjdk"catalina.sh run"9 minutes ago Up 9 minutes 8080/tcp tomcat.2.gq0ogycj7orbu4ua1dwk140as
[root@sm3 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9d0c412717d7 tomcat:7.0.96-jdk8-openjdk"catalina.sh run"9 minutes ago Up 9 minutes 8080/tcp tomcat.1.fsx1fnssbmtgv3qh84fgqknlh
s31z75rniv4p53ycbqch3xbqm overall progress: 3 out of 3 tasks 1/3: running [==================================================>] 2/3: running [==================================================>] 3/3: running [==================================================>] verify: Service converged
4.13.1.3 验证是否使用本地目录
[root@sm1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS s31z75rniv4p nginx-svc replicated 3/3192.168.10.15/library/nginx:v1 *:80->80/tcp
[root@sm1 ~]# docker service ps nginx-svc ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS vgfhk4lksbtp nginx-svc.1192.168.10.15/library/nginx:v1 sm2 Running Running 54 seconds ago v2bs9araxeuc nginx-svc.2192.168.10.15/library/nginx:v1 sw2 Running Running 59 seconds ago 1m7fobr3cscz nginx-svc.3192.168.10.15/library/nginx:v1 sm3 Running Running 59 seconds ago
[root@sm1 ~]# docker service create --name nginx-svc --replicas 3 --publish 80:80 --mount "type=volume,source=nginx_volume,target=/usr/share/nginx/html" 192.168.10.15/library/nginx:v1 uh6k84b87n8vciuirln4zqb4v overall progress: 3 out of 3 tasks 1/3: running [==================================================>] 2/3: running [==================================================>] 3/3: running [==================================================>] verify: Service converged
4.13.2.5 验证
[root@sm1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS uh6k84b87n8v nginx-svc replicated 3/3192.168.10.15/library/nginx:v1 *:80->80/tcp [root@sm1 ~]# docker service ps nginx-svc ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS k2vxpav5oadf nginx-svc.1192.168.10.15/library/nginx:v1 sw2 Running Running 43 seconds ago v8fh0r89wt5i nginx-svc.2192.168.10.15/library/nginx:v1 sw1 Running Running 43 seconds ago xb0nyft8ou4d nginx-svc.3192.168.10.15/library/nginx:v1 sm1 Running Running 43 seconds ago
[root@sm1 ~]# docker network ls NETWORK ID NAME DRIVER SCOPE 5ba369c13795 bridge bridge local 54568abb541a docker_gwbridge bridge local 4edcb5c4a324 host host local l6xmfxiiseqk ingress overlay swarm 此处 5d06d748c9c7 none null local mrkgccdfddy8 tomcat-net overlay swarm
[root@sm1 ~]# docker service ps test ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS x8nkifpdtyw5 test.1 busybox:latest sm2 Running Running about a minute ago
4, 去运行test服务的容器节点查找容器的名称
[root@sm2 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 8df13819bd5c busybox:latest "sleep 100000" About a minute ago Up About a minute test.1.x8nkifpdtyw5177zhr0r1lxad
5, 使用查找出来的容器名称,执行命令测试
[root@sm2 ~]# docker exec -it test.1.x8nkifpdtyw5177zhr0r1lxad ping -c 2 nginx-svc PING nginx-svc (192.168.100.2): 56data bytes 64 bytes from 192.168.100.2: seq=0 ttl=64 time=0.093 ms 64 bytes from 192.168.100.2: seq=1 ttl=64 time=0.162 ms
--- nginx-svc ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 0.093/0.127/0.162 ms
