Tomcat|LVS+KeepAlived+Nginx+Tomcat高可用解决方案 Tomcat|Nginx

1、环境： 1、操作系统：Centos7.6
2、服务器配置如下：

服务器	软件
192.168.0.13	LVS+KeepAlived
192.168.0.14	LVS+KeepAlived
192.168.0.15	Nginx
192.168.0.16	Nginx
192.168.0.17	Tomcat
192.168.0.18	Tomcat
VIP 192.168.1.200

2、安装lvs+keepalived 本文安装采用源代码编译方式进行安装。
2.1 Lvs
从2.4版本开始，linux内核默认支持LVS。要使用LVS的能力，只需安装一个LVS的管理工具：ipvsadm。

yum -y install ipvsadm

2.2 keepalived
同时在192.168.0.13和192.168.0.14两台服务器上操作：
技巧：在Xshell的撰写窗格里同时对两台服务器进行操作。
2.2.1 下载

`进入到/usr/local/src目录下` [root@henry004 ~]# cd /usr/local/src`下载keepalived` [root@henry004 src]# wget https://www.keepalived.org/software/keepalived-2.0.20.tar.gz`解压缩` [root@henry001 src]# tar -zxvf keepalived-2.0.20.tar.gz

2.2.2 安装

`在/usr/local目录下创建keepalived文件夹` [root@henry001 keepalived-2.0.20]# mkdir /usr/local/keepalived`将keepalived安装到/usr/local/keepalived下，conf配置文件指定到目录/etc下` [root@henry001 keepalived-2.0.20]# ./configure --prefix=/usr/local/keepalived --sysconf=/etc`编译安装` [root@henry004 keepalived-2.0.20]# make && make install

编译过程中可能会出现如下常见问题：
1、缺少OpenSSL

`-------错误信息---------------` hecking openssl/ssl.h usability... no checking openssl/ssl.h presence... no checking for openssl/ssl.h... no configure: error: !!! OpenSSL is not properly installed on your system. !!! !!! Can not include OpenSSL headers files.!!!`----- ---解决方案--------------------` yum -y install openssl-devel

2、缺少libnl/libnl-3

`--------错误信息---------------` *** WARNING - this build will not support IPVS with IPv6. Please install libnl/libnl-3 dev libraries to support IPv6 with IPVS.`-------解决方案--------------------` yum -y install libnl libnl-devel

2.2.3 配置

`进入安装后的路径 cd /data/program/keepalived, 创建软连接` [root@henry001 sbin]# ln -s /usr/local/keepalived/sbin/keepalived/sbin/`把 keepalived的启动文件复制到init.d下，加入开机启动项` [root@henry001 keepalived-2.0.20]# cp /usr/local/src/keepalived-2.0.20/keepalived/etc/init.d/keepalived /etc/init.d`添加keepalived到系统服务` [root@henry001 sbin]# chkconfig –add keepalived chkconfig version 1.7.4 - Copyright (C) 1997-2000 Red Hat, Inc. This may be freely redistributed under the terms of the GNU Public License.usage:chkconfig [--list] [--type ] [name] chkconfig --add chkconfig --del chkconfig --override chkconfig [--level ] [--type ] `检测是否添加成功` [root@henry001 sbin]# chkconfig keepalived on Note: Forwarding request to 'systemctl enable keepalived.service'. Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.

2.2.4 启动keepalived

`启动keepalived服务` [root@henry001 sbin]# systemctl start keepalived.service `查看keepalived状态` [root@henry001 sbin]# systemctl status keepalived.service ● keepalived.service - LVS and VRRP High Availability Monitor Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled) Active: active (running) since Sat 2020-03-07 22:13:54 CST; 3s ago Process: 25684 ExecStart=/usr/local/keepalived/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS) Main PID: 25685 (keepalived) CGroup: /system.slice/keepalived.service ├─25685 /usr/local/keepalived/sbin/keepalived -D ├─25686 /usr/local/keepalived/sbin/keepalived -D └─25687 /usr/local/keepalived/sbin/keepalived -D-------------------------------------------------

操作keepalived的命令有如下：

`----启动-----` systemctl start keepalived.service `----重启-----` systemctl restart keepalived.service `----停止-----` systemctl stop keepalived.service `----查看状态-----` systemctl status keepalived.service

2.3 防火墙
为方便测试，我直接关闭了防火墙，在实际应用中可以根据需要开启防火墙的端口，此外还要设置服务器的安全策略，我的是阿里云的服务器，就在阿里云服务器控制台设置了安全策略，开放了需要的端口。
关闭防火墙：

`关闭防火墙` [root@henry001 sysconfig]# systemctl stop firewalld`查看防火墙状态` [root@henry001 sysconfig]# systemctl status firewalld ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled) Active: `inactive (dead)` Docs: man:firewalld(1)Mar 07 22:39:08 henry001 systemd[1]: Starting firewalld - dynamic firewall daemon... Mar 07 22:39:08 henry001 systemd[1]: Started firewalld - dynamic firewall daemon. Mar 07 22:39:23 henry001 systemd[1]: Stopping firewalld - dynamic firewall daemon... Mar 07 22:39:24 henry001 systemd[1]: Stopped firewalld - dynamic firewall daemon.

3、安装Nginx 同时在192.168.0.15和192.168.0.16两台服务器上操作，对Nginx+Tomcat的安装请参考文章：
实现Nginx+Tomcat负载均衡 https://blog.csdn.net/qq_33996921/article/details/104999852
4、配置keepalived 4.1master服务器
先来配置192.168.0.13的主机，指定其为master服务器；

`进入配置文件目录` [root@henry001 ~]#cd /etc/keepalived [root@henry001 keepalived]# ls keepalived.confsamples `编辑配置文件信息` [root@henry001 keepalived]# vim keepalived.conf

【Tomcat|LVS+KeepAlived+Nginx+Tomcat高可用解决方案】编辑keepalived.conf文件

global_defs { #notification_email { #edisonchou@hotmail.com #} # notification_email_from sns-lvs@gmail.com # smtp_server 192.168.80.1 #smtp_connection_timeout 30 router_id LVS_DEVEL# 设置lvs的id，在一个网络内应该是唯一的 } vrrp_instance VI_1 { state MASTER#指定Keepalived的角色，MASTER为主，BACKUP为备记得大写 interface eth0#网卡id 不同的电脑网卡id会有区别可以使用:ip a查看 virtual_router_id 51#虚拟路由编号，主备要一致 priority 100#定义优先级，数字越大，优先级越高，主DR必须大于备用DR advert_int 1#检查间隔，默认为1s authentication {#这里配置的密码最多为8位，主备要一致，否则无法正常通讯 auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.0.200#定义虚拟IP(VIP)为192.168.1.200，可多设，每行一个 } } # 定义对外提供服务的LVS的VIP以及port virtual_server 192.168.0.200 80 { delay_loop 6 # 设置健康检查时间，单位是秒 lb_algo rr # 设置负载调度的算法为wlc lb_kind DR # 设置LVS实现负载的机制，有NAT、TUN、DR三个模式 #nat_mask 255.255.255.0 persistence_timeout 0 protocol TCP real_server 192.168.0.15 80 {# 指定real server1的IP地址 weight 3# 配置节点权值，数字越大权重越高 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 192.168.0.16 80 {# 指定real server2的IP地址 weight 3# 配置节点权值，数字越大权重越高 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } }

4.1 backup服务器
配置192.168.0.14的主机，指定其为backup服务器；

`进入配置文件目录` [root@henry004 ~]#cd /etc/keepalived [root@henry004 keepalived]# ls keepalived.confsamples `编辑配置文件信息` [root@henry004 keepalived]# vim keepalived.conf

编辑keepalived.conf 文件

global_defs { #notification_email { #edisonchou@hotmail.com #} # notification_email_from sns-lvs@gmail.com #smtp_server 192.168.80.1 #smtp_connection_timeout 30 router_id LVS_DEVEL# 设置lvs的id，在一个网络内应该是唯一的 } vrrp_instance VI_1 { state BACKUP #指定Keepalived的角色，MASTER为主，BACKUP为备记得大写 interface eth0#网卡id 不同的电脑网卡id会有区别可以使用:ip a查看 virtual_router_id 51#虚拟路由编号，主备要一致 priority 50#定义优先级，数字越大，优先级越高，主DR必须大于备用DR advert_int 1#检查间隔，默认为1s authentication {#这里配置的密码最多为8位，主备要一致，否则无法正常通讯 auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.0.200#定义虚拟IP(VIP)为192.168.1.200，可多设，每行一个 } } # 定义对外提供服务的LVS的VIP以及port virtual_server 192.168.0.200 80 { delay_loop 6 # 设置健康检查时间，单位是秒 lb_algo rr # 设置负载调度的算法为wlc lb_kind DR # 设置LVS实现负载的机制，有NAT、TUN、DR三个模式 #nat_mask 255.255.255.0 persistence_timeout 0 protocol TCP real_server 192.168.0.16 80 {# 指定real server1的IP地址 weight 3# 配置节点权值，数字越大权重越高 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 192.168.0.15 80 {# 指定real server2的IP地址 weight 3# 配置节点权值，数字越大权重越高 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } }

5、查看虚拟IP 查看master服务器：

[root@henry001 keepalived]# ip a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:16:3e:30:cc:a2 brd ff:ff:ff:ff:ff:ff inet 192.168.0.13/24 brd 192.168.0.255 scope global dynamic eth0 valid_lft 315332386sec preferred_lft 315332386sec `inet 192.168.1.200/32 scope global eth0` valid_lft forever preferred_lft forever

查看backup服务器

[root@henry004 keepalived]# ip a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:16:3e:30:9f:0f brd ff:ff:ff:ff:ff:ff inet 192.168.0.14/24 brd 192.168.0.255 scope global dynamic eth0 valid_lft 315348979sec preferred_lft 315348979sec

下面我们停止掉master服务器上的keepalived，虚拟ip将会漂移到backup服务器上

[root@henry004 keepalived]# ip a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: eth0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:16:3e:30:9f:0f brd ff:ff:ff:ff:ff:ff inet 192.168.0.14/24 brd 192.168.0.255 scope global dynamic eth0 valid_lft 315348845sec preferred_lft 315348845sec `inet 192.168.0.200/32 scope global eth0` valid_lft forever preferred_lft forever

至此,一个LVS+KeepAlived+Nginx+Tomcat的高可用方案就搭建完成了。