锐客网

  1. 首页 > 睿知 > it技术 > >

Keepalived+LVS实战案例（ 单主架构实现WEB负载均衡及可用）

IT知识keepalived+l实战案例

业无高卑志当坚，男儿有求安得闲？这篇文章主要讲述Keepalived+LVS实战案例： 单主架构实现WEB负载均衡及可用相关的知识，希望能为你提供帮助。


1.概述?Keepalived是通过vrrp 协议的实现高可用性，对网络比较了解的IT人，对这个技术应该非常熟悉了，早期核心交换机用来实现双机双线的标准协议，现在随着技术发展出现了更好的核心设备的双活技术，vrrp/hrrp慢慢被取代了，但目前在Linux主机类应用场景使用还比较广泛。它的原生设计目的为了解决 ipvs高可用性。?
?官网：http://keepalived.org/?
?功能：?

  • ?基于vrrp协议完成地址漂移；?
  • ?为vip地址所在的节点生成ipvs规则 (在配置文件中预先定义)；?
  • ?为ipvs集群的各RS做健康状态检测；（ keepalived可以搭配 LVS、haproxy等成为黄金组合，尤其是keepalived + haproxy 在很多企业生产中使用）?
  • ?基于脚本调用接口完成脚本中定义的功能，进而影响集群事务，以此支持nginx、haproxy等服务。?
2.工作原理及技术架构
?官方文档：?
https://keepalived.org/doc/
http://keepalived.org/documentation.html

  • ?用户空间核心组件：?
?vrrp stack：VIP消息通告；?
?checkers：监测real server；?
?system call：实现 vrrp 协议状态转换时调用脚本的功能；?
?SMTP：邮件组件；?
?IPVS wrapper：生成IPVS规则；?
?Netlink Reflector：网络接口；?
?WatchDog：监控进程；?
  • ?控制组件：提供keepalived.conf 的解析器，完成Keepalived配置。?
  • ?IO复用器：针对网络目的而优化的自己的线程抽象。?
  • ?内存管理组件：为某些通用的内存管理功能（例如分配，重新分配，发布等）提供访问权限。?
3.拓扑结构及主机准备
# 五台主机
12台web服务器 ：
主机名：WebServer-IP17
CentOS 7.9
IP:192.168.250.17

主机名：WebServer-IP27
CentOS 7.9
IP:192.168.250.27

22台 keepalived 服务器 ：
主机名： KA-IP18
CentOS 8.4
IP:192.168.250.18/24
Keepalived v2.1.5 (07/13,2020)

主机名： KA-IP28
CentOS 8.4
IP:192.168.250.28/24
Keepalived v2.1.5 (07/13,2020)

31台client主机 ：
主机名：Client-IP172-8
CentOS 8.4
IP:172.16.0.8/24NAT成192.168.250.254 访问192.168.250.X网段

4.后端WEB服务器准备4.1安装Apache httpd 完成基本的WWW配置
# 基础环境包括CentOS操作系统的优化、防火墙关闭、同步时间等都要做好，我们按照规划的架构图对四台服务器进行分组并重新命名
# 修改服务器名称
[root@centos79 < sub> ]# hostnamectl set-hostname WebServer-IP17
[root@centos79 < /sub> ]# exit
# 修改NTP服务器地址为阿里云的NTP启用时钟同步服务
[root@webserver-ip17 < sub> ]# timedatectl set-timezone Asia/Shanghai
[root@webserver-ip17 < /sub> ]#sed -i /^server/cserver ntp.aliyun.com iburst /etc/chrony.conf
[root@webserver-ip17 < sub> ]# systemctl enable --nowchronyd.service
# 安装Apache
[root@webserver-ip17 < /sub> ]# yum -y install httpd
# 定义web主页文件
[root@webserver-ip17 < sub> ]# hostname > /var/www/html/indexTmp.html
[root@webserver-ip17 < /sub> ]# hostname -I > > /var/www/html/indexTmp.html

# 将文件 /var/www/html/indexTmp.html 内两行文字合并成一行，便于后面测试观测
[root@webserver-ip17 < sub> ]# cat /var/www/html/indexTmp.html | xargs > /var/www/html/index.html
[root@webserver-ip17 < /sub> ]# ll /var/www/html/
total 8
-rw-r--r-- 1 root root 30 Mar 31 23:07 index.html
-rw-r--r-- 1 root root 31 Mar 31 23:07 indexTmp.html
[root@webserver-ip17 < sub> ]# rm -rf /var/www/html/indexTmp.html
[root@webserver-ip17 < /sub> ]# ll /var/www/html/
total 4
-rw-r--r-- 1 root root 30 Mar 31 23:07 index.html

# 启动Apache服务，并开机自启
[root@webserver-ip17 < sub> ]# systemctlenable --now httpd

# 验证
[root@webserver-ip17 < /sub> ]# curl 192.168.250.17
webserver-ip17 192.168.250.17
[root@webserver-ip17 < sub> ]#

#####################################################################################
#### 同样的方式完成另外一台 webserver-ip27 192.168.250.27 的Apache的配置和调试
# 修改主机名、同步时间等基础环境配置需要按常规完成好
[root@centos79 < /sub> ]# hostnamectl set-hostname WebServer-IP17
[root@centos79 < sub> ]# exit
# 修改NTP服务器地址为阿里云的NTP启用时钟同步服务
[root@webserver-ip27 < /sub> ]# timedatectl set-timezone Asia/Shanghai
[root@webserver-ip27 < sub> ]#sed -i /^server/cserver ntp.aliyun.com iburst /etc/chrony.conf
[root@webserver-ip27 < /sub> ]# systemctl enable --nowchronyd.service

# 安装Apache、配置主页、启动服务 一次搞定
[root@webserver-ip27 < sub> ]# yum -y install httpd; hostname > /var/www/html/indexTmp.html; hostname -I > > /var/www/html/indexTmp.html; cat /var/www/html/indexTmp.html | xargs > /var/www/html/index.html; rm -rf /var/www/html/indexTmp.html; systemctlenable --now httpd

# 验证
[root@webserver-ip27 < /sub> ]# curl 192.168.250.27
webserver-ip27 192.168.250.27
[root@webserver-ip27 ~]#

4.2配置与LVS相关的配置?简要概述：因为我们在利用keepalived实现LVS-DR模式的WEB服务负载均衡，所以需要对后端两台服务器的ARP宣告、VIP绑定做好相应的配置，这样在LVS + keepalived 配置好后就可以直接按照LVS-DR模式访问后端的服务器了，因为这个过程在我的博客的前面文章已经详细介绍过，对这两台服务器的配置直接用脚本来完成了。?
#### lvs_dr_rs.sh 内容,在VS-Code内修改好上传到两个WEB-RS服务器上运行

#!/bin/bash
vip=192.168.250.100
mask=255.255.255.255
dev=lo:1

case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
ifconfig $dev $vip netmask $mask
echo "The RS Server is Ready!"
; ;
stop)
ifconfig $dev down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "The RS Server is Canceled!"
; ;
*)
echo "Usage: $(basename $0) start|stop"
exit 1
; ;
esac

## IP192.168.250.17上的配置过程
[root@webserver-ip17 < sub> ]# rz
rz waiting to receive.
Starting zmodem transfer.Press Ctrl+C to cancel.
Transferring lvs_dr_rs.sh...
100%728 bytes728 bytes/sec 00:00:010 Errors

[root@webserver-ip17 < /sub> ]# bash
[root@webserver-ip17 < sub> ]# bash lvs_dr_rs.sh
Usage: lvs_dr_rs.sh start|stop
[root@webserver-ip17 < /sub> ]# bash lvs_dr_rs.sh start
The RS Server is Ready!
[root@webserver-ip17 < sub> ]# ip a
1: lo: < LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.250.100/32 scope global lo:1
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: < BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:a3:a8:67 brd ff:ff:ff:ff:ff:ff
inet 192.168.250.17/24 brd 192.168.250.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fea3:a867/64 scope link
valid_lft forever preferred_lft forever
[root@webserver-ip17 < /sub> ]#

## IP192.168.250.27上的配置过程
[root@webserver-ip27 < sub> ]# rz
rz waiting to receive.
Starting zmodem transfer.Press Ctrl+C to cancel.
Transferring lvs_dr_rs.sh...
100%728 bytes728 bytes/sec 00:00:010 Errors

[root@webserver-ip27 < /sub> ]# bash lvs_dr_rs.sh start
The RS Server is Ready!
[root@webserver-ip27 < sub> ]# ip a
1: lo: < LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.250.100/32 scope global lo:1
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: < BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:a3:fb:92 brd ff:ff:ff:ff:ff:ff
inet 192.168.250.27/24 brd 192.168.250.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fea3:fb92/64 scope link
valid_lft forever preferred_lft forever
[root@webserver-ip27 < /sub> ]#

5.配置keepalived服务器【Keepalived+LVS实战案例（ 单主架构实现WEB负载均衡及可用）】?第一步：先配置实现master/slave的 Keepalived 单主架构，并测试成功。第二步再配置和LVS的ipvs服务相关的，实现和后端WEB-RS的通信。?
5.1配置 Keepalived 实现master/slave的单主架构
#### IP 192.168.250.18 Keepalived-IP18 的 Keepalived 配置
[root@CentOS84-IP18 ]#hostnamectl set-hostname Keepalived-IP18
[root@CentOS84-IP18 ]#exit

[root@Keepalived-IP18 ]#timedatectl set-timezone Asia/Shanghai
[root@Keepalived-IP18 ]#sed -i /^server/cserver ntp.aliyun.com iburst /etc/chrony.conf
[root@Keepalived-IP18 ]#systemctl enable --nowchronyd.service
[root@Keepalived-IP18 ]#systemctl restartchronyd.service

[root@Keepalived-IP18 ]#ip a
1: lo: < LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: < BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:a3:e8:6b brd ff:ff:ff:ff:ff:ff
inet 192.168.250.18/24 brd 192.168.250.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fea3:e86b/64 scope link
valid_lft forever preferred_lft forever
[root@Keepalived-IP18 ]#

[root@Keepalived-IP18 ]#dnf info keepalived
Last metadata expiration check: 19:41:03 ago on Wed 30 Mar 2022 10:03:06 PM CST.
Installed Packages
Name: keepalived
Version: 2.1.5
....................

[root@Keepalived-IP18 ]#dnf install keepalived -y
[root@Keepalived-IP18 ]#keepalived -v
Keepalived v2.1.5 (07/13,2020)

Copyright(C) 2001-2020 Alexandre Cassen, < acassen@gmail.com>

Built with kernel headers for Linux 4.18.0
Running on Linux 4.18.0-305.3.1.el8.x86_64 #1 SMP Tue Jun 1 16:14:33 UTC 2021
......................
# 备份默认的 keepalived.conf 配置文件并按照规划拓扑结构修改相应的配置文件。配置文件详细的每行含义请参见文章末尾的详细说明。
[root@Keepalived-IP18 ]#cp /etc/keepalived/keepalived.conf,.bak
[root@Keepalived-IP18 ]#vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
# 全局设置：邮箱设置，要结合系统的/etc/mail.rc 和通知脚本来实现故障通知。
global_defs
notification_email
root@shone.cn

notification_email_from admin@shone.cn
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id KA-IP18
vrrp_skip_check_adv_addr
#vrrp_strict# 严格模式建议关闭掉
vrrp_garp_interval 0
vrrp_gna_interval 0
#vrrp_mcast_group4 234.0.0.66#可以用自定义多播地址通告VRRP信息，建议用下面的单播



vrrp_instance VI_IP100
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication
auth_type PASS
auth_pass shone888

virtual_ipaddress
192.168.250.100 dev eth0 label eth0:1

# 下面是单播的配置
unicast_src_ip 192.168.250.18
unicast_peer
192.168.250.28



# 全部配置好启动服务并设定开机自启
[root@Keepalived-IP28 ]#systemctl enable --now keepalived

#### IP 192.168.250.28 Keepalived-IP28 的 Keepalived 配置

[root@CentOS84-IP28 ]#hostnamectl set-hostname Keepalived-IP28
[root@CentOS84-IP28 ]#exit

[root@Keepalived-IP28 ]#timedatectl set-timezone Asia/Shanghai
[root@Keepalived-IP28 ]#sed -i /^server/cserver ntp.aliyun.com iburst /etc/chrony.conf
[root@Keepalived-IP28 ]#systemctl enable --nowchronyd.service
[root@Keepalived-IP28 ]#systemctl restartchronyd.service

[root@Keepalived-IP28 ]#ip a
1: lo: < LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: < BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:a3:e2:bf brd ff:ff:ff:ff:ff:ff
inet 192.168.250.28/24 brd 192.168.250.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fea3:e2bf/64 scope link
valid_lft forever preferred_lft forever
[root@Keepalived-IP28 ]#

[root@Keepalived-IP28 ]#dnf info keepalived
Last metadata expiration check: 19:41:03 ago on Wed 30 Mar 2022 10:03:06 PM CST.
Installed Packages
Name: keepalived
Version: 2.1.5
....................

[root@Keepalived-IP28 ]#dnf install keepalived -y
[root@Keepalived-IP28 ]#keepalived -v
Keepalived v2.1.5 (07/13,2020)
......................

# 备份默认的 keepalived.conf 配置文件并按照规划拓扑结构修改相应的配置文件。配置文件详细的每行含义请参见文章末尾的详细说明。
[root@Keepalived-IP28 ]#cp /etc/keepalived/keepalived.conf,.bak
[root@Keepalived-IP28 ]#vim /etc/keepalived/keepalived.conf
! Configuration File

    推荐阅读


    上一篇：Tars | 第2篇 TarsJava SpingBoot启动与负载均衡源码初探 #yyds干货盘点#

    下一篇：数仓建模—OneID