keepalived原理及实战部署
基于VRRP(Virtual Router Redundancy Protocol,虚拟路由冗余协议)的Keepalive机制是为了解决网络中静态默认网关单点故障的问题。在传统的网络架构中,如果作为默认网关的路由器出现故障,会导致整个子网失去对外通信的能力。而VRRP则通过引入主备模式来提供高可用性。
vrrp
VRRP(Virtual Router Redundancy Protocol,虚拟路由冗余协议)是一种网络层的容错协议。它的主要目的是在多个物理路由器之间提供一个虚拟路由器服务,以实现默认网关的高可用性。在传统的网络环境中,如果静态配置的默认网关出现故障,则会导致网络中的所有主机失去对外通信的能力。VRRP通过将多台路由器联合起来形成一个虚拟路由器,并选举出一台主路由器来处理所有的流量转发,而其他的备份路由器则随时待命,一旦主路由器出现故障,备份路由器能够迅速接替工作,从而确保网络连通性的持续性和可靠性。
工作原理
(1)虚拟路由器中的路由器根据优先级(priority)选举出Master。Master路由器通过发送免费ARP报文,将自己的虚拟MAC地址通知给与它连接的设备或者主机,从而承担报文转发任务;
(2) Master路由器周期性发送VRRP报文(心跳线),以公布其配置信息(优先级等)和工作状况;
(3) 如果Master路由器出现故障,虚拟路由器中的Backup路由器将根据优先级重新选举新的Master;
(4) 虚拟路由器状态切换时,Master路由器由一台设备切换为另外一台设备,新的Master路由器只是简单地发送一个携带虚拟路由器的MAC地址和虚拟IP地址信息的免费ARP报文,这样就可以更新与它连接的主机或设备中的ARP相关信息。网络中的主机感知不到Master路由器已经切换为另外一台设备。
(5) Backup路由器的优先级高于Master路由器时,由Backup路由器的工作方式(抢占方式和非抢占方式以及延时抢占)决定是否重新选举Master。
相关术语
虚拟路由器:Virtual Router 不是真实存在 ,虚构出来的
虚拟路由器标识:VRID(0-255),唯一标识虚拟路由器
VIP:Virtual IP
VMAC:Virutal MAC (00-00-5e-00-01-VRID)
物理路由器:
master:主设备
backup:备用设备
priority:优先级
keepalive简介
实现原理
1.首先根据优先级选出Master提供服务
2.Master默认每间隔1秒向Baskup发送一次VRRP报文,告知Baskup自己处于工作状态,并公布配置信息(优先级等)情况
3.如果Master路由器出现故障,Backup将根据优先级重新选举为新的Master
4.当原Master恢复工作时,将根据抢占模式(抢占方式和非抢占方式以及延时抢占)决定是否重新成为Master
安装
dnf install keepalived -y
开机自启
systemctl enable --now keepalived.service
相关文件
软件包名:keepalived
主程序文件:/usr/sbin/keepalived
主配置文件:/etc/keepalived/keepalived.conf
配置文件示例:/usr/share/doc/keepalived/
Unit File:/lib/systemd/system/keepalived.service
Unit File的环境配置文件:/etc/sysconfig/keepalived
主配置文件解析
全局配置
! Configuration File for keepalivedglobal_defs {
#keepalived 发生故障切换时邮件发送的目标邮箱,可以按行区分写多个notification_email {13279419836@163.com }#发邮件的地址notification_email_from k1@k1.org #邮件服务器地址smtp_server 127.0.0.1#设置SMTP服务器连接超时时间,单位为秒,这里为30秒。smtp_connect_timeout 30#每个keepalived主机唯一标识,建议使用当前主机名,但多节点重名不影响router_id k1#决定了设备是否跳过对 VRRP 通告报文中源 IP 地址的检查,启用它能让设备忽略通告报文源 IP 与虚拟 IP 是否一致的验证vrrp_skip_check_adv_addr#启用严格模式,要求配置必须符合RFC文档的标准规范,否则Keepalived服务可能不会启动。#启用此项后以下状况将无法启动服务:#1.无VIP地址#2.配置了单播邻居#3.在VRRP版本2中有IPv6地址#建议关闭vrrp_strict#免费 ARP(Gratuitous ARP)报文时间间隔
#免费 ARP用于通知网络中其他设备,某 IP地址对应的 MAC 地址发生了变化
#帮助网络设备更新 ARP 缓存,确保数据能正确转发到新的主节点vrrp_garp_interval 1#用于配置发送 Gratuitous NA(免费邻居通告)报文的时间间隔 ipv6vrrp_gna_interval 1#指定组播IP地址vrrp_mcast_group4 224.0.0.44#开启keepalived执行脚本功能enable_script_security#指定脚本执行用户身份script_user root
}
#导入子配置文件
include /etc/keepalived/conf.d/*.conf
vrrp实例配置
vrrp_instance VI_1 { #定义VRRP实例名称state MASTER #身份为master,优先处理请求interface eth0 #指定VRRP通告报文通过哪个网络接口发送,可以和vip不在一个网卡virtual_router_id 51 #虚拟路由器惟一标识,范围:0-255,每个虚拟路由器此值必须唯一,主备需要在同一个标识符内priority 100 #当前物理节点在此虚拟路由器的优先级,范围:1-254,值越大优先级越高advert_int 1 #vrrp通告的时间间隔,默认1sauthentication {auth_type PASS|AH #AH为IPSEC认证(不推荐),PASS为简单密码(建议使用)auth_pass 1111 #预共享密钥,仅前8位有效,保证各节点一致}track_script{ CHECK_HAPROXY #使用脚本
}virtual_ipaddress {<IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>#IP /掩码 广播地址 网卡 作用域 设置别名192.168.60.100/24 dev eth0 label eth0:0 }accept #开启vip对外响应ping包,需要关闭vrrp_strict#默认使用nftab策略禁用ping包响应,nft list ruleset 显示策略中即可看到
}
实现单主架构
master–k1
#打开配置文件
vim /etc/keepalived/keepalived.conf
#修改配置
global_defs {notification_email {13279419836@163.com}notification_email_from k1@k1.orgsmtp_server 127.0.0.1smtp_connect_timeout 30router_id k1vrrp_skip_check_adv_addr
# vrrp_strictvrrp_garp_interval 1vrrp_gna_interval 1vrrp_mcast_group4 224.0.0.44
}vrrp_instance VI_1 {state MASTERinterface eth0virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111 #主备设备认证密码需一致}virtual_ipaddress {192.168.60.100/24 dev eth0 label eth0:0 #vip }
}
backup–k2
#打开配置文件
vim /etc/keepalived/keepalived.conf
#修改配置
! Configuration File for keepalivedglobal_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 127.0.0.1smtp_connect_timeout 30router_id k2vrrp_skip_check_adv_addr#vrrp_strictvrrp_garp_interval 1vrrp_gna_interval 1vrrp_mcast_group4 224.0.0.44
}vrrp_instance VI_1 {state BACKUP #备interface eth0virtual_router_id 51priority 80 #优先级低于主advert_int 1authentication {auth_type PASSauth_pass 1111 #主备设备认证密码需一致}virtual_ipaddress {192.168.60.100/24 dev eth0 label eth0:0}
}
实现效果
master设备
#查看ip归属
[root@k1 ~]# ip a
...
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:89:c7:ed brd ff:ff:ff:ff:ff:ffaltname enp3s0altname ens160inet 192.168.60.50/24 brd 192.168.60.255 scope global noprefixroute eth0valid_lft forever preferred_lft foreverinet 192.168.60.100/24 scope global secondary eth0:0 #可以看到vip此时在k1-master设备上生效valid_lft forever preferred_lft foreverinet6 fe80::869f:ac0c:2ae7:302e/64 scope link noprefixroute valid_lft forever preferred_lft forever[root@k2 ~]# ip a
...inet 192.168.60.60/24 brd 192.168.60.255 scope global noprefixroute eth0 #仅一个IP
...
#关闭k1设备keepalived
[root@k1 ~]# systemctl stop keepalived.service
[root@k1 ~]# ip a
...inet 192.168.60.50/24 brd 192.168.60.255 scope global noprefixroute eth0
...
[root@k2 ~]# ip a
...inet 192.168.60.60/24 brd 192.168.60.255 scope global noprefixroute eth0valid_lft forever preferred_lft foreverinet 192.168.60.100/24 scope global secondary eth0:0 #可以看到此时vip飘到k2上valid_lft forever preferred_lft forever
非抢占模式与抢占延迟
默认为抢占模式preempt,即当高优先级的主机恢复在线后,会抢占低先级的主机的master角色,这样会使vip在KA主机中来回漂移,造成网络抖动,
建议设置为非抢占模式 nopreempt ,即高优先级主机恢复后,并不会抢占低优先级主机的master角色
非抢占模块下,如果原主机down机, VIP迁移至的新主机, 后续也发生down时,仍会将VIP迁移回原主机
注意:要关闭 VIP抢占,必须将各 keepalived 服务器state配置为BACKUP
非抢占模式
k1
#打开配置文件
vim /etc/keepalived/keepalived.conf
#修改配置
global_defs{...}
vrrp_instance VI_1 {state BACKUP #state设置为BACKUPinterface eth0virtual_router_id 51priority 100nopreempt #设置为非抢占模式advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.60.100/24 dev eth0 label eth0:0}
}
k2
#打开配置文件
vim /etc/keepalived/keepalived.conf
#修改配置
global_defs{...}
vrrp_instance VI_1 {state BACKUPinterface eth0virtual_router_id 51priority 80nopreempt #非抢占模式advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.60.100/24 dev eth0 label eth0:0}
}
测试
#关闭服务
[root@k1 ~]# systemctl stop keepalived.service
#可以看到ip飘到k2上
[root@k2 ~]# ip a
...inet 192.168.60.60/24 brd 192.168.60.255 scope global noprefixroute eth0valid_lft forever preferred_lft foreverinet 192.168.60.100/24 scope global secondary eth0:0valid_lft forever preferred_lft foreverinet6 fe80::71d1:24e2:443f:84de/64 scope link noprefixroute
...
#重新开启服务
[root@k1 ~]# systemctl start keepalived.service
#查看ip
[root@k1 ~]# ip a
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:89:c7:ed brd ff:ff:ff:ff:ff:ffaltname enp3s0altname ens160inet 192.168.60.50/24 brd 192.168.60.255 scope global noprefixroute eth0 #eth0网卡上仅一个ip
延迟抢占
k1
#打开配置文件
vim /etc/keepalived/keepalived.conf
#修改配置
global_defs{...}vrrp_instance VI_1 {state BACKUPinterface eth0virtual_router_id 51priority 100preempt_delay 10 #设置抢占延迟为10秒advert_int 1authentication {auth_type PASSauth_pass 1111}
k2
#打开配置文件
vim /etc/keepalived/keepalived.conf
#修改配置
global_defs{...}
vrrp_instance VI_1 {state BACKUPinterface eth0virtual_router_id 51priority 80preempt_delay 10advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.60.100/24 dev eth0 label eth0:0}
}
vip单播
k1
#打开配置文件
vim /etc/keepalived/keepalived.conf
#修改配置
global_defs{...}
vrrp_instance VI_1 {state MASTERinterface eth0virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.60.100/24 dev eth0 label eth0:0}unicast_src_ip 192.168.60.50unicast_peer {192.168.60.60}
}
k2
#打开配置文件
vim /etc/keepalived/keepalived.conf
#修改配置
global_defs{...}
vrrp_instance VI_1 {state BACKUPinterface eth0virtual_router_id 51priority 80preempt_delay 10advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.60.100/24 dev eth0 label eth0:0}unicast_src_ip 192.168.60.60unicast_peer {192.168.60.50}
}
测试
[root@k1 ~]# tcpdump -i eth0 -nn src host 192.168.60.50 and dst 192.168.60.60
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
20:53:52.038531 IP 192.168.60.50 > 192.168.60.60: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:53:53.044632 IP 192.168.60.50 > 192.168.60.60: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:53:54.049798 IP 192.168.60.50 > 192.168.60.60: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:53:55.050867 IP 192.168.60.50 > 192.168.60.60: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:53:56.057475 IP 192.168.60.50 > 192.168.60.60: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:53:57.061606 IP 192.168.60.50 > 192.168.60.60: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
^C
6 packets captured
6 packets received by filter
0 packets dropped by kernel
通知脚本配置
邮箱服务配置
详细配置参考配置qq邮箱收发功能,每天九点发邮件_qq邮箱每天定时发送邮件-CSDN博客
#下载s-nail
dnf install s-nail
#修改配置文件
vim /etc/s-nail.rc
# 最后一行添加以下内容:
set from=lxx1xxxxxxxxxxx@163.com
set smtp=smtp.163.com
set smtp-auth-user=lxx1065372838@163.com
set smtp-auth-password=HUaxxxxxxxxxxxxxxx
set smtp-auth=login
# 注意:该文件输入完毕有的系统需要强制保存退出,输入wq!
mail脚本配置
vim /etc/haproxy/mail.sh#!/bin/bash
mail_dest='13279419836@qq.com'
mail_send()
{
mail_subj="$HOSTNAME to be $1 vip 转移"
mail_mess="`date +%F\ %T`: vrrp 转移,$HOSTNAME 变为 $1"
echo "$mail_mess" | mail -s "$mail_subj" $mail_dest
}
case $1 in
master)
mail_send master
;;
backup)
mail_send backup
;;
fault)
mail_send fault
;;
*)
exit 1
;;
esac#添加执行权限
chmod +x /etc/haproxy/mail.sh
haproxy.conf
global_defs {notification_email {13279419836@163.com}notification_email_from k1@k1.orgsmtp_server 127.0.0.1smtp_connect_timeout 30router_id k1vrrp_skip_check_adv_addr
# vrrp_strictvrrp_garp_interval 1vrrp_gna_interval 1vrrp_mcast_group4 224.0.0.44enable_script_security #开启脚本执行功能script_user root #使用root身份执行脚本
}
vrrp_instance VI_1 {state MASTERinterface eth0virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.60.100/24 dev eth0 label eth0:0}notify_master "/etc/keepalived/notify.sh master" #调用脚本notify_backup "/etc/keepalived/notify.sh backup"notify_fault "/etc/keepalived/notify.sh fault"
}
测试
#重启keepalived服务
systemctl restart keepalived.service
双主架构配置
master/slave的单主架构,同一时间只有一个Keepalived对外提供服务,此主机繁忙,而另一台主机却
很空闲,利用率低下,可以使用master/master的双主架构,解决此问题。
master/master 的双主架构:
即将两个或以上VIP分别运行在不同的keepalived服务器,以实现服务器并行提供web访问的目的,提高
服务器资源利用率
k1
global_defs {notification_email {13279419836@163.com}notification_email_from k1@k1.orgsmtp_server 127.0.0.1smtp_connect_timeout 30router_id k1vrrp_skip_check_adv_addr
# vrrp_strictvrrp_garp_interval 1vrrp_gna_interval 1vrrp_mcast_group4 224.0.0.44
}vrrp_instance VI_1 {state MASTERinterface eth0virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.60.100/24 dev eth0 label eth0:0}
}vrrp_instance VI_60 {state BACKUPinterface eth0virtual_router_id 52priority 80 advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.60.200/24 dev eth0 label eth0:1}
}
k2
#打开配置文件
vim /etc/keepalived/keepalived.conf
#修改配置
global_defs{...}
vrrp_instance VI_1 {state BACKUP interface eth0virtual_router_id 51priority 80preempt_delay 10advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.60.100/24 dev eth0 label eth0:0}
}
vrrp_instance VI_60 {state MASTER interface eth0virtual_router_id 52priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.60.200/24 dev eth0 label eth0:1}
}
查看
k1:192.168.60.100
k2:192.168.60.200
[root@k1 keepalived]# ip a
...inet 192.168.60.50/24 brd 192.168.60.255 scope global noprefixroute eth0valid_lft forever preferred_lft foreverinet 192.168.60.100/24 scope global secondary eth0:0
...[root@k2 ~]# ip a
...inet 192.168.60.60/24 brd 192.168.60.255 scope global noprefixroute eth0valid_lft forever preferred_lft foreverinet 192.168.60.200/24 scope global secondary eth0:1valid_lft forever preferred_lft forever
...
ipvs高可用
双主分别实现httpd和mysql服务的调度
global_defs{...
}
vrrp_instance VI_1 {...
}
vrrp_instance VI_60 {...
}
#http
virtual_server 192.168.60.100 80 {delay_loop 6lb_algo rrlb_kind DRprotocol TCPreal_server 192.168.60.10 80 {weight 1TCP_CHECK {connect_port 80connect_timeout 5retry 3delay_before_retry 3}}real_server 192.168.60.20 80 {weight 1HTTP_GET {url {path /status_code 200}connect_timeout 1retry 3delay_before_retry 1}}}#mysql
virtual_server 192.168.60.200 3306 {delay_loop 6lb_algo rrlb_kind DRprotocol TCPreal_server 192.168.60.10 3306 {weight 1TCP_CHECK {connect_port 3306connect_timeout 10retry 3delay_before_retry 3}}real_server 192.168.60.20 3306 {weight 1TCP_CHECK {connect_timeout 10retry 3delay_before_retry 1connect_port 3306}}}
k2
global_defs{...
}
vrrp_instance VI_1 {...
}
vrrp_instance VI_60 {...
}
#http
virtual_server 192.168.60.100 80 {delay_loop 6lb_algo rrlb_kind DRpersistence_timeout 50protocol TCPreal_server 192.168.60.10 80 {weight 1TCP_CHECK {connect_port 80connect_timeout 5retry 3delay_before_retry 3}}real_server 192.168.60.20 80 {weight 1HTTP_GET {url {path /status_code 200}connect_timeout 1retry 3delay_before_retry 1}}}
#mysql
virtual_server 192.168.60.200 3306 {delay_loop 6lb_algo rrlb_kind DRprotocol TCPreal_server 192.168.60.10 3306 {weight 1TCP_CHECK {connect_port 3306connect_timeout 10retry 3delay_before_retry 3}}real_server 192.168.60.20 3306 {weight 1TCP_CHECK {connect_timeout 10retry 3delay_before_retry 1connect_port 3306}}
}
rs1/rs2
#安装nginx
dnf install nginx -y
#开启
systemctl enable --now nginx
#写入默认页面
echo rs11-192.168.60.10 > /usr/share/nginx/html/static/index.html
#安装mariadb
dnf install mariadb -y
#修改配置文件
vim /etc/my.cnf
server-id=1
#创建账户并授权
grant ALL on *.* to dll@'%' identified by 'dll'
#添加ip
ip a a 192.168.60.100/32 dev lo
ip a a 192.168.60.200/32 dev lo
#设置lo不对外响应
vim /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore =1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_ignore =1
net.ipv4.conf.lo.arp_announce =2
测试
[root@ceshi ~]# curl 192.168.60.100
rs22-192.168.60.20
[root@ceshi ~]# curl 192.168.60.100
rs11-192.168.60.10
[root@ceshi ~]# curl 192.168.60.100
rs22-192.168.60.20
[root@ceshi ~]# curl 192.168.60.100
rs11-192.168.60.10
[root@ceshi ~]# curl 192.168.60.100
rs22-192.168.60.20[root@ceshi ~]# mysql -udll -pdll -h192.168.60.200 -e 'select @@server_id'
+-------------+
| @@server_id |
+-------------+
| 20 |
+-------------+
[root@ceshi ~]# mysql -udll -pdll -h192.168.60.200 -e 'select @@server_id'
+-------------+
| @@server_id |
+-------------+
| 10 |
+-------------+
[root@ceshi ~]# mysql -udll -pdll -h192.168.60.200 -e 'select @@server_id'
+-------------+
| @@server_id |
+-------------+
| 20 |
+-------------+
[root@ceshi ~]# mysql -udll -pdll -h192.168.60.200 -e 'select @@server_id'
+-------------+
| @@server_id |
+-------------+
| 10 |
+-------------+
[root@k1 /]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.60.100:80 rr-> 192.168.60.10:80 Route 1 0 2 -> 192.168.60.20:80 Route 1 0 3
TCP 192.168.60.200:3306 rr-> 192.168.60.10:3306 Route 1 0 0 -> 192.168.60.20:3306 Route 1 0 0 [root@k2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.60.100:80 rr persistent 50-> 192.168.60.10:80 Route 1 0 0 -> 192.168.60.20:80 Route 1 0 0
TCP 192.168.60.200:3306 rr-> 192.168.60.10:3306 Route 1 0 2 -> 192.168.60.20:3306 Route 1 0 2 #关闭k1 keepalived服务
[root@k1 /]# systemctl stop keepalived.service
测试 可以看到web服务仍可以访问
[root@ceshi ~]# curl 192.168.60.100
rs11-192.168.60.10
[root@ceshi ~]# curl 192.168.60.100
rs22-192.168.60.20
[root@ceshi ~]# curl 192.168.60.100
rs11-192.168.60.10
[root@ceshi ~]# curl 192.168.60.100
rs22-192.168.60.20
[root@ceshi ~]# curl 192.168.60.100
#在k2上查看
[root@k2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.60.100:80 rr-> 192.168.60.10:80 Route 1 0 3 -> 192.168.60.20:80 Route 1 0 3
TCP 192.168.60.200:3306 rr-> 192.168.60.10:3306 Route 1 0 0 -> 192.168.60.20:3306 Route 1 0 0