负载均衡LVS(DR模式)安装实战
来源:互联网 发布:密度峰值聚类算法 编辑:程序博客网 时间:2024/05/28 15:29
1.编译安装ipvsadm
首先从LVS官网下载tarball,解压后make && make install即可。
要注意的是LVS的依赖有:popt-static、libnl、kernel。如果yum源里有就直接安装,如果没有就自行下载,但一定要对应上已有软件如popt或Linux内核的版本。
$ tar -xzvf ipvsadm-1.26.tar.gz$ cd ipvsadm-1.26$ makemake -C libipvsmake[1]: Entering directory `/root/lvs/ipvsadm-1.26/libipvs'gcc -Wall -Wunused -Wstrict-prototypes -g -fPIC -DLIBIPVS_USE_NL -DHAVE_NET_IP_VS_H -c -o libipvs.o libipvs.cgcc -Wall -Wunused -Wstrict-prototypes -g -fPIC -DLIBIPVS_USE_NL -DHAVE_NET_IP_VS_H -c -o ip_vs_nl_policy.o ip_vs_nl_policy.car rv libipvs.a libipvs.o ip_vs_nl_policy.oar: creating libipvs.aa - libipvs.oa - ip_vs_nl_policy.ogcc -shared -Wl,-soname,libipvs.so -o libipvs.so libipvs.o ip_vs_nl_policy.omake[1]: Leaving directory `/root/lvs/ipvsadm-1.26/libipvs'gcc -Wall -Wunused -Wstrict-prototypes -g -DVERSION=\"1.26\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DPE_LIST=\""sip"\" -DHAVE_POPT -DHAVE_NET_IP_VS_H -c -o ipvsadm.o ipvsadm.cipvsadm.c: In function âprint_largenumâ:ipvsadm.c:1383: warning: field width should have type âintâ, but argument 2 has type âsize_tâgcc -Wall -Wunused -Wstrict-prototypes -g -DVERSION=\"1.26\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DPE_LIST=\""sip"\" -DHAVE_POPT -DHAVE_NET_IP_VS_H -c -o config_stream.o config_stream.cgcc -Wall -Wunused -Wstrict-prototypes -g -DVERSION=\"1.26\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DPE_LIST=\""sip"\" -DHAVE_POPT -DHAVE_NET_IP_VS_H -c -o dynamic_array.o dynamic_array.cgcc -Wall -Wunused -Wstrict-prototypes -g -o ipvsadm ipvsadm.o config_stream.o dynamic_array.o libipvs/libipvs.a -lpopt -lnl
最后make install,安装ipvsadm、ipvsadm-save等命令,并添加ipvsadm到系统服务,方便了后续使用。
$ make installmake -C libipvsmake[1]: Entering directory `/root/lvs/ipvsadm-1.26/libipvs'make[1]: Nothing to be done for `all'.make[1]: Leaving directory `/root/lvs/ipvsadm-1.26/libipvs'if [ ! -d /sbin ]; then mkdir -p /sbin; fiinstall -m 0755 ipvsadm /sbininstall -m 0755 ipvsadm-save /sbininstall -m 0755 ipvsadm-restore /sbin[ -d /usr/man/man8 ] || mkdir -p /usr/man/man8install -m 0644 ipvsadm.8 /usr/man/man8install -m 0644 ipvsadm-save.8 /usr/man/man8install -m 0644 ipvsadm-restore.8 /usr/man/man8[ -d /etc/rc.d/init.d ] || mkdir -p /etc/rc.d/init.dinstall -m 0755 ipvsadm.sh /etc/rc.d/init.d/ipvsadm
最后,关闭防火墙和selinux:service iptables stop && setenforce 0
2.DR模式配置
2.1 Ipvsadm参数解释
$ ipvsadm -C$ ipvsadm -A -t 192.168.100.20:80 -s rr $ ipvsadm -a -t 192.168.100.20:80 -r 192.168.100.116:80 -g -p60 $ ipvsadm -a -t 192.168.100.20:80 -r 192.168.100.117:80 -g -p60
ipvsadm参数含义:
- -C:清除已有规则。
- -A:添加VIP服务,后跟服务的访问地址。
- -t:TCP协议,还是UDP协议(-u)。
- -s:负载均衡算法,rr表示RoundRobin。
- -a:添加RealServer到VIP,后跟虚地址。
- -r:添加RealServer到VIP,后跟实地址。
- -g:透传模式(-g表示Direct Routing即DR模式,-i表示ipip封装即Tunneling模式,-m表示Network Access Translation即NAT模式)
- -p:Session粘连,同一客户端的请求在一段时间内都负载到同一RealServer。
2.2 Director配置
使用DR模式必须在主机上开启IP转发。
$ vi /etc/sysctl.conf ...# Controls IP packet forwardingnet.ipv4.ip_forward = 1net.ipv4.conf.all.send_redirects = 0net.ipv4.conf.default.send_redirects = 0net.ipv4.conf.em1.send_redirects = 0 ...$ sysctl -pnet.ipv4.ip_forward = 1net.ipv4.conf.all.send_redirects = 0net.ipv4.conf.default.send_redirects = 0net.ipv4.conf.em1.send_redirects = 0net.ipv4.conf.default.rp_filter = 1net.ipv4.conf.default.accept_source_route = 0kernel.sysrq = 0kernel.core_uses_pid = 1net.ipv4.tcp_syncookies = 1error: "net.bridge.bridge-nf-call-ip6tables" is an unknown keyerror: "net.bridge.bridge-nf-call-iptables" is an unknown keyerror: "net.bridge.bridge-nf-call-arptables" is an unknown keykernel.msgmnb = 65536kernel.msgmax = 65536kernel.shmmax = 68719476736kernel.shmall = 4294967296vm.swappiness = 0
网络配置:
$ ifconfig em1:0 192.168.242.101 broadcast 192.168.242.255 netmask 255.255.255.0 up[root@node12 lvs]# ifconfigem1 Link encap:Ethernet HWaddr B0:83:FE:DB:48:DF inet addr:192.168.242.102 Bcast:192.168.242.255 Mask:255.255.255.0 inet6 addr: fe80::b283:feff:fedb:48df/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 ...em1:0 Link encap:Ethernet HWaddr B0:83:FE:DB:48:DF inet addr:192.168.242.101 Bcast:192.168.242.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Interrupt:40 Memory:95000000-957ffffflo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 ...$ route add -host 192.168.242.101 dev em1:0$ routeKernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface192.168.242.101 * 255.255.255.255 UH 0 0 0 em1192.168.242.0 * 255.255.255.0 U 0 0 0 em1192.168.180.0 * 255.255.255.0 U 0 0 0 em3link-local * 255.255.0.0 U 1002 0 0 em1link-local * 255.255.0.0 U 1004 0 0 em3default 192.168.180.253 0.0.0.0 UG 0 0 0 em3
接下来用ipvsadm配置LVS:
$ ipvsadm -A -t 192.168.242.101:8088 -s rr$ ipvsadm -a -t 192.168.242.101:8088 -r 192.168.242.105:8088 -g$ ipvsadm -a -t 192.168.242.101:8088 -r 192.168.242.106:8088 -g$ ipvsadm -LnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 192.168.242.101:8088 rr -> 192.168.242.105:8088 Route 1 0 0 -> 192.168.242.106:8088 Route 1 0 0 $ ipvsadm-save > /etc/sysconfig/ipvsadm$ cat /etc/sysconfig/ipvsadm-A -t 192.168.242.101:radan-http -s rr-a -t 192.168.242.101:radan-http -r 192.168.242.105:radan-http -g -w 1-a -t 192.168.242.101:radan-http -r 192.168.242.106:radan-http -g -w 1$ service ipvsadm startClearing the current IPVS table: [ OK ]Applying IPVS configuration: [ OK ]
2.4 RealServer配置
首先配置内核屏蔽掉Director发来的ARP消息:
$ vi /etc/sysctl.conf ...net.ipv4.conf.all.arp_ignore = 1net.ipv4.conf.all.arp_announce = 2net.ipv4.conf.em1.arp_ignore = 1net.ipv4.conf.em1.arp_announce = 2 ...$ sysctl -pnet.ipv4.conf.all.arp_ignore = 1net.ipv4.conf.all.arp_announce = 2net.ipv4.conf.em1.arp_ignore = 1net.ipv4.conf.em1.arp_announce = 2net.ipv4.ip_forward = 1net.ipv4.conf.default.rp_filter = 1net.ipv4.conf.default.accept_source_route = 0kernel.sysrq = 0kernel.core_uses_pid = 1net.ipv4.tcp_syncookies = 1error: "net.bridge.bridge-nf-call-ip6tables" is an unknown keyerror: "net.bridge.bridge-nf-call-iptables" is an unknown keyerror: "net.bridge.bridge-nf-call-arptables" is an unknown keykernel.msgmnb = 65536kernel.msgmax = 65536kernel.shmmax = 68719476736kernel.shmall = 4294967296vm.swappiness = 0
将VIP添加到loopback接口,并且netmask必须是255.255.255.255。
$ ifconfig lo:0 192.168.242.101 broadcast 192.168.242.255 netmask 255.255.255.255 up$ ifconfigem1 Link encap:Ethernet HWaddr XX:XX:XX:XX:XX:XX inet addr:192.168.242.105 Bcast:192.168.242.255 Mask:255.255.255.0 inet6 addr: xxxx::xxxx:xxxx:xxxx:xxxx/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 ...lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 ...lo:0 Link encap:Local Loopback inet addr:192.168.242.102 Mask:255.255.255.255 UP LOOPBACK RUNNING MTU:16436 Metric:1$ route add -host 192.168.242.101 dev lo:0$ routeKernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface192.168.242.101 * 255.255.255.255 UH 0 0 0 lo192.168.242.0 * 255.255.255.0 U 0 0 0 em1192.168.180.0 * 255.255.255.0 U 0 0 0 em3link-local * 255.255.0.0 U 1002 0 0 em1link-local * 255.255.0.0 U 1004 0 0 em3default 192.168.180.253 0.0.0.0 UG 0 0 0 em3
因为RealServer上不需要安装启动ipvsadm服务,所以我们要手动加载ip_vs到内核(Director上是ipvsadm自动帮我们加载的):
$ modprobe ip_vs$ lsmod | grep ipip_vs 125220 0 ip6t_REJECT 4628 2 nf_conntrack_ipv6 8748 2 nf_defrag_ipv6 11182 1 nf_conntrack_ipv6nf_conntrack 79758 2 nf_conntrack_ipv6,xt_stateip6table_filter 2889 1 ip6_tables 18732 1 ip6table_filteripv6 317340 399 ip_vs,ip6t_REJECT,nf_conntrack_ipv6,nf_defrag_ipv6libcrc32c 1246 2 ip_vs,bnx2x
3.负载均衡测试
在两台RealServer上通过tcpdump tcp -A -ni em1 port 8088监听,不断访问VIP就会发现请求轮询转发到两台RealServer上,实现了负载均衡。注意监听的是em1而不是lo:1。
测试时犯了一个错误,我一直在VIP本机上测试,发现怎么都不通,以为LVS没配好于是就一直配啊配,结果从其他机器访问了一下VIP发现其实早就配好了…
经过这次配置学习,以及上面监听em1而非lo:1和VIP本机访问的问题,都证明了我对LVS和网络基础的不够熟悉,还要强化学习一下!
- 负载均衡LVS(DR模式)安装实战
- 均衡负载LVS-DR模式
- 负载均衡集群企业级应用实战:LVS-DR模式
- 负载均衡(keepalived)+lvs/dr模式
- 使用Lvs DR模式实现负载均衡
- 使用Lvs DR模式实现负载均衡
- 负载均衡之LVS/DR模式
- LVS+keepalived实现DR模式负载均衡
- Lvs负载均衡(dr模式)
- lvs负载均衡集群之DR模式
- LVS DR模式负载均衡配置
- LVS负载均衡DR模式部署
- LVS负载均衡群集--DR模式
- Linux-LVS负载均衡DR模式
- LVS的DR模式实现负载均衡
- LVS-DR负载均衡-01
- LVS-DR负载均衡-02
- LVS-DR负载均衡-03
- GCD介绍(二): 多核心的性能
- 日经春秋 20150808
- 如何从app业务逻辑提炼api接口
- svn : 如果自己忘了自己服务器上的svn库url和口令
- 日经社説 20150808 天然ガスの安定調達へアジアは連携を
- 负载均衡LVS(DR模式)安装实战
- 南邮 OJ 2046 FREETALK的小游戏
- 浅析UICollectionView的点滴
- hdu1010
- 2251POJ
- Linux在应用层通过mmap映射后,读写寄存器
- 南邮 OJ 2047 地铁环线
- hud5358 数学,指针
- 2015.8.8 暑期培训第八天