LVS Tutorial

• LVS is only aware of TCP layer/IP layer, not aware of MAC layer
• LVS is a natural module in Linux Kernel, "ipvsadm" utility is used to manage this module
• Three work mode: Nat(m), Tunnel(g), Direct-Routing(i)

NAT Work Mode:

(CSRC > DR-IP)  ---- [ LVS DR ] --- (CSRC > REAL-IP) --- [ REAL-SERVER] --- (REAL-IP > CSRC) --- [ LVS DR] --- ( DR-IP > CSRC)

Note: 

1. [ REAL-SERVER ] : [ LVS DR] should be config as a gateway to all CSRC in 

2. [ LVS DR ]: ip_forward show be open, because [ LVS DR ] will be a gate way to [ REAL-SERVER ]

3. A VIP is no really required

Direct-Routing Mode:

(CSRC > VIP) -- [ LVS DR ] -- (CSRC > VIP) -- [ REAL-SERVER ] -- ( VIP > CSRC )

Note:

1. VIP should be bind in [ LVS DR ] and all [ REAL - SERVERS ] 

2. [ REAL-SERVER ] : arp query to VIP should be disable

Tunel Mode:

(CSRC > VIP) -- [ LVS DR ] -- (DR-IP > REAL-IP)[encapsulated CSRC > VIP ] -- [ REAL-SERVER ] -- ( VIP > CSRC )

Note:

1. VIP should be bind in [ LVS DR ] and all [ REAL - SERVERS ] 

2. [ REAL-SERVER ] : arp query to VIP should be disable

sched.sh:

export VIP=172.17.60.201
export PORT=8000
export NODES="192.168.135.80"
export TYPE='i'

# /sbin/ifconfig eth0:1 down
/sbin/ifconfig tunl0 down
# /sbin/ifconfig eth0:1 $VIP broadcast $VIP netmask 255.255.255.255 up 
/sbin/ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up
# /sbin/route add -host $VIP dev eth0:1 
/sbin/route add -host $VIP dev tunl0

/sbin/ipvsadm -C 
/sbin/ipvsadm -A -t $VIP:$PORT
for NODE in $NODES; do
    /sbin/ipvsadm -a -t $VIP:$PORT -r $NODE:$PORT -$TYPE
done

node.sh:

export VIP=172.17.60.201

/sbin/ifconfig lo:0 down
# /sbin/ifconfig tunl0 down
/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
# /sbin/ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:0
# /sbin/route add -host $VIP dev tunl0

# echo "1" > /proc/sys/net/ipv4/conf/tunl0/arp_ignore
# echo "2" > /proc/sys/net/ipv4/conf/tunl0/arp_announce
echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore 
echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce 
echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore 
echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce

# net.ipv4.conf.all.arp_ignore = 1
# net.ipv4.conf.all.arp_announce = 2
# net.ipv4.conf.lo.arp_ignore = 1
# net.ipv4.conf.lo.arp_announce = 2

sysctl -p

=================

keepalived is another tool other than ipvsadm to manage LVS

* LVS schedule schema is config in file keepalived.conf

* dynamically add/remove realserver entry according to realserver health condition

* have a support for master-slave director failed switch

keepalived installation:

yum install kernel-headers kernel-devel

wget keepalived-1.1.20.tar.gz ( keepalive-1.2 can not be build in centos-5)

./configure --with-kernel-dir=/lib/modules/$(uname -r)/build (kernel headers needed for LVS management)

ln -s /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
ln -s /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
ln -s /usr/local/sbin/keepalived /sbin/

=================

two box LVS

* let heartbeat to manage LVS VIP on eth0 between to box

* LVS is not installed on slave, slave only configed lo:0 LVS realserver VIP

* keepalived is installed on master, but only used to manage LVS schedule schema and realserver failed condition. keepalived knows nothing about VIP