libvirt-sanlock in ubuntu配置使用

I. 编译libvirt包

libvirt-sanlock在ubuntu没有可以直接安装的包，sanlock功能无法直接使用，需要另外重新编译libvirt包，将sanlock功能编译进去，步骤如下：
注意：由于重新编译libvirt包需要拉很多生产环境server下不需要的包和组件，因此请在虚拟机或者另外一台实体机中完成编译，整个编译过程请使用普通用户。

1. 配置好源，/etc/apt/source.list使用ubuntu官方源即可，需要src的源下载源码包：

deb http://cn.archive.ubuntu.com/ubuntu/ trusty main restricted
deb-src http://cn.archive.ubuntu.com/ubuntu/ trusty main restricted
deb http://cn.archive.ubuntu.com/ubuntu/ trusty-updates main restricted
deb-src http://cn.archive.ubuntu.com/ubuntu/ trusty-updates main restricted
deb http://cn.archive.ubuntu.com/ubuntu/ trusty universe
deb-src http://cn.archive.ubuntu.com/ubuntu/ trusty universe
deb http://cn.archive.ubuntu.com/ubuntu/ trusty-updates universe
deb-src http://cn.archive.ubuntu.com/ubuntu/ trusty-updates universe
当然，你也可以自己手动下载libvirt源码包：https://launchpad.net/ubuntu/trusty/+source/libvirt

2. 更新apt源并安装build工具，注意使用普通用户：

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential devscripts

3. 获取源码包并安装build的依赖包：

apt-get source libvirt
sudo apt-get build-dep libvirt
sudo apt-get install libsanlock-dev

4. 更改source文件配置：

cd libvirt-1.2.2/debian
(1)先从rules文件开始
vim rules
添加“--with-sanlock \” 至DEB_CONFIGURE_EXTRA_FLAGS变量下并保存文件
(2)然后 修改control文件
vim control
添加“libsanlock-dev,” 至Build-Depends列表下. (在“# For make check”前添加即可.)
添加“libsanlock-client1,”至package libvirt-bin下的Depends列表下，保存

5. 回上一层目录并更新changelog：

cd ..
dch -i
选择一个编辑器打开，添加自己的更改，例如”Add sanlock“，保存

6. 编译libvirt包

debuild -us -uc -b

7. 若一切正常在上一层目录会生成5个deb的安装包：

ls ../*.deb
../libvirt-bin_1.2.2-0ubuntu13.1.18_amd64.deb ../libvirt-doc_1.2.2-0ubuntu13.1.18_all.deb ../libvirt0_1.2.2-0ubuntu13.1.18_amd64.deb
../libvirt-dev_1.2.2-0ubuntu13.1.18_amd64.deb ../libvirt0-dbg_1.2.2-0ubuntu13.1.18_amd64.deb
注：我使用的是1.2.2-0ubuntu13.1.17的源码包，所以编译过后的包版本是13.1.18

8. 至此重新编译libvirt包已经完成，将这些package安装至目标机上：

拷贝所有deb包至目标机，然后安装
sudo dpkg -i *.deb

9. 通过apt-get install -f解决依赖包问题：

sudo apt-get install -f

10. (可选)最后，将这些安装的包标志为hold，不让apt-get升级：

sudo apt-mark hold libvirt0
sudo apt-mark hold libvirt-bin
sudo apt-mark hold libvirt-dev
sudo apt-mark hold libvirt-dbg
sudo apt-mark hold libvirt-doc

II. 安装sanlock包
apt-get install libaio1 libblkid1 libc6 libgcc1 libuuid1 sanlock libsanlock-client1

III. mount一个nfs到lease dir

1. 配置NFS server：

在一台服务器上安装nfs-server相关包
sudo apt-get install nfs-kernel-server
暴露nfs server
vim /etc/exports
添加如下行，并保存
/var/lib/libvirt/images *(rw,sync,no_root_squash,no_subtree_check)
重启nfs-server服务
# /etc/init.d/nfs-kernel-server restart

• Stopping NFS kernel daemon [ OK ]
• Unexporting directories for NFS kernel daemon... [ OK ]
• Exporting directories for NFS kernel daemon... [ OK ]
• Starting NFS kernel daemon [ OK ]

查看可用的nfs server
# showmount -e
Export list for $HOST:
/var/lib/libvirt/images *

2. 在目标机上mount：

安装nfs client相关包
# apt-get install nfs-common
挂载nfs
# mount -t nfs $NFS_SERVER_IP:/var/lib/libvirt/images /var/lib/libvirt/sanlock

IV. 配置libvirt-sanlock
编辑conf文件：
# grep "lock_manager" /etc/libvirt/qemu.conf
lock_manager = "sanlock"

# vim /etc/libvirt/qemu-sanlock.conf
auto_disk_leases = 1
disk_lease_dir = "/var/lib/libvirt/sanlock"
host_id = 1
user = "sanlock"
group = "sanlock"

V. 配置sanlock
# /etc/init.d/sanlock start
Starting sanlock: [ OK ]
# ps -axuwf | grep sanlock
root 3953 2.0 5.8 325232 46748 ? SLsl 12:13 0:00 sanlock daemon
-w 0

VI. 重启libvirt-bin
# service libvirt-bin restart
libvirt-bin stop/waiting
libvirt-bin stop/post-start, (post-start) process 12953
# ps aux | grep libvirtd
root 12953 0.0 0.5 966740 20692 ? Sl 10:09 0:00 /usr/sbin/libvirtd -d
此时查看lease目录，会产生一个__LIBVIRT__DISKS__的文件，表明配置成功
# ll /var/lib/libvirt/sanlock/
-rw-r----- 1 sanlock sanlock 1048576 Mar 15 12:47 __LIBVIRT__DISKS__

VII. 新建一个disk iamge文件，使两个guest都用同一个image作为disk
# qemu-img create /var/lib/libvirt/images/disk.img 100M
Formatting '/var/lib/libvirt/images/disk.img', fmt=raw size=104857600

VIII. 使用xml方式定义虚拟机
# vim demo1.xml 添加如下xml配置
<domain type='qemu'>
<name>demo1</name>
<memory>219200</memory>
<vcpu>1</vcpu>
<os>
<type arch='x86_64'>hvm</type>
<boot dev='cdrom'/>
</os>
<devices>
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/var/lib/libvirt/images/disk.img'/>
<target dev='vda' bus='virtio'/>
</disk>
<input type='mouse' bus='ps2'/>
<graphics type='spice' autoport='yes' listen='0.0.0.0'/>
</devices>
</domain>

# vim demo2.xml
<domain type='qemu'>
<name>demo2</name>
<memory>219200</memory>
<vcpu>1</vcpu>
<os>
<type arch='x86_64'>hvm</type>
<boot dev='cdrom'/>
</os>
<devices>
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/var/lib/libvirt/images/disk.img'/>
<target dev='vda' bus='virtio'/>
</disk>
<input type='mouse' bus='ps2'/>
<graphics type='spice' autoport='yes' listen='0.0.0.0'/>
</devices>
</domain>

# virsh create demo1.xml
Domain demo1 created from demo1.xml
demo1定义成功，并处于running状态

查看lease目录，会产生一个租赁文件：
# ll /var/lib/libvirt/sanlock/
-rw------- 1 sanlock sanlock 1048576 Mar 15 11:04 9ba8f0215a51a6dc0aa077c427b749c8
此文件和/var/lib/libvirt/images/disk.img一一对应

# virsh create demo2.xml
error: Failed to create domain from demo2.xml
error: Failed to acquire lock: File exists
sanlock工作：由于demo1使用了同一个image并处于running状态，demo2不可使用