openstack havana块存储Cinder磁盘加密方法研究

在openstack havana的release note中有如下介绍“Attached Cinder volumes can now be encrypted. Data is decrypted as needed at read and write time while presenting instances with a normal block storage device”。

众所周知，在以前版本的openstack中，块存储创建出volume后，将其挂载给虚拟机，就可以启动虚拟机。但这种实现方式也存在一些安全隐患：若存在不怀好意的系统管理员或者黑客在攻陷nova节点时，就可以从宿主系统中，查看guest os中用户存放的数据。如果用户在其磁盘中存在敏感私人数据时，则存在数据泄密的风险。

关于磁盘加密的BluePrint链接：https://wiki.openstack.org/wiki/VolumeEncryption ，通过分析其实现方式可以发现，cinder中的卷在挂载到HyperVisor主机上时，对其进行加密，再讲加密后的块设备提供给虚拟机使用，最终虚拟机使用的加密后的磁盘。如果此时黑客攻陷主机的话，他看到的将是加密后的volume，如果不知道磁盘的加密密码，他看到将是没有任何意义的一堆数据。

通过分析nova中代码实现，可以发现磁盘加密的实现细节，openstack中的磁盘加密使用了Linux中的一个加密组件：cryptsetup！

    def _open_volume(self, passphrase, **kwargs):        """Opens the LUKS partition on the volume using the specified        passphrase.        :param passphrase: the passphrase used to access the volume        """        LOG.debug(_("opening encrypted volume %s"), self.dev_path)        # NOTE(joel-coffman): cryptsetup will strip trailing newlines from        # input specified on stdin unless --key-file=- is specified.        cmd = ["cryptsetup", "create", "--key-file=-"]        cipher = kwargs.get("cipher", None)        if cipher is not None:            cmd.extend(["--cipher", cipher])        key_size = kwargs.get("key_size", None)        if key_size is not None:            cmd.extend(["--key-size", key_size])        cmd.extend([self.dev_name, self.dev_path])        utils.execute(*cmd, process_input=passphrase,                      check_exit_code=True, run_as_root=True)    def attach_volume(self, context, **kwargs):        """Shadows the device and passes an unencrypted version to the        instance.        Transparent disk encryption is achieved by mounting the volume via        dm-crypt and passing the resulting device to the instance. The        instance is unaware of the underlying encryption due to modifying the        original symbolic link to refer to the device mounted by dm-crypt.        """        key = self._get_key(context).get_encoded()        passphrase = self._get_passphrase(key)        self._open_volume(passphrase, **kwargs)        # modify the original symbolic link to refer to the decrypted device        utils.execute('ln', '--symbolic', '--force',                      '/dev/mapper/%s' % self.dev_name, self.symlink_path,                      run_as_root=True, check_exit_code=True)

可见，在attach_volume方法的_open_volume中，会调用cryptsetup系统命令对输入的待挂载设备进行加密，再将加密的设备生成挂载给虚拟机的软链接，这样虚拟机在启动时感知不到加密方法的存在，即磁盘加密对虚拟机是透明的。cryptsetup

由于手头没有havana的环境，下面打算根据cinder的实现原理对磁盘加密进行模拟，同时研究其实现：

1、首先使用truncate命令生成一个空洞文件

[root@armstrong tmp]# truncate --size=10G 10G[root@armstrong tmp]#  ll /tmp/10G-rw-r--r--. 1 root root 10737418240 12月  3 23:37 /tmp/10G[root@armstrong tmp]#

2、将上述空洞文件转化为loop设备

[root@armstrong tmp]#[root@armstrong tmp]# losetup -f /tmp/10G[root@armstrong tmp]#[root@armstrong tmp]# losetup -a/dev/loop0: [0030]:33006 (/tmp/10G)[root@armstrong tmp]#

3、在上述loop设备上创建pv、vg

[root@armstrong tmp]# pvcreate qixiaozhen /dev/loop0  Device qixiaozhen not found (or ignored by filtering).  Physical volume "/dev/loop0" successfully created[root@armstrong tmp]#[root@armstrong tmp]# vgcreate qixiaozhen /dev/loop0  Volume group "qixiaozhen" successfully created[root@armstrong tmp]# 

4、创建用以测试用的lv

[root@armstrong tmp]# lvcreate -n test001 -L 100M qixiaozhen  Logical volume "test001" created[root@armstrong tmp]#

5、使用cryptsetup对上述lv进行加密（注意此处需要用户输入密码）

[root@armstrong tmp]#[root@armstrong tmp]# cryptsetup luksFormat /dev/qixiaozhen/test001WARNING!========This will overwrite data on /dev/qixiaozhen/test001 irrevocably.Are you sure? (Type uppercase yes): YESEnter LUKS passphrase:Verify passphrase:[root@armstrong tmp]#

6、打开加密后的磁盘，生成供虚拟机使用磁盘（需要输入密码）

[root@armstrong tmp]#[root@armstrong tmp]# cryptsetup luksOpen /dev/qixiaozhen/test001 qixiaozhen_deviceEnter passphrase for /dev/qixiaozhen/test001:[root@armstrong tmp]#

7、在/dev/mapper路径下可以找到上述生成的qixiaozhen_device设备

[root@armstrong tmp]#[root@armstrong tmp]# ll /dev/mapper/qixiaozhen_devicelrwxrwxrwx. 1 root root 7 12月  3 23:48 /dev/mapper/qixiaozhen_device -> ../dm-4[root@armstrong tmp]#[root@armstrong tmp]#

8、分析逻辑卷test001与qixiaozhen_device设备间的关系

[root@armstrong tmp]#[root@armstrong tmp]# dmsetup tablefedora_armstrong-swap: 0 12255232 linear 8:9 2048fedora_armstrong-root: 0 104857600 linear 8:9 995198976qixiaozhen-test001: 0 204800 linear 7:0 2048qixiaozhen_device: 0 200704 crypt aes-cbc-essiv:sha256 0000000000000000000000000000000000000000000000000000000000000000 0 253:3 4096fedora_armstrong-home: 0 982941696 linear 8:9 12257280[root@armstrong tmp]#

可见qixiaozhen_device设备是由test001设备偏移4096个扇区（2MB）而生成。

9、往qixiaozhen_device中写入部分数据，本文中通过格式化成ext4文件系统进行测试

[root@armstrong tmp]# mkfs.ext4 /dev/mapper/qixiaozhen_devicemke2fs 1.42.5 (29-Jul-2012)文件系统标签=OS type: Linux块大小=1024 (log=0)分块大小=1024 (log=0)Stride=0 blocks, Stripe width=0 blocks25168 inodes, 100352 blocks5017 blocks (5.00%) reserved for the super user第一个数据块=1Maximum filesystem blocks=6737100813 block groups8192 blocks per group, 8192 fragments per group1936 inodes per groupSuperblock backups stored on blocks:        8193, 24577, 40961, 57345, 73729Allocating group tables: 完成正在写入inode表: 完成Creating journal (4096 blocks): 完成Writing superblocks and filesystem accounting information: 完成[root@armstrong tmp]#

10、下面分析使用dd命令读取test001偏移2MB和qixiaozhen_device进行比较。

[root@armstrong tmp]# dd if=/dev/qixiaozhen/test001 bs=1M count=1 iflag=direct  skip=2 | hexdump -C | less00000000  4c a7 db 72 01 e4 18 1e  bc 1e b9 bf fa 3b 25 01  |L..r.........;%.|00000010  cc 22 46 96 aa b9 e8 2c  53 60 7c b3 61 4a 02 33  |."F....,S`|.aJ.3|00000020  5d 39 4b 4f 6d b9 5a 57  16 fb a5 f9 c8 7f 9d 65  |]9KOm.ZW.......e|00000030  3d 25 38 f5 4b e2 b0 6e  ef 92 24 bd fc cf 56 07  |=%8.K..n..$...V.|00000040  4b 5f d9 85 ad f4 01 5d  dd a3 94 db 8b 58 0f a0  |K_.....].....X..|00000050  03 0f 08 ad 7a 66 a8 3a  72 16 47 58 0c ba f9 d0  |....zf.:r.GX....|00000060  60 34 b4 ba 69 32 27 8c  f8 97 cd 58 86 3e ce 34  |`4..i2'....X.>.4|00000070  ae 28 4f e1 c8 a9 90 5c  08 f9 30 9f a7 4b 6b 21  |.(O....\..0..Kk!|00000080  ba d0 39 45 7a 0d a4 c5  a6 c9 a8 40 a2 56 c9 27  |..9Ez......@.V.'|00000090  9d d4 77 05 77 ae b0 2a  eb fa 3a 1d 1b d0 19 9d  |..w.w..*..:.....|000000a0  fe c3 d9 1c ca 3c 51 fa  84 d7 d8 a8 8d 42 1f e0  |.....<Q......B..|000000b0  bd b0 09 84 69 25 6d 70  9f b6 78 25 87 52 07 70  |....i%mp..x%.R.p|000000c0  e1 9c d9 9c 42 2d 1d fb  92 fd 9d 62 d3 58 27 79  |....B-.....b.X'y|000000d0  2c b3 a5 a8 67 7d 3e 5d  01 15 3d ac 7e 44 77 67  |,...g}>]..=.~Dwg|000000e0  8f 4f 71 e9 4e 63 06 4d  9a 78 7c 0f 60 4a da 3b  |.Oq.Nc.M.x|.`J.;|000000f0  0d f2 46 ac c0 c7 44 0c  61 dc 6c ef 00 c0 7b 9f  |..F...D.a.l...{.|00000100  60 55 94 7f 1f c2 87 1b  10 93 c0 78 92 6a 8f bf  |`U.........x.j..|00000110  6d 08 b1 b7 51 86 6e 83  46 26 cb fe 0d 0f c4 72  |m...Q.n.F&.....r|00000120  b1 48 88 c5 05 be a9 f1  8a ac a2 ed f6 73 40 f5  |.H...........s@.|00000130  74 51 76 ba f4 db 8c be  f8 9d 73 14 e8 7b 8c 0f  |tQv.......s..{..|00000140  06 bb 89 f9 e4 2e 81 8e  5f c8 f4 a2 70 27 03 a5  |........_...p'..|00000150  3e 93 80 b1 38 4d ad 12  9f ca 4c e8 01 5b 26 41  |>...8M....L..[&A|00000160  73 f9 3f db 75 c5 d5 d9  c7 25 b9 6d 03 4d 39 98  |s.?.u....%.m.M9.|00000170  72 7e 5f 10 31 be e9 92  fa 5c d5 54 9f 61 65 00  |r~_.1....\.T.ae.|00000180  bd e0 4a 8f 8a 84 b4 41  5d b5 6c 42 f5 d5 ff 7a  |..J....A].lB...z|00000190  7f 97 c5 dc 73 74 c8 94  f7 03 f0 bd 8d 1d 69 9e  |....st........i.|000001a0  93 c4 52 52 d8 9e 21 6a  74 2d 1e d4 29 1e 1e 92  |..RR..!jt-..)...|000001b0  8a 61 03 9d ed 0c ca c0  e5 33 38 49 d3 7e 5f 94  |.a.......38I.~_.|000001c0  44 f9 ae 5d 5f 9c 5a 51  02 20 77 46 9c 46 23 6b  |D..]_.ZQ. wF.F#k|000001d0  8b 5e 15 28 4f 09 ce 3e  36 e8 62 21 e4 08 43 e9  |.^.(O..>6.b!..C.|000001e0  b3 53 90 e2 68 fb 28 4d  fa 24 e8 7a 3a ce b1 0a  |.S..h.(M.$.z:...|000001f0  f8 f1 ae a0 c9 57 82 9d  0e 56 6e f8 d9 d5 81 f8  |.....W...Vn.....|00000200  9d f6 fa 8b d2 f2 bd cf  d3 f0 ff e8 52 1f d1 f3  |............R...|00000210  c1 aa ed db 78 55 f8 e3  7c 85 bf ec d2 a3 15 ec  |....xU..|.......|00000220  8b 6d 04 cf 2d 14 37 1b  f4 22 06 5f c1 8d eb df  |.m..-.7.."._....|00000230  77 7b b9 f2 36 f7 b2 63  ad 23 06 ff 23 a1 b5 dc  |w{..6..c.#..#...|00000240  56 67 85 1f a2 f5 59 4c  cc a0 c0 97 c1 b1 7e b2  |Vg....YL......~.|00000250  e5 7f 6e 3c d2 ee 65 fa  52 10 ea b8 fb e0 38 e8  |..n<..e.R.....8.|00000260  cf 95 9e 80 72 e4 5b 22  2b 9f 72 f6 22 2f e1 f7  |....r.["+.r."/..|00000270  ec f4 49 cf c3 d2 f4 34  de 00 55 17 e9 1c cf 5a  |..I....4..U....Z|00000280  19 7f a0 f4 48 14 f5 ee  2f 75 76 ca 0e 94 9f 16  |....H.../uv.....|00000290  51 aa 80 03 a9 e6 2c 69  31 d4 35 d5 d5 b7 e9 58  |Q.....,i1.5....X|000002a0  bc a1 f5 8f 56 bc 12 94  9e d7 d8 cc 7d f7 61 bd  |....V.......}.a.|

[root@armstrong tmp]# dd if=/dev/mapper/qixiaozhen_device bs=1M count=1 iflag=direct | hexdump -C | less00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|*00000400  50 62 00 00 00 88 01 00  99 13 00 00 81 65 01 00  |Pb...........e..|00000410  45 62 00 00 01 00 00 00  00 00 00 00 00 00 00 00  |Eb..............|00000420  00 20 00 00 00 20 00 00  90 07 00 00 00 00 00 00  |. ... ..........|00000430  8c ff 9d 52 00 00 ff ff  53 ef 01 00 01 00 00 00  |...R....S.......|00000440  8c ff 9d 52 00 00 00 00  00 00 00 00 01 00 00 00  |...R............|00000450  00 00 00 00 0b 00 00 00  80 00 00 00 3c 00 00 00  |............<...|00000460  42 02 00 00 79 00 00 00  0e 6c 42 94 c7 e6 4b 18  |B...y....lB...K.|00000470  99 55 8d 81 b6 2c 5a 9a  00 00 00 00 00 00 00 00  |.U...,Z.........|00000480  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|*000004c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 01  |................|000004d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|000004e0  08 00 00 00 00 00 00 00  00 00 00 00 25 2a 68 6f  |............%*ho|000004f0  14 38 4a d7 b2 35 fb 62  20 5c ec 36 01 01 00 00  |.8J..5.b \.6....|00000500  0c 00 00 00 00 00 00 00  8c ff 9d 52 0a f3 01 00  |...........R....|00000510  04 00 00 00 00 00 00 00  00 00 00 00 00 10 00 00  |................|00000520  01 c0 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|00000530  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|00000540  00 00 00 00 00 00 00 00  00 00 00 00 00 00 40 00  |..............@.|00000550  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|00000560  01 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|00000570  00 00 00 00 04 00 00 00  5f 11 00 00 00 00 00 00  |........_.......|00000580  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|*00000800  03 01 00 00 13 01 00 00  23 01 00 00 8c 12 85 07  |........#.......|00000810  02 00 00 00 00 00 00 00  00 00 00 00 85 07 b9 cc  |................|00000820  04 01 00 00 14 01 00 00  15 02 00 00 fe 1e 90 07  |................|00000830  00 00 01 00 00 00 00 00  00 00 00 00 90 07 f4 ce  |................|00000840  05 01 00 00 15 01 00 00  07 03 00 00 00 20 90 07  |............. ..|00000850  00 00 03 00 00 00 00 00  00 00 00 00 90 07 77 60  |..............w`|00000860  06 01 00 00 16 01 00 00  f9 03 00 00 fe 1e 90 07  |................|00000870  00 00 01 00 00 00 00 00  00 00 00 00 90 07 36 89  |..............6.|00000880  07 01 00 00 17 01 00 00  eb 04 00 00 00 20 90 07  |............. ..|00000890  00 00 03 00 00 00 00 00  00 00 00 00 90 07 d2 35  |...............5|000008a0  08 01 00 00 18 01 00 00  dd 05 00 00 fe 1e 90 07  |................|000008b0  00 00 01 00 00 00 00 00  00 00 00 00 90 07 e0 b7  |................|000008c0  09 01 00 00 19 01 00 00  cf 06 00 00 00 10 90 07  |................|000008d0  00 00 01 00 00 00 00 00  00 00 00 00 90 07 ad 33  |...............3|000008e0  0a 01 00 00 1a 01 00 00  c1 07 00 00 fe 1e 90 07  |................|000008f0  00 00 01 00 00 00 00 00  00 00 00 00 90 07 ee 88  |................|00000900  0b 01 00 00 1b 01 00 00  b3 08 00 00 00 20 90 07  |............. ..|

结论：可见，直接在主机上读取裸设备上的裸数据时，从加密前设备和加密后设备上的是完全不同的。使用磁盘加密的方法，在删除磁盘时，不需要对磁盘进行格式化，直接把加密用的密码删除即可，再也不用往裸设备中填0实现格式化。