Linux 下磁盘分区偏移对于磁盘性能的影响

操作环境：

OpenStack Havana

Centos 6.0

fio 2.1.2

 

本次通过centos6下磁盘分区的偏移不同，测试偏移不同对于磁盘性能的影响

 

首先使用默认创建磁盘分区方式（默认情况下偏移量为63 block），磁盘分区不对齐

libaio.dat: (groupid=0, jobs=1): err= 0: pid=4777: Thu Dec 19 03:14:48 2013  read : io=6194.5MB, bw=10571KB/s, iops=165, runt=600052msec    clat (usec): min=902, max=1535.4K, avg=3496.93, stdev=23268.84     lat (usec): min=902, max=1535.4K, avg=3497.34, stdev=23268.84    clat percentiles (usec):     |  1.00th=[  956],  5.00th=[  972], 10.00th=[  988], 20.00th=[  996],     | 30.00th=[ 1004], 40.00th=[ 1020], 50.00th=[ 1032], 60.00th=[ 1048],     | 70.00th=[ 1080], 80.00th=[ 1176], 90.00th=[ 8160], 95.00th=[10048],     | 99.00th=[13120], 99.50th=[42752], 99.90th=[374784], 99.95th=[561152],     | 99.99th=[888832]    bw (KB  /s): min=   63, max=32256, per=100.00%, avg=11429.35, stdev=8259.00  write: io=6202.9MB, bw=10585KB/s, iops=165, runt=600052msec    clat (msec): min=1, max=1231, avg= 2.53, stdev=16.29     lat (msec): min=1, max=1231, avg= 2.54, stdev=16.29    clat percentiles (usec):     |  1.00th=[ 1080],  5.00th=[ 1096], 10.00th=[ 1112], 20.00th=[ 1128],     | 30.00th=[ 1144], 40.00th=[ 1144], 50.00th=[ 1160], 60.00th=[ 1192],     | 70.00th=[ 1208], 80.00th=[ 1240], 90.00th=[ 1496], 95.00th=[ 8512],     | 99.00th=[15552], 99.50th=[18048], 99.90th=[228352], 99.95th=[362496],     | 99.99th=[831488]    bw (KB  /s): min=   53, max=32768, per=100.00%, avg=11476.66, stdev=8307.46    lat (usec) : 1000=12.61%    lat (msec) : 2=74.04%, 4=0.86%, 10=8.17%, 20=3.83%, 50=0.12%    lat (msec) : 100=0.09%, 250=0.13%, 500=0.09%, 750=0.03%, 1000=0.01%    lat (msec) : 2000=0.01%  cpu          : usr=0.63%, sys=1.52%, ctx=199539, majf=0, minf=26  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%     issued    : total=r=99111/w=99245/d=0, short=r=0/w=0/d=0Run status group 0 (all jobs):   READ: io=6194.5MB, aggrb=10570KB/s, minb=10570KB/s, maxb=10570KB/s, mint=600052msec, maxt=600052msec  WRITE: io=6202.9MB, aggrb=10585KB/s, minb=10585KB/s, maxb=10585KB/s, mint=600052msec, maxt=600052msecDisk stats (read/write):  vdc: ios=99036/99425, merge=0/181, ticks=342033/465754, in_queue=810322, util=98.37%

 

在磁盘分区前63个block是用于存放MBR信息的，下面设置为偏移量为64 block进行性能测试

设置偏移量为64 block，操作如下：

[root@centos6 ~]# fdisk /dev/vdcWARNING: DOS-compatible mode is deprecated. It's strongly recommended to         switch off the mode (command 'c') and change display units to         sectors (command 'u').Command (m for help): nCommand action   e   extended   p   primary partition (1-4)pPartition number (1-4): 1First cylinder (1-624152, default 1): Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-624152, default 624152): Using default value 624152Command (m for help): xExpert command (m for help): bPartition number (1-4): 1New beginning of data (1-629145215, default 63): 64Expert command (m for help): pDisk /dev/vdc: 16 heads, 63 sectors, 624152 cylindersNr AF  Hd Sec  Cyl  Hd Sec  Cyl     Start      Size ID 1 00   1   1    0  15  63 1023         64  629145152 83 2 00   0   0    0   0   0    0          0          0 00 3 00   0   0    0   0   0    0          0          0 00 4 00   0   0    0   0   0    0          0          0 00

测试结果如下：

 

libaio.dat: (groupid=0, jobs=1): err= 0: pid=6786: Thu Dec 19 03:56:57 2013  read : io=8240.4MB, bw=14063KB/s, iops=219, runt=600001msec    clat (usec): min=913, max=1621.6K, avg=3340.33, stdev=23281.19     lat (usec): min=913, max=1621.6K, avg=3340.74, stdev=23281.19    clat percentiles (usec):     |  1.00th=[  964],  5.00th=[  980], 10.00th=[  988], 20.00th=[  996],     | 30.00th=[ 1012], 40.00th=[ 1020], 50.00th=[ 1032], 60.00th=[ 1048],     | 70.00th=[ 1080], 80.00th=[ 1112], 90.00th=[ 7328], 95.00th=[ 9664],     | 99.00th=[12864], 99.50th=[52480], 99.90th=[358400], 99.95th=[497664],     | 99.99th=[897024]    bw (KB  /s): min=   62, max=33792, per=100.00%, avg=15060.70, stdev=9208.06  write: io=8228.9MB, bw=14044KB/s, iops=219, runt=600001msec    clat (msec): min=1, max=906, avg= 1.19, stdev= 2.74     lat (msec): min=1, max=906, avg= 1.20, stdev= 2.74    clat percentiles (usec):     |  1.00th=[ 1096],  5.00th=[ 1112], 10.00th=[ 1112], 20.00th=[ 1128],     | 30.00th=[ 1144], 40.00th=[ 1144], 50.00th=[ 1160], 60.00th=[ 1176],     | 70.00th=[ 1192], 80.00th=[ 1208], 90.00th=[ 1240], 95.00th=[ 1272],     | 99.00th=[ 1368], 99.50th=[ 1640], 99.90th=[ 2704], 99.95th=[ 3472],     | 99.99th=[ 8896]    bw (KB  /s): min=   45, max=32000, per=100.00%, avg=15107.37, stdev=9207.78    lat (usec) : 1000=10.87%    lat (msec) : 2=81.83%, 4=0.45%, 10=4.71%, 20=1.80%, 50=0.07%    lat (msec) : 100=0.08%, 250=0.08%, 500=0.07%, 750=0.02%, 1000=0.01%    lat (msec) : 2000=0.01%  cpu          : usr=0.85%, sys=2.00%, ctx=264954, majf=0, minf=26  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%     issued    : total=r=131845/w=131662/d=0, short=r=0/w=0/d=0Run status group 0 (all jobs):   READ: io=8240.4MB, aggrb=14063KB/s, minb=14063KB/s, maxb=14063KB/s, mint=600001msec, maxt=600001msec  WRITE: io=8228.9MB, aggrb=14043KB/s, minb=14043KB/s, maxb=14043KB/s, mint=600001msec, maxt=600001msecDisk stats (read/write):  vdc: ios=131804/131853, merge=0/177, ticks=434689/446135, in_queue=880684, util=98.54%

 

设置偏移量为128 block进行性能测试：

设置步骤如下：

[root@centos6 ~]# fdisk /dev/vdcWARNING: DOS-compatible mode is deprecated. It's strongly recommended to         switch off the mode (command 'c') and change display units to         sectors (command 'u').Command (m for help): nCommand action   e   extended   p   primary partition (1-4)pPartition number (1-4): 1First cylinder (1-624152, default 1): Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-624152, default 624152): Using default value 624152Command (m for help): xExpert command (m for help): bPartition number (1-4): 1New beginning of data (1-629145215, default 63): 128Expert command (m for help): pDisk /dev/vdc: 16 heads, 63 sectors, 624152 cylindersNr AF  Hd Sec  Cyl  Hd Sec  Cyl     Start      Size ID 1 00   1   1    0  15  63 1023        128  629145088 83 2 00   0   0    0   0   0    0          0          0 00 3 00   0   0    0   0   0    0          0          0 00 4 00   0   0    0   0   0    0          0          0 00Expert command (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.

测试结果如下：

libaio.dat: Laying out IO file(s) (1 file(s) / 4096MB)Jobs: 1 (f=1): [m] [100.0% done] [26752KB/30528KB/0KB /s] [418/477/0 iops] [eta 00m:00s]libaio.dat: (groupid=0, jobs=1): err= 0: pid=7836: Thu Dec 19 04:29:07 2013  read : io=10562MB, bw=18025KB/s, iops=281, runt=600002msec    clat (usec): min=907, max=855615, avg=2343.27, stdev=12381.39     lat (usec): min=907, max=855616, avg=2343.68, stdev=12381.39    clat percentiles (usec):     |  1.00th=[  956],  5.00th=[  980], 10.00th=[  988], 20.00th=[  996],     | 30.00th=[ 1012], 40.00th=[ 1020], 50.00th=[ 1032], 60.00th=[ 1048],     | 70.00th=[ 1064], 80.00th=[ 1096], 90.00th=[ 4128], 95.00th=[ 8384],     | 99.00th=[11328], 99.50th=[16512], 99.90th=[183296], 99.95th=[292864],     | 99.99th=[489472]    bw (KB  /s): min=  114, max=33024, per=100.00%, avg=18514.19, stdev=9072.17  write: io=10531MB, bw=17973KB/s, iops=280, runt=600002msec    clat (usec): min=1046, max=277697, avg=1184.54, stdev=948.23     lat (usec): min=1056, max=277713, avg=1196.02, stdev=948.30    clat percentiles (usec):     |  1.00th=[ 1080],  5.00th=[ 1112], 10.00th=[ 1112], 20.00th=[ 1128],     | 30.00th=[ 1144], 40.00th=[ 1160], 50.00th=[ 1160], 60.00th=[ 1176],     | 70.00th=[ 1192], 80.00th=[ 1208], 90.00th=[ 1240], 95.00th=[ 1272],     | 99.00th=[ 1368], 99.50th=[ 1816], 99.90th=[ 2640], 99.95th=[ 3056],     | 99.99th=[ 8256]    bw (KB  /s): min=  118, max=32000, per=100.00%, avg=18474.97, stdev=9036.99    lat (usec) : 1000=10.97%    lat (msec) : 2=83.13%, 4=0.82%, 10=3.68%, 20=1.18%, 50=0.06%    lat (msec) : 100=0.06%, 250=0.07%, 500=0.03%, 750=0.01%, 1000=0.01%  cpu          : usr=1.10%, sys=2.59%, ctx=339298, majf=0, minf=25  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%     issued    : total=r=168987/w=168497/d=0, short=r=0/w=0/d=0Run status group 0 (all jobs):   READ: io=10562MB, aggrb=18025KB/s, minb=18025KB/s, maxb=18025KB/s, mint=600002msec, maxt=600002msec  WRITE: io=10531MB, aggrb=17972KB/s, minb=17972KB/s, maxb=17972KB/s, mint=600002msec, maxt=600002msecDisk stats (read/write):  vdc: ios=168874/168617, merge=0/145, ticks=388717/445504, in_queue=838295, util=98.15%

 

测试分析：

从测试结果可以得出结论是，在数据请求快大小为64KB的情况下，磁盘分区偏移量在128 block的时候性能是最好的，默认 63 block的时候是最差的。