flash cache tier下放flush实验

来源：互联网发布：c语言命名规则编辑：程序博客网时间：2024/05/19 02:43

创建3台vm

分别是ceph01、ceph02、ceph03

a)后台手动部署ceph

b)后台部署完毕后创建一个HDD池

rados mkpool HDD

然后定制crushmap

1.ceph osd getcrushmap -o crushmap 获取当前crushmap

2.crushtool -d crushmap -o crushmap.txt 将二进制的crushmap转换成文本格式便于修改

# begin crush map

tunable choose_local_tries 0

tunable choose_local_fallback_tries 0

tunable choose_total_tries 50

tunable chooseleaf_descend_once 1

# devices

device 0 osd.0

device 1 osd.1

device 2 osd.2

device 3 osd.3

device 4 osd.4

device 5 osd.5

# types

type 0 osd

type 1 host

type 2 chassis

type 3 rack

type 4 row

type 5 pdu

type 6 pod

type 7 room

type 8 datacenter

type 9 region

type 10 root

# buckets

host ceph01 {

id -2 # do not change unnecessarily

# weight 0.000

alg straw

hash 0 # rjenkins1

item osd.0 weight 0.000

item osd.1 weight 0.000

}

host ceph02 {

id -3 # do not change unnecessarily

# weight 0.000

alg straw

hash 0 # rjenkins1

item osd.2 weight 0.000

item osd.3 weight 0.000

}

host ceph03 {

id -4 # do not change unnecessarily

# weight 0.000

alg straw

hash 0 # rjenkins1

item osd.4 weight 0.000

item osd.5 weight 0.000

}

root default {

id -1 # do not change unnecessarily

# weight 0.000

alg straw

hash 0 # rjenkins1

item ceph01 weight 0.000

item ceph02 weight 0.000

item ceph03 weight 0.000

}

root flash {

id -5 # do not change unnecessarily

# weight 0.000

alg straw

hash 0 # rjenkins1

item osd.0 weight 0.000

item osd.2 weight 0.000

item osd.4 weight 0.000

}

root hdd {

id -6 # do not change unnecessarily

# weight 0.000

alg straw

hash 0 # rjenkins1

item osd.1 weight 0.000

item osd.3 weight 0.000

item osd.5 weight 0.000

}

# rules

rule replicated_ruleset {

ruleset 0

type replicated

min_size 1

max_size 10

step take default

step chooseleaf firstn 0 type host

step emit

}

rule hdd_ruleset {

ruleset 1

type replicated

min_size 1

max_size 10

step take hdd

step chooseleaf firstn 0 type osd

step emit

}

rule flash_ruleset {

ruleset 2

type replicated

min_size 1

max_size 10

step take flash

step chooseleaf firstn 0 type osd

step emit

}

# end crush map

3.crushtool -c crushmap.txt -o ncrushmap 将新的crushmap反编译回二进制

4.ceph osd setcrushmap -i ncrushmap 将新的映射到集群中

(ceph osd pool set hdd crush_ruleset 1

ceph osd pool set sad crush_ruleset 2)将生成的新rules使能到pool中

5.添加cachetier之前测试一下空间可用

echo 'test' | rados -p HDD put test_object - 添加一个对象

rados -p HDD get test_object - 查看该对象的内容

rados df

rados -p HDD rm test_object 删除该测试对象

6.ceph osd tier add HDD flash 将flash pool添加到HDD pool上的cache tier

ceph osd tier cache-mode flash writeback 设置flashpool的cache模式

ceph osd tier set-overlay HDD flash

获取参数值的命令：ceph osd pool get {cachepool} {key}

设置参数命令ceph osd pool set {cachepool} {key} {value}

74 ceph osd pool get flash hit_set_type

75 ceph osd pool set flash hit_set_type bloom

76 ceph osd pool set flash hit_set_count 0

77 ceph osd pool set flash hit_set_period 3600

78 ceph osd pool set flash target_max_bytes 10000000000

79 ceph osd pool set flash cache_target_dirty_ratio 0.4

80 ceph osd pool set flash cache_target_full_ratio 0.8

81 ceph osd pool set flash target_max_objects 1000000

82 ceph osd pool set flash cache_min_flush_age 600

83 ceph osd pool set flash cache_min_evict_age 1200

7.找一台客户端机挂载一个文件系统

在挂载点下写一个shell脚本循环写入100个文件

此时rados df

能看到flash的pool中写入了很多对象

8.将flash pool的数据刷到hdd上（操作之前将/wyz挂载点umont掉）

rados -p flash cache-flush-evict-all

刷结束后再mount上，发现数据完好都在

阅读全文

0 0