如何在Kafka上对一个Topic增加replicas

来源：互联网发布：专业的vb振动电机编辑：程序博客网时间：2024/05/01 07:45

操作步骤如下：

操作，是指手动写扩充replicas的配置文件，然后使用工具进行操作

1.查看topic的详细信息

lizhitao@users-MacBook-Pro-2:~$ ./bin/kafka-topics.sh –zookeeper 192.168.2.225:2183/config/mobile/mq –describe –topic test.example
Topic:test.example PartitionCount:12 ReplicationFactor:1 Configs:
Topic: test.example Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: test.example Partition: 1 Leader: 1 Replicas: 1 Isr: 1
Topic: test.example Partition: 2 Leader: 2 Replicas: 2 Isr: 2
Topic: test.example Partition: 3 Leader: 0 Replicas: 0 Isr: 0
Topic: test.example Partition: 4 Leader: 1 Replicas: 1 Isr: 1
Topic: test.example Partition: 5 Leader: 2 Replicas: 2 Isr: 2
Topic: test.example Partition: 6 Leader: 0 Replicas: 0 Isr: 0
Topic: test.example Partition: 7 Leader: 1 Replicas: 1 Isr: 1
Topic: test.example Partition: 8 Leader: 2 Replicas: 2 Isr: 2
Topic: test.example Partition: 9 Leader: 0 Replicas: 0 Isr: 0
Topic: test.example Partition: 10 Leader: 1 Replicas: 1 Isr: 1
Topic: test.example Partition: 11 Leader: 2 Replicas: 2 Isr: 2

2.修改配置文件

将原有replicas为[0]扩充为[0,4], [1]扩充为[1,5],[2]扩充为[2,3]
[sankuai@data-kafka01 kafka]$ cat partitions-to-move.json
{
“partitions”:
[
{
“topic”: “test.example”,
“partition”: 0,
“replicas”: [0,4]
},
{
“topic”: “test.example”,
“partition”: 1,
“replicas”: [1,5]
},
{
“topic”: “test.example”,
“partition”: 2,
“replicas”: [2,3]
},
{
“topic”: “test.example”,
“partition”: 3,
“replicas”: [0,4]
},
{
“topic”: “test.example”,
“partition”: 4,
“replicas”: [1,5]
},
{
“topic”: “test.example”,
“partition”: 5,
“replicas”: [2,3]
},
{
“topic”: “test.example”,
“partition”: 6,
“replicas”: [0,4]
},
{
“topic”: “test.example”,
“partition”: 7,
“replicas”: [1,5]
},
{
“topic”: “test.example”,
“partition”: 8,
“replicas”: [2,3]
},
{
“topic”: “test.example”,
“partition”: 9,
“replicas”: [0,4]
},
{
“topic”: “test.example”,
“partition”: 10,
“replicas”: [1,5]
},
{
“topic”: “test.example”,
“partition”: 11,
“replicas”: [2,3]
}
],
“version”:1
}

3.执行
./bin/kafka-reassign-partitions.sh –zookeeper 192.168.2.225:2183/config/mobile/mq –reassignment-json-file partitions-to-move.json –execute

4.检查修改情况
[sankuai@data-kafka01 kafka]$ ./bin/kafka-topics.sh –zookeeper 192.168.2.225:2183/config/mobile/mq –describe –topic test.example
Topic:test.example PartitionCount:12 ReplicationFactor:2 Configs:
Topic: test.example Partition: 0 Leader: 0 Replicas: 0,4 Isr: 0,4
Topic: test.example Partition: 1 Leader: 1 Replicas: 1,5 Isr: 1,5
Topic: test.example Partition: 2 Leader: 2 Replicas: 2,3 Isr: 2,3
Topic: test.example Partition: 3 Leader: 0 Replicas: 0,4 Isr: 0,4
Topic: test.example Partition: 4 Leader: 1 Replicas: 1,5 Isr: 1,5
Topic: test.example Partition: 5 Leader: 2 Replicas: 2,3 Isr: 2,3
Topic: test.example Partition: 6 Leader: 0 Replicas: 0,4 Isr: 0,4
Topic: test.example Partition: 7 Leader: 1 Replicas: 1,5 Isr: 1,5
Topic: test.example Partition: 8 Leader: 2 Replicas: 2,3 Isr: 2,3
Topic: test.example Partition: 9 Leader: 0 Replicas: 0,4 Isr: 0,4
Topic: test.example Partition: 10 Leader: 1 Replicas: 1,5 Isr: 1,5
Topic: test.example Partition: 11 Leader: 2 Replicas: 2,3 Isr: 2,3

0 0