kafka 3.0 水平扩展/迁移数据

> kafka增加新节点后，数据不会自动迁移过去。我们可以手动在broker之间以replica 为单位迁移数据。如果减少节点的话，我们可以用同样的方法把上面的数据迁移走，再关机。

#参考文档：
http://kafka.apache.org/0101/documentation.html#basic_ops_cluster_expansion

***

#使用kafka-reassign-partitions.sh
##**-generate:**生成当前的配置文件，你可以修改它。
编辑配置文件（要迁移哪些topic）
```
[leo@test1 bin]$ cat /tmp/kafka.json
{"topics": [{"topic": "lala"}],
  "version":1
 }
```
跑脚本
--topics-to-move-json-file 后面跟配置文件
--broker-list  后面跟要把数据迁移到的broker 的id,逗号分隔
```
[leo@test1 bin]$ ./kafka-reassign-partitions.sh --zookeeper localhost:2181 --topics-to-move-json-file /tmp/kafka.json  --broker-list '2' --generate
Current partition replica assignment
{"version":1,"partitions":[{"topic":"lala","partition":0,"replicas":[1]}]}

Proposed partition reassignment configuration
{"version":1,"partitions":[{"topic":"lala","partition":0,"replicas":[2]}]}
```
***

##**-execute:**提交迁移数据的任务，后台运行，可以把generate 生成的文件加以修改后作为参数。（要用--reassignment-json-file选项）
把generate 的输出保存到新配置文件中
```
[leo@test1 bin]$ cat /tmp/kafka_exe.json
{"version":1,"partitions":[{"topic":"lala","partition":0,"replicas":[2]}]}
```
跑脚本
```
[leo@test1 bin]$ ./kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file /tmp/kafka_exe.json --execute
Current partition replica assignment

{"version":1,"partitions":[{"topic":"lala","partition":0,"replicas":[1]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions.
```
###如果想增加某一个topic 的 replication 数量，手动修改配置文件，增加replicas的数量
```
[leo@test1 bin]$ cat /tmp/kafka_exe.json
{"version":1,"partitions":[{"topic":"lala","partition":0,"replicas":[2,3]}]}
[leo@test1 bin]$ ./kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file /tmp/kafka_exe.json --execute
Current partition replica assignment

{"version":1,"partitions":[{"topic":"lala","partition":0,"replicas":[2]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions.
[leo@test1 bin]$ ./kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file /tmp/kafka_exe.json --execute
Current partition replica assignment

{"version":1,"partitions":[{"topic":"lala","partition":0,"replicas":[2]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions.
```
***

##**-verify:**检查上一个-execute 的结果。
```
[leo@test1 bin]$ ./kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file /tmp/kafka_exe.json --verify
Status of partition reassignment:
Reassignment of partition [lala,0] completed successfully
```

乱读天书， 不求甚解

乱读天书，不求甚解