乱读天书, 不求甚解
周祎骏的个人云笔记
Toggle navigation
乱读天书, 不求甚解
主页
Linux:系统配置
Linux:用户管理
Linux:优化排错
Linux:进程调度
Linux:文件系统
Linux:网络
Linux:系统服务
Linux:安全
Linux:内核
容器:Docker
容器:containerd
容器编排:Kubernetes
IAC:Terraform
大数据:Hadoop
大数据:Zookeeper
大数据:Hbase
消息队列:rsyslog
消息队列:kafka
数据库:MySQL
数据库:MongoDB
搜索引擎:Elasticsearch
时序数据库:OpenTSDB
网站服务:Nginx
编程:Bash
编程:Perl
编程:Python
编程:C
编程:JAVA
编程:Rust
版本控制:gitlab
知识管理:docusaurus
常用小工具
关于我
标签
kafka 3.2 镜像
2017-04-23 14:58:44
56
0
0
admin
> 在不同的kafka集群之间同步数据 **配置数据源**(同步数据的进程是数据源的consumer) ``` [leo@test1 config]$ cat consumer.properties # Zookeeper connection string # comma separated host:port pairs, each corresponding to a zk # server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" zookeeper.connect=test1:2181 # timeout in ms for connecting to zookeeper zookeeper.connection.timeout.ms=6000 #consumer group id group.id=test-consumer-group #consumer timeout #consumer.timeout.ms=5000 ``` **配置被同步数据的kaka集群**(同步数据的进程是被同步数据的kafka集群的producer) ``` [leo@test1 config]$ cat producer.properties # list of brokers used for bootstrapping knowledge about the rest of the cluster # format: host1:port1,host2:port2 ... bootstrap.servers=test4:9092 # specify the compression codec for all data generated: none, gzip, snappy, lz4 compression.type=none # name of the partitioner class for partitioning events; default partition spreads data randomly #partitioner.class= # the maximum amount of time the client will wait for the response of a request #request.timeout.ms= # how long `KafkaProducer.send` and `KafkaProducer.partitionsFor` will block for #max.block.ms= # the producer will wait for up to the given delay to allow other records to be sent so that the sends can be batched together #linger.ms= # the maximum size of a request in bytes #max.request.size= # the default batch size in bytes when batching multiple records sent to a partition #batch.size= # the total bytes of memory the producer can use to buffer records waiting to be sent to the server #buffer.memory= ``` **启动同步进程** ``` ./kafka-mirror-maker.sh --consumer.config ../config/consumer.properties --producer.config ../config/producer.properties --whitelist lala & ``` 这里的--whitelist 支持正则表达式,比如“*” 或者“topicA|topicB”
上一篇:
kafka 3.1 负载平衡
下一篇:
kafka 4.0 通过JMX 监控kafka
文档导航