kafka问题列表:

来源:互联网 发布:苹果手机称重软件 编辑:程序博客网 时间:2024/06/05 16:49

kafka问题列表:
1. kafka分区的个数根据什么设置,是不是越多越好?
2. 同一个组的两个消费者能消费同一个分区吗?
参考:This API is centered around iterators, implemented by the KafkaStream class. Each KafkaStream
represents the stream of messages from one or more partitions on one or more servers. Each stream
is used for single threaded processing, so the client can provide the number of desired streams in the create call. Thus a stream may represent the merging of
multiple server partitions (to correspond to the number of processing threads), but each partition only goes to one stream.
3.怎么创建消费者才更好:
[a].The createMessageStreams call registers the consumer for the topic, which results in rebalancing the consumer/broker assignment.
The API encourages creating many topic streams in a single call in order to minimize this rebalancing. The createMessageStreamsByFilter
call (additionally) registers watchers to discover new topics that match its filter. Note that each stream that
createMessageStreamsByFilter returns may iterate over messages from multiple topics (i.e., if multiple topics are allowed by the filter).
[b].The consumers in a group divide up the partitions as fairly as possible, each partition is consumed by exactly one consumer in a consumer group.
参考:http://kafka.apache.org/documentation/#majordesignelements Consumers and Consumer Groups

0 0