跟踪kafka配置max.poll.records参数有否生效

kafka消费客户端使用Spring-kafka第三方库的开源jar包，引入Maven

        <dependency>            <groupId>org.apache.kafka</groupId>            <artifactId>kafka_2.10</artifactId>            <version>0.10.2.1</version>        </dependency>        <dependency>            <groupId>org.springframework.kafka</groupId>            <artifactId>spring-kafka</artifactId>            <version>1.2.1.RELEASE</version>        </dependency>        <dependency>            <groupId>org.apache.kafka</groupId>            <artifactId>kafka-clients</artifactId>            <version>0.10.2.1</version>        </dependency>

配置kafka参数max.poll.records 为10

断点调试org.springframework.kafka.listener.KafkaMessageListenerContainer里的run()函数里的以下代码

long lastReceive = System.currentTimeMillis();long lastAlertAt = lastReceive;while (isRunning()) {try {if (!this.autoCommit) {processCommits();}processSeeks();if (this.logger.isTraceEnabled()) {this.logger.trace("Polling (paused=" + this.paused + ")...");}ConsumerRecords<K, V> records = this.consumer.poll(this.containerProperties.getPollTimeout()); //拉取数据在这行代码if (records != null && this.logger.isDebugEnabled()) {this.logger.debug("Received: " + records.count() + " records");}if (records != null && records.count() > 0) {if (this.containerProperties.getIdleEventInterval() != null) {lastReceive = System.currentTimeMillis();}// if the container is set to auto-commit, then execute in the// same thread// otherwise send to the buffering queueif (this.autoCommit) {invokeListener(records);}else {if (sendToListener(records)) {if (this.assignedPartitions != null) {// avoid group management rebalance due to a slow// consumerthis.consumer.pause(this.assignedPartitions);this.paused = true;this.unsent = records;}}}}else {if (this.containerProperties.getIdleEventInterval() != null) {long now = System.currentTimeMillis();if (now > lastReceive + this.containerProperties.getIdleEventInterval()&& now > lastAlertAt + this.containerProperties.getIdleEventInterval()) {publishIdleContainerEvent(now - lastReceive);lastAlertAt = now;if (this.theListener instanceof ConsumerSeekAware) {seekPartitions(getAssignedPartitions(), true);}}}}this.unsent = checkPause(this.unsent);}catch (WakeupException e) {this.unsent = checkPause(this.unsent);}catch (Exception e) {if (this.containerProperties.getGenericErrorHandler() != null) {this.containerProperties.getGenericErrorHandler().handle(e, null);}else {this.logger.error("Container exception", e);}}}

消费者拉取Kafka broker数据在ConsumerRecords<K, V> records = this.consumer.poll(this.containerProperties.getPollTimeout()); 这行代码

注意：kafka在0.9版本无max.poll.records参数，默认拉取记录是500，直到0.10版本才引入该参数，所以在0.9版本配置是无效的。

在ConsumerConfig.java类里有做默认配置拉取默认500

 .define(MAX_POLL_RECORDS_CONFIG,                                        Type.INT,                                        500,                                        atLeast(1),                                        Importance.MEDIUM,                                        MAX_POLL_RECORDS_DOC)