[ https://issues.apache.org/jira/browse/KAFKA-5093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027541#comment-16027541 ]
ASF GitHub Bot commented on KAFKA-5093: --------------------------------------- GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3160 KAFKA-5093: Avoid loading full batch data when possible when iterating FileRecords You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5093 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3160.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3160 ---- commit a453e37032a8e0db7dde69f2c76d22070cabe80a Author: Jason Gustafson <ja...@confluent.io> Date: 2017-05-27T07:56:55Z KAFKA-5093: Avoid loading full batch data when possible when iterating FileRecords ---- > Load only batch header when rebuilding producer ID map > ------------------------------------------------------ > > Key: KAFKA-5093 > URL: https://issues.apache.org/jira/browse/KAFKA-5093 > Project: Kafka > Issue Type: Sub-task > Reporter: Jason Gustafson > Priority: Blocker > Labels: exactly-once > Fix For: 0.11.0.0 > > > When rebuilding the producer ID map for KIP-98, we unnecessarily load the > full record data into memory when scanning through the log. It would be > better to only load the batch header since it is all that is needed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)