[jira] [Commented] (KAFKA-5093) Load only batch header when rebuilding producer ID map
[ https://issues.apache.org/jira/browse/KAFKA-5093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16031975#comment-16031975 ] ASF GitHub Bot commented on KAFKA-5093: --- Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3160 > Load only batch header when rebuilding producer ID map > -- > > Key: KAFKA-5093 > URL: https://issues.apache.org/jira/browse/KAFKA-5093 > Project: Kafka > Issue Type: Sub-task >Reporter: Jason Gustafson >Assignee: Jason Gustafson >Priority: Blocker > Labels: exactly-once > Fix For: 0.11.0.0 > > > When rebuilding the producer ID map for KIP-98, we unnecessarily load the > full record data into memory when scanning through the log. It would be > better to only load the batch header since it is all that is needed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KAFKA-5093) Load only batch header when rebuilding producer ID map
[ https://issues.apache.org/jira/browse/KAFKA-5093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027696#comment-16027696 ] Umesh Chaudhary commented on KAFKA-5093: No worries at all [~hachikuji] :) > Load only batch header when rebuilding producer ID map > -- > > Key: KAFKA-5093 > URL: https://issues.apache.org/jira/browse/KAFKA-5093 > Project: Kafka > Issue Type: Sub-task >Reporter: Jason Gustafson >Assignee: Jason Gustafson >Priority: Blocker > Labels: exactly-once > Fix For: 0.11.0.0 > > > When rebuilding the producer ID map for KIP-98, we unnecessarily load the > full record data into memory when scanning through the log. It would be > better to only load the batch header since it is all that is needed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KAFKA-5093) Load only batch header when rebuilding producer ID map
[ https://issues.apache.org/jira/browse/KAFKA-5093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027555#comment-16027555 ] Jason Gustafson commented on KAFKA-5093: [~umesh9...@gmail.com] Apologies. I decided to pick this up yesterday, but forgot to assign myself. > Load only batch header when rebuilding producer ID map > -- > > Key: KAFKA-5093 > URL: https://issues.apache.org/jira/browse/KAFKA-5093 > Project: Kafka > Issue Type: Sub-task >Reporter: Jason Gustafson >Priority: Blocker > Labels: exactly-once > Fix For: 0.11.0.0 > > > When rebuilding the producer ID map for KIP-98, we unnecessarily load the > full record data into memory when scanning through the log. It would be > better to only load the batch header since it is all that is needed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KAFKA-5093) Load only batch header when rebuilding producer ID map
[ https://issues.apache.org/jira/browse/KAFKA-5093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027541#comment-16027541 ] ASF GitHub Bot commented on KAFKA-5093: --- GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3160 KAFKA-5093: Avoid loading full batch data when possible when iterating FileRecords You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-5093 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3160.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3160 commit a453e37032a8e0db7dde69f2c76d22070cabe80a Author: Jason Gustafson Date: 2017-05-27T07:56:55Z KAFKA-5093: Avoid loading full batch data when possible when iterating FileRecords > Load only batch header when rebuilding producer ID map > -- > > Key: KAFKA-5093 > URL: https://issues.apache.org/jira/browse/KAFKA-5093 > Project: Kafka > Issue Type: Sub-task >Reporter: Jason Gustafson >Priority: Blocker > Labels: exactly-once > Fix For: 0.11.0.0 > > > When rebuilding the producer ID map for KIP-98, we unnecessarily load the > full record data into memory when scanning through the log. It would be > better to only load the batch header since it is all that is needed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KAFKA-5093) Load only batch header when rebuilding producer ID map
[ https://issues.apache.org/jira/browse/KAFKA-5093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027362#comment-16027362 ] Umesh Chaudhary commented on KAFKA-5093: Tried to co-relate it with KIP but was unable to locate the intended piece of code to tweak. If possible, can you please point that? > Load only batch header when rebuilding producer ID map > -- > > Key: KAFKA-5093 > URL: https://issues.apache.org/jira/browse/KAFKA-5093 > Project: Kafka > Issue Type: Sub-task >Reporter: Jason Gustafson >Priority: Blocker > Labels: exactly-once > Fix For: 0.11.0.0 > > > When rebuilding the producer ID map for KIP-98, we unnecessarily load the > full record data into memory when scanning through the log. It would be > better to only load the batch header since it is all that is needed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
