[jira] [Commented] (KAFKA-3007) Implement max.poll.records for new consumer (KIP-41)
[ https://issues.apache.org/jira/browse/KAFKA-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159888#comment-15159888 ] ASF GitHub Bot commented on KAFKA-3007: --- Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/931 > Implement max.poll.records for new consumer (KIP-41) > > > Key: KAFKA-3007 > URL: https://issues.apache.org/jira/browse/KAFKA-3007 > Project: Kafka > Issue Type: Improvement > Components: consumer >Affects Versions: 0.9.0.0 >Reporter: aarti gupta >Assignee: Jason Gustafson > Fix For: 0.9.1.0 > > > Currently, the consumer.poll(timeout) > returns all messages that have not been acked since the last fetch > The only way to process a single message, is to throw away all but the first > message in the list > This would mean we are required to fetch all messages into memory, and this > coupled with the client being not thread-safe, (i.e. we cannot use a > different thread to ack messages, makes it hard to consume messages when the > order of message arrival is important, and a large number of messages are > pending to be consumed) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3007) Implement max.poll.records for new consumer (KIP-41)
[ https://issues.apache.org/jira/browse/KAFKA-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151340#comment-15151340 ] ASF GitHub Bot commented on KAFKA-3007: --- GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/931 KAFKA-3007: implement max.poll.records (KIP-41) You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-3007 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/931.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #931 commit 11a977b69b57bee650bc7b2d4e96abc2cb901d6f Author: Jason GustafsonDate: 2016-02-17T19:25:31Z KAFKA-3007: implement max.poll.records (KIP-41) > Implement max.poll.records for new consumer (KIP-41) > > > Key: KAFKA-3007 > URL: https://issues.apache.org/jira/browse/KAFKA-3007 > Project: Kafka > Issue Type: Improvement > Components: consumer >Affects Versions: 0.9.0.0 >Reporter: aarti gupta >Assignee: Jason Gustafson > > Currently, the consumer.poll(timeout) > returns all messages that have not been acked since the last fetch > The only way to process a single message, is to throw away all but the first > message in the list > This would mean we are required to fetch all messages into memory, and this > coupled with the client being not thread-safe, (i.e. we cannot use a > different thread to ack messages, makes it hard to consume messages when the > order of message arrival is important, and a large number of messages are > pending to be consumed) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3007) Implement max.poll.records for new consumer (KIP-41)
[ https://issues.apache.org/jira/browse/KAFKA-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147845#comment-15147845 ] Erik Pettersson commented on KAFKA-3007: No problem! Was just thinking about if I should implement it if you had other things to do :) > Implement max.poll.records for new consumer (KIP-41) > > > Key: KAFKA-3007 > URL: https://issues.apache.org/jira/browse/KAFKA-3007 > Project: Kafka > Issue Type: Improvement > Components: consumer >Affects Versions: 0.9.0.0 >Reporter: aarti gupta >Assignee: Jason Gustafson > > Currently, the consumer.poll(timeout) > returns all messages that have not been acked since the last fetch > The only way to process a single message, is to throw away all but the first > message in the list > This would mean we are required to fetch all messages into memory, and this > coupled with the client being not thread-safe, (i.e. we cannot use a > different thread to ack messages, makes it hard to consume messages when the > order of message arrival is important, and a large number of messages are > pending to be consumed) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3007) Implement max.poll.records for new consumer (KIP-41)
[ https://issues.apache.org/jira/browse/KAFKA-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147713#comment-15147713 ] Jason Gustafson commented on KAFKA-3007: [~eripe...@gmail.com] Apologies for the delay. I was planning to implement this week. > Implement max.poll.records for new consumer (KIP-41) > > > Key: KAFKA-3007 > URL: https://issues.apache.org/jira/browse/KAFKA-3007 > Project: Kafka > Issue Type: Improvement > Components: consumer >Affects Versions: 0.9.0.0 >Reporter: aarti gupta >Assignee: Jason Gustafson > > Currently, the consumer.poll(timeout) > returns all messages that have not been acked since the last fetch > The only way to process a single message, is to throw away all but the first > message in the list > This would mean we are required to fetch all messages into memory, and this > coupled with the client being not thread-safe, (i.e. we cannot use a > different thread to ack messages, makes it hard to consume messages when the > order of message arrival is important, and a large number of messages are > pending to be consumed) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3007) Implement max.poll.records for new consumer (KIP-41)
[ https://issues.apache.org/jira/browse/KAFKA-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147662#comment-15147662 ] Erik Pettersson commented on KAFKA-3007: Has any work been done to this issue? > Implement max.poll.records for new consumer (KIP-41) > > > Key: KAFKA-3007 > URL: https://issues.apache.org/jira/browse/KAFKA-3007 > Project: Kafka > Issue Type: Improvement > Components: consumer >Affects Versions: 0.9.0.0 >Reporter: aarti gupta >Assignee: Jason Gustafson > > Currently, the consumer.poll(timeout) > returns all messages that have not been acked since the last fetch > The only way to process a single message, is to throw away all but the first > message in the list > This would mean we are required to fetch all messages into memory, and this > coupled with the client being not thread-safe, (i.e. we cannot use a > different thread to ack messages, makes it hard to consume messages when the > order of message arrival is important, and a large number of messages are > pending to be consumed) -- This message was sent by Atlassian JIRA (v6.3.4#6332)