Hi there, Thanks for the KIP.
We’ve run into issues with this at Mailchimp so something to address consuming behavior would save us from having to always ensure we’re running enough consumers that each consumer has only one partition (which is our usual MO). I wonder though if it would be simpler and more powerful to define the maximum number of records the consumer should pull from one partition before pulling some records from another? So if you set max.poll.records to 500 and then some new setting, max.poll.records.per.partition, to 100 then the Consumer would switch what partition it reads from every 100 records - looping back around to the first partition that had records if there aren’t 5 or more partitions with records. What do you think? On Mon, Nov 19, 2018 at 9:11 AM ChienHsing Wu <chien...@opentext.com> wrote: > Hi, could anyone please review this KIP? > > Thanks, ChienHsing > > From: ChienHsing Wu > Sent: Friday, November 09, 2018 1:10 PM > To: dev@kafka.apache.org > Subject: RE: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions > in KafkaConsumer > > Just to check: Will anyone review this? It's been silent for a week... > Thanks, ChienHsing > > From: ChienHsing Wu > Sent: Monday, November 05, 2018 4:18 PM > To: 'dev@kafka.apache.org' <dev@kafka.apache.org<mailto: > dev@kafka.apache.org>> > Subject: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in > KafkaConsumer > > Hi I just put together the KIP page as requested. This email is to start > the discussion thread. > > KIP: KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer< > https://cwiki.apache.org/confluence/display/KAFKA/KIP-387%3A+Fair+Message+Consumption+Across+Partitions+in+KafkaConsumer > > > Pull Request: https://github.com/apache/kafka/pull/5838 > Jira: https://issues.apache.org/jira/browse/KAFKA-3932 > > Thanks, CH >
