Thanks Matthias. I just assigned that ticket to myself. This ticket is about what returns from the poll call from the client's perspective. I will ask the reporter to confirm.
I will request to enter a KIP. CH -----Original Message----- From: Matthias J. Sax <matth...@confluent.io> Sent: Tuesday, October 30, 2018 2:44 PM To: dev@kafka.apache.org Subject: Re: [EXTERNAL] - Re: KAFKA-3932 - Consumer fails to consume in a round robin fashion I added you to the list of contributes. You can now self-assign ticket to yourself. Before you start working on this, we need to understand what the actual issue is in detail. Note, that sending fetch request to partitions and what poll() returns is two different things by design. You might also want to read https://cwiki.apache.org/confluence/display/KAFKA/KIP-227%3A+Introduce+Incremental+FetchRequests+to+Increase+Partition+Scalability I am not sure, if KAFKA-3923 requests to change the fetch request logic (what was done already) or what poll() returns from the buffer, or maybe both. It would be good to ask for clarification on the ticket to see what the reported actually means, and if the current behavior meets there requirements and if not, why. Overall, this change seems to require a KIP anyway. Hope this helps. -Matthias On 10/30/18 9:38 AM, ChienHsing Wu wrote: > I just looked at the release schedule. I guess the 2.2 is around > Feb/2019, right? --CH > > -----Original Message----- > From: ChienHsing Wu <chien...@opentext.com> > Sent: Tuesday, October 30, 2018 10:56 AM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: KAFKA-3932 - Consumer fails to consume > in a round robin fashion > > Hi Matthias, > > Sorry about the late reply. > > I have a Jira account. It's chienhsw. I am using the latest version 2.0. > Would it be possible to add that to 2.0 as a minor release? > > Thanks, ChienHsing > > -----Original Message----- > From: Matthias J. Sax <matth...@confluent.io> > Sent: Wednesday, October 24, 2018 6:41 PM > To: dev@kafka.apache.org > Subject: [EXTERNAL] - Re: KAFKA-3932 - Consumer fails to consume in a > round robin fashion > > CH, > > Thanks for contributing to Kafka. Do you have a Jira account already? If yes, > what is your account id? If not, you need to create one first and share your > id so we can grant permission to self-assign tickets. > > I was just looking into the ticket itself, and it's marked as 0.10.0.0. > You say you encountered this issues. Do you use 0.10.0.x version? AFAIK, the > consumer was updated in later versions, and the behavior should be different. > Before you start working on the ticket, we should verify that it is not > already fixed. For this case, we would just resolve the ticket with > corresponding fixed version. > > Note, that the behavior (at least from my point of view) is not a bug, > but addressing it would be an improvement. Thus, if you work on it, > the patch would be released with 2.2.0 version, but _not_ with a > potential > 0.10.0.2 release. > > Does this make sense? > > > -Matthias > > On 10/24/18 6:27 AM, ChienHsing Wu wrote: >> I don't see any comments/concerns. I would like to implement and commit to >> this ticket. Could anyone let me know how to request for the permission to >> assign that ticket to me? >> >> Thanks, CH >> >> From: ChienHsing Wu >> Sent: Monday, October 22, 2018 1:40 PM >> To: 'dev@kafka.apache.org' <dev@kafka.apache.org> >> Subject: KAFKA-3932 - Consumer fails to consume in a round robin >> fashion >> >> >> Hi, >> >> >> >> I encountered the issue documented in the jira >> KAFKA-3932<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_KAFKA-2D3932-3Fjql-3Dtext-2520-7E-2520-2522round-2520robin-2520consumer-2522&d=DwIFAg&c=ZgVRmm3mf2P1-XDAyDsu4A&r=Az03wMrbL9ToLW0OFyo3wo3985rhAKPMLmmg6RA3V7I&m=HLTuFMPfB0s5o_WTleQx1c5YRdKxDWwoJRC4iDkPopw&s=9KsjjzejGA5jiySmpE3WR0wqAoOKfZhju-8dUhZZoD4&e=>. >> Upon studying the source code and the >> PIP<https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D41-253A-2BKafkaConsumer-2BMax-2BRecords&d=DwIFAg&c=ZgVRmm3mf2P1-XDAyDsu4A&r=Az03wMrbL9ToLW0OFyo3wo3985rhAKPMLmmg6RA3V7I&m=HLTuFMPfB0s5o_WTleQx1c5YRdKxDWwoJRC4iDkPopw&s=L7JVo7fEjkgHrA0jN5SUmCIrxlQuIRWCxG_iiBD-zdw&e=>, >> I think the issues is the statement in PIP: "As before, we'd keep track of >> which partition we left off at so that the next iteration would begin >> there." I think it should NOT use the last partition in the next iteration; >> it should pick the next one instead. >> >> If this behavior is agreeable, the simplest solution to impose the order to >> pick the next one is to use the order the consumer.internals.Fetcher >> receives the partition messages, as determined by completedFetches queue in >> that class. To avoid parsing the partition messages repeatedly. we can save >> those parsed fetches to a list and maintain the next partition to get >> messages there. >> >> Does it sound like a good approach? If this is not the right place to >> discuss the design please let me know where to engage. If this is agreeable >> I can contribute the implementation. >> >> >> >> Thanks, CH >> >> >
