The wiki you pointed to is no longer maintained and fell out of sync with the code and protocol.
You may want to refer to: https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol On Mon, Jan 4, 2016 at 4:38 AM, Jens Rantil <jens.ran...@tink.se> wrote: > Hi guys, > > I realized I never thanked yall for your input - thanks! > Jason: I apologize for assuming your stance on the issue! Feels like we all > agreed on the solution. +1 > > Follow-up: Jason made a point about defining prefetch and fairness > behaviour in the KIP. I am now working on putting that down in writing. To > do be able to do this I think I need to understand the current prefetch > behaviour in the new consumer API (0.9) a bit better. Some specific > questions: > > - How does a specific consumer balance incoming messages from multiple > partitions? Is the consumer simply issuing Multi-Fetch requests[1] for > the > consumed assigned partitions of the relevant topics? Or is the consumer > fetching from one partition at a time and balancing between them > internally? That is, is the responsibility of partition balancing (and > fairness) on the broker side or consumer side? > - Is the above documented somewhere? > > [1] > > https://cwiki.apache.org/confluence/display/KAFKA/Writing+a+Driver+for+Kafka > , > see "Multi-Fetch". > > Thanks, > Jens > > On Wed, Dec 23, 2015 at 2:44 AM, Ismael Juma <ism...@juma.me.uk> wrote: > > > On Wed, Dec 23, 2015 at 1:24 AM, Gwen Shapira <g...@confluent.io> wrote: > > > > > Given the background, it sounds like you'll generally want each call to > > > poll() to return the same number of events (which is the number you > > planned > > > on having enough memory / time for). It also sounds like tuning the > > number > > > of events will be closely tied to tuning the session timeout. That is - > > if > > > I choose to lower the session timeout for some reason, I will have to > > > modify the number of records returning too. > > > > > > If those assumptions are correct, I think a configuration makes more > > sense. > > > 1. We are unlikely to want this parameter to be change at the lifetime > of > > > the consumer > > > 2. The correct value is tied to another configuration parameter, so > they > > > will be controlled together. > > > > > > > I was thinking the same thing. > > > > Ismael > > > > > > -- > Jens Rantil > Backend engineer > Tink AB > > Email: jens.ran...@tink.se > Phone: +46 708 84 18 32 > Web: www.tink.se > > Facebook <https://www.facebook.com/#!/tink.se> Linkedin > < > http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary > > > Twitter <https://twitter.com/tink> >
