Yes you understand correctly that batch == request -hans
> On Apr 30, 2017, at 11:58 AM, Petr Novak <oss.mli...@gmail.com> wrote: > > Thank you a lot. > > How requests in max.in.flight.requests.per.connection relates to batches? 1 > request precisely means 1 batch? It would make sense if I think about it. > Just to ensure I understand correctly. > > Petr > > From: Michal Borowiecki [mailto:michal.borowie...@openbet.com] > Sent: 30. dubna 2017 20:51 > To: users@kafka.apache.org > Subject: Re: Does Kafka producer waits till previous batch returns responce > before sending next one? > > Yes, that's what the docs say in both places: > > max.in.flight.requests.per.connection > The maximum number of unacknowledged requests the client will send on a > single connection before blocking. Note that if this setting is set to be > greater than 1 and there are failed sends, there is a risk of message > re-ordering due to retries (i.e., if retries are enabled). > > retries > Setting a value greater than zero will cause the client to resend any record > whose send fails with a potentially transient error. Note that this retry is > no different than if the client resent the record upon receiving the error. > Allowing retries without setting max.in.flight.requests.per.connection to 1 > will potentially change the ordering of records because if two batches are > sent to a single partition, and the first fails and is retried but the second > succeeds, then the records in the second batch may appear first. > Cheers, > Michał > > > On 30/04/17 19:32, Jun MA wrote: > Does this mean that if the client have retry > 0 and > max.in.flight.requests.per.connection > 1, then even if the topic only have > one partition, there’s still no guarantee of the ordering? > > Thanks, > Jun > On Apr 30, 2017, at 7:57 AM, Hans Jespersen <h...@confluent.io> wrote: > > There is a parameter that controls this behavior called max.in. > flight.requests.per.connection > > If you set max.in. flight.requests.per.connection = 1 then the producer waits > until previous produce requests returns a response before sending the next > one (or retrying). The retries parameter controller the number of times to > attempt to produce a batch after a failure. > > If flight.requests.per.connection = 1 and retries is get to the maximum then > ordering is guaranteed. > > If there is a timeout then the producer library would try again and again to > produce the message and will not skip over to try and produce the next > message. > > If you set flight.requests.per.connection > 1 (I think the default is 5) then > you can get a commit log with messages out of order wrt the original > published order (because retries are done in parallel rather then in series) > > -hans > > > > On Apr 30, 2017, at 3:13 AM, Petr Novak <oss.mli...@gmail.com> wrote: > > Hello, > > Does Kafka producer waits till previous batch returns response before > sending next one? Do I assume correctly that it does not when retries can > change ordering? > > > > Hence batches delay is introduced only by producer internal send loop time > and linger? > > > > If a timeout would be localized only to a single batch send request for some > reason, does it affect the next batch (assuming this batch can go through > successfully)? > > > > Many thanks, > > Petr > > > > -- > > > Michal Borowiecki > Senior Software Engineer L4 > > T: > +44 208 742 1600 > +44 203 249 8448 > > > E: > michal.borowie...@openbet.com > > W: > www.openbet.com > > OpenBet Ltd > Chiswick Park Building 9 > 566 Chiswick High Rd > London > W4 5XT > UK > > This message is confidential and intended only for the addressee. If you have > received this message in error, please immediately notify the > postmas...@openbet.com and delete it from your system as well as any copies. > The content of e-mails as well as traffic data may be monitored by OpenBet > for employment and security purposes. To protect the environment please do > not print this e-mail unless necessary. OpenBet Ltd. Registered Office: > Chiswick Park Building 9, 566 Chiswick High Road, London, W4 5XT, United > Kingdom. A company registered in England and Wales. Registered no. 3134634. > VAT no. GB927523612 >
