One thing that changed in 0.9.0.0 is the introduction of `max.block.ms`,
which is 60 seconds by default. Since you have "block.on.buffer.full=false",
you may want to tweak `max.block.ms`.

Ismael

On Mon, Sep 5, 2016 at 1:54 PM, Ismael Juma <[email protected]> wrote:

> Hi Yifan,
>
> Just to make sure I understand, the comparison is between 0.8.2.1 and
> 0.10.0.0 clients (both) connecting to a 0.10.0.0 cluster? We haven't
> received reports of increased average producer latency and I'm not aware of
> a change that could cause such a dramatic difference. Is compression being
> used and did you paste the full producer config?
>
> Ismael
>
> On Fri, Sep 2, 2016 at 6:31 PM, Yifan Ying <[email protected]> wrote:
>
>> The load before and after the upgrade are pretty similar. And all these
>> configs stayed same too.
>>
>> Also please correct me if I am wrong. I think the request latency =
>> response received time - request created time. And request created time is
>> created when the records are ready to be sent after 'lingering'.
>>
>> Yifan
>>
>> On Thu, Sep 1, 2016 at 11:01 PM, Gerard Klijs <[email protected]>
>> wrote:
>>
>> > With a linger of 5 seconds, 2-3 seconds would make sense when the load
>> is
>> > smaller, are are sure the measurements with 0.8.2.1 where with the same
>> > load and/or linger worked correctly there?
>> >
>> > On Fri, Sep 2, 2016 at 1:12 AM Yifan Ying <[email protected]> wrote:
>> >
>> > > We tried to upgrade the Kafka clients dependency from 0.8.2.1 to
>> > 0.10.0.0.
>> > > As I understand, the producer client doesn't have major changes in
>> 0.10,
>> > so
>> > > we kept the same producer config in the upgrade:
>> > >
>> > > retries 3
>> > > retry.backoff.ms 5000
>> > > timeout.ms 10000
>> > > block.on.buffer.full false
>> > > linger.ms 5000
>> > > metadata.fetch.timeout.ms 30000
>> > >
>> > > However, after the upgrade, the avg producer request latency
>> increased by
>> > > 5×, from a few hundreds milliseconds to 2-3 seconds. Anyone has seen
>> this
>> > > issue? Is there any change in 0.9/0.10 that would cause this?
>> > >
>> > > Thanks!
>> > >
>> > > --
>> > > Yifan
>> > >
>> >
>>
>>
>>
>> --
>> Yifan
>>
>
>

Reply via email to