Re: Reg : Slowness in Kafka

[Ismael Juma]

Hi James,

A JIRA would be helpful. It looks like something we should fix.

Ismael

On Mon, Mar 30, 2020 at 8:17 PM James Olsen  wrote:

> Christophe,
>
> See "Problems when Consuming from multiple Partitions” in the list
> archive.  I’ll forward you the full conversation privately.  It includes
> debug logs that demonstrate fetches being discarded and refetched, but only
> after one or more full expirations of the fetch.max.wait.ms even though
> messages are available. It appears to be due to the re-fetch only querying
> one of the Partitions allocated to the Consumer, so if that Partition is
> empty you get the full delay even though messages are available (and in
> fact already fetched) from another Partition.
>
> The exact behaviour differs depending on the server and client versions.
> I tried 2.2.1 and 2.3.1 servers with a 2.4.0 client.  Both introduced
> delays and/or some partitions not being processed at all.
>
> The problems were only observed where a Consumer subscribed to multiple
> Partitions of the same Topic.
>
> I haven’t raised an issue for it as I would have no expectation of it
> being fixed.  I just made my client and server versions match and
> implemented a workaround for the issue I wanted the 2.4.0 client to resolve.
>
> Regards, James.
>
> On 31/03/2020, at 12:16, JALADY Christophe  <mailto:christophe.jal...@laposte.fr>> wrote:
>
> There are serious latency issues when mixing different client and server
> version
>
> Could you be more specific  ? Link to any issue ?
>
> Thanks by advance !
>
> Christophe
> 
> De : James Olsen mailto:ja...@inaseq.com>>
> Envoyé : vendredi 27 mars 2020 01:48
> À : users@kafka.apache.org<mailto:users@kafka.apache.org> <
> users@kafka.apache.org<mailto:users@kafka.apache.org>>
> Objet : Re: Reg : Slowness in Kafka
>
> Also check your Kafka Client and Server versions.  There are serious
> latency issues when mixing different client and server versions IF your
> consumers handle multiple partitions.
>
> On 27/03/2020, at 12:59, Chris Larsen  clar...@confluent.io>> wrote:
>
> Hi Vidhya,
>
> How many tasks are you running against the topic? How many partitions are
> on the topic?  Can you post the connector config anonymized?
>
> Best,
> Chris
>
> On Thu, Mar 26, 2020 at 17:58 Vidhya Sakar  sakar.bl...@gmail.com>> wrote:
>
> Hi Team,
>
> The Kafka consumer is reading only 8 records per second.We have implemented
> apache Kafka and confluent connect S3. The confluent connect S3 collects
> the records and pushes it to S3 bucket.
> In this process, we are seeing some slowness like on an average only 8
> records is being processed for a second. am planning to have a higher
> throughout results, so that where there is a higher data load, it should
> process more number of records.
>
> Thanks in advance.
>
> --
>
> *Chris Larsen*
>
> Systems Engineer | Confluent
>
> 847.274.3735
>
> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> <https://www.confluent.io/blog/>
>
>
> Post-scriptum La Poste
>
> Ce message est confidentiel. Sous reserve de tout accord conclu par
> ecrit entre vous et La Poste, son contenu ne represente en aucun cas un
> engagement de la part de La Poste. Toute publication, utilisation ou
> diffusion, meme partielle, doit etre autorisee prealablement. Si vous
> n'etes pas destinataire de ce message, merci d'en avertir immediatement
> l'expediteur.
>
>

Re: Reg : Slowness in Kafka

[James Olsen]

Christophe,

See "Problems when Consuming from multiple Partitions” in the list archive.  
I’ll forward you the full conversation privately.  It includes debug logs that 
demonstrate fetches being discarded and refetched, but only after one or more 
full expirations of the fetch.max.wait.ms even though messages are available. 
It appears to be due to the re-fetch only querying one of the Partitions 
allocated to the Consumer, so if that Partition is empty you get the full delay 
even though messages are available (and in fact already fetched) from another 
Partition.

The exact behaviour differs depending on the server and client versions.  I 
tried 2.2.1 and 2.3.1 servers with a 2.4.0 client.  Both introduced delays 
and/or some partitions not being processed at all.

The problems were only observed where a Consumer subscribed to multiple 
Partitions of the same Topic.

I haven’t raised an issue for it as I would have no expectation of it being 
fixed.  I just made my client and server versions match and implemented a 
workaround for the issue I wanted the 2.4.0 client to resolve.

Regards, James.

On 31/03/2020, at 12:16, JALADY Christophe 
mailto:christophe.jal...@laposte.fr>> wrote:

There are serious latency issues when mixing different client and server version

Could you be more specific  ? Link to any issue ?

Thanks by advance !

Christophe

De : James Olsen mailto:ja...@inaseq.com>>
Envoyé : vendredi 27 mars 2020 01:48
À : users@kafka.apache.org<mailto:users@kafka.apache.org> 
mailto:users@kafka.apache.org>>
Objet : Re: Reg : Slowness in Kafka

Also check your Kafka Client and Server versions.  There are serious latency 
issues when mixing different client and server versions IF your consumers 
handle multiple partitions.

On 27/03/2020, at 12:59, Chris Larsen 
mailto:clar...@confluent.io>> wrote:

Hi Vidhya,

How many tasks are you running against the topic? How many partitions are
on the topic?  Can you post the connector config anonymized?

Best,
Chris

On Thu, Mar 26, 2020 at 17:58 Vidhya Sakar 
mailto:sakar.bl...@gmail.com>> wrote:

Hi Team,

The Kafka consumer is reading only 8 records per second.We have implemented
apache Kafka and confluent connect S3. The confluent connect S3 collects
the records and pushes it to S3 bucket.
In this process, we are seeing some slowness like on an average only 8
records is being processed for a second. am planning to have a higher
throughout results, so that where there is a higher data load, it should
process more number of records.

Thanks in advance.

--

*Chris Larsen*

Systems Engineer | Confluent

847.274.3735

Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
<https://www.confluent.io/blog/>


Post-scriptum La Poste

Ce message est confidentiel. Sous reserve de tout accord conclu par
ecrit entre vous et La Poste, son contenu ne represente en aucun cas un 
engagement de la part de La Poste. Toute publication, utilisation ou diffusion, 
meme partielle, doit etre autorisee prealablement. Si vous n'etes pas 
destinataire de ce message, merci d'en avertir immediatement
l'expediteur.

RE: Reg : Slowness in Kafka

[JALADY Christophe]

> There are serious latency issues when mixing different client and server 
> version

Could you be more specific  ? Link to any issue ?

Thanks by advance !

Christophe

De : James Olsen 
Envoyé : vendredi 27 mars 2020 01:48
À : users@kafka.apache.org 
Objet : Re: Reg : Slowness in Kafka

Also check your Kafka Client and Server versions.  There are serious latency 
issues when mixing different client and server versions IF your consumers 
handle multiple partitions.

> On 27/03/2020, at 12:59, Chris Larsen  wrote:
>
> Hi Vidhya,
>
> How many tasks are you running against the topic? How many partitions are
> on the topic?  Can you post the connector config anonymized?
>
> Best,
> Chris
>
> On Thu, Mar 26, 2020 at 17:58 Vidhya Sakar  wrote:
>
>> Hi Team,
>>
>> The Kafka consumer is reading only 8 records per second.We have implemented
>> apache Kafka and confluent connect S3. The confluent connect S3 collects
>> the records and pushes it to S3 bucket.
>> In this process, we are seeing some slowness like on an average only 8
>> records is being processed for a second. am planning to have a higher
>> throughout results, so that where there is a higher data load, it should
>> process more number of records.
>>
>> Thanks in advance.
>>
> --
>
> *Chris Larsen*
>
> Systems Engineer | Confluent
>
> 847.274.3735
>
> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> <https://www.confluent.io/blog/>


Post-scriptum La Poste

Ce message est confidentiel. Sous reserve de tout accord conclu par
ecrit entre vous et La Poste, son contenu ne represente en aucun cas un 
engagement de la part de La Poste. Toute publication, utilisation ou diffusion, 
meme partielle, doit etre autorisee prealablement. Si vous n'etes pas 
destinataire de ce message, merci d'en avertir immediatement
l'expediteur.

Re: Reg : Slowness in Kafka

[James Olsen]

Also check your Kafka Client and Server versions.  There are serious latency 
issues when mixing different client and server versions IF your consumers 
handle multiple partitions.

> On 27/03/2020, at 12:59, Chris Larsen  wrote:
> 
> Hi Vidhya,
> 
> How many tasks are you running against the topic? How many partitions are
> on the topic?  Can you post the connector config anonymized?
> 
> Best,
> Chris
> 
> On Thu, Mar 26, 2020 at 17:58 Vidhya Sakar  wrote:
> 
>> Hi Team,
>> 
>> The Kafka consumer is reading only 8 records per second.We have implemented
>> apache Kafka and confluent connect S3. The confluent connect S3 collects
>> the records and pushes it to S3 bucket.
>> In this process, we are seeing some slowness like on an average only 8
>> records is being processed for a second. am planning to have a higher
>> throughout results, so that where there is a higher data load, it should
>> process more number of records.
>> 
>> Thanks in advance.
>> 
> -- 
> 
> *Chris Larsen*
> 
> Systems Engineer | Confluent
> 
> 847.274.3735
> 
> Follow us: Twitter  | blog
> 

Re: Reg : Slowness in Kafka

[Chris Larsen]

Hi Vidhya,

How many tasks are you running against the topic? How many partitions are
on the topic?  Can you post the connector config anonymized?

Best,
Chris

On Thu, Mar 26, 2020 at 17:58 Vidhya Sakar  wrote:

> Hi Team,
>
> The Kafka consumer is reading only 8 records per second.We have implemented
> apache Kafka and confluent connect S3. The confluent connect S3 collects
> the records and pushes it to S3 bucket.
> In this process, we are seeing some slowness like on an average only 8
> records is being processed for a second. am planning to have a higher
> throughout results, so that where there is a higher data load, it should
> process more number of records.
>
> Thanks in advance.
>
-- 

*Chris Larsen*

Systems Engineer | Confluent

847.274.3735

Follow us: Twitter  | blog

Re: Reg : Slowness in Kafka

[Hans Jespersen]

Yes it should be going much faster than that. Something is wrong in your setup.

-hans

> On Mar 26, 2020, at 5:58 PM, Vidhya Sakar  wrote:
> 
> Hi Team,
> 
> The Kafka consumer is reading only 8 records per second.We have implemented
> apache Kafka and confluent connect S3. The confluent connect S3 collects
> the records and pushes it to S3 bucket.
> In this process, we are seeing some slowness like on an average only 8
> records is being processed for a second. am planning to have a higher
> throughout results, so that where there is a higher data load, it should
> process more number of records.
> 
> Thanks in advance.

Reg : Slowness in Kafka

[Vidhya Sakar]

Hi Team,

The Kafka consumer is reading only 8 records per second.We have implemented
apache Kafka and confluent connect S3. The confluent connect S3 collects
the records and pushes it to S3 bucket.
In this process, we are seeing some slowness like on an average only 8
records is being processed for a second. am planning to have a higher
throughout results, so that where there is a higher data load, it should
process more number of records.

Thanks in advance.