Re: Kafka connector throughput reduction upon avro schema change

Bumping this. Has anyone here observed this in their Kafka connect deployments? Thanks, Dave On 5/26/17, 1:44 PM, "Dave Hamilton" <dhamil...@nanigans.com> wrote: We are currently using the Kafka S3 connector to ship Avro data to S3. We made a change to one of our Avro

Re: Kafka connector throughput reduction upon avro schema change

Hi, does anyone have advice on how to deal with this issue? Is it possible that changing a schema compatibility setting could correct it? Thanks, Dave On 5/26/17, 1:44 PM, "Dave Hamilton" <dhamil...@nanigans.com> wrote: We are currently using the Kafka S3 connector to ship

Kafka connector throughput reduction upon avro schema change

We are currently using the Kafka S3 connector to ship Avro data to S3. We made a change to one of our Avro schemas and have noticed consumer throughput on the Kafka connector drop considerably. I am wondering if there is anything we can do to avoid such issues when we update schemas in the

Kafka log.message.format.version and consumer client versions

Hi, I have a question about the performance implications of upgrading the Kafka message format relating to the following from the upgrade documentation: The message format in 0.10.0 includes a new timestamp field and uses relative offsets for compressed messages. The on disk message format can

Re: Error finding consumer coordinators after restart

Just wanted to close the loop on this. It seems the consumer offset logs might have been corrupted by the system restart. Deleting the topic logs and restarting the Kafka service cleared up the problem. Thanks, Dave On 1/12/17, 2:29 PM, "Dave Hamilton" <dhamil...@naniga

Error finding consumer coordinators after restart

Hello, we ran into a memory issue on a Kafka 0.10.0.1 broker we are running that required a system restart. Since bringing Kafka back up it seems the consumers are having issues finding their coordinators. Here are some errors we’ve seen in our server logs after restarting: [2017-01-12

Re: Unclaimed partitions

, is there anything to indicate that the rebalance process on those hosts even started? Does this happen occasionally or every time you start the consumer group? Can you paste the output of ConsumerOffsetChecker and describe topic? Thanks, Aditya From: Dave Hamilton

Re: Unclaimed partitions

Hi, would anyone be able to help me with this issue? Thanks. - Dave On Tue, Apr 28, 2015 at 1:32 PM -0700, Dave Hamilton dhamil...@nanigans.commailto:dhamil...@nanigans.com wrote: 1. We’re using version 0.8.1.1. 2. No failures in the consumer logs 3. We’re using the ConsumerOffsetChecker

Re: Unclaimed partitions

From: Dave Hamilton [dhamil...@nanigans.com] Sent: Tuesday, April 28, 2015 10:19 AM To: users@kafka.apache.org Subject: Re: Unclaimed partitions I’m sorry, I forgot to specify that these processes are in the same consumer group. Thanks, Dave On 4/28/15, 1:15 PM, Aditya Auradkar aaurad

Re: Unclaimed partitions

are assigning partitions in your application code. Did you mean to say that you are using the high level consumer API? Thanks, Aditya From: Dave Hamilton [dhamil...@nanigans.com] Sent: Tuesday, April 28, 2015 7:58 AM To: users@kafka.apache.org Subject