Re: Kafka Producer Errors

r than the `request.timeout.ms`. Check your `jvm.log` and update the `request.timeout.ms`. The same property is applicable to producer, consumer and broker. Increase the config only for the KafkaProducer. -- Kamal On Sat, Aug 12, 2017 at 12:19 AM, Saladi Naidu < naidusp2...@yahoo.com.inval

Re: Kafka Producer Errors

000 and batch.size - < 0 Any clue on why linger.ms 0 or btach.size 0 is not taken into consideration ???   Naidu Saladi On Thursday, August 10, 2017 1:57 PM, Saladi Naidu <naidusp2...@yahoo.com.INVALID> wrote: We have recently upgraded our Kafka cluster from 0.8.2 to 0.1

Kafka Producer Errors

We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part of the upgrade our clients upgraded the client libraries as well. After the upgrade we are facing following issue org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for

FetchRequestPurgatory

We have Kafka clusters in three datacenters with Mirror Maker replicating across three data centers. We are monitoring JMX metrics of our clusters. I have question regarding FetchRequestPurgatory (kafka.server:type=FetchRequestPurgatory,name=PurgatorySize), description on Wiki says Number of

Re: Optimal number of partitions for topic

In general partitions are to improve throughput by parallelism. From your explanation below yes partitions are written to different physical locations but still append only. With write ahead buffering and append only writes, having partitions still will increase throughput. Below is an

Kafka Monitoring using JMX

We are planning to monitor health of our Kafka environment using JMX. I have looked at below links to find what is available via https://cwiki.apache.org/confluence/display/KAFKA/Available+Metrics  https://cwiki.apache.org/confluence/display/KAFKA/JMX+Reporters why some of the kafka objects

No broker partitions consumed by consumer thread

We are running 0.8.1.1, am experiencing following issue, any clue I have 7 topics in my Kafka cluster of 5 nodes. I pushed  1 million messages to each topic using perf producer tool (no compression, no acknowledgement) . I saw old messages about a bug in 0.8 Consumer code and it is fixed but I

Fw: How to measure performance of Mirror Maker

Any suggestions on how to measure throughput of the Mirror Maker Naidu Saladi - Forwarded Message - From: Saladi Naidu naidusp2...@yahoo.com To: users@kafka.apache.org users@kafka.apache.org Sent: Monday, March 16, 2015 10:31 PM Subject: How to measure performance of Mirror

How to measure performance of Mirror Maker

We have three Kafka clusters deployed in 3 DC's, each one having their own topics. We are using Mirror Maker to keep all the three clusters up to date with continuous  replication using Mirror Maker. We used Perf producer and Perf consumer to conduct basic testing, all seems to work great. Our

Out of Disk Space - Infinite loop

We have 3 DC's and created 5 node Kafka cluster in each DC, connected these 3 DC's using Mirror Maker for replication. We were conducting performance testing using Kafka Producer Performance tool to load 100 million rows into 7 topics. We expected that data will be loaded evenly across 7 topics