RE: how to programatically monitor Kafka availability

2015-12-21 Thread Hohl, Ken
We're doing this - every half second For each broker get the value of this MBean: kafka.server:type=KafkaServer,name=BrokerState if the value is 4 (this broker is the leader), assume the cluster is available if either couldn't get the MBean from

Exceptions when programmatically start multiple kafka brokers

2015-12-21 Thread hsy...@gmail.com
I'm trying to start 2 brokers in my kafka ingestion unit test and I got exception javax.management.InstanceAlreadyExistsException: kafka.server:type=app-info,id=0 at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at

Closing socket for 1.2.4.5 because of error (kafka.network.Processor)

2015-12-21 Thread Arathi Maddula
Hi, I found this error in my kafka.out. How can I find what is causing this error? Kafka jar : kafka_2.10-0.8.2.2.3.2.0-2950.jar Closing socket for /1.2.3.4 because of error (kafka.network.Processor) java.lang.NullPointerException at kafka.network.Processor.run(SocketServer.scala:404)

Re: Measuring Kafka Producer request latency when it is less than 1ms

2015-12-21 Thread Ismael Juma
Hi Alexey, Could you please report a bug in JIRA for the NaN result? We should handle that better. Thanks, Ismael On Mon, Dec 21, 2015 at 9:12 AM, Alexey Pirogov wrote: > I'm looking for help with a question regarding measuring of producer > request latency. > I

Re: Re: Measuring Kafka Producer request latency when it is less than 1ms

2015-12-21 Thread Alexey Pirogov
Ismael, thanks for reply. Jire created https://issues.apache.org/jira/browse/KAFKA-3028. Thank you, Alexey

Measuring Kafka Producer request latency when it is less than 1ms

2015-12-21 Thread Alexey Pirogov
I'm looking for help with a question regarding measuring of producer request latency. I expected that "producer.request-latency-avg/max" will do a good job for me. But seems that if latencies less than 1ms in most cases, this metric will emit NaN(as it doesn't support float values). We need this

Unifying kafka-clients call signatures

2015-12-21 Thread Pierre-Yves Ritschard
Hi list, I've been working on an issue at https://issues.apache.org/jira/browse/KAFKA-3006 and it is now a good time to ask for feedback. The attached PR moves all signatures which accepted either arrays or java.util.List to accept java.util.Collection. The aim is to provide consumers of

Kafka 0.9.0 New Java Consumer API fetching duplicate records

2015-12-21 Thread pradeep kumar
Can someone please help me on this. http://stackoverflow.com/questions/34405124/kafka-0-9-0-new-java-consumer-api-fetching-duplicate-records Thanks, Pradeep