Unifying kafka-clients call signatures

2015-12-21 Thread Pierre-Yves Ritschard
Hi list, I've been working on an issue at https://issues.apache.org/jira/browse/KAFKA-3006 and it is now a good time to ask for feedback. The attached PR moves all signatures which accepted either arrays or java.util.List to accept java.util.Collection. The aim is to provide consumers of kafka-cl

Kafka 0.9.0 New Java Consumer API fetching duplicate records

2015-12-21 Thread pradeep kumar
Can someone please help me on this. http://stackoverflow.com/questions/34405124/kafka-0-9-0-new-java-consumer-api-fetching-duplicate-records Thanks, Pradeep

Re: Re: Measuring Kafka Producer request latency when it is less than 1ms

2015-12-21 Thread Alexey Pirogov
Ismael, thanks for reply. Jire created https://issues.apache.org/jira/browse/KAFKA-3028. Thank you, Alexey

Closing socket for 1.2.4.5 because of error (kafka.network.Processor)

2015-12-21 Thread Arathi Maddula
Hi, I found this error in my kafka.out. How can I find what is causing this error? Kafka jar : kafka_2.10-0.8.2.2.3.2.0-2950.jar Closing socket for /1.2.3.4 because of error (kafka.network.Processor) java.lang.NullPointerException at kafka.network.Processor.run(SocketServer.scala:404)

Exceptions when programmatically start multiple kafka brokers

2015-12-21 Thread hsy...@gmail.com
I'm trying to start 2 brokers in my kafka ingestion unit test and I got exception javax.management.InstanceAlreadyExistsException: kafka.server:type=app-info,id=0 at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.re

RE: how to programatically monitor Kafka availability

2015-12-21 Thread Hohl, Ken
We're doing this - every half second For each broker get the value of this MBean: kafka.server:type=KafkaServer,name=BrokerState if the value is 4 (this broker is the leader), assume the cluster is available if either couldn't get the MBean from a

Re: Fallout from upgrading to kafka 0.9 from 0.8.2.3

2015-12-21 Thread Jun Rao
Rajiv, Upgrading from 0.8.2.1 to 0.9.0.0 should also be fine. If you can reproduce this issue in a test environment, that would be great. The following may be helpful in figuring out the issue. 1. Use ack=1 instead of ack=0 will allow the producer to see the error code when the send fails. 2. If

Re: Measuring Kafka Producer request latency when it is less than 1ms

2015-12-21 Thread Ismael Juma
Hi Alexey, Could you please report a bug in JIRA for the NaN result? We should handle that better. Thanks, Ismael On Mon, Dec 21, 2015 at 9:12 AM, Alexey Pirogov wrote: > I'm looking for help with a question regarding measuring of producer > request latency. > I expected that "producer.request

Measuring Kafka Producer request latency when it is less than 1ms

2015-12-21 Thread Alexey Pirogov
I'm looking for help with a question regarding measuring of producer request latency. I expected that "producer.request-latency-avg/max" will do a good job for me. But seems that if latencies less than 1ms in most cases, this metric will emit NaN(as it doesn't support float values). We need this me