Consumer stops consuming after one of the broker stopped

2017-08-10 Thread Rakesh Singh
I have installed kafka_2.11-0.11.0.0 and using java to create producer and consumer I have 3 brokers server running on the same unix machine . I created a topic with 3 partition with a replication factor of 2. I started a very simple producer with key/value as auto incrementing integers. I h

RE: How performant is the confluent REST proxy

2017-08-10 Thread Tauzell, Dave
I don't have any concrete numbers but the REST proxy is quite a bit slower. That said, it can still be fast and can scale out so it might meet your needs. -Dave -Original Message- From: Affan Syed [mailto:as...@an10.io] Sent: Thursday, August 10, 2017 1:32 AM To: users@kafka.apache.org

Kafka Producer Errors

2017-08-10 Thread Saladi Naidu
We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part of the upgrade our clients upgraded the client libraries as well. After the upgrade we are facing following issue org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for OM-LOCAL-DC3-REQUISITION-SUBMISSIO

How to clear a particular partition?

2017-08-10 Thread Sven Ludwig
Hello,   assume that all producers and consumers regarding a topic-partition have been shutdown.   Is it possible in this situation to empty that topic-partition, while the other topic-partitions keep working?   Like for example, is it possible to trigger a log truncation to 0 on the leader for

Re: Create Topic Error: Create Topic Error and cannot write to console producer

2017-08-10 Thread Ascot Moss
Could point me where the document is? On Thu, Aug 10, 2017 at 6:50 PM, M. Manna wrote: > This is due to partitions you are consuming from. Documentation section > explains what needs to be done. > > > > > On 10 August 2017 at 11:43, Ascot Moss wrote: > > > A question: > > > > (input order) > >

Re: Create Topic Error: Create Topic Error and cannot write to console producer

2017-08-10 Thread M. Manna
This is due to partitions you are consuming from. Documentation section explains what needs to be done. On 10 August 2017 at 11:43, Ascot Moss wrote: > A question: > > (input order) > test1 > test2 > test3 > test 2017-08-10 > |2017-08-10 test1 > 2017-08-10 test2 > > > If get them using > *--

Re: Create Topic Error: Create Topic Error and cannot write to console producer

2017-08-10 Thread Ascot Moss
A question: (input order) test1 test2 test3 test 2017-08-10 |2017-08-10 test1 2017-08-10 test2 If get them using *--from-beginning* (received order) test1 test 2017-08-10 2017-08-10 test1 test2 test3 2017-08-10 test2 Any idea how to get the message in the original order as input?

Re: Random consumer offset rewinds/resets

2017-08-10 Thread Ismael Juma
Great to hear, thanks for the update. Ismael On Wed, Aug 9, 2017 at 4:14 PM, Christiane Lemke wrote: > Hi Ismael, > > thanks a lot for your for your answer, it was indeed exactly the issue we > had! > > We did see the ticket for the issue before, but the steps to reproduce > included a consumer

Re: Create Topic Error: Create Topic Error and cannot write to console producer

2017-08-10 Thread Ascot Moss
A question: (input order) test1 test2 test3 test 2017-08-10 |2017-08-10 test1 2017-08-10 test2 If get them using *--from-beginning* (received order) test1 test 2017-08-10 2017-08-10 test1 test2 test3 2017-08-10 test2 Any idea how to get the message in the correct order as inputted? On T

Re: Create Topic Error: Create Topic Error and cannot write to console producer

2017-08-10 Thread Ascot Moss
Works! Many thanks On Thu, Aug 10, 2017 at 4:33 PM, M. Manna wrote: > you missed port - comment that out too. > > Debugging can enabled by > > 1) Setting root logger to DEBUG - more information on you cluster > 2) SSL debugging - edit kafka-run-class - to add -Djavax.security.debug=all > (see s

Re: Kafka Replication modes

2017-08-10 Thread Sameer Kumar
please refer to the following doc. https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Replication It talks about both the modes, I am not sure of which one is followed by Kafka. Also, is there a property to switch between the two. -Sameer. On Wed, Aug 9, 2017 at 3:24 PM, Manikumar wrote:

Re: Create Topic Error: Create Topic Error and cannot write to console producer

2017-08-10 Thread M. Manna
you missed port - comment that out too. Debugging can enabled by 1) Setting root logger to DEBUG - more information on you cluster 2) SSL debugging - edit kafka-run-class - to add -Djavax.security.debug=all (see some examples of how some other values are configured) could you please set: zookee