Allow parallel processing

2019-11-18 Thread pwozniak
Hi all, He is my usecase: I have three message producers that submits batch of messages to Kafka from time to time. Let's assume now that one of them just submitted 1k messages, second one submitted some number of messages after that and third one also submitted some messages. I would like

Re: [kafka-clients] [VOTE] 2.4.0 RC0

2019-11-18 Thread Bruno Cadonna
Hi, ClientMetricsTest.shouldAddCommitIdMetric should only fail if executed from an IDE. The test fails because the test expects a file on the class path which is not there when the test is executed from the IDE, but is there when the test is executed from gradle. I will try to fix the test so that

Re: [External] Allow parallel processing

2019-11-18 Thread Tauzell, Dave
I would go with #1: 1. It will be easier to add new "batch producers" since you won't need to worry about re-partitioning 2. You have more control over the parallelism since you can have different numbers of partitions for each topic 3. You can easily split out your consumer into N consumers if

Re: [kafka-clients] [VOTE] 2.4.0 RC0

2019-11-18 Thread Vahid Hashemian
Thanks Bruno, Just to clarify, I ran the tests from the command line: ./gradlew streams:test Regards, --Vahid On Mon, Nov 18, 2019 at 6:16 AM Bruno Cadonna wrote: > Hi, > > ClientMetricsTest.shouldAddCommitIdMetric should only fail if executed > from an IDE. The test fails because the test exp

Re: Kafka Broker do not recover after crash

2019-11-18 Thread Eric Azama
Hi Oliver, Your first line of log has a timestamp of 19:15:42 and the last few logs show that the container received a SIGTERM at 19:16:10. That looks suspiciously close to 30 seconds after kubernetes initiated the pod. Does your deployment have a timeout that terminates a container if it's not re

Re: [External] Allow parallel processing

2019-11-18 Thread Eric Azama
I second Dave's suggestion. With regards to the consumers round-robining between topics, they usually round-robin in batches. So you'll probably see a consumer work on a large batch of records from TopicA before moving on to TopicB. Depending on the behavior of the producers this might appear the

Re: [kafka-clients] [VOTE] 2.4.0 RC0

2019-11-18 Thread Eric Lalonde
This test has been failing when executed from the command line. I have not run this test in an IDE. > On Nov 18, 2019, at 6:16 AM, Bruno Cadonna wrote: > > Hi, > > ClientMetricsTest.shouldAddCommitIdMetric should only fail if executed > from an IDE. The test fails because the test expects a f

setting Kafdrops 3 UI properties

2019-11-18 Thread khoi dinh
How can I set the truststore and keystore password? Every time I run the program, I get SSL handshake error. Did anyone encounter this error and know how to fix it?