Re: ConcurrentModificationException in Kafka Producer 2.1.0

2018-12-05 Thread Guozhang Wang
Hmm.. seems like a regression introduced in since "getExpiredInflightBatches". Actually, just found someone already file a JIRA with a PR: https://github.com/apache/kafka/pull/6005/files Guozhang On Wed, Dec 5, 2018 at 6:54 AM Mark Anderson wrote: > Hi, > > I'm periodically seeing

ConcurrentModificationException in Kafka Producer 2.1.0

2018-12-05 Thread Mark Anderson
Hi, I'm periodically seeing ConcurrentModificationExceptions in the producer when records are expired e.g. ERROR Dec 05 11:56:13.033 388753 [kafka-producer-network-thread | analogDataProducer] com.x.AnalogMessageBundler Exception org.apache.kafka.common.errors.TimeoutException: Expiring 1

Re: Kafka producer throw "TimeoutException: Batch Expired" when consumer begin to read data

2018-12-05 Thread Ismael Juma
This is likely the issue fixed by KIP-91. You'll get the right behaviour if you upgrade to 2.1 or you can increase the request timeout. Ismael On Tue, Dec 4, 2018, 10:12 PM JF Chen I have a system keep sending data to kafka continuously, and a spark > streaming application reading these data

Problem with Kafka Connector

2018-12-05 Thread Федор Чернилин
Hello! I have question. We have cluster with several connect workers. And we have many different connectors. We need to set for each connector its own settings, max.in.flight.requests.per.connection , partitioner.class, acks. But I have difficulties. How can I do that? Thanks

Re: How to pull the data from mobile app directly into Kafka broker

2018-12-05 Thread Satendra Pratap Singh
Hi Sönke, yeah i have done that, i don't understand couple of thing like how to configure with mobile app like in my app whatever url i will put it hit the logs there. what url it should be? and how rest will post that log to kafka. My problem is quite straight forward how to push messages using

Re: How to pull the data from mobile app directly into Kafka broker

2018-12-05 Thread Sönke Liebau
Hi Satendra, have you tried following the quickstart [1] on the Confluent page? This will help you get a feel for the rest proxy and how to interact with it. Configuration is also explained on the same page [2] - it doesn't take much beyond your Kafka brokers and Zookeeper nodes to get you