Re: Kafka Streams not allowed to change the number of input topic partitions

That is correct. And there is no WIP to address this atm. It's a pretty complicated problem to allow for dynamic scaling. -Matthias On 11/13/19 9:41 AM, ChienHsing Wu wrote: > Hi, > > I came across the links below that indicates it's not simple to increase > partitions in an input topic

Re: producer transaction and threading model

That is not possible. A producer can only have a single open transaction. If your example, the call of thread-2 to start a new transactions would fail and an exception would be thrown. -Matthias On 11/13/19 10:52 AM, Anindya Haldar wrote: > The producer API doc >

Re: kafka-console-consumer --value-deserializer with access to headers

Hi, Sorry for the late reply, here is the command: kafka-console-consumer.cmd --consumer.config my-config.properties --bootstrap-server kafka-host:9443 --topic my-topic --value-deserializer my.BasicDeserializer --group my-console-group --property print.key=true my-config.properties only

produce very slow after upgrade to 2.3.0

hi: recently we tried to upgrade our kafka cluster 0.10.0.1 to 2.3.0. we know kafka upgrade should: 1.use 2.3.0.jar and restart all brokers 2.unset inter.broker.protocol.version=0.10.0.1 and restart all brokers 3.unset log.message.format.version=0.10.0.1 and restart all

Re: help for regarding my question

true On Thu, Nov 14, 2019 at 2:03 PM prashanth sri wrote: > i am new to kafka , can any one help me , regarding my problem. >

help for regarding my question

i am new to kafka , can any one help me , regarding my problem.

Anybody using uReplicator?

Hi, There are lot of articles which talk about known limitations of MirrorMaker 1 (MM1) and how Uber has overcome them with uReplicator. I am also aware that MM2 is getting ready. I have few questions: 1. When will MM2 be GA? 2. Anybody using uReplicator in production other than Uber? If yes,

Re: Partition Reassignment is getting stuck

If only one broker isn't in sync, it can caused by a dead replica fetcher thread in my experience. I fixed it by restarting the affected broker, but this was on 0.11, so YMMV. On Thu, Nov 14, 2019 at 9:35 AM Koushik Chitta wrote: > The topic partition having the ISR issue might be on a

RE: Partition Reassignment is getting stuck

The topic partition having the ISR issue might be on a offline directory. Look into the metric "offlineLogDirectoryCount" or use kafka-log-dirs.sh to understand the issue with that directory. In most cases, it would be the a KafkaStorage Exception. The partition reassignment would also be

producer transaction and threading model

The producer API doc (https://kafka.apache.org/23/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html ) says that KafkaProducer is thread safe. What I was trying to figure

Kafka Streams not allowed to change the number of input topic partitions

Hi, I came across the links below that indicates it's not simple to increase partitions in an input topic afterwards. Is this still true in the current kafka version 2.3.1 and soon to be available 2.4?

Cannot resolve org.apache.kafka:connect-json:5.5.0-ccs-SNAPSHOT

Hi, I'm trying to compile Kafka Connect Connector for S3 ( https://github.com/confluentinc/kafka-connect-storage-cloud) I saw the following things on the kafka-connect-storage-common FAQ - Kafka - clone

Re: Partition Reassignment is getting stuck

On Wed, 13 Nov 2019 at 13:10, Ashutosh singh wrote: > Yeah, Although it wouldn't have any impact but I will have to try this > tonight as it is peak business hours now. > Instead deleting all data I will try to delete topic partitions which are > having issues and then restart the broker. I

Re: Partition Reassignment is getting stuck

Yeah, Although it wouldn't have any impact but I will have to try this tonight as it is peak business hours now. Instead deleting all data I will try to delete topic partitions which are having issues and then restart the broker. I believe it should catch up but I will let you know. On Wed,

Re: Partition Reassignment is getting stuck

On Wed, 13 Nov 2019 at 12:41, Ashutosh singh wrote: > Hi, > > All of a sudden I see under replicated partition in our Kafka cluster and > it is not getting replicated. It seems it is getting stuck somewhere. In > sync replica is missing only form one of the broker it seems there is some >

Partition Reassignment is getting stuck

Hi, All of a sudden I see under replicated partition in our Kafka cluster and it is not getting replicated. It seems it is getting stuck somewhere. In sync replica is missing only form one of the broker it seems there is some issue with that broker but other hand there are many others topic on

Re: Install kafka-connect-storage-cloud

Thank you for yout help. That worked. Regarding this: I need a sink connector that reads json from kafka topics and writes to s3 on parquet files. I need to read several topics and the files are going to the same bucket on different paths. Do you know anything that can do the task? It seems that