Re: [VOTE] 2.3.1 RC2

It appears issue 8950 is merged and in the rc2 release but I'm not seeing it in the release notes: https://issues.apache.org/jira/browse/KAFKA-8950 Would be great to get confirmation that it will be included and would be awesome to see it in the release notes. Thanks! Jonathan On Fri, Oct 18,

Re: Help Needed - Message Loss for Consumer groups in 2.3.0 (client 2.2.0)

The issue was due to the fact the some messages were corrupted and didn't get processed at the subscriber end, so nothing to do with Kafka. Since the error logs were not stored anywhere, we had to troubleshoot the live system in debug mode to capture message logs. On Mon, 21 Oct 2019 at 01:13,

Kafka-View - Free Browsing of Cluster

Hi All, https://github.com/SourceLabOrg/kafka-webview Not sure if anyone has come across this. Very nice tool indeed and has Spring boot baseline. another one is - Kafka Magic viewer http://www.kafkamagic.com/ Are they worth covering in cwiki? Thanks,

Re: Allow keys to specify partitionKey

Hey Jan, correct me if I'm wrong, my understanding of your problem is as follows: 1. This is a Kafka Streams question 2. You want to preserve the same primary key, but to repartition based on some certain field with key/value If this is the case, have you considered using `through(topic,

Re: Running a stand alone Mirror Maker 2.0 cluster and Issues

Vishal, the number of tasks created per source->target herder is determined by both tasks.max and the total number of topic-partitions being replicated. In order to use all 12 worker nodes, you'd need tasks.max >= 12 and number of topic-partitions >= 12. From previous emails it sounds like you

Re: Kafka Streams State stuck in rebalancing after one of the StreamThread encounters java.lang.IllegalStateException: No current assignment for partition

Hey Amuthan, I replied you on the Slack community. Feel free to choose either continue the discussion here or in Slack. Boyang On Mon, Oct 21, 2019 at 1:06 PM Amuthan wrote: > I have a Kafka stream application that stores the incoming messages into a > state store, and later during the

Kafka Streams State stuck in rebalancing after one of the StreamThread encounters java.lang.IllegalStateException: No current assignment for partition

I have a Kafka stream application that stores the incoming messages into a state store, and later during the punctuation period, we store them into a big data persistent store after processing the messages. The application consumes from 120 partitions distributed across 40 instances. The

Allow keys to specify partitionKey

The default partitioner takes a hash of the key of a topic to determine the partition number. It would be useful for a key to be able to specify the object on which the default partitioner should base its hash on. This would allow us to use different composite keys and still be certain that

Re: Impact on having large number of consumers on producers / brokers

What are you trying to do here ? whats your objective ? On Sat, Oct 19, 2019 at 8:45 PM Hrishikesh Mishra wrote: > Can anyone please help me with this? > > On Fri, 18 Oct 2019 at 2:58 PM, Hrishikesh Mishra > wrote: > > > Hi all, > > > > I wanted to understand, having large numbers of consumers

NotEnoughReplicasException: Messages are rejected since there are fewer in-sync replicas than required..

Hi there, While rolling deploying SSL changes to get Kafka working over SSL + inter broker protocol updated from SASL_PLAINTEXT to SSL, getting like problems to publish messages with: org.apache.kafka.common.errors.NotEnoughReplicasException: Messages are rejected since there are fewer