Dynamic update of server-side keystores - apache kafka 2.3.1

2020-08-21 Thread PraneethKumar
I have few queries related to enabling SSL/TLS on kafka with dynamic update of certs using kafka-configs.sh and i am running into issues when certs are expired Server Side === I have generated short lived keystore and truststore certificates (4hrs expiry time) and placed details in

Re: Kafka Producer - Producing to Multiple Topics

2020-08-21 Thread SenthilKumar K
it would be great if someone provides input(s)/hint :) thanks! --Senthil On Fri, Aug 21, 2020 at 3:28 PM SenthilKumar K wrote: > Updating the Kafka broker version: > > Kafka Version: 2.4.1 > > On Fri, Aug 21, 2020 at 3:21 PM SenthilKumar K > wrote: > >> Hi Team, We have deployed 150 node Kafk

Re: Leader count is skewed on kafka brokers

2020-08-21 Thread rohit garg
Hello, If it is getting skewed means brokers are restarting . You can prepare manual assignment once and rerun same assignment again for preferred election. On Fri, Aug 21, 2020, 19:57 aishwary thakur wrote: > Hi, > I have deployed Kafka on K8S cluster over 3 nodes. > Leaders are not getting e

Re: MirrorMaker 2.0 - Translating offsets for remote topics and consumer groups

2020-08-21 Thread Ryanne Dolan
Josh, make sure there is a consumer in cluster B subscribed to A.topic1. Wait a few seconds for a checkpoint to appear upstream on cluster A, and then translateOffsets() will give you the correct offsets. By default MM2 will block consumers that look like kafka-console-cosumer, so make sure you sp

Leader count is skewed on kafka brokers

2020-08-21 Thread aishwary thakur
Hi, I have deployed Kafka on K8S cluster over 3 nodes. Leaders are not getting equally distributed over the brokers. I have tried kafka-preferred-replica-election.sh script also. Sometime it works and sometime it doesn't. Even when it works, over time it again gets skewed. Any help would be appreci

Re: Kafka Producer - Producing to Multiple Topics

2020-08-21 Thread SenthilKumar K
Updating the Kafka broker version: Kafka Version: 2.4.1 On Fri, Aug 21, 2020 at 3:21 PM SenthilKumar K wrote: > Hi Team, We have deployed 150 node Kafka cluster on production for our > use case. Recently I have seen issue(s) in Kafka Producer Client. > > Use Case: > --> (Consume)Stream App (M

Kafka Producer - Producing to Multiple Topics

2020-08-21 Thread SenthilKumar K
Hi Team, We have deployed 150 node Kafka cluster on production for our use case. Recently I have seen issue(s) in Kafka Producer Client. Use Case: --> (Consume)Stream App (Multiple Topologies) (Transform) --> Kafka Producer Topology ( Produce it to Multiple Topics) Initially, the data is writte