Re: Help with log.dirs please

2022-09-15 Thread Peter Bukowinski
The next thing I’d check is the broker logs. The parameters read from the config should appear in the logs when kafka starts up. Search the logs for ‘log.dirs’ and ensure the correct configs are loaded. -- Peter > On Sep 15, 2022, at 11:10 PM, Chris Peart wrote: > > Hi Peter, > > Thanks for

Re: Help with log.dirs please

2022-09-15 Thread Chris Peart
Hi Peter, Thanks for your response, we have the following configuration: Partition count=4 Replication factor=3 All four brokers have topics-partitions in /data/1/kafka/data and are receiving data. Each server has 6 x 2TB disks for kaka data. Many Thanks, Chris > On 16 Sep 2022, at 1:56 am

Re: Kafka consumer ensure offsets committed before partition rebalance to avoid duplicates

2022-09-15 Thread Luke Chen
Hi Pushkar, Here's the answer to your questions: > 1. During scale-down operation, I am adding a shutdown hook to the Java Runtime, and calling close on the consumer. As per kafka docs, close provides 30 sec to commit current offsets if auto.commit is enabled: so, i assume that it will process th

Re: Help with log.dirs please

2022-09-15 Thread Peter Bukowinski
Hi Chris, Can you share the partition count and replication factor of your partitions? Also, do all four brokers contain topic-partition directories in /data/1/kafka/data or just a single broker? Depending on your topic config, it may be entirely normal that his has happened. — Peter Bukowins

mTLS client hostname validation is not working

2022-09-15 Thread James Ziesig
Hi, I have configured mTLS on a three server Kafka cluster. The servers and clients are all communicating properly, except I am having trouble with client hostname validation when the client is using a cert from a different host. I would expect this to fail on handshake like it does when the cer

Mirror maker disable topic creation

2022-09-15 Thread Mcs Vemuri
Hello, Is there any way to disable topic creation in MM2? I tried setting the topic.creation.enable to False but the MM-offsets/configs/status topics are still created(broker auto topic creation is also set to false) I found this- https://groups.google.com/g/confluent-platform/c/JRZmpCEZElo  whi

Re: Mirror Maker is not replicating WRITE ACLs for topics and consumer group ACLs

2022-09-15 Thread John Lewis
Hello, this is a gentle reminder. Any response will be appreciated. John ср, 7 сент. 2022 г. в 15:03, John Lewis : > Hi everyone, > > I installed and ran Mirror Maker 2 to replicate all possible data from > Active cluster to Standby one. It's replicating topics, consumer groups, > topic READ AC

Re: Missing partitions after rebalancing

2022-09-15 Thread Daria Bulanova
Hi again, I have a lot of logs. Would someone like to collaborate on this strange situation? I feel like it's very interesting. We have been missing several partitions for a month. Which is very strange for fault-tolerated products)). I'm sure that I have some missing parts from this puzzle that'

Help with log.dirs please

2022-09-15 Thread Chris Peart
Hi All, I have a 4 node kafka cluster running version 2.8.1, we have started pushing data to the cluster but can only see one disk being used. We had 6 disk configured as non-raid and 1 partition per disk, we have the following in fstab: /dev/mapper/VolGroup01-data/data/1xfsno