Re: [DISCUSS] Minimum constraint for segment.ms

2024-03-13 Thread Divij Vaidya
+ users@kafka Hi users of Apache Kafka With the upcoming 4.0 release, we have an opportunity to improve the constraints and default values for various Kafka configurations. We are soliciting your feedback and suggestions on configurations where the default values and/or constraints should be

Re: Kafka Producer avoid sending more records when met exception while also keeping performance

2024-03-13 Thread William Lee
Hi Richard, Thanks for replying. > but I close the KafkaProducer inside the send > callback. > ... > Combined with idempotence enabled > and max inflight set to 5 (the maximum for idempotence tracking) it gave me > relatively good performance. Yes, I also find that closing the KafkaProducer

Re: Kakfa Topic gets auto-created post deletion

2024-03-13 Thread Abhishek Singla
Issue is now resolved by upgrading zookeeper version from 3.4.14 to 3.8.0 On Sun, Feb 18, 2024 at 6:15 PM sunil chaudhari wrote: > Hi Abhishek, > as I told you before, topic is getting created only because of consumer > polling. > The only way is to stop consumer, remove topic name from the

Re: [EXTERNAL] Re: Kafka Streams 3.5.1 based app seems to get stalled

2024-03-13 Thread Bruno Cadonna
Hi Venkatesh, Extending on what Matthias replied, a metadata refresh might trigger a rebalance if the metadata changed. However, a metadata refresh that does not show a change in the metadata will not trigger a rebalance. In this context, i.e., config METADATA_MAX_AGE_CONFIG, the metadata is

Re: Kafka Producer avoid sending more records when met exception while also keeping performance

2024-03-13 Thread Richard Bosch
Hi WIlliam, I see from your example that you close the kafka producer in the send loop, based on the content of sendException that is used in the callback of the KafkaProducer send. Since your send loop is a different thread than the KafkaProducer uses to send you will encounter race conditions