Does the kafka controller protocol have a backup?

2024-04-18 Thread Dima Brodsky
Hello, I am in the process of setting up a kafka cluster which is configured to use KRaft. There is a set of three controller nodes and a set of six brokers. Both the controllers and the brokers are configured to use mTLS (Mutual TLS). So the part of the controller config looks like:

Re: Kraft controller readiness checks

2024-04-18 Thread Luke Chen
Hello Frank, That's a good question. I think we all know there is no "correct" answer for this question. But I can share with you what our team did for it. Readiness: controller is listening on the controller.listener.names The rationale behind it is: 1. The last step for the controller node

Kraft controller readiness checks

2024-04-18 Thread Francesco Burato
Hello, I have a question regarding the deployment of Kafka using Kraft controllers in a Kubernetes environment. Our current Kafka cluster is deployed on K8S clusters as statefulsets without operators and our brokers are configured to use a script which marks the containers as unready if

Re: Streams group final result: EmitStrategy vs Suppressed

2024-04-18 Thread Matthias J. Sax
The main difference is the internal implementation. Semantically, both are equivalent. suppress() uses an in-memory buffer, while `emitStrategy()` does not, but modifies the upstream aggregation operator impl, and waits to send results downstream, and thus, it's RocksDB based. -Matthias

Re: Is there any recommendation about header max size?

2024-04-18 Thread Matthias J. Sax
I don't think that there is any specific recommendation. However, there is an overall max-message-size config that you need to keep in mind. -Matthias On 4/16/24 9:42 AM, Gabriel Giussi wrote: I have logic in my service to capture exceptions being thrown during message processing and produce

Re: [ANNOUNCE] New Kafka PMC Member: Greg Harris

2024-04-18 Thread Matthias J. Sax
Congrats Greg! On 4/15/24 10:44 AM, Hector Geraldino (BLOOMBERG/ 919 3RD A) wrote: Congrats! Well deserved From: d...@kafka.apache.org At: 04/13/24 14:42:22 UTC-4:00To: d...@kafka.apache.org Subject: [ANNOUNCE] New Kafka PMC Member: Greg Harris Hi all, Greg Harris has been a Kafka