KIP-794 inhibits Partitioner delegation with no fallback option

2022-12-15 Thread James Olsen
Prior to KIP-794 it was possible to create a custom Partitioner that could delegate to the DefaultPartitioner. This has been deprecated so we can now only delegate to BuiltInPartitioner.partitionForKey which does not handle a non-keyed message. Hence there is now no mechanism for a custom Part

[VOTE] 3.3.2 RC0

2022-12-15 Thread Chris Egerton
Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 3.3.2. This is a bugfix release with several fixes since the release of 3.3.1. A few of the major issues include: * KAFKA-14358 Users should not be able to create a regular topic name __c

Kafka's scalability

2022-12-15 Thread Péter Sinóros-Szabó
Hi, We have a Kafka cluster that's growing a lot year to year and I was wondering how much it can handle without custom changes. If you're running a large Kafka cluster or if you're aware of one, please share some data around it: - how many brokers is in the cluster and CPUs / RAM used - how many

Need clarity on a statement in the Upgrade path

2022-12-15 Thread Swathi Mocharla
Hello, In the upgrade docs, it is mentioned "Restart the brokers one by one for the new protocol version to take effect. *Once the brokers begin using the latest protocol version, it will no longer be possible to downgrade the cluster to an older version.*" What does this mean? Is this statement a

Is there an option to ignore the apiVersion between Kafka versions

2022-12-15 Thread Swathi Mocharla
Hello, We are trying to upgrade between 2 versions of kafka where the apiVersions of multiple apis have changed. During upgrade we notice a number of UnsupportedVersionExceptions which we want to ignore (during upgrade). Few questions: 1. Is it ok to ignore mismatch of apiVersions? 2. Is the