GitHub user lhotari added a comment to the discussion: Key_Shared consumer scale behavior with a partitioned topic
There's yet another aspect of the individual hash tracking. Currently, when new consumers join, all message delivery is blocked for the consumer until the inflight messages that were sent before the consumer joined have been processed. The reason for this behavior is to prevent out-of-order handling in cases since the hash assignments change in `KeySharedMode.AUTO_SPLIT` when consumers join and leave. This solution would also benefit of individual hash tracking. [PIP-282: Change definition of the recently joined consumers position](https://github.com/apache/pulsar/blob/master/pip/pip-282.md) will be delivered in Pulsar 4.0 and it contains important improvements. However, it seems that the solution isn't yet the most optimal design that would be possible and practical. The updated design should also allow sending events to Pulsar clients which hash assignments change. There's an open issue https://github.com/apache/pulsar/issues/6555 about this. The common use case is to keep a cache and for cache invalidation, there's a need to know when a message key will no longer receive updates on a specific consumer when using the AUTO_SPLIT mode. A side note here is that for caching use cases, the AUTO_SPLIT is usually not applicable and there's already the STICKY mode where hash assignments can be controlled by the application. The challenge with STICKY mode is that the application must make sure to cover the complete hash range. Implementing this on the application side is currently tedious and it would be useful to provide best practices and patterns of implementing this in Kubernetes. One possible solution is to use a Kubernetes StatefulSet and calculate the assigned sticky hash ranges based on the ordinal index of the pod. This way the hash range would always be handled by the same pod and there would be only a single instance. The challenge is when the StatefulSet is scaled up and down. The StatefulSet would have to be stopped completely before changing the StatefulSet size. The benefit of this solution is that it has worked already for years in Pulsar with Key_Shared subscriptions, but this pattern isn't well known. GitHub link: https://github.com/apache/pulsar/discussions/22912#discussioncomment-10589453 ---- This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@pulsar.apache.org
