Hi all,

I'd like to propose PIP-474: Key_Shared Hot Key Overflow Mechanism.

Key_Shared is Pulsar's only built-in solution for parallel consumption
with per-key ordering. But it has a critical production issue: a
single stuck consumer can starve ALL other keys across ALL partitions
within minutes, due to the containsStickyKeyHash ordering check
flooding the Replay queue.

This becomes especially urgent as AI inference workloads adopt MQ as
their transport layer — slow consumption (seconds per request) plus
strict per-key ordering is exactly what Key_Shared is designed for,
yet the hot-key starvation bug makes it unusable in production.

PIP-474 proposes diverting hot-key messages to an independent Overflow
ManagedLedger, unblocking Normal Read and mark-delete advancement
while preserving at-least-once delivery and per-key ordering. Zero
overhead when no hot keys are present.

PIP: https://github.com/apache/pulsar/pull/25706

Feedback welcome.

Thanks, Xiangying Meng

Reply via email to