Hi Pulsar community, I would like to start a discussion on PIP-497: Subscription-scoped internal statistics for persistent topics.
Today, diagnosing a single subscription cursor requires fetching the complete topic-level internalStats response. That response includes all managed-ledger cursors and unrelated topic-level internal state, and individuallyDeletedMessages can make it especially large. The goal of this PIP is to provide the same existing cursor diagnostics with a narrower request and response scope. The proposal is to: - add subscription-scoped REST, Java Admin, and pulsar-admin topics access; - reuse the existing ManagedLedgerInternalStats.CursorStats contract without adding new cursor fields; - omit individuallyDeletedMessages by default and expose it through an includeIndividualDeletedMessages option; - support both a non-partitioned topic or explicit partition and a partitioned base topic; - return independent per-partition results with explicit partial failures instead of inventing an aggregated cursor position; and - keep the operation read-only and use the existing TopicOperation.GET_STATS authorization model. The main points on which I would appreciate feedback are: 1. Should the first version provide broker-side fan-out for partitioned topics, or require callers to query explicit partitions? 2. Is a dedicated subscription resource preferable to adding filtering semantics to the existing topic-level internalStats resource? 3. Should the potentially large individuallyDeletedMessages value be opt-in, or should the new endpoint preserve full parity by default? 4. Is GET_STATS the appropriate authorization model, or should this be tied to subscription-level consume permission? The proposal intentionally does not add a generic API for arbitrary internal cursor names. Replication, deduplication, and transaction cursors have different semantics and may need component-specific diagnostics in separate proposals. PIP draft: https://github.com/apache/pulsar/pull/27014 Motivating prototype (the API shape may differ from the current PIP draft): https://github.com/apache/pulsar/pull/27010 Feedback is welcome in this thread. Inline comments on the draft PR are also appreciated. Thanks, void-ptr974
