Hi, Right now, Pulsar provides the topic's stats and stats-internal over HTTP admin API, and this stats data is used by user applications and also by Pulsar internal components such as Pulsar-functions to derive the certain states of the applications. for example, there are use cases where the application wants to check the topic's backlog, subscription's state (readPosition, list of subscriptions), numberOfEntriesSinceFirstNotAckedMessage, etc to bootstrap the application or handle the application’s resiliency and state dynamically. Applications can retrieve this stats information by using the broker’s admin HTTP APIs.
However, stats retrieval over HTTP API doesn’t work well in use cases when users would like to access this API at a higher scale when a large number of application nodes would like to use it over HTTP which could overload brokers and sometimes makes broker irresponsive and impact admin API performance. It also becomes difficult when Pulsar is deployed in the cloud behind the SNI proxy and applications also want to access large-scale stats information periodically over different HTTP ports. Instead it would be better if applications can fetch stats over on the same binary protocol for scalability and accessibility reasons. Therefore, there are multiple use cases where producer/consumer applications need stats information for topics using the client library over binary protocol. Hence, this PIP introduces client API for producers and consumers to access topic stats/internal-stats information which can be used by applications as needed. Please visit and review the PIP: https://github.com/apache/pulsar/issues/20265 Thanks, Rajan
