andrasbeni opened a new pull request, #16062: URL: https://github.com/apache/pulsar/pull/16062
Fixes #14505 ### Motivation When using a pattern-based subscription, clients poll for new topics that match the pattern. It can happen frequently that the first messages produced to the topic will be consumed with a significant delay because the client has not got information about the topic for a while. This commit introduces topic list watchers. By using these objects clients act as observers of the creation or deletion of topics closer to real-time. This reduces latency in consuming the first messages published to a topic when using a pattern-based subscription. ### Modifications - New commands were added to the binary protocol to enable registering and deregistering watchers. - Pattern-based consumers create TopicListWatcher objects if the broker supports this feature. Otherwise, they fall back to polling only. - The watchers use ConnectionHandler to obtain a connection to a broker. - Once connected, watchers register and wait for updates. - ServerCnx uses the newly created TopicListService to manage watchers. - TopicListService listens to metadata notifications and sends updates. ### Verifying this change - [ ] Make sure that the change passes the CI checks. This change added tests ### Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API: no - The schema: no - The default values of configurations: no - The wire protocol: yes - Adds new commands and a new optional field to an existing command - The rest endpoints: no - The admin cli options: no - Anything that affects deployment: no ### Documentation Check the box below or label this PR directly. Need to update docs? - [x] `doc-required` (Your PR needs to update docs and you will update later) - [ ] `doc-not-needed` (Please explain why) - [ ] `doc` (Your PR contains doc changes) - [ ] `doc-complete` (Docs have been already added) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org