[
https://issues.apache.org/jira/browse/GEODE-9347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413910#comment-17413910
]
ASF subversion and git services commented on GEODE-9347:
--------------------------------------------------------
Commit 5cbb203b2a267a92d440f2daed58f080312c5263 in geode's branch
refs/heads/develop from Darrel Schneider
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=5cbb203 ]
GEODE-9347: optimize pubSub (#6814)
* reworked PubSub data structures.
Each server keeps track of all its local subscriptions in a SubscriptionManager.
The SubscriptionManager keeps track of Clients subscribed in a
ClientSubscriptionManager.
Each Client keeps track of its subscription names in a set.
Client now keeps track of channel and patterns it has subscribed to making it
easier to check. This will improve performance of non-pubsub ops when a server
has existing subscriptions.
The internal Subscriptions class now uses maps instead of lists. Concurrent
maps are used and the synchronization in the pubsub internal implementation has
been removed.
* Some of the messages have been optimized by sending a canonical byte[]
instead of converting a String to a byte[] for every message.
* If many clients subscribe to the same pattern, publish will now only need to
check that pattern once. The older code checked once per client.
* The Subscription class is now smaller since some of its fields are now keys
in the Manager maps.
*Now we have a single channel shutdown listener per Client.
Before we had one for every Subscription and unsubscribe did
not clean them up. If the same client subscribed and unsubscribed this would
result in a memory leak.
> Make Publish and Subscribe use Map data structures
> --------------------------------------------------
>
> Key: GEODE-9347
> URL: https://issues.apache.org/jira/browse/GEODE-9347
> Project: Geode
> Issue Type: Improvement
> Components: redis
> Reporter: John Hutchison
> Assignee: Darrel Schneider
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> our current implementation of Publish and Subscribe use a list of
> Subscription objects to maintain state.
> @doevans noticed that native redis is using 2 maps to maintain state
> in an effort to mimic native redis data structures and hopefully improve
> performance, our publish and subscribe commands should also use maps 1)
> (channel -> all subscribers). 2) subsrciber(client) -> all subscribed channels
> Updated 08/20/2021:
> Redis handles subscriptions as follows:
> * Each client object maintains a set of channels to which it is subscribed
> * Each client object maintains a list of patterns to which it is subscribed
> * The server maintains a map of channels -> subscribed clients
> * The server maintains a list of all Pattern objects
> * Each Pattern object knows which client is subscribed to it and the byte[]
> pattern associated with it
> On subscribe:
> * Add the channel to the client's set of subscribed clients
> * Add the channel to the server's map of channels -> subscribed clients
> * Respond with the total number of subscribed channels and patterns for the
> client (sum the sizes of the client's channel set and pattern list)
> On psubscribe:
> * Add the byte[] pattern to the client's list of patterns
> * Create a Pattern object and add it to the server's list of Patterns
> * Respond with the total number of subscribed channels and patterns for the
> client (sum the sizes of the client's channel set and pattern list)
> On publish:
> * Notify all clients subscribed to the channel using the server's channels ->
> subscribed clients map
> * Notify all clients subscribed to matching channels by comparing the channel
> to the server's list of all patterns using Glob-style matching
> * Respond with the number of notifications sent
> For our implementation, a similar approach can be used, but this will entail
> significant refactoring of PubSub classes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
