[
https://issues.apache.org/jira/browse/SOLR-13439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16850988#comment-16850988
]
Gus Heck commented on SOLR-13439:
---------------------------------
I realized two things:
First, my changes for SOLR-13418 have made it possible to easily distinguish
deletion from removing the last property on a collection that still exists (via
a value of -1 for zk version that I chose to return in this case and the
existence of a prior ZK version other than -1 for our cached properties). That
allows us to solves all the sticky problems about how to get rid of stuff when
collections have been deleted.
Second I did misunderstand your proposal. When you said _optionally set a
“timed watch”_, I thought you meant a CollectionPropertiesWatcher similar to
your prior suggestions of "just setting a watch". I still had your old
suggestions in my head, but now I realize you only meant altering PropsWatcher,
which is much better.
Taken together, I think those realizations mean that your way works to my
satisfaction and is cleaner too... Most of my reason for the expiring caches
was to guard against orphaned watches and things that might accumulate, but we
now have good control of what happens when a collection disappears so the
"quiet" case where watches never fire is our worst case and that's not soluble
by either solution, but can be limited to just one watch per collection once
SOLR-13491 is in. Also the above discussion has helped clarify ZK's actual
behavior (to me at least :) ) and several things I worried about initially are
non-issues.
> Make collection properties easier and safer to use in code
> ----------------------------------------------------------
>
> Key: SOLR-13439
> URL: https://issues.apache.org/jira/browse/SOLR-13439
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Affects Versions: master (9.0)
> Reporter: Gus Heck
> Assignee: Gus Heck
> Priority: Major
> Attachments: SOLR-13439.patch, SOLR-13439.patch
>
>
> (breaking this out from SOLR-13420, please read there for further background)
> Before this patch the api is quite confusing (IMHO):
> # any code that wanted to know what the properties for a collection are
> could call zkStateReader.getCollectionProperties(collection) but this was a
> dangerous and trappy API because that was a query to zookeeper every time. If
> a naive user auto-completed that in their IDE without investigating, heavy
> use of zookeeper would ensue.
> # To "do it right" for any code that might get called on a per-doc or per
> request basis one had to cause caching by registering a watcher. At which
> point the getCollectionProperties(collection) magically becomes safe to use,
> but the watcher pattern probably looks famillar induces a user who hasn't
> read the solr code closely to create their own cache and update it when their
> watcher is notified. If the caching side effect of watches isn't understood
> this will lead to many in-memory copies of collection properties maintained
> in user code.
> # This also creates a task to be scheduled on a thread (PropsNotification)
> and induces an extra thread-scheduling lag before the changes can be observed
> by user code.
> # The code that cares about collection properties needs to have a lifecycle
> tied to either a collection or something other object with an even more
> ephemeral life cycle such as an URP. The user now also has to remember to
> ensure the watch is unregistered, or there is a leak.
> After this patch
> # Calls to getCollectionProperties(collection) are always safe to use in any
> code anywhere. Caching and cleanup are automatic.
> # Code that really actually wants to know if a collection property changes
> so it can wake up and do something (autoscaling?) still has the option of
> registering a watcher that will asynchronously send them a notification.
> # Updates can be observed sooner via getCollectionProperties with no need to
> wait for a thread to run. (vs a cache held in user code)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
