We can definitely fix it by making HelixPropertyStore.subscribe(String path, HelixPropertyListener listener) dynamic. The internal zk updated cache is quite complicated though (see ZkCallbackCache), so introducing another dynamic dimension will certainly introduce more race conditions and need more solid testing.
I would also like to know more details about Santi's use case. There are a couple issues need to be considered here: i) the internal zk cache in HelixPropertyStore has lock on most of the operations and zk callbacks, it will probably not perform well in use cases where data change is frequent. ii) zk cache registers zk-watchers on every znodes under subscription paths, so if the use case generates a large number of znodes, it will create a large number of watchers, which is problematic also. Thanks, Jason On Mon, Aug 5, 2013 at 3:45 PM, kishore g <[email protected]> wrote: > I agree with Santi. It is kind of odd that we can only use it by directly > instantiating propertystore but not via manager. Jason, can you provide > more details on why this is the case and how can we fix it. > > thanks, > Kishore G > > > On Mon, Aug 5, 2013 at 12:51 PM, Santiago Perez <[email protected] > >wrote: > > > Could we overload the getHelixPropertyStore() to an alternative that > takes > > these paths as argument? That way if I know beforehand the paths I'll > need > > I can still use this useful api rather than having to deal with lower > level > > objects (IZK*Listeners or ZkClient). > > > > > > > > > > On Mon, Aug 5, 2013 at 3:10 PM, Zhen Zhang <[email protected]> wrote: > > > > > Hi Santi, it's intended. The HelixPropertyListener used in > > > HelixPropertyStore#subscribe() provides callbacks on data > > > change/delete/create. This is based on a ZkCache built inside. The > > ZkCache > > > requires these paths to be initially setup instead of changing > > dynamically. > > > Note that the callback provided by HelixPropertyListener is different > > from > > > those by ZkClient, since it's invoked on every descendant and > > distinguishes > > > between different changes, i.e. when znode is deleted/created. We also > > > expose the original ZkClient callbacks: > > > subscribeDataChanges(String path, IZkDataListener listener) > > > subscribeChildChanges(String path, IZkChildListener listener) > > > > > > This doesn't have the limitation of registering paths in constructor, > but > > > they work only on children znodes and you need to figure out znode > > > delete/create by child change callbacks. > > > > > > I agree that subscribe api should fail when it's not going to work. > > > > > > > > > > > > > > > > > > On Mon, Aug 5, 2013 at 10:40 AM, Santiago Perez <[email protected] > > > >wrote: > > > > > > > I see. Is this intended this way or is it a bug? At the very least I > > > would > > > > expect the subscribe api to fail if it is not really going to work. > > > > > > > > Also it would be great if I didn't have to construct my own based on > a > > > > ZkClient. > > > > > > > > Are there other alternatives for listening to property store changes > > that > > > > can be solved directly from the HelixManager? > > > > > > > > > > > > On Mon, Aug 5, 2013 at 2:25 PM, Zhen Zhang <[email protected]> > > wrote: > > > > > > > > > Hi Santi, you are right. The property store obtained from > > HelixManager > > > is > > > > > not registering with any paths. You should construct your own > > property > > > > > store. > > > > > > > > > > > > > > > On Mon, Aug 5, 2013 at 9:57 AM, Santiago Perez < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > I'm not following. When calling subscribe I'm providing relative > > > path, > > > > > the > > > > > > problem is that the store obtained from the HelixManager is not > > > > listening > > > > > > to subscribed to anything and never will be. > > > > > > > > > > > > > > > > > > On Mon, Aug 5, 2013 at 1:54 PM, kishore g <[email protected]> > > > wrote: > > > > > > > > > > > > > You should probably provide only relative path, not absolute. > If > > > > thats > > > > > > not > > > > > > > the case, we should fix it. > > > > > > > > > > > > > > > > > > > > > On Mon, Aug 5, 2013 at 9:51 AM, Santiago Perez < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > I see, but I'm obtaining the property store via the > > HelixManager. > > > > > > Should > > > > > > > I > > > > > > > > be constructing it myself? > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Aug 5, 2013 at 1:43 PM, Zhen Zhang < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > > > yes. you need to construct with a list of paths. subscribe > () > > > > works > > > > > > > with > > > > > > > > > any sub paths of the constructed paths. > > > > > > > > > On Aug 5, 2013 9:34 AM, "Santiago Perez" < > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hey guys, > > > > > > > > > > > > > > > > > > > > I'm trying to rely on > > ZkHelixPropertyStore.subscribe(String, > > > > > > > > > > HelixPropertyListener) to listen for changes to certain > > paths > > > > in > > > > > > the > > > > > > > > > > property store but I'm never getting any callbacks. > > > > > > > > > > > > > > > > > > > > Looking into the code I noticed that > > > > > > > > HelixManager.getHelixPropertyStore() > > > > > > > > > > is creating a ZkHelixPropertyStore with null in the > > > > > subscribedPaths > > > > > > > > > > argument. This prevents the event thread from being > started > > > and > > > > > in > > > > > > > fact > > > > > > > > > the > > > > > > > > > > log confirms that with line: > > > > > > > > > > "ZkCachePaths is null or empty. Will not start > > > > > ZkCacheEventThread" > > > > > > > > > > > > > > > > > > > > So everything seems to indicate that this is the cause > why > > I > > > > will > > > > > > > never > > > > > > > > > get > > > > > > > > > > a callback on property store changes. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Santiago > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
