On Mon, Oct 24, 2011 at 4:42 PM, Sanne Grinovero <sa...@infinispan.org> wrote: > On 24 October 2011 12:58, Dan Berindei <dan.berin...@gmail.com> wrote: >> Hi Galder >> >> On Mon, Oct 24, 2011 at 1:46 PM, Galder Zamarreño <gal...@redhat.com> wrote: >>> >>> On Oct 24, 2011, at 12:04 PM, Dan Berindei wrote: >>> >>>> ISPN-1470 (https://issues.jboss.org/browse/ISPN-1470) raises an >>>> interesting question: if the preloading happens before joining, the >>>> preloading code won't know anything about the consistent hash. It will >>>> load everything from the cache store, including the keys that are >>>> owned by other nodes. >>> >>> It's been defined to work that way: >>> https://docs.jboss.org/author/display/ISPN/CacheLoaders >>> >>> Tbh, that will only happen in shared cache stores. In non-shared ones, >>> you'll only have data that belongs to that node. >>> >> >> Not really... in distributed mode, every time the cache starts it will >> have another position on the hash wheel. >> That means even with a non-shared cache store, it's likely most of the >> stored keys will no longer be local. >> >> Actually I just noticed that you've fixed ISPN-1404, which looks like >> it would solves my problem when the cache is created by a HotRod >> server. I would like to extend it to work like this by default, e.g. >> by using the transport's nodeName as the seed. >> >>>> I think there is a check in place already so that the joiner won't >>>> push stale data from its cache store to the other nodes, but we should >>>> also discard the keys that don't map locally or we'll have stale data >>>> (since we don't have a way to check if those keys are stale and >>>> register to receive invalidations for those keys). >>> >>> +1, only for shared cache stores. >>> >>>> >>>> What do you think, should I discard the non-local keys with the fix >>>> for ISPN-1470 or should I let them be and warn the user about >>>> potentially stale data? >>> >>> Discard only for shared cache stores. >>> >>> Cache configurations should be symmetrical, so if other nodes preload, >>> they'll preload only data local to them with your change. >>> >> >> Discarding works fine from the correctness POV, but for performance >> it's not that great: we may do a lot of work to preload keys and have >> nothing to show for it at the end. > > Can't you just skip loading state and be happy with the state you > receive from peers? More data will be lazily loaded. > Applying of course only when you're not the only/first node in the > grid, in which case you have to load. >
Right, we could preload only on the first node. With a shared cache store this should work great, we just have to start preloading after we connect to the cluster and before we send the join request. But I have trouble visualizing how a persistent (purgeOnStartup = false) non-shared cache store should to work until we have some validation mechanism like in https://issues.jboss.org/browse/ISPN-1195. Should we even allow this kind of setup? > The only alternative I see is to be able to find the boundaries of > keys you own, and change the CacheLoader API to load keys by the > identified range - should work with multiple boundaries too for > virtualnodes, but this is something that not all CacheLoaders will be > able to implement, so it should be an optional API; for now I'd stick > with the first option above as I don't see how we can be more > efficient in loading the state from CacheLoaders than via JGroups. > > Sanne > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev