Hi Galder On Mon, Oct 24, 2011 at 1:46 PM, Galder Zamarreño <gal...@redhat.com> wrote: > > On Oct 24, 2011, at 12:04 PM, Dan Berindei wrote: > >> ISPN-1470 (https://issues.jboss.org/browse/ISPN-1470) raises an >> interesting question: if the preloading happens before joining, the >> preloading code won't know anything about the consistent hash. It will >> load everything from the cache store, including the keys that are >> owned by other nodes. > > It's been defined to work that way: > https://docs.jboss.org/author/display/ISPN/CacheLoaders > > Tbh, that will only happen in shared cache stores. In non-shared ones, you'll > only have data that belongs to that node. >
Not really... in distributed mode, every time the cache starts it will have another position on the hash wheel. That means even with a non-shared cache store, it's likely most of the stored keys will no longer be local. Actually I just noticed that you've fixed ISPN-1404, which looks like it would solves my problem when the cache is created by a HotRod server. I would like to extend it to work like this by default, e.g. by using the transport's nodeName as the seed. >> I think there is a check in place already so that the joiner won't >> push stale data from its cache store to the other nodes, but we should >> also discard the keys that don't map locally or we'll have stale data >> (since we don't have a way to check if those keys are stale and >> register to receive invalidations for those keys). > > +1, only for shared cache stores. > >> >> What do you think, should I discard the non-local keys with the fix >> for ISPN-1470 or should I let them be and warn the user about >> potentially stale data? > > Discard only for shared cache stores. > > Cache configurations should be symmetrical, so if other nodes preload, > they'll preload only data local to them with your change. > Discarding works fine from the correctness POV, but for performance it's not that great: we may do a lot of work to preload keys and have nothing to show for it at the end. Enabling the fixed hash seed by default should make the performance issue go away. I think it would also require virtual nodes enabled by default and a way to ensure that the nodeNames are unique across the cluster. Cheers Dan >> >> Cheers >> Dan >> >> >> On Mon, Oct 3, 2011 at 3:09 AM, Manik Surtani <ma...@jboss.org> wrote: >>> >>> On 28 Sep 2011, at 10:56, Dan Berindei wrote: >>> >>> I'm not sure if the comment is valid though, since the old >>> StateTransferManager had priority 55 and it also cleared the data >>> container before applying the state from the coordinator. I'm not sure >>> how preloading and state transfer are supposed to interact, maybe >>> Manik can help clear this up? >>> >>> Hmm - this is interesting. I think preloading should happen first, since >>> the cache store may contain old data. >>> -- >>> Manik Surtani >>> ma...@jboss.org >>> twitter.com/maniksurtani >>> Lead, Infinispan >>> http://www.infinispan.org >>> >>> >>> > > -- > Galder Zamarreño > Sr. Software Engineer > Infinispan, JBoss Cache > > _______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev