I think I found it. During partial key removeall the Disk Element Descriptor, which identifies the slot on disk, was being added to the recylebin. Then the removeSingleItem was called for each match. The remove single item method also adds the item to the recylebin. The recyclebin is not a set. The spot would get entered twice. This is very bad. Here's why:
The spot gets used. Another put uses the same spot. Both keys in the key map point to the same spot. A get for the first key will pull the data for the second. Not so good. I'm verifying this and I'll have a fix soon. Aaron --- On Wed, 8/12/09, Aaron Smuts <[email protected]> wrote: > From: Aaron Smuts <[email protected]> > Subject: RE: index cache corruption > To: "JCS Users List" <[email protected]> > Date: Wednesday, August 12, 2009, 6:52 AM > > So were you getting back the wrong data or data that should > have been deleted? > > Are you running a recent temp build? > > I'll dig into the partial key removal. This should > probably deprecated for the remove matching method. . . > . Either way, it should work. > > Aaron > > --- On Tue, 8/11/09, Tim Cronin <[email protected]> > wrote: > > > From: Tim Cronin <[email protected]> > > Subject: RE: index cache corruption > > To: "JCS Users List" <[email protected]> > > Date: Tuesday, August 11, 2009, 2:30 PM > > Good to know, thanks. > > > > Yes that's when we see it too, under high load. > > > > For us it seems to happen around removes, we use the > key: > > linkage and > > It happens to data that's linked and would get cleared > with > > the key: > > call. > > > > > > -----Original Message----- > > From: Al Forbes [mailto:[email protected]] > > > > Sent: Tuesday, August 11, 2009 4:13 PM > > To: JCS Users List > > Subject: Re: index cache corruption > > > > Hi Tim, > > > > I had the same problem. It seems to happen under high > load, > > with fairly > > large objects, but I could not reproduce it offline. > I > > changed to using > > a > > JDBC cache and have not had a problem since then. > > > > We still use disk caches for more static data - mostly > read > > only, and > > the > > problem never happens there, so I assume it's the > writes > > that cause the > > corruption. > > > > Regards > > Al > > > > 2009/8/11 Tim Cronin <[email protected]> > > > > > We initially were using indexed disk cache but > ran > > into cache > > corruption > > > where the data returned for a key would not be > the > > data associated for > > > that key. I haven't been able to come up with a > good > > repro scenario... > > > > > > We had to work around it with the following > code: > > > > > > /** > > > * is the cache element not the > > correct object for the key > > > * @param key the current key > > > * @param element the current element > > associated with the key > > > * @param hasReadLock whether caller > > has read lock > > > * @return true if key mismatch els > > false > > > * @throws InterruptedException if > > locking fails > > > */ > > > private boolean isCorrupt(String key, > > ICacheElement element, boolean > > > hasReadLock) throws InterruptedException > > > { > > > boolean corrupt = > > !key.equals(element.getKey()); > > > if (corrupt) > > > { > > > mLogger.error("cache corruption!!! > > [" + (mCorruptionCounter++) + > > > "]"); > > > > > > if (mLogger.isDebugEnabled()) > > > { > > > mLogger.error("culprit > > stack...", new Exception("cache > > > corruption")); > > > } > > > > > > try > > > { > > > if (hasReadLock) > > > { > > > > > mLock.readLock().release(); > > > } > > > > > mLock.writeLock().acquire(); > > > try > > > { > > > > > mLogger.error("purging " + key); > > > mCache.remove(key); > > > mCache.remove(key + > > ":"); > > > } > > > catch (Exception e) > > > { > > > > > mLogger.error("failed to purge key " + key, e); > > > } > > > try > > > { > > > String k = > > (String)element.getKey(); > > > > > mLogger.error("purging " + k); > > > mCache.remove(k); > > > mCache.remove(k + > > ":"); > > > } > > > catch (Exception e) > > > { > > > > > mLogger.error("failed to purge key " + > element.getKey(), > > e); > > > } > > > } > > > finally > > > { > > > if (hasReadLock) > > > { > > > > > mLock.readLock().acquire(); > > > } > > > > > mLock.writeLock().release(); > > > } > > > } > > > return corrupt; > > > } > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
