Thanks Mike. If you look at my impl, I am using the getCoreCacheKey() only, but keyed on a ReaderClosedListener and purging it onClose(). When NRT does reopens, will it invoke the onClose() method for the expired-reader?. I saw that FieldCacheImpl is using a CoreClosedListener, whereas I am using a ReaderClosedListener. What is the difference between these two?
I will surely look at the FBS replacement you have pointed out. BTW, this method actually runs during opening of an index for the first-time. May be for clarity and organisation, I will refactor this code as you have suggested On Thursday, November 7, 2013, Michael McCandless wrote: > Hi, a few comments on quickly looking at the code... > > It's sort of strange, inside the Weight.scorer() method, to go and > build an IndexSearcher and run a whole new search, if the cache entry > is missing. Could you instead just do a top-level search, which then > populates the cache per-segment? > > Also, FixedBitSet is better here than OpenBitSet: it should be a bit > faster, because OpenBitSet is "elastic". > > When you use the core cache key, it does not change for a > SegmentReader that was reopened with new deletes; this is the whole > point of the core cache key. So, if deletes changed for a segment and > you reopened, your cache entry will reuse whatever was created the > first time for that segment. If deletes matter, then it's usually > best to look at liveDocs "live" and fold them in, instead of > regenerating the whole cache entry. > > > Mike McCandless > > http://blog.mikemccandless.com > > > On Thu, Nov 7, 2013 at 8:04 AM, Ravikumar Govindarajan > <ravikumar.govindara...@gmail.com <javascript:;>> wrote: > > Thanks Mike. Can you help me out with one more question? > > > > I have a sample impl as below, where I am adding a ReaderClosedListener > to > > purge the BitSet. > > > > When using NRT with applyAllDeletes, old-reader will get closed and > > new-reader will open. In such a case, will the below impl-cache also be > > purged and re-built? > > > > I also saw that FieldCache uses a CoreClosedListener, instead of > > ReaderClosedListener and I need such a functionality. It will be great to > > maintain the BitSet cache at the cost of taking extra hit for testing > > deletes. > > > > @Override > > > > public Scorer scorer(AtomicReaderContext context, boolean > scoreDocsInOrder, > > boolean topScorer, Bits acceptDocs) { > > > > Object key = context.getReader().getCoreCacheKey(); > > > > OpenBitSet bitSet = cacheMap.get(key); > > > > if (bitSet == null) { > > > > reader.addReaderClosedListener(new ReaderClosedListener() { > > > > @Override > > > > public void onClose(IndexReader reader) { > > > > Object key = reader.getCoreCacheKey(); > > > > cacheMap.remove(key); > > > > } > > > > }); > > > > final OpenBitSet bs = new OpenBitSet(reader.maxDoc()); > > > > //Add the empty bit-set first > > > > cacheMap.put(key, bs); > > > > IndexSearcher searcher = new IndexSearcher(reader); > > > > //Do a search and populate the bitset > > > > return bs; > > > > } > > > > //Proceed with scoring logic > > > > } > > > > -- > > > > Ravi > > > > > > On Thu, Nov 7, 2013 at 4:28 PM, Michael McCandless < > > luc...@mikemccandless.com <javascript:;>> wrote: > > > >> You need to call .getCoreCacheKey() on each of the sub-readers > >> (returned by IndexReader.leaves()), to play well with NRT. > >> > >> Typically you'd do so in a context that already sees each leaf, like a > >> custom Filter or a Collector. > >> > >> Mike McCandless > >> > >> http://blog.mikemccandless.com > >> > >> > >> On Thu, Nov 7, 2013 at 1:33 AM, Ravikumar Govindarajan > >> <ravikumar.govindara...@gmail.com <javascript:;>> wrote: > >> > I am trying to cache a BitSet by attaching to > >> IndexReader.addCloseListener, > >> > using the getCoreCacheKey() > >> > > >> > But, I find that getCoreCacheKey() returns the IndexReader object > itself > >> as > >> > the key. > >> > > >> > Whenever the IndexReader re-opens via NRT because of deletes, will it > >> mean > >> > that my cache will be purged, because a new IndexReader is opened? > >> > > >> > Are there ways to avoid this purging? > >> > > >> > -- > >> > Ravi > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: > >> java-user-unsubscr...@lucene.apache.org<javascript:;> > >> For additional commands, e-mail: > >> java-user-h...@lucene.apache.org<javascript:;> > >> > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org<javascript:;> > For additional commands, e-mail: > java-user-h...@lucene.apache.org<javascript:;> > >
