Hi,
How can I improve the performance of SortedSetDocValuesReaderState
which is responsible for computing '$facets' global ordinals at search time?
Is there any better way to warm up this state instead of putting it in
cache?
Any help is much appreciated.
Thanks & Regards,
Chitra
On Fri, Feb 3, 2017 at 11:32 AM, Chitra R <chithu.r...@gmail.com> wrote:
> Hi,
>
> We are going to implement faceted search (for both string & numeric)
> based on DocValuesField. For better performance, we decided to put the
> SortedSetDocValuesReaderState in cache ie
>
> *During first faceted search - new DocValuesReaderState will be opened
> and "$facets" values will be loaded from index and then we put that state
> in cache using ConcurrentHashMap
>
> *For next faceted search - instead of opening new DocValuesReaderState,
> we served the DocValuesReaderState from the ConcurrentHashMap.
>
> we will maintain the DocValuesReaderState in concurrentHashMap till 5
> minutes. After 5 minutes, an Old DocValuesReaderState will be removed from
> the ConcurrentHashMap.
>
> And for every search calls,
>
> DocValuesReaderState will be taken from cache like
>
> SortedSetDocValuesReaderStatereaderState oldReaderState = new
> DefaultSortedSetDocValuesReaderState(oldIndexSearcher.getIndexReader());
>
> readerStateCacheMap.put(key, oldReaderState);
>
> DocValuesReaderState will be maintained in readerStateCacheMap till 5
> minutes
>
> **** TopDocs will be computed using new IndexSearcher as
> FacetsCollector.search(newIndexSearcher, drillDownQuery, boolFilter,
> noofdocs, facetsCollector);
>
> ****After computing TopDocs, facets will be estimated from using old
> SortedSetDocValuesReaderState ad
> SortedSetDocValuesFacetCounts facetCount = new
> SortedSetDocValuesFacetCounts(oldReaderState, facetsCollector);
>
>
> 1. So, is there any chance to get conflicts during topDocs search or
> facets estimation (because of old indexSearcher in DocValuesReaderState and
> newIndexSearcher in topDocs computation)?
> 2. How can we maintain SortedSetDocValuesReaderState in cache?
>
>
> Kindly post your suggestions.
>
> Thanks & Regards,
> Chitra
>
>
