Digging deeper - hmmm... so there is a cache of ords in SlowCompositeReaderWrapper:
// TODO: consider ConcurrentHashMap ? // TODO: this could really be a weak map somewhere else on the coreCacheKey, // but do we really need to optimize slow-wrapper any more? final Map<String,OrdinalMap> cachedOrdMaps = new HashMap<>(); I wonder why this doesn't seem to be used from request to request in my case, eh. Dawid On Mon, Aug 29, 2022 at 3:07 PM Dawid Weiss <[email protected]> wrote: > > Hi, > > I have a situation here with Solr 8.11.2 in stand-alone mode, a large > (200GB+) index with multi-valued doc value string fields. The problem is > that faceting over these fields takes a long time. Before you say: "well, > duh, of course" I wanted to point out that it takes a long time for *every* > query, even those that collect facets from a relatively small subset of all > documents (say, one hundred). > > Looking and debugging the code, I see a few things that made my head > scratch but this one is particularly troubling. > > So, the faceting code goes into FacetFieldProcessorByArrayDV and then most > of the time is spent inside findStartAndEndOrds, looking basically for the > count of unique ordinals (for all segments). Now, because this is done with > a slow reader wrapper, it takes forever. And it's repeated for each and > every request - even though clearly the ordinal map (or the count of > values) won't change for the same reader: > > @Override > protected void findStartAndEndOrds() throws IOException { > if (multiValuedField) { > si = FieldUtil.getSortedSetDocValues(fcontext.qcontext, sf, null); > if (si instanceof MultiDocValues.MultiSortedSetDocValues) { > ordinalMap = ((MultiDocValues.MultiSortedSetDocValues)si).mapping; > } > > If there is any rationale behind not caching the ordinal map (or just the > size of all ords!) then I failed to see it. Otherwise it's a serious > concern and slowdown that I think could help many poor souls speed up Solr > faceting. > > Anybody familiar with that code who could comment on the above? > > Dawid >
