I'm working on a project / idea that will require it. I'll share more about that later when appropriate, but it's too early now.
At least the current state isn't bad. It works, especially with expanded functionality extending to schemas/use-cases with nested docs: SOLR-17170 -- fully merged back to 9.x. There's a usability issue that I hope Christine will prioritize improving. I've heard Adrien and others extol the virtues/benefits of index sorting but haven't yet had deployed it to share. I will when I can. On Wed, Jul 15, 2026 at 8:06 AM Jason Gerlowski <[email protected]> wrote: > Curious David - are you considering using index sorting for a > use-case, or have any experience with it to share in terms of > speedups, etc? Or more just something that struck your fancy? > > I remember index-sorting being something that came up in a Community > over Code "Birds of a Feather" session (in, I believe, Halifax?), as > one of the things Lucene supported that Solr should really consider > enabling out of the box for performance benefits. But I can't > remember the details of that discussion very well... > > Best, > > Jason > > On Thu, Jul 2, 2026 at 12:20 PM David Smiley <[email protected]> wrote: > > > > I started investigating the current state of Lucene's index sorting > support > > in Solr. I had Claude Opus write a report for me. Rather than hoard it > to > > myself, I'm sharing with everyone in case others are wondering what's up > as > > well. > > > > > > Background > > ---------- > > > > Lucene has supported index-level sorting since LUCENE-6766 (Lucene 6.2), > > where segments are internally sorted by a configurable field order at > > flush/merge time. This enables significant query-time optimizations -- > > when the query's sort matches the index sort, Lucene can skip entire > > segments or terminate collection early. > > > > > > Current Solr Support > > -------------------- > > > > Solr does support index sorting today, but through an indirect mechanism: > > > > Configuration is done via SortingMergePolicyFactory in solrconfig.xml: > > > > <mergePolicyFactory > > class="org.apache.solr.index.SortingMergePolicyFactory"> > > <str name="sort">timestamp desc</str> > > <str name="wrapped.prefix">inner</str> > > <str > > name="inner.class">org.apache.solr.index.TieredMergePolicyFactory</str> > > </mergePolicyFactory> > > > > Internally, SortingMergePolicy is a FilterMergePolicy that does nothing > > merge-policy-related -- it simply holds a Sort object. SolrIndexConfig > > then has a special instanceof check that extracts this Sort and calls > > IndexWriterConfig.setIndexSort(). The class itself has a TODO comment > > acknowledging this is a workaround: "remove this and add indexSort > > specification directly to solrconfig.xml?" > > > > Query-side integration exists via the "segmentTerminateEarly" query > > parameter, which wraps the collector in an > EarlyTerminatingSortingCollector. > > Note that this collector is @Deprecated -- modern Lucene's > TopFieldCollector > > handles early termination natively when it detects sorted segments. > > > > The /admin/segments API (with coreInfo=true) exposes the indexSort > > configuration and per-segment sort info. > > > > AtomicUpdateDocumentMerger correctly detects fields used for index > sorting > > and prevents DocValues-only updates on them (a Lucene limitation). > > > > > > Open Issues > > ----------- > > > > Several open JIRA issues relate to this area. > > > > SOLR-9108: Improve how index time sorting is configured > > https://issues.apache.org/jira/browse/SOLR-9108 > > > > Filed by Mike McCandless in 2016 right after LUCENE-6766. Proposes > > configuring index sort directly in solrconfig.xml alongside other > > IndexWriter settings rather than piggybacking on the merge policy. > > > > SOLR-13681: Make Lucene's index sorting directly configurable in Solr > > https://issues.apache.org/jira/browse/SOLR-13681 > > > > Filed by Christine Poerschke in 2019. Has a draft PR (#313) that adds > > a direct <indexSort> config element to solrconfig.xml. The PR has been > > stalled since 2021; the main open question is what should happen when > > index sorting is enabled on an existing collection that already has > > unsorted segments. Duplicates SOLR-12230 (deprecate > SortingMergePolicy). > > > > SOLR-12239: Enabling index sorting causes CorruptIndexException > > https://issues.apache.org/jira/browse/SOLR-12239 > > > > When index sorting is enabled on an existing collection with unsorted > > segments, reloading throws: "segment not sorted with indexSort=null". > > The current workaround is to delete all data and reindex from scratch. > > Notably, the related LUCENE-9484 ("Allow index sorting to happen after > > the fact") was fixed in Lucene 9.0, which allows merging unsorted > > segments into sorted ones retroactively. Solr has not wired this up. > > > > SOLR-17170: Support Blocks in Index Sorting > > https://issues.apache.org/jira/browse/SOLR-17170 > > > > Lucene 9.10+ supports block-aware presort during index sorting (via > > Lucene PR #12829). This is critical for nested/block-join documents. > > No Solr-side work has been done. > > > > > > Summary > > ------- > > > > Index sorting in Solr works for simple (non-nested) use cases via the > > SortingMergePolicyFactory, but the implementation is showing its age: > > > > - Configuration is indirect and hacky (merge policy as a Sort carrier) > > - Cannot be safely enabled on existing collections without full reindex, > > despite Lucene having solved this at the engine level since 9.0 > > - Incompatible with nested/block-join documents on Solr 10+ > > - The query-side early termination collector is deprecated > > - A draft PR for direct configuration has been stalled since 2021 > > > > ~ David Smiley > > Apache Lucene/Solr Search Developer > > http://www.linkedin.com/in/davidwsmiley > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
