[
https://issues.apache.org/jira/browse/SOLR-12239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18096828#comment-18096828
]
Serhiy Bzhezytskyy commented on SOLR-12239:
-------------------------------------------
I took a look at this and put up a PR with a working approach — filing it for
discussion since it overlaps with the index-sorting work others have been
doing: https://github.com/apache/solr/pull/4644
The problem (verified still present on main): enabling an index sort on an
existing collection fails on core reload with IndexWriter's validateIndexSort
("cannot change previous indexSort") — Lucene rejects the transition from
unsorted segments to a configured sort, so today you have to delete and reindex.
The mechanism answers the "might LUCENE-9484 solve this?" question from the
earlier comments — it does. LUCENE-9484 ("allow index sorting to happen after
the fact", Lucene 9.0) lets you wrap each existing segment reader in a
SortingCodecReader and merge into a fresh sort-configured IndexWriter via
addIndexes(CodecReader...). (addIndexes doesn't auto-sort — LUCENE-8505 removed
that — so the explicit wrap does the re-sort.)
What the PR adds: a RESORTINDEX core-admin action (v1 + v2) that re-sorts the
existing index into a target sort and swaps it in via modifyIndexProps
(RestoreCore's path), then reopens. Target sort from a sort param or the core's
configured SortingMergePolicy. Not supported in SolrCloud; child/nested-doc
indexes rejected. Tests + ref-guide + changelog included.
I see this as the migration piece that complements SOLR-13681 / #313 (making
the sort configurable) — resort an existing index, then declare the matching
sort in config so new docs stay sorted — rather than something separate. Does
this direction fit with how index sorting should evolve? Happy to adjust;
didn't want to step on the config work.
cc [~cpoerschke] [~dsmiley]
> Enabling index sorting causes "segment not sorted with indexSort=null"
> ----------------------------------------------------------------------
>
> Key: SOLR-12239
> URL: https://issues.apache.org/jira/browse/SOLR-12239
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 7.1
> Reporter: Ishan Chattopadhyaya
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When index sorting is enabled on an existing collection/index (using
> SortingMergePolicy), the collection reload causes the following exception:
> {code}
> java.util.concurrent.ExecutionException:
> org.apache.solr.common.SolrException: Unable to create core
> [mycoll_shard1_replica_n1]
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.solr.core.CoreContainer.lambda$load$14(CoreContainer.java:671)
> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.solr.common.SolrException: Unable to create core
> [mycoll_shard1_replica_n1]
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1045)
> at
> org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:642)
> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
> ... 5 more
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:989)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:844)
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1029)
> ... 7 more
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2076)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2196)
> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1072)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:961)
> ... 9 more
> Caused by: org.apache.lucene.index.CorruptIndexException: segment not sorted
> with indexSort=null (resource=_0(7.1.0):C1)
> at
> org.apache.lucene.index.IndexWriter.validateIndexSort(IndexWriter.java:1185)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1108)
> at
> org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:119)
> at
> org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:94)
> at
> org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:257)
> at
> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:131)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2037)
> ... 12 more
> {code}
> This means that the user actually needs to delete the index segments, reload
> the collection and then re-index. This is bad user experience.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
