[
https://issues.apache.org/jira/browse/SOLR-12509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16566931#comment-16566931
]
Steve Rowe commented on SOLR-12509:
-----------------------------------
Reproducing {{SolrIndexSplitterTest.testSplitAlternatelyLink()}} failure, from
[https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/2469/]:
{noformat}
Checking out Revision 600c15d14e73274d4152e8ef1b8c0d0aae69fd18
(refs/remotes/origin/branch_7x)
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=SolrIndexSplitterTest -Dtests.method=testSplitAlternatelyLink
-Dtests.seed=2EC831F1D9B21D7D -Dtests.multiplier=3 -Dtests.slow=true
-Dtests.locale=pl -Dtests.timezone=CTT -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 1.02s J1 | SolrIndexSplitterTest.testSplitAlternatelyLink
<<<
[junit4] > Throwable #1: java.lang.AssertionError: split index1 has wrong
number of documents expected:<5> but was:<6>
[junit4] > at
__randomizedtesting.SeedInfo.seed([2EC831F1D9B21D7D:9992FDD95CED0639]:0)
[junit4] > at
org.apache.solr.update.SolrIndexSplitterTest.doTestSplitAlternately(SolrIndexSplitterTest.java:272)
[junit4] > at
org.apache.solr.update.SolrIndexSplitterTest.testSplitAlternatelyLink(SolrIndexSplitterTest.java:247)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[...]
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70):
{id=PostingsFormat(name=Memory)},
docValues:{_version_=DocValuesFormat(name=Lucene70),
id=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=419,
maxMBSortInHeap=6.5586874621200195, sim=RandomSimilarity(queryNorm=false): {},
locale=pl, timezone=CTT
[junit4] 2> NOTE: Linux 4.15.0-29-generic amd64/Oracle Corporation
1.8.0_172 (64-bit)/cpus=8,threads=1,free=160622208,total=536870912
{noformat}
> Improve SplitShardCmd performance and reliability
> -------------------------------------------------
>
> Key: SOLR-12509
> URL: https://issues.apache.org/jira/browse/SOLR-12509
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Reporter: Andrzej Bialecki
> Assignee: Andrzej Bialecki
> Priority: Major
> Fix For: 7.5
>
> Attachments: SOLR-12509.patch, SOLR-12509.patch
>
>
> {{SplitShardCmd}} is currently quite complex.
> Shard splitting occurs on active shards, which are still being updated, so
> the splitting has to involve several carefully orchestrated steps, making
> sure that new sub-shard placeholders are properly created and visible, and
> then also applying buffered updates to the split leaders and performing
> recovery on sub-shard replicas.
> This process could be simplified in cases where collections are not actively
> being updated or can tolerate a little downtime - we could put the shard
> "offline", ie. disable writing while the splitting is in progress (in order
> to avoid users' confusion we should disable writing to the whole collection).
> The actual index splitting could perhaps be improved to use
> {{HardLinkCopyDirectoryWrapper}} for creating a copy of the index by
> hard-linking existing index segments, and then applying deletes to the
> documents that don't belong in a sub-shard. However, the resulting index
> slices that replicas would have to pull would be the same size as the whole
> shard.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
