AndreyBozhko commented on code in PR #2707:
URL: https://github.com/apache/solr/pull/2707#discussion_r1755326203
##########
solr/core/src/java/org/apache/solr/cloud/ZkShardTerms.java:
##########
@@ -421,6 +421,10 @@ private void registerWatcher() throws KeeperException {
return;
}
retryRegisterWatcher();
+ // if term node is deleted, refresh cannot possibly succeed
+ if (Watcher.Event.EventType.NodeDeleted == event.getType()) {
+ return;
+ }
Review Comment:
Test failure:
```
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
uncaught exception in thread: Thread[id=68, name=zkCallback-23-thread-1,
state=RUNNABLE, group=TGRP-TestConfigSetsAPI]
> at
__randomizedtesting.SeedInfo.seed([5ECEE82AA0B50470:5D74B82B43D74CFA]:0)
>
> Caused by:
> org.apache.solr.common.SolrException: Error updating shard term for
collection: newcollection
> at __randomizedtesting.SeedInfo.seed([5ECEE82AA0B50470]:0)
> at
app//org.apache.solr.cloud.ZkShardTerms.refreshTerms(ZkShardTerms.java:377)
> at
app//org.apache.solr.cloud.ZkShardTerms.lambda$registerWatcher$8(ZkShardTerms.java:426)
> at
app//org.apache.solr.common.cloud.SolrZkClient$ProcessWatchWithExecutor.lambda$process$1(SolrZkClient.java:1083)
> at
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:363)
> at
java.base@17.0.11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
java.base@17.0.11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base@17.0.11/java.lang.Thread.run(Thread.java:840)
>
> Caused by:
> org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for /collections/newcollection/terms/shard1
> at
app//org.apache.zookeeper.KeeperException.create(KeeperException.java:117)
> at
app//org.apache.zookeeper.KeeperException.create(KeeperException.java:53)
> at
app//org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1972)
> at
app//org.apache.solr.common.cloud.SolrZkClient.lambda$getData$6(SolrZkClient.java:448)
> at
app//org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
> at
app//org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:448)
> at
app//org.apache.solr.cloud.ZkShardTerms.refreshTerms(ZkShardTerms.java:373)
> ... 6 more
2> NOTE: reproduce with: gradlew test --tests TestConfigSetsAPI.testUpload
-Dtests.seed=5ECEE82AA0B50470 -Dtests.locale=mer-Latn-KE
-Dtests.timezone=Asia/Magadan -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
```
##########
solr/core/src/java/org/apache/solr/cloud/ZkController.java:
##########
@@ -729,14 +729,16 @@ public void preClose() {
try {
synchronized (collectionToTerms) {
- customThreadPool.submit(
- () ->
collectionToTerms.values().parallelStream().forEach(ZkCollectionTerms::close));
+ collectionToTerms
+ .values()
+ .forEach(zkCollectionTerms ->
customThreadPool.execute(zkCollectionTerms::close));
}
- customThreadPool.submit(
- () ->
- replicateFromLeaders.values().parallelStream()
- .forEach(ReplicateFromLeader::stopReplication));
+ replicateFromLeaders
+ .values()
+ .forEach(
+ replicateFromLeader ->
+
customThreadPool.execute(replicateFromLeader::stopReplication));
Review Comment:
Test failure:
```
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured
an uncaught exception in thread: Thread[id=428,
name=preCloseThreadPool-354-thread-2, state=RUNNABLE,
group=TGRP-TestCollectionAPI]
> at
__randomizedtesting.SeedInfo.seed([F3266546A761EFBD:7B725A9C099D8245]:0)
>
> Caused by:
> java.security.AccessControlException: access denied
("java.lang.RuntimePermission" "modifyThread")
> at __randomizedtesting.SeedInfo.seed([F3266546A761EFBD]:0)
> at
java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:488)
> at
java.base/java.security.AccessController.checkPermission(AccessController.java:1071)
> at
java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:411)
> at
java.base/java.util.concurrent.ThreadPoolExecutor.checkShutdownAccess(ThreadPoolExecutor.java:764)
> at
java.base/java.util.concurrent.ThreadPoolExecutor.shutdown(ThreadPoolExecutor.java:1394)
> at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor.shutdown(ScheduledThreadPoolExecutor.java:842)
> at
java.base/java.util.concurrent.Executors$DelegatedExecutorService.shutdown(Executors.java:759)
> at
org.apache.solr.handler.ReplicationHandler$1.preClose(ReplicationHandler.java:1415)
> at
org.apache.solr.handler.ReplicationHandler.shutdown(ReplicationHandler.java:1447)
> at
org.apache.solr.cloud.ReplicateFromLeader.stopReplication(ReplicateFromLeader.java:184)
> at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
java.base/java.util.concurrent.ConcurrentHashMap$ValueSpliterator.forEachRemaining(ConcurrentHashMap.java:3612)
> at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
> at
java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
> at
java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
> at
java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
> at
java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
> at
java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
> at
java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
> at
java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
Reproduce with: ./gradlew :solr:core:test --tests
"org.apache.solr.cloud.api.collections.TestCollectionAPI.test" -Ptests.jvms=6
"-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC
-XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
-Ptests.seed=985B588C69AC93B2 -Ptests.file.encoding=ISO-8859-1
```
##########
solr/core/src/test/org/apache/solr/cloud/TestConfigSetsAPI.java:
##########
@@ -936,7 +936,7 @@ private void testNewSingleFileAfterSchemaAPI(boolean v2)
throws Exception {
String collectionName = "newcollection";
String configsetName = "regular";
String configsetSuffix = "testSinglePathNew-1-" + v2;
- createConfigSet(null, configsetName + configsetSuffix, null,
cluster.getSolrClient(), "solr");
+ uploadConfigSetWithAssertions(configsetName, configsetSuffix, "solr");
Review Comment:
Test failure:
```
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured
an uncaught exception in thread: Thread[id=288, name=zkCallback-23-thread-4,
state=RUNNABLE, group=TGRP-TestConfigSetsAPI]
> at
__randomizedtesting.SeedInfo.seed([985B588C69AC93B2:F96D87EFF9EC7FC7]:0)
>
> Caused by:
> org.apache.solr.common.SolrException: Can't load schema null:
org.apache.solr.core.SolrResourceNotFoundException: Can't find resource
'synonyms.txt' in classpath or '/configs/regulartestSinglePathNew-1-false',
cwd=/home/abozhko/projects/solr/solr/core/build/tmp/tests-cwd
> at __randomizedtesting.SeedInfo.seed([985B588C69AC93B2]:0)
> at
app//org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:663)
> at
app//org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:193)
> at
app//org.apache.solr.schema.ManagedIndexSchema.<init>(ManagedIndexSchema.java:113)
> at
app//org.apache.solr.schema.ZkIndexSchemaReader.updateSchema(ZkIndexSchemaReader.java:197)
> at
app//org.apache.solr.schema.ZkIndexSchemaReader$SchemaWatcher.process(ZkIndexSchemaReader.java:145)
> at
app//org.apache.solr.common.cloud.SolrZkClient$ProcessWatchWithExecutor.lambda$process$1(SolrZkClient.java:1083)
> at
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$1(ExecutorUtil.java:449)
> at
java.base@21.0.4/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
java.base@21.0.4/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base@21.0.4/java.lang.Thread.run(Thread.java:1583)
>
> Caused by:
> org.apache.solr.common.SolrException:
org.apache.solr.core.SolrResourceNotFoundException: Can't find resource
'synonyms.txt' in classpath or '/configs/regulartestSinglePathNew-1-false',
cwd=/home/abozhko/projects/solr/solr/core/build/tmp/tests-cwd
> at
app//org.apache.solr.schema.IndexSchema.informResourceLoaderAwareObjectsInChain(IndexSchema.java:2098)
> at
app//org.apache.solr.schema.IndexSchema.informResourceLoaderAwareObjectsForFieldType(IndexSchema.java:2017)
> at
java.base@21.0.4/java.util.HashMap$Values.forEach(HashMap.java:1073)
> at
app//org.apache.solr.schema.IndexSchema.postReadInform(IndexSchema.java:685)
> at
app//org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:659)
> ... 9 more
>
> Caused by:
> org.apache.solr.core.SolrResourceNotFoundException:
Can't find resource 'synonyms.txt' in classpath or
'/configs/regulartestSinglePathNew-1-false',
cwd=/home/abozhko/projects/solr/solr/core/build/tmp/tests-cwd
> at
app//org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:135)
> at
app//org.apache.lucene.analysis.synonym.SynonymGraphFilterFactory.loadSynonyms(SynonymGraphFilterFactory.java:191)
> at
app//org.apache.lucene.analysis.synonym.SynonymGraphFilterFactory.inform(SynonymGraphFilterFactory.java:161)
> at
app//org.apache.solr.core.SolrResourceLoader.informAware(SolrResourceLoader.java:784)
> at
app//org.apache.solr.schema.IndexSchema.informResourceLoaderAwareObjectsInChain(IndexSchema.java:2096)
> ... 13 more
Reproduce with: ./gradlew :solr:core:test --tests
"org.apache.solr.cloud.TestConfigSetsAPI.testNewSingleFileAfterSchemaAPIV2"
-Ptests.jvms=6 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC
-XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
-Ptests.seed=985B588C69AC93B2 -Ptests.file.encoding=ISO-8859-1
```
##########
solr/core/src/java/org/apache/solr/cloud/ZkShardTerms.java:
##########
@@ -421,6 +421,10 @@ private void registerWatcher() throws KeeperException {
return;
}
retryRegisterWatcher();
+ // if term node is deleted, refresh cannot possibly succeed
+ if (Watcher.Event.EventType.NodeDeleted == event.getType()) {
+ return;
+ }
Review Comment:
Proposed fix: don't attempt to refresh terms when term node is deleted - the
operation cannot possibly succeed.
##########
solr/core/src/java/org/apache/solr/handler/admin/PrepRecoveryOp.java:
##########
@@ -84,11 +84,15 @@ public void execute(CallInfo it) throws Exception {
TimeUnit.MILLISECONDS,
(n, c) -> {
if (c == null) return false;
+ if (coreContainer.isShutDown()) {
+ log.info("Not going to wait for replica to recover - Solr is
shutting down");
+ return false;
+ }
Review Comment:
Test failure:
```
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured
an uncaught exception in thread: Thread[id=99, name=zkCallback-47-thread-1,
state=RUNNABLE, group=TGRP-TestPrepRecovery]
>
> Caused by:
> org.apache.solr.common.SolrException: Solr is shutting down
> at __randomizedtesting.SeedInfo.seed([493CC1B130218651]:0)
> at
app//org.apache.solr.handler.admin.PrepRecoveryOp.lambda$execute$0(PrepRecoveryOp.java:199)
> at
app//org.apache.solr.common.cloud.ZkStateReader.lambda$waitForState$10(ZkStateReader.java:1641)
> at
app//org.apache.solr.common.cloud.ZkStateReader$DocCollectionAndLiveNodesWatcherWrapper.onChange(ZkStateReader.java:2145)
> at
app//org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$2(ZkStateReader.java:828)
> at
java.base@21.0.4/java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4709)
> at
app//org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:826)
> at
app//org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1381)
> at
app//org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1376)
> at
app//org.apache.solr.common.cloud.SolrZkClient$ProcessWatchWithExecutor.lambda$process$1(SolrZkClient.java:1083)
> at
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$1(ExecutorUtil.java:449)
> at
java.base@21.0.4/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
java.base@21.0.4/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base@21.0.4/java.lang.Thread.run(Thread.java:1583)
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured
an uncaught exception in thread: Thread[id=145, name=zkCallback-47-thread-3,
state=RUNNABLE, group=TGRP-TestPrepRecovery]
>
> Caused by:
> org.apache.solr.common.SolrException: core not found:
testLeaderNotResponding_shard1_replica_n1
> at __randomizedtesting.SeedInfo.seed([493CC1B130218651]:0)
> at
app//org.apache.solr.handler.admin.PrepRecoveryOp.lambda$execute$0(PrepRecoveryOp.java:94)
> at
app//org.apache.solr.common.cloud.ZkStateReader.lambda$waitForState$10(ZkStateReader.java:1641)
> at
app//org.apache.solr.common.cloud.ZkStateReader$DocCollectionAndLiveNodesWatcherWrapper.onChange(ZkStateReader.java:2145)
> at
app//org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$2(ZkStateReader.java:828)
> at
java.base@21.0.4/java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4709)
> at
app//org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:826)
> at
app//org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1381)
> at
app//org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1376)
> at
app//org.apache.solr.common.cloud.SolrZkClient$ProcessWatchWithExecutor.lambda$process$1(SolrZkClient.java:1083)
> at
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$1(ExecutorUtil.java:449)
> at
java.base@21.0.4/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
java.base@21.0.4/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base@21.0.4/java.lang.Thread.run(Thread.java:1583)
Reproduce with: ./gradlew :solr:core:test --tests
"org.apache.solr.cloud.TestPrepRecovery.testLeaderNotResponding" -Ptests.jvms=6
"-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC
-XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m"
-Ptests.seed=493CC1B130218651 -Ptests.file.encoding=ISO-8859-1
```
##########
solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ZkStateReader.java:
##########
@@ -790,7 +790,7 @@ private void refreshLiveNodes(Watcher watcher) throws
KeeperException, Interrupt
try {
List<String> nodeList = zkClient.getChildren(LIVE_NODES_ZKNODE,
watcher, true);
newLiveNodes = new TreeSet<>(nodeList);
- } catch (KeeperException.NoNodeException e) {
+ } catch (KeeperException.NoNodeException | AlreadyClosedException e) {
Review Comment:
Test failure:
```
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured
an uncaught exception in thread: Thread[id=4610, name=zkCallback-3128-thread-1,
state=RUNNABLE, group=TGRP-TestLeaderElectionZkExpiry]
> at
__randomizedtesting.SeedInfo.seed([CE3C0A1A450B88C4:18E32B4073169B5C]:0)
>
> Caused by:
> org.apache.solr.common.AlreadyClosedException
> at __randomizedtesting.SeedInfo.seed([CE3C0A1A450B88C4]:0)
> at
app//org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:68)
> at
app//org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:412)
> at
app//org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:791)
> at
app//org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1381)
> at
app//org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1376)
> at
app//org.apache.solr.common.cloud.SolrZkClient$ProcessWatchWithExecutor.lambda$process$1(SolrZkClient.java:1083)
> at
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$1(ExecutorUtil.java:449)
> at
java.base@21.0.4/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
java.base@21.0.4/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base@21.0.4/java.lang.Thread.run(Thread.java:1583)
> NOTE: reproduce with: gradlew test --tests
TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry
-Dtests.seed=CE3C0A1A450B88C4 -Dtests.locale=sr-RS -Dtests.timezone=Libya
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
```
##########
solr/core/src/test/org/apache/solr/pkg/TestPackages.java:
##########
@@ -810,10 +813,18 @@ public void testSchemaPlugins() throws Exception {
":fieldType:_packageinfo_:version",
"1.0"));
+ JettySolrRunner jetty =
+ cluster.getJettySolrRunners().stream()
+ .dropWhile(j -> j.getCoreContainer().getAllCoreNames().isEmpty())
+ .findFirst()
+ .orElseThrow();
+
+ IndexSchema schemaBeforePackageUpdate = withOnlyCoreInJetty(jetty,
SolrCore::getLatestSchema);
+
add = new PackagePayload.AddVersion();
add.version = "2.0";
add.pkg = "schemapkg";
- add.files = Arrays.asList(FILE1);
+ add.files = Arrays.asList(FILE1, FILE2);
Review Comment:
Test failure:
```
> Caused by:
> org.apache.solr.common.SolrException: Can't load schema null:
PACKAGE_LOADER:
schemapkg:{"package":"schemapkg","version":"2.0","files":["/schemapkg/schema-plugins.jar"]}
Error loading class 'com.o19s.payloads.Base64Encoder'
> at __randomizedtesting.SeedInfo.seed([982BE2CBECB9D539]:0)
> at
app//org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:663)
> at
app//org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:193)
> at
app//org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:109)
> at
app//org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:370)
> at
app//org.apache.solr.core.ConfigSetService.lambda$loadConfigSet$0(ConfigSetService.java:281)
> at
app//org.apache.solr.core.ConfigSet.getIndexSchema(ConfigSet.java:66)
> at
app//org.apache.solr.core.SolrCore.fetchLatestSchema(SolrCore.java:3068)
> at
app//org.apache.solr.core.SolrResourceLoader.lambda$createSchemaLoader$7(SolrResourceLoader.java:979)
> at
app//org.apache.solr.pkg.PackageListeningClassLoader.lambda$new$0(PackageListeningClassLoader.java:68)
> at
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$1(ExecutorUtil.java:449)
> at
java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base@11.0.17/java.lang.Thread.run(Thread.java:829)
>
> Caused by:
> org.apache.solr.common.SolrException: PACKAGE_LOADER:
schemapkg:{"package":"schemapkg","version":"2.0","files":["/schemapkg/schema-plugins.jar"]}
Error loading class 'com.o19s.payloads.Base64Encoder'
> at
app//org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:551)
> at
app//org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:472)
> at
app//org.apache.solr.pkg.PackageListeningClassLoader.findClass(PackageListeningClassLoader.java:150)
> at
app//org.apache.solr.core.SolrResourceLoader.getPackageClass(SolrResourceLoader.java:589)
> at
app//org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:507)
> at
app//org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:613)
> at
app//org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:607)
> at
app//org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:599)
> at
app//org.apache.lucene.analysis.payloads.DelimitedPayloadTokenFilterFactory.inform(DelimitedPayloadTokenFilterFactory.java:82)
> at
app//org.apache.solr.core.SolrResourceLoader.informAware(SolrResourceLoader.java:784)
> at
app//org.apache.solr.schema.IndexSchema.informResourceLoaderAwareObjectsInChain(IndexSchema.java:2096)
> at
app//org.apache.solr.schema.IndexSchema.informResourceLoaderAwareObjectsForFieldType(IndexSchema.java:2008)
> at
java.base@11.0.17/java.util.HashMap$Values.forEach(HashMap.java:977)
> at
app//org.apache.solr.schema.IndexSchema.postReadInform(IndexSchema.java:685)
> at
app//org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:659)
> ... 12 more
>
> Caused by:
> java.lang.ClassNotFoundException:
com.o19s.payloads.Base64Encoder
> at
java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
> at
java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
> at
java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:904)
> at
java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
> at java.base/java.lang.Class.forName0(Native Method)
> at java.base/java.lang.Class.forName(Class.java:398)
> at
org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:535)
> ... 26 more
2> NOTE: reproduce with: gradlew test --tests
TestPackages.testSchemaPlugins -Dtests.seed=982BE2CBECB9D539 -Dtests.locale=pa
-Dtests.timezone=America/Eirunepe -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
```
##########
solr/core/src/test/org/apache/solr/cloud/TestConfigSetsAPI.java:
##########
@@ -936,7 +936,7 @@ private void testNewSingleFileAfterSchemaAPI(boolean v2)
throws Exception {
String collectionName = "newcollection";
String configsetName = "regular";
String configsetSuffix = "testSinglePathNew-1-" + v2;
- createConfigSet(null, configsetName + configsetSuffix, null,
cluster.getSolrClient(), "solr");
+ uploadConfigSetWithAssertions(configsetName, configsetSuffix, "solr");
Review Comment:
The test creates a configset from the `_default` one which has a lot of
files, and looks like the test needs to wait until all files are available in
ZK before proceeding.
Updated the test to use a different configset as the base (with fewer files)
- this doesn't really change the semantics of the test, and the test is passing.
##########
solr/core/src/java/org/apache/solr/handler/admin/PrepRecoveryOp.java:
##########
@@ -84,11 +84,15 @@ public void execute(CallInfo it) throws Exception {
TimeUnit.MILLISECONDS,
(n, c) -> {
if (c == null) return false;
+ if (coreContainer.isShutDown()) {
+ log.info("Not going to wait for replica to recover - Solr is
shutting down");
+ return false;
+ }
Review Comment:
Not sure what the best approach could be here - sounds like the
PrepRecoveryOp watcher can't do much if the CoreContainer already shut down.
##########
solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ZkStateReader.java:
##########
@@ -790,7 +790,7 @@ private void refreshLiveNodes(Watcher watcher) throws
KeeperException, Interrupt
try {
List<String> nodeList = zkClient.getChildren(LIVE_NODES_ZKNODE,
watcher, true);
newLiveNodes = new TreeSet<>(nodeList);
- } catch (KeeperException.NoNodeException e) {
+ } catch (KeeperException.NoNodeException | AlreadyClosedException e) {
Review Comment:
Updated `refreshLiveNodes` to handle AlreadyClosedException as well.
##########
solr/core/src/java/org/apache/solr/cloud/ZkController.java:
##########
@@ -729,14 +729,16 @@ public void preClose() {
try {
synchronized (collectionToTerms) {
- customThreadPool.submit(
- () ->
collectionToTerms.values().parallelStream().forEach(ZkCollectionTerms::close));
+ collectionToTerms
+ .values()
+ .forEach(zkCollectionTerms ->
customThreadPool.execute(zkCollectionTerms::close));
}
- customThreadPool.submit(
- () ->
- replicateFromLeaders.values().parallelStream()
- .forEach(ReplicateFromLeader::stopReplication));
+ replicateFromLeaders
+ .values()
+ .forEach(
+ replicateFromLeader ->
+
customThreadPool.execute(replicateFromLeader::stopReplication));
Review Comment:
Proposed fix: avoid using `parallelStream`.
`parallelStream` _may_ choose to offload some of its work to ForkJoinPool,
but ForkJoinPool threads run without any security permissions by design. So
they should not be used to close thread pools (not when the Security Manager is
enabled).
##########
solr/core/src/test/org/apache/solr/pkg/TestPackages.java:
##########
@@ -810,10 +813,18 @@ public void testSchemaPlugins() throws Exception {
":fieldType:_packageinfo_:version",
"1.0"));
+ JettySolrRunner jetty =
+ cluster.getJettySolrRunners().stream()
+ .dropWhile(j -> j.getCoreContainer().getAllCoreNames().isEmpty())
+ .findFirst()
+ .orElseThrow();
+
+ IndexSchema schemaBeforePackageUpdate = withOnlyCoreInJetty(jetty,
SolrCore::getLatestSchema);
+
add = new PackagePayload.AddVersion();
add.version = "2.0";
add.pkg = "schemapkg";
- add.files = Arrays.asList(FILE1);
+ add.files = Arrays.asList(FILE1, FILE2);
Review Comment:
This test was always faulty - it attempts to register package version 2.0,
but FILE1 does not have all necessary classes for that to ever work. So the
async schema reload fails, and the core is still associated with the same
instance of IndexSchema.
I fixed the test, and added a new assertion to check that the core schema
was indeed reloaded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
