[ https://issues.apache.org/jira/browse/SOLR-10916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119082#comment-16119082 ]
Steve Rowe edited comment on SOLR-10916 at 8/8/17 9:52 PM: ----------------------------------------------------------- {{TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete()}} has been failing pretty regularly (11 out of 100 iterations when I beasted it on master just now). The problem: a doc addition fails after intentionally killing a couple of solr's jetties, apparently because the client attempts to send an update to a replica that isn't there. I'm beasting a change that calls {{client.getZkStateReader().forceUpdateCollection()}} before adding the doc, but it seems like CloudSolrClient ought to be able to detect this problem and retry after refreshing the cluster state? >From [https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/223/] this morning: {noformat} Checking out Revision 95af49e5882226be52141a26565d8d2f99b76aaf (refs/remotes/origin/branch_7x) [...] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCollectionsAPIViaSolrCloudCluster -Dtests.method=testCollectionCreateSearchDelete -Dtests.seed=2EB43E7310F7BC75 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fil -Dtests.timezone=Canada/Pacific -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] ERROR 12.8s J2 | TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete <<< [junit4] > Throwable #1: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:37257/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html> [junit4] > <head> [junit4] > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> [junit4] > <title>Error 404 </title> [junit4] > </head> [junit4] > <body> [junit4] > <h2>HTTP ERROR: 404</h2> [junit4] > <p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason: [junit4] > <pre> Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p> [junit4] > <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/> [junit4] > </body> [junit4] > </html> [junit4] > at __randomizedtesting.SeedInfo.seed([2EB43E7310F7BC75:8D4E90D6971F56D0]:0) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793) [junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178) [junit4] > at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233) [junit4] > at org.apache.solr.cloud.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:167) [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:564) [junit4] > at java.base/java.lang.Thread.run(Thread.java:844) [junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37257/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html> [junit4] > <head> [junit4] > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> [junit4] > <title>Error 404 </title> [junit4] > </head> [junit4] > <body> [junit4] > <h2>HTTP ERROR: 404</h2> [junit4] > <p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason: [junit4] > <pre> Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p> [junit4] > <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/> [junit4] > </body> [junit4] > </html> [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:591) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516) [junit4] > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188) [junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1161) [junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [junit4] > ... 1 more [...] [junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=Lucene50(blocksize=128)}, docValues:{}, maxPointsInLeafNode=710, maxMBSortInHeap=7.885441283456621, sim=RandomSimilarity(queryNorm=true): {}, locale=fil, timezone=Canada/Pacific [junit4] 2> NOTE: Linux 4.10.0-27-generic i386/Oracle Corporation 9 (32-bit)/cpus=8,threads=1,free=224954368,total=518979584 {noformat} was (Author: steve_rowe): {{TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete()}} has been failing pretty regularly (11 out of 100 iterations when I beasted it on master just now). The problem: a doc addition fails after intentionally killing a couple of solr's jetties, apparently because the client attempts to send an update to a shard that isn't there. I'm beasting a change that calls {{client.getZkStateReader().forceUpdateCollection()}} before adding the doc, but it seems like CloudSolrClient ought to be able to detect this problem and retry after refreshing the cluster state? >From [https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/223/] this morning: {noformat} Checking out Revision 95af49e5882226be52141a26565d8d2f99b76aaf (refs/remotes/origin/branch_7x) [...] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCollectionsAPIViaSolrCloudCluster -Dtests.method=testCollectionCreateSearchDelete -Dtests.seed=2EB43E7310F7BC75 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fil -Dtests.timezone=Canada/Pacific -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] ERROR 12.8s J2 | TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete <<< [junit4] > Throwable #1: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:37257/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html> [junit4] > <head> [junit4] > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> [junit4] > <title>Error 404 </title> [junit4] > </head> [junit4] > <body> [junit4] > <h2>HTTP ERROR: 404</h2> [junit4] > <p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason: [junit4] > <pre> Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p> [junit4] > <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/> [junit4] > </body> [junit4] > </html> [junit4] > at __randomizedtesting.SeedInfo.seed([2EB43E7310F7BC75:8D4E90D6971F56D0]:0) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793) [junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178) [junit4] > at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233) [junit4] > at org.apache.solr.cloud.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:167) [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:564) [junit4] > at java.base/java.lang.Thread.run(Thread.java:844) [junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37257/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html> [junit4] > <head> [junit4] > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> [junit4] > <title>Error 404 </title> [junit4] > </head> [junit4] > <body> [junit4] > <h2>HTTP ERROR: 404</h2> [junit4] > <p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason: [junit4] > <pre> Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p> [junit4] > <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/> [junit4] > </body> [junit4] > </html> [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:591) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516) [junit4] > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188) [junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1161) [junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [junit4] > ... 1 more [...] [junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=Lucene50(blocksize=128)}, docValues:{}, maxPointsInLeafNode=710, maxMBSortInHeap=7.885441283456621, sim=RandomSimilarity(queryNorm=true): {}, locale=fil, timezone=Canada/Pacific [junit4] 2> NOTE: Linux 4.10.0-27-generic i386/Oracle Corporation 9 (32-bit)/cpus=8,threads=1,free=224954368,total=518979584 {noformat} > Any Solr test using MiniSolrCloud or Solr Core's should extend > SolrTestCaseJ4/SolrCloudTestCase > ----------------------------------------------------------------------------------------------- > > Key: SOLR-10916 > URL: https://issues.apache.org/jira/browse/SOLR-10916 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Reporter: Hoss Man > Assignee: Steve Rowe > Fix For: master (8.0), 7.1 > > Attachments: SOLR-10916.patch, SOLR-10916.patch, SOLR-10916.patch > > > We have a non-trivial number of tests that extend LuceneTestCase directly. > For "utility" method minded tests, this is fine - but for any test that wants > to instantiate re-use shared config files to instantiate SolrCores, or > instances of MiniSolrCloudCluster, this makes these tests really cumbersome > to maintain and deal with because htye don't leverage the existing > randomization setup logic in SolrTestCaseJ4. > we should fix these tests where applicable -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org