I ran BasicDistributedZk2Test
* 5 times with SolrTestCaseJ4.ALLOW_SSL = false: 5 green runs
* 5 times with SolrTestCaseJ4.ALLOW_SSL = true: 2 green and 3 red runs
Do not know if that is statistics enough to conclude anything, but it
smells a little like a SSL issue, if you ask me.
Mac OSX Maverick (10.9.1)
java -version
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
Regards, Per Steffensen
On 11/02/14 17:14, Steve Davids wrote:
A way to isolate if this is an SSL issue, you can turn off SSL for
that specific test by adding:
static {
ALLOW_SSL = false;
}
to the test class. Although, I have seen that test error out every so
often in non-SSL mode. Will take a look later on tonight though.
-Steve
On Tue, Feb 11, 2014 at 11:06 AM, Uwe Schindler <[email protected]
<mailto:[email protected]>> wrote:
Hi,
this looks like some bug in the MacOSX libc. Sometimes, Java 7
also crashes on OSX in this code part (because it segfaults when
producing the error message). There is already a bug open at
Oracle, but they have no idea how to fix. Currently it is
impossible to run something like Tomcat or Jetty on an OSX server
in production...
https://bugs.openjdk.java.net/browse/JDK-8024045
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [email protected] <mailto:[email protected]>
> -----Original Message-----
> From: Per Steffensen [mailto:[email protected]
<mailto:[email protected]>]
> Sent: Tuesday, February 11, 2014 4:46 PM
> To: [email protected] <mailto:[email protected]>
> Subject: Trunk tests fail (on Mac OSX)
>
> I am sure you have noticed it from Jenkins, but tests fail (most
of the
> time) on trunk. E.g. I have been running BasicDistributedZk2Test
numerous
> times from Eclipse on my Mac. Revision 1567049 on trunk.
> Sometimes the test is green, but most of the times it is not. It
is random
> exactly where it fails, but there is an example below. I believe
that when it
> fails it is always with the kind of exception
> (java.net.SocketException: Invalid argument) shown at the bottom. My
> guess it that is has to do with SOLR-3854.
>
> Anyone working on fixing this? Any current knowledge about what the
> problem is? An estimate on when things are consistently green
again on
> trunk?
>
> Regards, Per Steffensen
>
> --------- An example of where the test fails
-----------------------------
> org.apache.solr.client.solrj.SolrServerException: No live
SolrServers available
> to handle this request:[https://127.0.0.1:58496/qls/z/collection1,
> https://127.0.0.1:58500/qls/z/collection1,
> https://127.0.0.1:58493/qls/z/collection1,
> https://127.0.0.1:58490/qls/z/collection1]
> at
> __randomizedtesting.SeedInfo.seed([F0E9F84409868187:710F765C7ED9E1BB
> ]:0)
> at
>
org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.j
> ava:352)
> at
>
org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.jav
> a:635)
> at
>
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.ja
> va:90)
> at
org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
> at
>
org.apache.solr.cloud.AbstractFullDistribZkTestBase.queryServer(AbstractFul
> lDistribZkTestBase.java:1356)
> at
>
org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearc
> hTestCase.java:561)
> at
>
org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearc
> hTestCase.java:543)
> at
>
org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearc
> hTestCase.java:522)
> at
>
org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkQueries(AbstractF
> ullDistribZkTestBase.java:754)
> at
>
org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Te
> st.java:107)
> at
>
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistri
> butedSearchTestCase.java:868)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
> ava:57)
> at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> sorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomize
> dRunner.java:1559)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(Rando
> mizedRunner.java:79)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Rando
> mizedRunner.java:737)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Rando
> mizedRunner.java:773)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Rando
> mizedRunner.java:787)
> at
>
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.
> evaluate(SystemPropertiesRestoreRule.java:53)
> at
>
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRule
> SetupTeardownChained.java:50)
> at
>
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCa
> cheSanity.java:51)
> at
>
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA
> fterRule.java:46)
> at
>
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1
> .evaluate(SystemPropertiesInvariantRule.java:55)
> at
>
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleTh
> readAndTestName.java:49)
> at
>
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule
> IgnoreAfterMaxFailures.java:70)
> at
>
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure
> .java:48)
> at
>
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
> ementAdapter.java:36)
> at
>
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.
> run(ThreadLeakControl.java:358)
> at
>
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask
> (ThreadLeakControl.java:782)
> at
>
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadL
> eakControl.java:442)
> at
>
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Ran
> domizedRunner.java:746)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(Rando
> mizedRunner.java:648)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Rando
> mizedRunner.java:682)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Rando
> mizedRunner.java:693)
> at
>
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
> ementAdapter.java:36)
> at
>
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.
> evaluate(SystemPropertiesRestoreRule.java:53)
> at
>
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeA
> fterRule.java:46)
> at
>
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreCl
> assName.java:42)
> at
>
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1
> .evaluate(SystemPropertiesInvariantRule.java:55)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet
> hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMet
> hodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at
>
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
> ementAdapter.java:36)
> at
>
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss
> ertionsRequired.java:43)
> at
>
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure
> .java:48)
> at
>
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule
> IgnoreAfterMaxFailures.java:70)
> at
>
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore
> TestSuites.java:55)
> at
>
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
> ementAdapter.java:36)
> at
>
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.
> run(ThreadLeakControl.java:358)
> at java.lang.Thread.run(Thread.java:744)
> Caused by:
>
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
> org.apache.solr.client.solrj.SolrServerException: No live
SolrServers available
> to handle this request:[https://127.0.0.1:58496/qls/z/collection1,
> https://127.0.0.1:58500/qls/z/collection1,
> https://127.0.0.1:58493/qls/z/collection1,
> https://127.0.0.1:58490/qls/z/collection1]
> at
>
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:
> 498)
> at
>
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:
> 199)
> at
>
org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.j
> ava:283)
> ... 50 more
>
> --------------------- What I saw in the log during this test-run
> ----------------------
> 21901 T71 C29 P58493 oasc.SolrException.log ERROR
> org.apache.solr.common.SolrException:
> org.apache.solr.client.solrj.SolrServerException: No live
SolrServers available
> to handle this request:[https://127.0.0.1:58496/qls/z/collection1,
> https://127.0.0.1:58500/qls/z/collection1,
> https://127.0.0.1:58493/qls/z/collection1,
> https://127.0.0.1:58490/qls/z/collection1]
> at
>
org.apache.solr.handler.component.SearchHandler.handleRequestBody(Sea
> rchHandler.java:309)
> at
>
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl
> erBase.java:135)
> at
org.apache.solr.core.SolrCore.execute(SolrCore.java:1916)
> at
>
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:780
> )
> at
>
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427
> )
> at
>
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:217
> )
> at
>
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandle
> r.java:1419)
> at
>
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(
> JettySolrRunner.java:134)
> at
>
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandle
> r.java:1419)
> at
>
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at
>
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.ja
> va:229)
> at
>
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:
> 137)
> at
>
org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
> at
>
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.j
> ava:1077)
> at
>
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at
>
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.jav
> a:193)
> at
>
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.ja
> va:1009)
> at
>
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:
> 135)
> at
>
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.ja
> va:116)
> at org.eclipse.jetty.server.Server.handle(Server.java:368)
> at
>
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHtt
> pConnection.java:489)
> at
>
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractH
> ttpConnection.java:942)
> at
>
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerCo
> mplete(AbstractHttpConnection.java:1004)
> at
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
> at
>
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> at
>
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.
> java:82)
> at
>
org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196)
> at
>
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoi
> nt.java:628)
> at
>
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint
> .java:52)
> at
>
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.j
> ava:608)
> at
>
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.ja
> va:543)
> at java.lang.Thread.run(Thread.java:744)
> Caused by:
org.apache.solr.client.solrj.SolrServerException: No live
> SolrServers available to handle this
> request:[https://127.0.0.1:58496/qls/z/collection1,
> https://127.0.0.1:58500/qls/z/collection1,
> https://127.0.0.1:58493/qls/z/collection1,
> https://127.0.0.1:58490/qls/z/collection1]
> at
>
org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.j
> ava:352)
> at
>
org.apache.solr.handler.component.HttpShardHandlerFactory.makeLoadBal
> ancedRequest(HttpShardHandlerFactory.java:205)
> at
>
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHand
> ler.java:161)
> at
>
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHand
> ler.java:1)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
>
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
>
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
> a:1145)
> at
>
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
> va:615)
> ... 1 more
> Caused by: org.apache.solr.client.solrj.SolrServerException:
> IOException occured when talking to server at:
> https://127.0.0.1:58490/qls/z/collection1
> at
>
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:
> 510)
> at
>
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:
> 199)
> at
>
org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.j
> ava:283)
> ... 9 more
> Caused by: java.net.SocketException: Invalid argument
> at java.net.SocketInputStream.socketRead0(Native Method)
> at
java.net.SocketInputStream.read(SocketInputStream.java:152)
> at
java.net.SocketInputStream.read(SocketInputStream.java:122)
> at
sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
> at sun.security.ssl.InputRecord.read(InputRecord.java:480)
> at
> sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
> at
>
sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
> at
sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
> at
>
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessio
> nInputBuffer.java:160)
> at
>
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:
> 84)
> at
>
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessio
> nInputBuffer.java:273)
> at
>
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultH
> ttpResponseParser.java:140)
> at
>
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultH
> ttpResponseParser.java:57)
> at
>
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessagePars
> er.java:260)
> at
>
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader
> (AbstractHttpClientConnection.java:283)
> at
>
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader
> (DefaultClientConnection.java:251)
> at
>
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponse
> Header(ManagedClientConnectionImpl.java:197)
> at
>
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRe
> questExecutor.java:271)
> at
>
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecut
> or.java:123)
> at
>
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequ
> estDirector.java:682)
> at
>
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequest
> Director.java:486)
> at
>
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient
> .java:863)
> at
>
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient
> .java:82)
> at
>
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient
> .java:106)
> at
>
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient
> .java:57)
> at
>
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:
> 395)
> ... 11 more
>
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
<mailto:[email protected]> For additional
> commands, e-mail: [email protected]
<mailto:[email protected]>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
<mailto:[email protected]>
For additional commands, e-mail: [email protected]
<mailto:[email protected]>
