[
https://issues.apache.org/jira/browse/SOLR-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589557#comment-13589557
]
Markus Jelsma commented on SOLR-4509:
-------------------------------------
Hi Ryan,
Until now i've not seen real significant latency improvements when performing
concurrent stress tests. The response time still hovers between 70ms and 100ms
with and without the patch using today's trunk. I did see some disturbing
exceptions, it looks like sometimes connections are reused that are already
dead, returning a connection refused exception. Sometimes shards are also
reported dead yielding the dreaded no servers hosting shard error.
Here are some exceptions:
{code}
Caused by: org.apache.solr.client.solrj.SolrServerException: No live
SolrServers available to handle this request:[...hosts here...]
at
org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:333)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:171)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:135)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException
occured when talking to server at: http://host/solr/shard_d
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:416)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at
org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:264)
... 10 more
Caused by: org.apache.http.NoHttpResponseException: The target server failed to
respond
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:95)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
... 12 more
{code}
{code}
2013-02-28 13:00:15,666 WARN [solr.cloud.RecoveryStrategy] - [main-EventThread]
- : Stopping recovery for
zkNodeName=178.21.118.192:8080_solr_openindex_dcore=openindex_d
2013-02-28 13:00:17,199 WARN [solr.update.PeerSync] - [main-EventThread] - :
PeerSync: core=shard_d url=http://host/solr couldn't connect to
http://host/shard/, counting as success
2013-02-28 13:00:17,201 ERROR [solr.cloud.SyncStrategy] - [main-EventThread] -
: Sync request error: org.apache.solr.client.solrj.SolrServerException: Server
refused connection at: http://host/shard
2013-02-28 13:00:17,205 ERROR [solr.cloud.SyncStrategy] -
[recoveryCmdExecutor-17-thread-1] - : http://host/shard/: Could not tell a
replica to recover:org.apache.solr.
client.solrj.SolrServerException: Server refused connection at: http://host
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:409)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.http.conn.HttpHostConnectException: Connection to
http://host refused
at
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
at
org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at
org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
... 5 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at
org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
... 12 more
{code}
I'm not sure yet why these occur, maybe it's my Tomcat settings that do not
match Jetty's defaults.
> Disable Stale Check - Distributed Search (Performance)
> ------------------------------------------------------
>
> Key: SOLR-4509
> URL: https://issues.apache.org/jira/browse/SOLR-4509
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: 5 node SmartOS cluster (all nodes living in same global
> zone - i.e. same physical machine)
> Reporter: Ryan Zezeski
> Priority: Minor
> Attachments: SOLR-4509.patch
>
>
> By disabling the Apache HTTP Client stale check I've witnessed a 2-4x
> increase in throughput and reduction of over 100ms. This patch was made in
> the context of a project I'm leading, called Yokozuna, which relies on
> distributed search.
> Here's the patch on Yokozuna: https://github.com/rzezeski/yokozuna/pull/26
> Here's a write-up I did on my findings:
> http://www.zinascii.com/2013/solr-distributed-search-and-the-stale-check.html
> I'm happy to answer any questions or make changes to the patch to make it
> acceptable.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
