Thank you Mike.Sure enough,we are running into the same issue you mentoined.Is there a quick fix for this other than the patch.I do not see the tlogs getting replayed at all.It is doing a full index recovery from the leader and our index size is around 200G.Would lowering the autocommit settings help(where the replica would go for a tlog replay as the tlogs I see are not huge).
Thanks, Nishanth On Tue, Jan 20, 2015 at 10:46 AM, Mike Drob <md...@apache.org> wrote: > Are we sure this isn't SOLR-6931? > > On Tue, Jan 20, 2015 at 11:39 AM, Nishanth S <nishanth.2...@gmail.com> > wrote: > > > Hello All, > > > > We are running solr cloud 4.4 with 30 shards and 3 replicas with real > time > > indexing on rhel 6.5.The indexing rate is 3K Tps now.We are running into > an > > issue with replicas going into recovery mode due to connection reset > > errors.Soft commit time is 2 min and auto commit is set as 5 minutes.I > have > > seen that replicas do a full index recovery which takes a long > > time(days).Below is the error trace that I see.I would really appreciate > > any help in this case. > > > > g.apache.solr.client.solrj.SolrServerException: IOException occured when > > talking to server at: http://xxx:8083/solr/log_pn_shard20_replica2 > > at > > > > > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:435) > > at > > > > > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) > > at > > > > > org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401) > > at > > > > > org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375) > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > at > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > at > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:745) > > Caused by: java.net.SocketException: Connection reset > > at java.net.SocketInputStream.read(SocketInputStream.java:196) > > at java.net.SocketInputStream.read(SocketInputStream.java:122) > > at > > > > > org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166) > > at > > > > > org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90) > > at > > > > > org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281) > > at > > > > > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92) > > at > > > > > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62) > > at > > > > > org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254) > > at > > > > > org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289) > > at > > > > > org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252) > > at > > > > > org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191) > > at > > > > > org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300) > > at > > > > > org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127) > > at > > > > > org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717) > > at > > > > > org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522) > > at > > > > > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) > > at > > > > > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) > > at > > > > > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) > > at > > > > > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) > > ... 9 more > > > > > > Thanks, > > Nishanth > > >
