You are probably running into https://issues.apache.org/jira/browse/SOLR-6931
On Sat, Jan 24, 2015 at 12:09 AM, Mike Drob <mad...@cloudera.com> wrote: > I'm not sure what a reasonable workaround would be. Perhaps somebody else > can brainstorm and make a suggestion, sorry. > > On Tue, Jan 20, 2015 at 12:56 PM, Nishanth S <nishanth.2...@gmail.com> > wrote: > > > Thank you Mike.Sure enough,we are running into the same issue you > > mentoined.Is there a quick fix for this other than the patch.I do not see > > the tlogs getting replayed at all.It is doing a full index recovery from > > the leader and our index size is around 200G.Would lowering the > autocommit > > settings help(where the replica would go for a tlog replay as the tlogs I > > see are not huge). > > > > Thanks, > > Nishanth > > > > On Tue, Jan 20, 2015 at 10:46 AM, Mike Drob <md...@apache.org> wrote: > > > > > Are we sure this isn't SOLR-6931? > > > > > > On Tue, Jan 20, 2015 at 11:39 AM, Nishanth S <nishanth.2...@gmail.com> > > > wrote: > > > > > > > Hello All, > > > > > > > > We are running solr cloud 4.4 with 30 shards and 3 replicas with real > > > time > > > > indexing on rhel 6.5.The indexing rate is 3K Tps now.We are running > > into > > > an > > > > issue with replicas going into recovery mode due to connection reset > > > > errors.Soft commit time is 2 min and auto commit is set as 5 > minutes.I > > > have > > > > seen that replicas do a full index recovery which takes a long > > > > time(days).Below is the error trace that I see.I would really > > appreciate > > > > any help in this case. > > > > > > > > g.apache.solr.client.solrj.SolrServerException: IOException occured > > when > > > > talking to server at: http://xxx:8083/solr/log_pn_shard20_replica2 > > > > at > > > > > > > > > > > > > > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:435) > > > > at > > > > > > > > > > > > > > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) > > > > at > > > > > > > > > > > > > > org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:401) > > > > at > > > > > > > > > > > > > > org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:375) > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > > > at > > > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > > > at > > > > > > > > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > at > > > > > > > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:745) > > > > Caused by: java.net.SocketException: Connection reset > > > > at > java.net.SocketInputStream.read(SocketInputStream.java:196) > > > > at > java.net.SocketInputStream.read(SocketInputStream.java:122) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191) > > > > at > > > > > > > > > > > > > > org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300) > > > > at > > > > > > > > > > > > > > org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) > > > > at > > > > > > > > > > > > > > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) > > > > at > > > > > > > > > > > > > > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) > > > > ... 9 more > > > > > > > > > > > > Thanks, > > > > Nishanth > > > > > > > > > > -- Regards, Shalin Shekhar Mangar.
