It seems like reasonable behavior. SolrWriter hangs while import is running, it holds DistributedZkUpdateProcessor, which holds SolrCmdDistributor, which keep client connection to shards open, which causes timeout. It might be worked around by supplying custom SolrWriter which finishes UpdateProcessor's chain from time to time and recreate it. However, that exception shouldn't cause any problem or it does? Also, it's worth to track as a jira, or mentioned in the ticket regarding adjusting DIH for Cloud.
On Mon, Sep 2, 2019 at 9:44 AM Vadim Ivanov < vadim.iva...@spb.ntk-intourist.ru> wrote: > I’ve raised that timeout from 5 min to 40 min. > > It somehow mitigated the issue in my use case. > > Problem occurs when some updates goes to one shard in the beginning of > long DIH session, then all updates goes to other shards for more than 300 > sec. > > And when ( after 300 sec) some updates goes again to the first shard it > faces closed connection problem. > > > > Either it’s right behavior or not – it’s hard to say…. > > > > > > -- > > Vadim > > *From:* Mikhail Khludnev [mailto:m...@apache.org] > *Sent:* Monday, September 02, 2019 1:31 AM > *To:* solr-user > *Cc:* vadim.iva...@spb.ntk-intourist.ru > *Subject:* Re: Idle Timeout while DIH indexing and implicit sharding in > 7.4 > > > > Giving that > > org.apache.solr.common.util.FastInputStream.peek(FastInputStream.java:60) > at > > org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:107) > > > > > JavabinLoader hangs on Stream.peek(), awaiting -1, and hit timeout. I > guess it's might be related with "closing sockets". I've looked through > CHANGES after 6.5.1, here's what might be examined for impacting this: > > > > * SOLR-10779: JavaBinCodec should use close consistently > > * SOLR-12290: Do not close any servlet streams and improve our servlet > stream > > Description Resource Path Location > * SOLR-12477: An update would return a client error(400) if it hit a > AlreadyClos > > * SOLR-12897: Introduce AlreadyClosedException to clean up silly close / > shutd > > > > Have nothing more than that so far. > > > > On Sun, Sep 1, 2019 at 8:50 PM swapna.minnaka...@copart.com < > swapna.minnaka...@copart.com> wrote: > > I am facing same exact issue. We never had any issue with 6.5.1 when doing > full index (initial bulk load) > After upgrading to 7.5.0, getting below exception and indexing is taking a > very long time > > 2019-09-01 10:11:27.436 ERROR (qtp1650813924-22) [c:c_collection s:shard1 > r:core_node3 x:c_collection_shard1_replica_n1] o.a.s.h.RequestHandlerBase > java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout > expired: 300000/300000 ms > at > > org.eclipse.jetty.server.HttpInput$ErrorState.noContent(HttpInput.java:1080) > at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:313) > at > > org.apache.solr.servlet.ServletInputStreamWrapper.read(ServletInputStreamWrapper.java:74) > at > > org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:100) > at > > org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputStream.java:79) > at > org.apache.solr.common.util.FastInputStream.refill(FastInputStream.java:88) > at > org.apache.solr.common.util.FastInputStream.peek(FastInputStream.java:60) > at > > org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:107) > at > org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55) > at > > org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97) > at > > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68) > at > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2541) > at > org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515) > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) > at > > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) > at > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > at > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > at > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) > at > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) > at > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > at > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) > at > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > at > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219) > at > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at > > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at org.eclipse.jetty.server.Server.handle(Server.java:531) > at > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) > at > org.eclipse.jetty.io > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) > at org.eclipse.jetty.io > .FillInterest.fillable(FillInterest.java:102) > at > org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > > > > -- > Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html > > > > > -- > > Sincerely yours > Mikhail Khludnev > -- Sincerely yours Mikhail Khludnev
