Ok, that stack trace shows where the problem. I'll investigate and report back.
Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Sep 27, 2018 at 4:01 PM Gaini Rajeshwar < raja.rajeshwar2...@gmail.com> wrote: > @Joel: On some of the shards, i am seeing the following error in logs. > > null:java.lang.NullPointerException > at > org.apache.lucene.util.BitSetIterator.<init>(BitSetIterator.java:61) > at > org.apache.solr.handler.ExportWriter.writeDocs(ExportWriter.java:243) > at > org.apache.solr.handler.ExportWriter.lambda$null$1(ExportWriter.java:222) > at > org.apache.solr.response.JSONWriter.writeIterator(JSONResponseWriter.java:515) > at > org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:179) > at > org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:551) > at > org.apache.solr.handler.ExportWriter.lambda$null$2(ExportWriter.java:222) > at > org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:539) > at > org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:181) > at > org.apache.solr.response.JSONWriter$2.put(JSONResponseWriter.java:551) > at > org.apache.solr.handler.ExportWriter.lambda$write$3(ExportWriter.java:220) > at > org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:539) > at > org.apache.solr.handler.ExportWriter.write(ExportWriter.java:218) > at org.apache.solr.core.SolrCore$4.write(SolrCore.java:2686) > at > org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:49) > at > org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:787) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:524) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > at org.eclipse.jetty.server.Server.handle(Server.java:531) > at > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) > at org.eclipse.jetty.io > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) > at org.eclipse.jetty.io > .FillInterest.fillable(FillInterest.java:102) > at org.eclipse.jetty.io > .ChannelEndPoint$2.run(ChannelEndPoint.java:118) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > at > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) > at java.lang.Thread.run(Thread.java:748) > > > On Fri, Sep 28, 2018 at 1:20 AM Gaini Rajeshwar < > raja.rajeshwar2...@gmail.com> wrote: > > > I've tried with 7.3 version also and encountering same issue. > > > > On Thu, Sep 27, 2018 at 11:02 PM Joel Bernstein <joels...@gmail.com> > > wrote: > > > >> That is weird, I've not encountered this behavior. There has been some > >> changes in 7.4 to the export handler, and I'm wondering if a bug was > >> introduced. The stack trace you are posting is coming from the node > where > >> the expression is being run. Can you check the logs from the shards to > see > >> if there are errors on the shards? > >> > >> > >> Joel Bernstein > >> http://joelsolr.blogspot.com/ > >> > >> > >> On Thu, Sep 27, 2018 at 12:10 PM Gaini Rajeshwar < > >> raja.rajeshwar2...@gmail.com> wrote: > >> > >> > Also, to mention, if i give q="*:*" it is working fine (which is kind > of > >> > weird) > >> > > >> > On Thu, Sep 27, 2018 at 9:35 PM Gaini Rajeshwar < > >> > raja.rajeshwar2...@gmail.com> wrote: > >> > > >> > > Hi All, > >> > > > >> > > I am using solr 7.4 version. I am trying to export data using > >> streaming > >> > > expressions. > >> > > > >> > > Following is the simple curl query that i am using. I am indexing my > >> data > >> > > and testing the following query. > >> > > > >> > > curl --data-urlencode 'expr=search(collection1, > >> > >> q="text:solar", > >> > >> fl="id", > >> > >> sort="id asc", > >> > >> qt="/export")' > http://localhost:8983/solr/collection1/stream > >> > > > >> > > > >> > > when i run above query with initial few documents indexed, it is > >> working > >> > > fine. After indexing few thousand documents (around , i am getting > the > >> > > following error for the same query. > >> > > > >> > > EXCEPTION":"java.util.concurrent.ExecutionException: > >> > >> java.io.IOException: --> http://localhost:8983/solr/collection1/: > An > >> > >> exception has occurred on the server, refer to server log for > >> details." > >> > > > >> > > > >> > > I've looked into my solr logs and the following is the error info > >> > > > >> > > java.io.IOException: java.util.concurrent.ExecutionException: > >> > >> java.io.IOException: --> http://localhost:8983/solr/collection1/: > An > >> > >> exception has occurred on the server, refer to server log for > >> details. > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:400) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.CloudSolrStream.open(CloudSolrStream.java:275) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:54) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:397) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.TupleStream.writeMap(TupleStream.java:83) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:539) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:181) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:209) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:325) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:120) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:71) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:787) > >> > >> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:524) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) > >> > >> at org.eclipse.jetty.server.Server.handle(Server.java:531) > >> > >> at > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) > >> > >> at > >> > >> org.eclipse.jetty.io > >> > > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) > >> > >> at org.eclipse.jetty.io > .FillInterest.fillable(FillInterest.java:102) > >> > >> at org.eclipse.jetty.io > >> .ChannelEndPoint$2.run(ChannelEndPoint.java:118) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) > >> > >> at > >> > >> > >> > > >> > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) > >> > >> at java.lang.Thread.run(Thread.java:748) > >> > >> Caused by: java.util.concurrent.ExecutionException: > >> java.io.IOException: > >> > >> --> http://localhost:8983/solr/collection1/: An exception has > >> occurred > >> > >> on the server, refer to server log for details. > >> > >> at java.util.concurrent.FutureTask.report(FutureTask.java:122) > >> > >> at java.util.concurrent.FutureTask.get(FutureTask.java:192) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:394) > >> > >> ... 49 more > >> > >> Caused by: java.io.IOException: --> > >> > >> http://localhost:8983/solr/collection1/: An exception has occurred > >> on > >> > >> the server, refer to server log for details. > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:225) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.CloudSolrStream$TupleWrapper.next(CloudSolrStream.java:484) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:507) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:494) > >> > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) > >> > >> at > >> > >> > >> > > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > >> > >> at > >> > >> > >> > > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > >> > >> ... 1 more > >> > >> Caused by: java.io.IOException: JSONTupleStream: expected > >> OBJECT_START > >> > >> but got EOF > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.JSONTupleStream.expect(JSONTupleStream.java:99) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.JSONTupleStream.advanceToDocs(JSONTupleStream.java:179) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.JSONTupleStream.next(JSONTupleStream.java:77) > >> > >> at > >> > >> > >> > > >> > org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:194) > >> > >> ... 8 more > >> > > > >> > > > >> > > Is it because new documents have something unusual in their content? > >> Even > >> > > if that is the case, i am requesting only one field which is **id** > >> and > >> > > there is nothing unusual about it. > >> > > > >> > > Any idea what what is issue here ? > >> > > > >> > > >> > > >
