Dmitry,
The last NPE in HighlightingComponent is just a sad coding issue.
few rows later we can see that developer expected to have some docs not
found
// remove nulls in case not all docs were able to be retrieved
rb.rsp.add("highlighting", SolrPluginUtils.removeNulls(new
SimpleOrderedMap(arr)));
But as you already know he forgot to check if(sdoc!=null){.
Is there anything that stopping you from contributing the patch, beside of
the lack of time, of course?
about the core issue I can't get into it and, particularly, how the using
disjunction query in place of IDS can help you. Could you please provide
more detailed info like stacktraces, etc. Btw, have you checked trunk for
your case?
On Thu, Apr 12, 2012 at 7:08 PM, Dmitry Kan <dmitry....@gmail.com> wrote:
> Can anyone help me out with this? Is this too complicated / unclear? I
> could share more detail if needed.
>
> On Wed, Apr 11, 2012 at 3:16 PM, Dmitry Kan <dmitry....@gmail.com> wrote:
>
> > Hello,
> >
> > Hopefully this question is not too complex to handle, but I'm currently
> > stuck with it.
> >
> > We have a system with nTiers, that is:
> >
> > Solr front base ---> Solr front --> shards
> >
> > Inside QueryComponent there is a method
> createRetrieveDocs(ResponseBuilder
> > rb) which collects doc ids of each shard and sends them in different
> > queries using the ids parameter:
> >
> > [code]
> > sreq.params.add(ShardParams.IDS, StrUtils.join(ids, ','));
> > [/code]
> >
> > This actually produces NPE (same as in
> > https://issues.apache.org/jira/browse/SOLR-1477) in the first tier,
> > because Solr front (on the second tier) fails to process such a query. I
> > have tried to fix this by using a unique field with a value of ids ORed
> > (the following code substitutes the code above):
> >
> > [code]
> > StringBuffer idsORed = new StringBuffer();
> > for (Iterator<String> iterator = ids.iterator();
> iterator.hasNext();
> > ) {
> > String next = iterator.next();
> >
> > if (iterator.hasNext()) {
> > idsORed.append(next).append(" OR ");
> > } else {
> > idsORed.append(next);
> > }
> > }
> >
> > sreq.params.add(rb.req.getSchema().getUniqueKeyField().getName(),
> > idsORed.toString());
> > [/code]
> >
> > This works perfectly if for rows=n there is n or less hits from a
> > distributed query. However, if there are more than 2*n hits, the querying
> > fails with an NPE in a completely different component, which is
> > HighlightComponent (highlights are requested in the same query with
> >
> hl=true&hl.fragsize=50000&hl.requireFieldMatch=true&hl.fl=targetTextField):
> >
> > SEVERE: java.lang.NullPointerException
> > at
> >
> org.apache.solr.handler.component.HighlightComponent.finishStage(HighlightComponent.java:161)
> > at
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> > at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> > at
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > at
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> > at
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> > at
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> > at
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> > at
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> > at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> > at
> >
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
> > at
> >
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
> > at
> > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
> > at java.lang.Thread.run(Thread.java:619)
> >
> > It sounds like the ids of documents somehow get shuffled and the
> > instruction (only a hypothesis)
> >
> > [code]
> > ShardDoc sdoc = rb.resultIds.get(id);
> > [/code]
> >
> > returns sdoc=null, which causes the next line of code to fail with an
> NPE:
> >
> > [code]
> > int idx = sdoc.positionInResponse;
> > [/code]
> >
> > Am I missing anything? Can something be done for solving this issue?
> >
> > Thanks.
> >
> > --
> > Regards,
> >
> > Dmitry Kan
> >
>
>
>
> --
> Regards,
>
> Dmitry Kan
>
--
Sincerely yours
Mikhail Khludnev
ge...@yandex.ru
<http://www.griddynamics.com>
<mkhlud...@griddynamics.com>
