[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105249#comment-13105249 ] Jasper van Veghel commented on SOLR-2066: - Martijn, would group.truncate be something that can also be added in with the same restrictions as group.ngroups (documents within the same group must reside on the same shard)? Would this be particularly tricky to add to the patch? It would definitely be nice to have, as people seem to rather care a lot about exact facet counts. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.5, 4.0 Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[
https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105460#comment-13105460
]
Martijn van Groningen commented on SOLR-2066:
-
{quote}
Martijn, would group.truncate be something that can also be added in with the
same restrictions as group.ngroups (documents within the same group must reside
on the same shard)? Would this be particularly tricky to add to the patch? It
would definitely be nice to have, as people seem to rather care a lot about
exact facet counts.
{quote}
No that is not that difficult. I was planning on adding it anyway.
{quote}
I took a quick peek at this, and I see some changes to how distrib search works
(ShardRequestFactory).
Could you give a brief explanation about the need for that and how it works?
Maybe changes like this should be in their own issue so it's easy to tell other
refactoring vs what's needed just for grouping.
{quote}
Sure! In the initial versions of the patch most of the code was in the
QueryComponent. The class became very large and I found it very difficult to
change / maintain. So I decided to move responsibilities to behind interfaces
and have different concrete implementations for different behavior. During
coding I also noticed that a few concepts (like ShardRequestFactory,
ShardResponseProcessor and ShardResultTransformer) are very useable for non
grouping distributed search. I chose not to refactor the current regular
distributed grouping, because otherwise this patch becomes to large. I think
after this is committed that we should refactor the non grouped distributed
search to use these concepts or similar.
Currently the regular distributed search implementation hasn't changed.
Concepts that can also be used for non grouped distributed searches:
* ShardRequestFactory is responsible for creating requests to shards in the
cluster based on the incoming request from the client.
* ShardResultTransformer. Transforming a NamedList response from the client in
for example SearchGroup or TopDocs instance.
* ShardResponseProcessor. Basically merges the shard responses. The
ShardReponseProcessor uses a ShardResultTransformer to transform the shard
response into a native structure (SearchGroup / TopGroups).
Grouping related concepts:
* EndResultTransformer. Basically form the end result structure. There're three
implementations for grouped, simple and main results.
* Command. Same responsibility as in Grouping.Command. Currently more focussed
on distributed search.
Search Grouping: support distributed search
---
Key: SOLR-2066
URL: https://issues.apache.org/jira/browse/SOLR-2066
Project: Solr
Issue Type: Sub-task
Reporter: Yonik Seeley
Fix For: 3.5, 4.0
Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch
Support distributed field collapsing / search grouping.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105474#comment-13105474 ] Jason Rutherglen commented on SOLR-2066: +1 on Concepts that can also be used for non grouped distributed searches in a separate issue. The Solr distributed search code is overly complicated. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.5, 4.0 Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105694#comment-13105694 ] Yonik Seeley commented on SOLR-2066: bq. I chose not to refactor the current regular distributed grouping, because otherwise this patch becomes to large. Ah, ok. So this currently just looks like an implementation detail for QueryComponent (and only used when grouping now), and not some change to the distrib search architecture. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.5, 4.0 Attachments: SOLR-2066-3x.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105708#comment-13105708 ] Martijn van Groningen commented on SOLR-2066: - Yes it is an implementation detail of the QueryComponent for grouping. But I think we should open a new issue and look what we can refactor, based on the concepts in this issue. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.5, 4.0 Attachments: SOLR-2066-3x.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104917#comment-13104917 ] Yonik Seeley commented on SOLR-2066: I took a quick peek at this, and I see some changes to how distrib search works (ShardRequestFactory). Could you give a brief explanation about the need for that and how it works? Maybe changes like this should be in their own issue so it's easy to tell other refactoring vs what's needed just for grouping. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.5, 4.0 Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[
https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103573#comment-13103573
]
Jasper van Veghel commented on SOLR-2066:
-
Apologies for continuing to bring up issues ;-) but it seems that facets also
cause problems with empty result sets. Modify the TestDistributedGrouping test
as follows and you'll see:
{code}// Test distributed grouping with empty indices
query(q, *:*, rows, 100, fl, id, + i1, group, true,
group.field, i1, group.limit, 10, sort, i1 + asc, id asc);
query(q, *:*, rows, 100, fl, id, + i1, group, true,
group.field, i1, group.limit, 10, sort, i1 + asc, id asc,
hl,true,hl.fl,t1);
query(q, *:*, rows, 100, fl, id, + i1, group, true,
group.field, i1, group.limit, 10, sort, i1 + asc, id asc, facet,
true, facet.field, t1);{code}
Stacktrace:
{code}[junit] Caused by: org.apache.solr.common.SolrException: null
java.lang.NullPointerException
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:409)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:194)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1407)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
null java.lang.NullPointerException
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:409)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:194)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1407)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.{code}
Search Grouping: support distributed search
---
Key: SOLR-2066
URL: https://issues.apache.org/jira/browse/SOLR-2066
Project: Solr
Issue Type: Sub-task
Reporter: Yonik Seeley
Fix For: 3.5, 4.0
Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch
Support distributed field collapsing / search grouping.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[
https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103067#comment-13103067
]
Jasper van Veghel commented on SOLR-2066:
-
Martijn, the problem seems to be with highlighting combined with empty result
sets. When I modify the TestDistributedGrouping test as follows:
{code}// Test distributed grouping with empty indices
query(q, *:*, rows, 100, fl, id, + i1, group, true,
group.field, i1, group.limit, 10, sort, i1 + asc, id asc);
query(q, *:*, rows, 100, fl, id, + i1, group, true,
group.field, i1, group.limit, 10, sort, i1 + asc, id asc,
hl,true,hl.fl,t1);{code}
I can reproduce the exact stacktrace. The exception doesn't occur with a
populated index.
Search Grouping: support distributed search
---
Key: SOLR-2066
URL: https://issues.apache.org/jira/browse/SOLR-2066
Project: Solr
Issue Type: Sub-task
Reporter: Yonik Seeley
Fix For: 3.5, 4.0
Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch
Support distributed field collapsing / search grouping.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[
https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13102261#comment-13102261
]
Jasper van Veghel commented on SOLR-2066:
-
You're more than welcome! Having distributed grouping will be a great addition
to have. As for the patch, could it be that you've modified a previous version
rather than the latest one that includes the highlighting fixes? I'm getting
the same highlighting-related stacktrace as before. ;-)
{code}SEVERE: java.lang.NullPointerException
at
org.apache.solr.handler.component.HighlightComponent.finishStage(HighlightComponent.java:156)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1407)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:680){code}
Search Grouping: support distributed search
---
Key: SOLR-2066
URL: https://issues.apache.org/jira/browse/SOLR-2066
Project: Solr
Issue Type: Sub-task
Reporter: Yonik Seeley
Fix For: 3.5, 4.0
Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch
Support distributed field collapsing / search grouping.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13102307#comment-13102307 ] Martijn van Groningen commented on SOLR-2066: - Jasper, does the exception occur occur for the same queries? I did add a test for this. Can you run the TestDistributedSearch test? Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.5, 4.0 Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[
https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13102000#comment-13102000
]
Jasper van Veghel commented on SOLR-2066:
-
With the latest patch I'm seeing this exception, even without any special
parameters like ngroups or hl:
{code}java.lang.NullPointerException at
org.apache.solr.search.grouping.firstphase.SearchGroupShardResponseProcessor.process(SearchGroupShardResponseProcessor.java:88)
at
org.apache.solr.handler.component.QueryComponent.handleGroupedResponses(QueryComponent.java:679)
at
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:662)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:292)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1407)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:680){code}
Search Grouping: support distributed search
---
Key: SOLR-2066
URL: https://issues.apache.org/jira/browse/SOLR-2066
Project: Solr
Issue Type: Sub-task
Reporter: Yonik Seeley
Fix For: 3.5, 4.0
Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch
Support distributed field collapsing / search grouping.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[
https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13101113#comment-13101113
]
Jasper van Veghel commented on SOLR-2066:
-
Great - ngroups is coming through now! Another thing I noticed is that
highlighting doesn't work on account of the resultIds not getting set in the
ResponseBuilder. It only happens in combination with _distributed_ grouping -
so it works when I do this:
http://localhost:8983/solr/foo/select?wt=jsonrows=2group=truegroup.field=dcterms_sourcegroup.ngroups=truehl=true
Or this:
http://localhost:8983/solr/foo/select?wt=jsonrows=2shards=localhost:8983/solr/foo,localhost:8983/solr/barhl=true
But not this:
http://localhost:8983/solr/foo/select?wt=jsonrows=2group=truegroup.field=dcterms_sourcegroup.ngroups=trueshards=localhost:8983/solr/foo,localhost:8983/solr/barhl=true
Stacktrace:
{code}SEVERE: java.lang.NullPointerException
at
org.apache.solr.handler.component.HighlightComponent.finishStage(HighlightComponent.java:156)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1407)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:680){code}
Search Grouping: support distributed search
---
Key: SOLR-2066
URL: https://issues.apache.org/jira/browse/SOLR-2066
Project: Solr
Issue Type: Sub-task
Reporter: Yonik Seeley
Fix For: 3.5, 4.0
Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch
Support distributed field collapsing / search grouping.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13100372#comment-13100372 ] Jasper van Veghel commented on SOLR-2066: - Thanks for adding the ngroups feature Martijn - it's really much appreciated; with it I can move forward with some of the functionality that I've been looking to add to a project of mine. The only thing is that I haven't gotten it to work quite yet. When I do: http://localhost:8983/solr/foo/select?wt=xmlrows=2group=truegroup.field=dcterms_sourcegroup.ngroups=true I get an ngroup-count along with the matches-count just fine, but when I then do: http://localhost:8983/solr/foo/select?wt=xmlrows=2group=truegroup.field=dcterms_sourcegroup.ngroups=trueshards=localhost:8383/solr/foo,localhost:8983/solr/bar It's not added in. Any thoughts? Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.4, 4.0 Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13100381#comment-13100381 ] Martijn van Groningen commented on SOLR-2066: - I noticed this also yesterday. I do merge the group count, but I forgot the add it to the final response... I'll update this patch later today. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.4, 4.0 Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[
https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13098878#comment-13098878
]
Michael McCandless commented on SOLR-2066:
--
The changes to modules/grouping look good...
There's a small typo in the comment (In case groups where indexed...
- In case groups were indexed...). Also I wouldn't call this
properly? Ie, it's OK if the same group value appears on more than
one shard? It just means we can't compute the exact total group count
(we can compute an upper bound).
Can you update the .merge javadocs? Now it says it cannot compute
totalGroupCount; I think just change that to say that it'll produce
the sum of the total group counts, which is an upper bound on the true
value? And this will be exact if you know the groups are orthogonal
across shards?
Maybe, for better readability, change this:
{noformat}
if (groupValue != null ? !groupValue.equals(that.groupValue) :
that.groupValue != null) return false;
{noformat}
to this:
{noformat}
if (groupValue == null) {
if (that.groupValue != null) {
return false;
}
} else if (!groupValue.equals(other.groupValue)) {
return false;
}
{noformat}
?
Search Grouping: support distributed search
---
Key: SOLR-2066
URL: https://issues.apache.org/jira/browse/SOLR-2066
Project: Solr
Issue Type: Sub-task
Reporter: Yonik Seeley
Fix For: 3.4, 4.0
Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch,
SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch
Support distributed field collapsing / search grouping.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13099153#comment-13099153 ] Martijn van Groningen commented on SOLR-2066: - Thanks for the review! I'll update the patch. The SearchGroup#equals method was generated, so that is why it isn't very readable. bq. And this will be exact if you know the groups are orthogonal across shards? Yes, if all the documents of a group reside on the same shard, the groupCount will be correct. bq. I think just change that to say that it'll produce the sum of the total group counts, which is an upper bound on the true value? If documents of a group do reside on more than one shard then the count will always be higher then it would be if all documents of a group stay on one shard. So it is an upper bound. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.4, 4.0 Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13097855#comment-13097855 ] Matt Beaumont commented on SOLR-2066: - Found two issues with this patch: 1. Using Faceting with the combination of sharding and grouping in our queries an error occurs. 2. When a shard returns no results and other shards do an error occurs. Thanks Matt. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.4, 4.0 Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13098249#comment-13098249 ] Martijn van Groningen commented on SOLR-2066: - Thanks for reporting these issues Matt! I'll update the patch soon. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.4, 4.0 Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13096930#comment-13096930 ] Jasper van Veghel commented on SOLR-2066: - Martijn, would it be possible to support group.ngroups for distributed grouping? I noticed that when merging TopGroups it's stated that totalGroupCount cannot be merged, but wouldn't it just be a matter of adding up the numbers (since groups shouldn't overlap across shards)? I'm probably missing something, but having a total number of groups - also across shards - is a tremendous help for paging; despite the performance penalty involved. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.4, 4.0 Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13096935#comment-13096935 ] Martijn van Groningen commented on SOLR-2066: - bq. I noticed that when merging TopGroups it's stated that totalGroupCount cannot be merged, but wouldn't it just be a matter of adding up the numbers (since groups shouldn't overlap across shards) If group.ngroups is specified we can just add the total group count together, but the Solr user is then responsible for making sure that the documents belonging to a group end up in the same shard. If this is not done, then the group counts will be incorrect. bq. I'm probably missing something, but having a total number of groups - also across shards - is a tremendous help for paging; despite the performance penalty involved. All shards need to send all the unique groups over the wire to the shard the is responsible for the group merging. For indices with a lot a of unique groups this will be very expensive. In the current solution only the top N groups (N being equals to rows param) are sent over the wire from the individual shards to the shard that is responsible for the group merging. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Fix For: 3.4, 4.0 Attachments: SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch, SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006453#comment-13006453 ] Harish Agarwal commented on SOLR-2066: -- Hi Otis - Sorry for the late reply, I've been travelling. I'll have a look at SOLR-1682 and try to build on top of that patch (I did build on top of the trunk originally, but it may be easier to work with the CollapseComponent). This patch was incomplete, I'd just wanted to get advice on implementation before moving forward with it whole hog. Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Attachments: SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] Commented: (SOLR-2066) Search Grouping: support distributed search
[ https://issues.apache.org/jira/browse/SOLR-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12996682#comment-12996682 ] Otis Gospodnetic commented on SOLR-2066: Harish - I haven't looked at this patch, but since you are not mentioning SOLR-1682, I'm assuming your patch doesn't build on top of what's in trunk (which is what SOLR-1682 provided). although I see it applies to today's trunk. If the above is correct, would it be possible for you to make a patch that adds support for distributed search to trunk? Also, here's feedback from somebody who applied and tried your patch today: ??The patch applied cleanly against the nightly build from last night, but a query resulted in solr hanging, then running out of heap and dumping.?? Search Grouping: support distributed search --- Key: SOLR-2066 URL: https://issues.apache.org/jira/browse/SOLR-2066 Project: Solr Issue Type: Sub-task Reporter: Yonik Seeley Attachments: SOLR-2066.patch Support distributed field collapsing / search grouping. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
