[jira] [Commented] (SOLR-10059) In SolrCloud, every fq added via is computed twice.
[
https://issues.apache.org/jira/browse/SOLR-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221474#comment-17221474
]
Christine Poerschke commented on SOLR-10059:
{quote}... The FacetComponent corrects this automatically by removing
duplicates. ...
{quote}
Thanks for sharing that detail!
[https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.3/solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java#L86-L101]
looks to be the area of code where duplicate facet parameters are removed.
I don't know how far or close we are here w.r.t. changing the existing
behaviour for every handler that inherits from {{RequestHandlerBase}} but
complementary to any changes, if a protected (say)
{{RequestHandlerBase.prepareRequestParams}} method was factored out then that
would slightly shorten the {{handleRequest}} method –
[https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.3/solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java#L189-L276]
– and custom handlers could override this method e.g. to de-duplicate
parameters.
How would having a custom handler differ from locally patching the
{{RequestHandlerBase}} code itself?
* Any {{RequestHandlerBase}} change automatically applies to all handlers that
inherit from it, no configuration changes would be needed but a locally built
Solr needs to be deployed.
* A custom handler means that changes only apply to the handler being
customised but a configuration change would be needed to use the custom
handler. Build wise the custom handler could be built separately and deployed
as a plugin i.e. no need to locally build Solr itself.
Another approach perhaps might be to have custom search component – a bit like
[~janhoy]'s
[RequestSanitizerComponent|https://github.com/cominvent/request-sanitizer-component]
– which dedupes parameters before any other component gets to see the request
and its parameters?
> In SolrCloud, every fq added via is computed twice.
>
>
> Key: SOLR-10059
> URL: https://issues.apache.org/jira/browse/SOLR-10059
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Affects Versions: 6.4
>Reporter: Marc Morissette
>Priority: Major
> Labels: performance
> Attachments:
> SOLR-10059-RequestHandlerBase-prepareRequestParams.patch, SOLR-10059_7x.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While researching another issue, I noticed that parameters appended to a
> query via SearchHandler's are added to the query twice
> in SolrCloud: once on the aggregator and again on the shard.
> The FacetComponent corrects this automatically by removing duplicates. Field
> queries added in this fashion are however computed twice and that hinders
> performance on filter queries that aren't simple bitsets such as those
> produced by the CollapsingQueryParser.
> To reproduce the issue, simply test this handler on a large enough
> collection, then replace "appends" with "defaults". You'll notice significant
> performance improvements.
> {code}
>
>
> {!collapse field=routingKey hint=top_fc}
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-10059) In SolrCloud, every fq added via is computed twice.
[
https://issues.apache.org/jira/browse/SOLR-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213937#comment-17213937
]
Johannes Baiter commented on SOLR-10059:
I believe SOLR-14931 describes exactly the issue [~tboeghk] described in the
above comment, thank you [~cpoerschke]!
> In SolrCloud, every fq added via is computed twice.
>
>
> Key: SOLR-10059
> URL: https://issues.apache.org/jira/browse/SOLR-10059
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Affects Versions: 6.4
>Reporter: Marc Morissette
>Priority: Major
> Labels: performance
> Attachments: SOLR-10059_7x.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While researching another issue, I noticed that parameters appended to a
> query via SearchHandler's are added to the query twice
> in SolrCloud: once on the aggregator and again on the shard.
> The FacetComponent corrects this automatically by removing duplicates. Field
> queries added in this fashion are however computed twice and that hinders
> performance on filter queries that aren't simple bitsets such as those
> produced by the CollapsingQueryParser.
> To reproduce the issue, simply test this handler on a large enough
> collection, then replace "appends" with "defaults". You'll notice significant
> performance improvements.
> {code}
>
>
> {!collapse field=routingKey hint=top_fc}
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-10059) In SolrCloud, every fq added via is computed twice.
[
https://issues.apache.org/jira/browse/SOLR-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213880#comment-17213880
]
Christine Poerschke commented on SOLR-10059:
Just adding a note here to cross-reference SOLR-10059 and SOLR-14931 as
potentially having things in common; haven't looked too closely though, only
going by _"SolrCloud" and "appends" and "twice"_ mentionings.
> In SolrCloud, every fq added via is computed twice.
>
>
> Key: SOLR-10059
> URL: https://issues.apache.org/jira/browse/SOLR-10059
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Affects Versions: 6.4
>Reporter: Marc Morissette
>Priority: Major
> Labels: performance
> Attachments: SOLR-10059_7x.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While researching another issue, I noticed that parameters appended to a
> query via SearchHandler's are added to the query twice
> in SolrCloud: once on the aggregator and again on the shard.
> The FacetComponent corrects this automatically by removing duplicates. Field
> queries added in this fashion are however computed twice and that hinders
> performance on filter queries that aren't simple bitsets such as those
> produced by the CollapsingQueryParser.
> To reproduce the issue, simply test this handler on a large enough
> collection, then replace "appends" with "defaults". You'll notice significant
> performance improvements.
> {code}
>
>
> {!collapse field=routingKey hint=top_fc}
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-10059) In SolrCloud, every fq added via is computed twice.
[
https://issues.apache.org/jira/browse/SOLR-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139917#comment-17139917
]
Torsten Bøgh Köster commented on SOLR-10059:
I attached a patch for this issue, which still exists in 7.x and 8.x.
In a distributes request, pre-configured query params in the "appends"-section
get re-appended on the shards. If those parameters furthermore reference other
parameters (like $qq), these do not get dereferenced. In our case, this broke
the collapse component.
The patch skips re-appending on the shards (_isShard=true_) if the parameter
_shards.handler.skipAppends=true_. The latter defaults to _false_.
> In SolrCloud, every fq added via is computed twice.
>
>
> Key: SOLR-10059
> URL: https://issues.apache.org/jira/browse/SOLR-10059
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Affects Versions: 6.4
>Reporter: Marc Morissette
>Priority: Major
> Labels: performance
> Attachments: SOLR-10059_7x.patch
>
>
> While researching another issue, I noticed that parameters appended to a
> query via SearchHandler's are added to the query twice
> in SolrCloud: once on the aggregator and again on the shard.
> The FacetComponent corrects this automatically by removing duplicates. Field
> queries added in this fashion are however computed twice and that hinders
> performance on filter queries that aren't simple bitsets such as those
> produced by the CollapsingQueryParser.
> To reproduce the issue, simply test this handler on a large enough
> collection, then replace "appends" with "defaults". You'll notice significant
> performance improvements.
> {code}
>
>
> {!collapse field=routingKey hint=top_fc}
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
