[jira] [Updated] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-6930: --- Attachment: SOLR-6930.patch > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch, SOLR-6930.patch, SOLR-6930.patch, > SOLR-6930.patch, SOLR-6930.patch, SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-6930: --- Attachment: SOLR-6930.patch > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch, SOLR-6930.patch, SOLR-6930.patch, > SOLR-6930.patch, SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[
https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919526#comment-16919526
]
Dr Oleg Savrasov commented on SOLR-6930:
Request Parameters are supposed to be configured like
{code:java}
{"params":{
"circuit.breaker":{
"circuit.breaker.enabled":"true",
"threshold.unInvertedField":"30%",
"":{"v":0}
}
}}{code}
> Provide "Circuit Breakers" For Expensive Solr Queries
> -
>
> Key: SOLR-6930
> URL: https://issues.apache.org/jira/browse/SOLR-6930
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: Mike Drob
>Priority: Major
> Attachments: SOLR-6930.patch, SOLR-6930.patch, SOLR-6930.patch,
> SOLR-6930.patch
>
>
> Ref:
> http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html
> ES currently allows operators to configure "circuit breakers" to preemptively
> fail queries that are estimated too large rather than allowing an OOM
> Exception to happen. We might be able to do the same thing.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[
https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911237#comment-16911237
]
Dr Oleg Savrasov edited comment on SOLR-6930 at 8/30/19 12:47 PM:
--
Our customers very often experience OOM because of caching too large
UnInvertedField items. This happens when they try to count facets or sort by
not docvalues field. We don't want to fully disable UnInvertedFields usage
because in some cases they work better than docvalues. For example, if the
field is sparse and defined just in a few documents out of the millions, it
seems better to create and cache relatively small UnInvertedField rather than
traverse full docvalues structure. So to prevent OOM we need a kind of circuit
breaker which would count memory utilized by UnInvertedField items and forbid
caching them, when some configured threshold is exceeded. As you can see, with
such an approach we are not trying to predict memory consumption, we're just
restricting UnInvertedField cache by consumed memory.
Since such a breakers try to control memory usage, ideally they should be part
of some Global Resource Manager, see
https://issues.apache.org/jira/browse/SOLR-13578. But we don't have such a
manager so far, so for the time being we've tried to rely on existing Metric
framework. Memory consumption is controlled by custom MetricReporter, so to
enable circuit breaker it's necessary to configure appropriate unInvertedField
metric reporter in solr.xml, for example:
{code:java}
20%
{code}
was (Author: osavrasov):
Our customers very often experience OOM because of caching too large
UnInvertedField items. This happens when they try to count facets or sort by
not docvalues field. We don't want to fully disable UnInvertedFields usage
because in some cases they work better than docvalues. For example, if the
field is sparse and defined just in a few documents out of the millions, it
seems better to create and cache relatively small UnInvertedField rather than
traverse full docvalues structure. So to prevent OOM we need a kind of circuit
breaker which would count memory utilized by UnInvertedField items and forbid
caching them, when some configured threshold is exceeded. As you can see, with
such an approach we are not trying to predict memory consumption, we're just
restricting UnInvertedField cache by consumed memory.
Since such a breakers try to control memory usage, ideally they should be part
of some Global Resource Manager, see
https://issues.apache.org/jira/browse/SOLR-13578. But we don't have such a
manager so far, so for the time being we've tried to rely on existing Metric
framework. Memory consumption is controlled by custom MetricReporter, so to
enable circuit breaker it's necessary to configure appropriate unInvertedField
metric reporter in solr.xml, for example:
{code:java}
20%
{code}
> Provide "Circuit Breakers" For Expensive Solr Queries
> -
>
> Key: SOLR-6930
> URL: https://issues.apache.org/jira/browse/SOLR-6930
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: Mike Drob
>Priority: Major
> Attachments: SOLR-6930.patch, SOLR-6930.patch, SOLR-6930.patch,
> SOLR-6930.patch
>
>
> Ref:
> http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html
> ES currently allows operators to configure "circuit breakers" to preemptively
> fail queries that are estimated too large rather than allowing an OOM
> Exception to happen. We might be able to do the same thing.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919491#comment-16919491 ] Dr Oleg Savrasov commented on SOLR-6930: Patch is reworked to use Request Parameters API in order to modify Memory Circuit Breaker configuration on the fly. > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch, SOLR-6930.patch, SOLR-6930.patch, > SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-6930: --- Attachment: SOLR-6930.patch > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch, SOLR-6930.patch, SOLR-6930.patch, > SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16915614#comment-16915614 ] Dr Oleg Savrasov commented on SOLR-6930: Kill-switch config property has been added > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch, SOLR-6930.patch, SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-6930: --- Attachment: SOLR-6930.patch > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch, SOLR-6930.patch, SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914097#comment-16914097 ] Dr Oleg Savrasov commented on SOLR-6930: Patch is reworked in order to avoid finalize() invocation > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch, SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-6930: --- Attachment: SOLR-6930.patch > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch, SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-6930: --- Attachment: (was: SOLR-6930.patch) > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-6930: --- Attachment: SOLR-6930.patch > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch, SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[
https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911237#comment-16911237
]
Dr Oleg Savrasov edited comment on SOLR-6930 at 8/20/19 11:37 AM:
--
Our customers very often experience OOM because of caching too large
UnInvertedField items. This happens when they try to count facets or sort by
not docvalues field. We don't want to fully disable UnInvertedFields usage
because in some cases they work better than docvalues. For example, if the
field is sparse and defined just in a few documents out of the millions, it
seems better to create and cache relatively small UnInvertedField rather than
traverse full docvalues structure. So to prevent OOM we need a kind of circuit
breaker which would count memory utilized by UnInvertedField items and forbid
caching them, when some configured threshold is exceeded. As you can see, with
such an approach we are not trying to predict memory consumption, we're just
restricting UnInvertedField cache by consumed memory.
Since such a breakers try to control memory usage, ideally they should be part
of some Global Resource Manager, see
https://issues.apache.org/jira/browse/SOLR-13578. But we don't have such a
manager so far, so for the time being we've tried to rely on existing Metric
framework. Memory consumption is controlled by custom MetricReporter, so to
enable circuit breaker it's necessary to configure appropriate unInvertedField
metric reporter in solr.xml, for example:
{code:java}
20%}}
{code}
was (Author: osavrasov):
Our customers very often experience OOM because of caching too large
UnInvertedField items. This happens when they try to count facets or sort by
not docvalues field. We don't want to fully disable UnInvertedFields usage
because in some cases they work better than docvalues. For example, if the
field is sparse and defined just in a few documents out of the millions, it
seems better to create and cache relatively small UnInvertedField rather than
traverse full docvalues structure. So to prevent OOM we need a kind of circuit
breaker which would count memory utilized by UnInvertedField items and forbid
caching them, when some configured threshold is exceeded. As you can see, with
such an approach we are not trying to predict memory consumption, we're just
restricting UnInvertedField cache by consumed memory.
Since such a breakers try to control memory usage, ideally they should be part
of some Global Resource Manager, see
https://issues.apache.org/jira/browse/SOLR-13578. But we don't have such a
manager so far, so for the time being we've tried to rely on existing Metric
framework. Memory consumption is controlled by custom MetricReporter, so to
enable circuit breaker it's necessary to configure appropriate unInvertedField
metric reporter in solr.xml, for example:
{{ {{
{{ {{
{{ {{ 20%
{{ }}
{{ {{
{{ {{
> Provide "Circuit Breakers" For Expensive Solr Queries
> -
>
> Key: SOLR-6930
> URL: https://issues.apache.org/jira/browse/SOLR-6930
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: Mike Drob
>Priority: Major
> Attachments: SOLR-6930.patch
>
>
> Ref:
> http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html
> ES currently allows operators to configure "circuit breakers" to preemptively
> fail queries that are estimated too large rather than allowing an OOM
> Exception to happen. We might be able to do the same thing.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[
https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911237#comment-16911237
]
Dr Oleg Savrasov edited comment on SOLR-6930 at 8/20/19 11:37 AM:
--
Our customers very often experience OOM because of caching too large
UnInvertedField items. This happens when they try to count facets or sort by
not docvalues field. We don't want to fully disable UnInvertedFields usage
because in some cases they work better than docvalues. For example, if the
field is sparse and defined just in a few documents out of the millions, it
seems better to create and cache relatively small UnInvertedField rather than
traverse full docvalues structure. So to prevent OOM we need a kind of circuit
breaker which would count memory utilized by UnInvertedField items and forbid
caching them, when some configured threshold is exceeded. As you can see, with
such an approach we are not trying to predict memory consumption, we're just
restricting UnInvertedField cache by consumed memory.
Since such a breakers try to control memory usage, ideally they should be part
of some Global Resource Manager, see
https://issues.apache.org/jira/browse/SOLR-13578. But we don't have such a
manager so far, so for the time being we've tried to rely on existing Metric
framework. Memory consumption is controlled by custom MetricReporter, so to
enable circuit breaker it's necessary to configure appropriate unInvertedField
metric reporter in solr.xml, for example:
{code:java}
20%
{code}
was (Author: osavrasov):
Our customers very often experience OOM because of caching too large
UnInvertedField items. This happens when they try to count facets or sort by
not docvalues field. We don't want to fully disable UnInvertedFields usage
because in some cases they work better than docvalues. For example, if the
field is sparse and defined just in a few documents out of the millions, it
seems better to create and cache relatively small UnInvertedField rather than
traverse full docvalues structure. So to prevent OOM we need a kind of circuit
breaker which would count memory utilized by UnInvertedField items and forbid
caching them, when some configured threshold is exceeded. As you can see, with
such an approach we are not trying to predict memory consumption, we're just
restricting UnInvertedField cache by consumed memory.
Since such a breakers try to control memory usage, ideally they should be part
of some Global Resource Manager, see
https://issues.apache.org/jira/browse/SOLR-13578. But we don't have such a
manager so far, so for the time being we've tried to rely on existing Metric
framework. Memory consumption is controlled by custom MetricReporter, so to
enable circuit breaker it's necessary to configure appropriate unInvertedField
metric reporter in solr.xml, for example:
{code:java}
20%}}
{code}
> Provide "Circuit Breakers" For Expensive Solr Queries
> -
>
> Key: SOLR-6930
> URL: https://issues.apache.org/jira/browse/SOLR-6930
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: Mike Drob
>Priority: Major
> Attachments: SOLR-6930.patch
>
>
> Ref:
> http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html
> ES currently allows operators to configure "circuit breakers" to preemptively
> fail queries that are estimated too large rather than allowing an OOM
> Exception to happen. We might be able to do the same thing.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[
https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911237#comment-16911237
]
Dr Oleg Savrasov edited comment on SOLR-6930 at 8/20/19 11:35 AM:
--
Our customers very often experience OOM because of caching too large
UnInvertedField items. This happens when they try to count facets or sort by
not docvalues field. We don't want to fully disable UnInvertedFields usage
because in some cases they work better than docvalues. For example, if the
field is sparse and defined just in a few documents out of the millions, it
seems better to create and cache relatively small UnInvertedField rather than
traverse full docvalues structure. So to prevent OOM we need a kind of circuit
breaker which would count memory utilized by UnInvertedField items and forbid
caching them, when some configured threshold is exceeded. As you can see, with
such an approach we are not trying to predict memory consumption, we're just
restricting UnInvertedField cache by consumed memory.
Since such a breakers try to control memory usage, ideally they should be part
of some Global Resource Manager, see
https://issues.apache.org/jira/browse/SOLR-13578. But we don't have such a
manager so far, so for the time being we've tried to rely on existing Metric
framework. Memory consumption is controlled by custom MetricReporter, so to
enable circuit breaker it's necessary to configure appropriate unInvertedField
metric reporter in solr.xml, for example:
{{ {{
{{ {{
{{ {{ 20%
{{ }}
{{ {{
{{ {{
was (Author: osavrasov):
Our customers very often experience OOM because of caching too large
UnInvertedField items. This happens when they try to count facets or sort by
not docvalues field. We don't want to fully disable UnInvertedFields usage
because in some cases they work better than docvalues. For example, if the
field is sparse and defined just in a few documents out of the millions, it
seems better to create and cache relatively small UnInvertedField rather than
traverse full docvalues structure. So to prevent OOM we need a kind of circuit
breaker which would count memory utilized by UnInvertedField items and forbid
caching them, when some configured threshold is exceeded. As you can see, with
such an approach we are not trying to predict memory consumption, we're just
restricting UnInvertedField cache by consumed memory.
Since such a breakers try to control memory usage, ideally they should be part
of some Global Resource Manager, see
https://issues.apache.org/jira/browse/SOLR-13578. But we don't have such a
manager so far, so for the time being we've tried to rely on existing Metric
framework. Memory consumption is controlled by custom MetricReporter, so to
enable circuit breaker it's necessary to configure appropriate unInvertedField
metric reporter in solr.xml, for example:
{{}}
{{ }}
{{ }}
{{ 20%}}
{{ }}
{{ }}
{{}}
> Provide "Circuit Breakers" For Expensive Solr Queries
> -
>
> Key: SOLR-6930
> URL: https://issues.apache.org/jira/browse/SOLR-6930
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: Mike Drob
>Priority: Major
> Attachments: SOLR-6930.patch
>
>
> Ref:
> http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html
> ES currently allows operators to configure "circuit breakers" to preemptively
> fail queries that are estimated too large rather than allowing an OOM
> Exception to happen. We might be able to do the same thing.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[
https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911237#comment-16911237
]
Dr Oleg Savrasov commented on SOLR-6930:
Our customers very often experience OOM because of caching too large
UnInvertedField items. This happens when they try to count facets or sort by
not docvalues field. We don't want to fully disable UnInvertedFields usage
because in some cases they work better than docvalues. For example, if the
field is sparse and defined just in a few documents out of the millions, it
seems better to create and cache relatively small UnInvertedField rather than
traverse full docvalues structure. So to prevent OOM we need a kind of circuit
breaker which would count memory utilized by UnInvertedField items and forbid
caching them, when some configured threshold is exceeded. As you can see, with
such an approach we are not trying to predict memory consumption, we're just
restricting UnInvertedField cache by consumed memory.
Since such a breakers try to control memory usage, ideally they should be part
of some Global Resource Manager, see
https://issues.apache.org/jira/browse/SOLR-13578. But we don't have such a
manager so far, so for the time being we've tried to rely on existing Metric
framework. Memory consumption is controlled by custom MetricReporter, so to
enable circuit breaker it's necessary to configure appropriate unInvertedField
metric reporter in solr.xml, for example:
{{}}
{{ }}
{{ }}
{{ 20%}}
{{ }}
{{ }}
{{}}
> Provide "Circuit Breakers" For Expensive Solr Queries
> -
>
> Key: SOLR-6930
> URL: https://issues.apache.org/jira/browse/SOLR-6930
> Project: Solr
> Issue Type: Improvement
> Components: search
>Reporter: Mike Drob
>Priority: Major
> Attachments: SOLR-6930.patch
>
>
> Ref:
> http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html
> ES currently allows operators to configure "circuit breakers" to preemptively
> fail queries that are estimated too large rather than allowing an OOM
> Exception to happen. We might be able to do the same thing.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6930) Provide "Circuit Breakers" For Expensive Solr Queries
[ https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-6930: --- Attachment: SOLR-6930.patch > Provide "Circuit Breakers" For Expensive Solr Queries > - > > Key: SOLR-6930 > URL: https://issues.apache.org/jira/browse/SOLR-6930 > Project: Solr > Issue Type: Improvement > Components: search >Reporter: Mike Drob >Priority: Major > Attachments: SOLR-6930.patch > > > Ref: > http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html > ES currently allows operators to configure "circuit breakers" to preemptively > fail queries that are estimated too large rather than allowing an OOM > Exception to happen. We might be able to do the same thing. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13500) Vector Search with Solr
[ https://issues.apache.org/jira/browse/SOLR-13500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907000#comment-16907000 ] Dr Oleg Savrasov commented on SOLR-13500: - At first glance this looks like duplicate of https://issues.apache.org/jira/browse/SOLR-12890 > Vector Search with Solr > --- > > Key: SOLR-13500 > URL: https://issues.apache.org/jira/browse/SOLR-13500 > Project: Solr > Issue Type: New Feature >Reporter: Dr Oleg Savrasov >Priority: Major > > In machine learning problems we usually need to find nearest neighbors in > dense high dimensional vector space. Often, this retrieval had to be > augmented with traditional Boolean retrieval. > While no efficient KNN implementation is available in Solr yet, in practice > people are using brute-force vector distance sorting after Boolean retrieval > limits number of candidate vectors. > It's proposed to add a clear way to implement such a solution with Solr. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13500) Vector Search with Solr
Dr Oleg Savrasov created SOLR-13500: --- Summary: Vector Search with Solr Key: SOLR-13500 URL: https://issues.apache.org/jira/browse/SOLR-13500 Project: Solr Issue Type: New Feature Security Level: Public (Default Security Level. Issues are Public) Reporter: Dr Oleg Savrasov In machine learning problems we usually need to find nearest neighbors in dense high dimensional vector space. Often, this retrieval had to be augmented with traditional Boolean retrieval. While no efficient KNN implementation is available in Solr yet, in practice people are using brute-force vector distance sorting after Boolean retrieval limits number of candidate vectors. It's proposed to add a clear way to implement such a solution with Solr. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13081) In-Place Update doesn't work when route.field is defined
[ https://issues.apache.org/jira/browse/SOLR-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16824864#comment-16824864 ] Dr Oleg Savrasov commented on SOLR-13081: - Hi, [~munendrasn], your understanding is correct. Thank you for reviewing the patch, I've modified it according to your comments. > In-Place Update doesn't work when route.field is defined > > > Key: SOLR-13081 > URL: https://issues.apache.org/jira/browse/SOLR-13081 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: update >Reporter: Dr Oleg Savrasov >Priority: Major > Attachments: SOLR-13081.patch, SOLR-13081.patch, SOLR-13081.patch, > SOLR-13081.patch > > > As soon as cloud collection is configured with route.field property, In-Place > Updates are not applied anymore. This happens because > AtomicUpdateDocumentMerger skips only id and version fields and doesn't > verify configured route.field. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13081) In-Place Update doesn't work when route.field is defined
Dr Oleg Savrasov created SOLR-13081: --- Summary: In-Place Update doesn't work when route.field is defined Key: SOLR-13081 URL: https://issues.apache.org/jira/browse/SOLR-13081 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: update Reporter: Dr Oleg Savrasov As soon as cloud collection is configured with route.field property, In-Place Updates are not applied anymore. This happens because AtomicUpdateDocumentMerger skips only id and version fields and doesn't verify configured route.field. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-5211) updating parent as childless makes old children orphans
[ https://issues.apache.org/jira/browse/SOLR-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16690359#comment-16690359 ] Dr Oleg Savrasov edited comment on SOLR-5211 at 11/17/18 3:57 AM: -- [~moshebla], don't have time these days, sorry. I'll try to return to the feature in December-January if it's not too late. was (Author: osavrasov): [~moshebla], don't have time these days, sorry. > updating parent as childless makes old children orphans > --- > > Key: SOLR-5211 > URL: https://issues.apache.org/jira/browse/SOLR-5211 > Project: Solr > Issue Type: Sub-task > Components: update >Affects Versions: 4.5, 6.0 >Reporter: Mikhail Khludnev >Assignee: David Smiley >Priority: Blocker > Fix For: master (8.0) > > Attachments: SOLR-5211.patch, SOLR-5211.patch, SOLR-5211.patch > > > if I have parent with children in the index, I can send update omitting > children. as a result old children become orphaned. > I suppose separate \_root_ fields makes much trouble. I propose to extend > notion of uniqueKey, and let it spans across blocks that makes updates > unambiguous. > WDYT? Do you like to see a test proves this issue? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5211) updating parent as childless makes old children orphans
[ https://issues.apache.org/jira/browse/SOLR-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16690359#comment-16690359 ] Dr Oleg Savrasov commented on SOLR-5211: [~moshebla], don't have time these days, sorry. > updating parent as childless makes old children orphans > --- > > Key: SOLR-5211 > URL: https://issues.apache.org/jira/browse/SOLR-5211 > Project: Solr > Issue Type: Sub-task > Components: update >Affects Versions: 4.5, 6.0 >Reporter: Mikhail Khludnev >Assignee: David Smiley >Priority: Blocker > Fix For: master (8.0) > > Attachments: SOLR-5211.patch, SOLR-5211.patch, SOLR-5211.patch > > > if I have parent with children in the index, I can send update omitting > children. as a result old children become orphaned. > I suppose separate \_root_ fields makes much trouble. I propose to extend > notion of uniqueKey, and let it spans across blocks that makes updates > unambiguous. > WDYT? Do you like to see a test proves this issue? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5211) updating parent as childless makes old children orphans
[ https://issues.apache.org/jira/browse/SOLR-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16521999#comment-16521999 ] Dr Oleg Savrasov commented on SOLR-5211: The patch is reworked according to the latest changes in DirectUpdateHandler2. Added test for the case when update has child docs, but there is no _root_ in schema. Personally, I'd prefer to move getLuceneDocument and getLuceneDocsIfNested from AddUpdateCommand to DirectUpdateHandler2. I believe that logic of documents conversion is not responsibility of the command, and should be done by handler. Ready to rework the patch if needed. > updating parent as childless makes old children orphans > --- > > Key: SOLR-5211 > URL: https://issues.apache.org/jira/browse/SOLR-5211 > Project: Solr > Issue Type: Sub-task > Components: update >Affects Versions: 4.5, 6.0 >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-5211.patch, SOLR-5211.patch, SOLR-5211.patch > > > if I have parent with children in the index, I can send update omitting > children. as a result old children become orphaned. > I suppose separate \_root_ fields makes much trouble. I propose to extend > notion of uniqueKey, and let it spans across blocks that makes updates > unambiguous. > WDYT? Do you like to see a test proves this issue? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5211) updating parent as childless makes old children orphans
[ https://issues.apache.org/jira/browse/SOLR-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5211: --- Attachment: SOLR-5211.patch > updating parent as childless makes old children orphans > --- > > Key: SOLR-5211 > URL: https://issues.apache.org/jira/browse/SOLR-5211 > Project: Solr > Issue Type: Sub-task > Components: update >Affects Versions: 4.5, 6.0 >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-5211.patch, SOLR-5211.patch, SOLR-5211.patch > > > if I have parent with children in the index, I can send update omitting > children. as a result old children become orphaned. > I suppose separate \_root_ fields makes much trouble. I propose to extend > notion of uniqueKey, and let it spans across blocks that makes updates > unambiguous. > WDYT? Do you like to see a test proves this issue? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-12490) referring/excluding clauses from JSON query DSL in JSON facets.
[
https://issues.apache.org/jira/browse/SOLR-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16515095#comment-16515095
]
Dr Oleg Savrasov edited comment on SOLR-12490 at 6/17/18 2:45 PM:
--
I'd also propose to add ability to exclude tagged query from domain filter, i.e
{code:java}
"domain" : {
"excludeTags":"top", // we need to drop top-level parent
query
"filter":[
{"param":"childFq", // referring to .queries.childFq
"exclude":"color"} // excluding one child filter clause
]
}
{code}
was (Author: osavrasov):
I'd also propose to add ability to exclude tagged query from domain filter, i.e
{code:java}
"domain" : {
"excludeTags":"top", // we need to drop top-level parent
query
"filter":[
{"param":"childFq", // referring to .queries.childFq
"exclude":"color"} // excluding one child filter clause
]
}
{code}
> referring/excluding clauses from JSON query DSL in JSON facets.
>
>
> Key: SOLR-12490
> URL: https://issues.apache.org/jira/browse/SOLR-12490
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting
>Reporter: Mikhail Khludnev
>Priority: Major
>
> It's spin off from the
> [discussion|https://issues.apache.org/jira/browse/SOLR-9685?focusedCommentId=16508720&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16508720].
>
> h2. Problem
> # after SOLR-9685 we can tag separate clauses in hairish queries like
> {{parent}}, {{bool}}
> # we can {{domain.excludeTags}}
> # we are looking for child faceting with exclusions, see SOLR-9510, SOLR-8998
>
> # but we can refer only separate params in {{domain.filter}}, it's not
> possible to refer separate clauses
> h2. Proposal
> # tag child clauses multiple times
> {code}
> {
> "query" : {
> "#top":{
> "parent": {
> "query": "sku-title:foo",
> "filters" : [
> "scope:sku",
> { "#sku,color" : "color:black" }, // multiple tags
> { "#sku,size" : "size:L" }
> ],
> "which": "scope:product"
>}
> }
> }
> }
> {code}
> # refer to sku clauses, either by
> ## (1) {{domain.filter.tag}} in addition to {{param}}, or
> ## (2) {{domain.includeTags}} mimicking {{excludeTags}}
> {code}
> "facet":{
> "sku_colors_in_prods":{
> "type" : "terms",
> "field" : "color",
>"domain" : {
> "excludeTags":["top","color"], // we need to drop top-level
> parent query
> "filter":[
> {"tag":"sku"} // (1)
>],
> "includeTags":"sku" // (2)
>},
> "facet":"uniqueBlock(_root_)"
>}
> }
> {code}
> WDYT, [~osavrasov], [~ysee...@gmail.com]?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-12490) referring/excluding clauses from JSON query DSL in JSON facets.
[
https://issues.apache.org/jira/browse/SOLR-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16515095#comment-16515095
]
Dr Oleg Savrasov edited comment on SOLR-12490 at 6/17/18 2:44 PM:
--
I'd also propose to add ability to exclude tagged query from domain filter, i.e
{code:java}
"domain" : {
"excludeTags":"top", // we need to drop top-level parent
query
"filter":[
{"param":"childFq", // referring to .queries.childFq
"exclude":"color"} // excluding one child filter clause
]
}
{code}
was (Author: osavrasov):
I'd also propose to add ability to exclude tagged query from domain filter, i.e
{code:java}
"domain" : {
"excludeTags":"top", // we need to drop top-level parent
query
"filter":[
{"param":"childFq", // referring to .queries.childFq
"exclude":"color"} // excluding one child filter clause
]
}
{code}
> referring/excluding clauses from JSON query DSL in JSON facets.
>
>
> Key: SOLR-12490
> URL: https://issues.apache.org/jira/browse/SOLR-12490
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting
>Reporter: Mikhail Khludnev
>Priority: Major
>
> It's spin off from the
> [discussion|https://issues.apache.org/jira/browse/SOLR-9685?focusedCommentId=16508720&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16508720].
>
> h2. Problem
> # after SOLR-9685 we can tag separate clauses in hairish queries like
> {{parent}}, {{bool}}
> # we can {{domain.excludeTags}}
> # we are looking for child faceting with exclusions, see SOLR-9510, SOLR-8998
>
> # but we can refer only separate params in {{domain.filter}}, it's not
> possible to refer separate clauses
> h2. Proposal
> # tag child clauses multiple times
> {code}
> {
> "query" : {
> "#top":{
> "parent": {
> "query": "sku-title:foo",
> "filters" : [
> "scope:sku",
> { "#sku,color" : "color:black" }, // multiple tags
> { "#sku,size" : "size:L" }
> ],
> "which": "scope:product"
>}
> }
> }
> }
> {code}
> # refer to sku clauses, either by
> ## (1) {{domain.filter.tag}} in addition to {{param}}, or
> ## (2) {{domain.includeTags}} mimicking {{excludeTags}}
> {code}
> "facet":{
> "sku_colors_in_prods":{
> "type" : "terms",
> "field" : "color",
>"domain" : {
> "excludeTags":["top","color"], // we need to drop top-level
> parent query
> "filter":[
> {"tag":"sku"} // (1)
>],
> "includeTags":"sku" // (2)
>},
> "facet":"uniqueBlock(_root_)"
>}
> }
> {code}
> WDYT, [~osavrasov], [~ysee...@gmail.com]?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-12490) referring/excluding clauses from JSON query DSL in JSON facets.
[
https://issues.apache.org/jira/browse/SOLR-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16515095#comment-16515095
]
Dr Oleg Savrasov edited comment on SOLR-12490 at 6/17/18 2:44 PM:
--
I'd also propose to add ability to exclude tagged query from domain filter, i.e
{code:java}
"domain" : {
"excludeTags":"top", // we need to drop top-level parent
query
"filter":[
{"param":"childFq", // referring to .queries.childFq
"exclude":"color"} // excluding one child filter clause
]
}
{code}
was (Author: osavrasov):
I'd also propose to add ability to exclude tagged query from domain filter, i.e
"domain" : { "excludeTags":"top", // we need to
drop top-level parent query"filter":[
{"param":"childFq", // referring to .queries.childFq
"exclude":"color"} // excluding one child filter clause
]
}
> referring/excluding clauses from JSON query DSL in JSON facets.
>
>
> Key: SOLR-12490
> URL: https://issues.apache.org/jira/browse/SOLR-12490
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting
>Reporter: Mikhail Khludnev
>Priority: Major
>
> It's spin off from the
> [discussion|https://issues.apache.org/jira/browse/SOLR-9685?focusedCommentId=16508720&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16508720].
>
> h2. Problem
> # after SOLR-9685 we can tag separate clauses in hairish queries like
> {{parent}}, {{bool}}
> # we can {{domain.excludeTags}}
> # we are looking for child faceting with exclusions, see SOLR-9510, SOLR-8998
>
> # but we can refer only separate params in {{domain.filter}}, it's not
> possible to refer separate clauses
> h2. Proposal
> # tag child clauses multiple times
> {code}
> {
> "query" : {
> "#top":{
> "parent": {
> "query": "sku-title:foo",
> "filters" : [
> "scope:sku",
> { "#sku,color" : "color:black" }, // multiple tags
> { "#sku,size" : "size:L" }
> ],
> "which": "scope:product"
>}
> }
> }
> }
> {code}
> # refer to sku clauses, either by
> ## (1) {{domain.filter.tag}} in addition to {{param}}, or
> ## (2) {{domain.includeTags}} mimicking {{excludeTags}}
> {code}
> "facet":{
> "sku_colors_in_prods":{
> "type" : "terms",
> "field" : "color",
>"domain" : {
> "excludeTags":["top","color"], // we need to drop top-level
> parent query
> "filter":[
> {"tag":"sku"} // (1)
>],
> "includeTags":"sku" // (2)
>},
> "facet":"uniqueBlock(_root_)"
>}
> }
> {code}
> WDYT, [~osavrasov], [~ysee...@gmail.com]?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12490) referring/excluding clauses from JSON query DSL in JSON facets.
[
https://issues.apache.org/jira/browse/SOLR-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16515095#comment-16515095
]
Dr Oleg Savrasov commented on SOLR-12490:
-
I'd also propose to add ability to exclude tagged query from domain filter, i.e
"domain" : { "excludeTags":"top", // we need to
drop top-level parent query"filter":[
{"param":"childFq", // referring to .queries.childFq
"exclude":"color"} // excluding one child filter clause
]
}
> referring/excluding clauses from JSON query DSL in JSON facets.
>
>
> Key: SOLR-12490
> URL: https://issues.apache.org/jira/browse/SOLR-12490
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting
>Reporter: Mikhail Khludnev
>Priority: Major
>
> It's spin off from the
> [discussion|https://issues.apache.org/jira/browse/SOLR-9685?focusedCommentId=16508720&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16508720].
>
> h2. Problem
> # after SOLR-9685 we can tag separate clauses in hairish queries like
> {{parent}}, {{bool}}
> # we can {{domain.excludeTags}}
> # we are looking for child faceting with exclusions, see SOLR-9510, SOLR-8998
>
> # but we can refer only separate params in {{domain.filter}}, it's not
> possible to refer separate clauses
> h2. Proposal
> # tag child clauses multiple times
> {code}
> {
> "query" : {
> "#top":{
> "parent": {
> "query": "sku-title:foo",
> "filters" : [
> "scope:sku",
> { "#sku,color" : "color:black" }, // multiple tags
> { "#sku,size" : "size:L" }
> ],
> "which": "scope:product"
>}
> }
> }
> }
> {code}
> # refer to sku clauses, either by
> ## (1) {{domain.filter.tag}} in addition to {{param}}, or
> ## (2) {{domain.includeTags}} mimicking {{excludeTags}}
> {code}
> "facet":{
> "sku_colors_in_prods":{
> "type" : "terms",
> "field" : "color",
>"domain" : {
> "excludeTags":["top","color"], // we need to drop top-level
> parent query
> "filter":[
> {"tag":"sku"} // (1)
>],
> "includeTags":"sku" // (2)
>},
> "facet":"uniqueBlock(_root_)"
>}
> }
> {code}
> WDYT, [~osavrasov], [~ysee...@gmail.com]?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-9685) tag a query in JSON syntax
[
https://issues.apache.org/jira/browse/SOLR-9685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508720#comment-16508720
]
Dr Oleg Savrasov edited comment on SOLR-9685 at 6/11/18 8:56 PM:
-
[~mkhludnev], sorry for invalid JSON, my fault. An attempt to make it valid
didn't help as well
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
query : "{!parent tag=top filters=$child.fq which=scope:product
v=$childquery}",
filter : "{!tag=top}category:clothes",
params:{
childquery : "scope:sku",
child.fq : { "#color" : "color:black" }
}
}'{code}
I've tried to rework the request according to JSON DSL and just want to share
my results, because I'm not sure if they are supposed to work or not.
This one works fine
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"query": "scope:sku",
"filters" : [
"{!tag=color}color:black",
"{!tag=size}size:L"
],
"which": "scope:product"
}
}
}'
{code}
while this
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"query": "scope:sku",
"filters" : [
{ "#color" : "color:black" },
{ "#size" : "size:L" }
],
"which": "scope:product"
}
}
}'{code}
responds with
{code:java}
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","org.apache.solr.search.SyntaxError"],
"msg":"org.apache.solr.search.SyntaxError: Missing end to unquoted value
starting at 6 str='{!tag=color'",
"code":400}}{code}
Absolutely the same results I've got with
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"which": "scope:product",
"query": {
"bool": {
"must":"scope:sku",
"filter":[
"{!tag=color}color:black",
"{!tag=size}size:L"
]
}
}
}
}
}'{code}
and
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"which": "scope:product",
"query": {
"bool": {
"must":"scope:sku",
"filter":[
{ "#color" : "color:black" },
{ "#size" : "size:L" }
]
}
}
}
}
}'{code}
Finally [~mkhludnev]'s helped to find a working version, which doesn't look
obvious for me.
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"which": "scope:product",
"query": {
"bool": {
"must":"scope:sku",
"filter":[
{ "#color" : {lucene: { "query":"color:black"}
}},
{ "#size" : {lucene: { "query":"size:L"} }}
]
}
}
}
}
}'{code}
Let me express my huge thanks to [~mkhludnev] for his incredible creativity and
support.
was (Author: osavrasov):
[~mkhludnev], sorry for invalid JSON, my fault. An attempt to make it valid
didn't help as well
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
query : "{!parent tag=top filters=$child.fq which=scope:product
v=$childquery}",
filter : "{!tag=top}category:clothes",
params:{
childquery : "scope:sku",
child.fq : { "#color" : "color:black" }
}
}'{code}
I've tried to rework the request according to JSON DSL and just want to share
my results, because I'm not sure if they are supposed to work or not.
This one works fine
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"query": "scope:sku",
"filters" : [
"{!tag=color}color:black",
"{!tag=size}size:L"
],
"which": "scope:product"
}
}
}'
{code}
while this
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"query": "scope:sku",
"filters" : [
{ "#color" : "color:black" },
{ "#size" : "size:L" }
],
"which": "scope:product"
}
}
}'{code}
responds with
[jira] [Commented] (SOLR-9685) tag a query in JSON syntax
[
https://issues.apache.org/jira/browse/SOLR-9685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508720#comment-16508720
]
Dr Oleg Savrasov commented on SOLR-9685:
[~mkhludnev], sorry for invalid JSON, my fault. An attempt to make it valid
didn't help as well
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
query : "{!parent tag=top filters=$child.fq which=scope:product
v=$childquery}",
filter : "{!tag=top}category:clothes",
params:{
childquery : "scope:sku",
child.fq : { "#color" : "color:black" }
}
}'{code}
I've tried to rework the request according to JSON DSL and just want to share
my results, because I'm not sure if they are supposed to work or not.
This one works fine
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"query": "scope:sku",
"filters" : [
"{!tag=color}color:black",
"{!tag=size}size:L"
],
"which": "scope:product"
}
}
}'
{code}
while this
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"query": "scope:sku",
"filters" : [
{ "#color" : "color:black" },
{ "#size" : "size:L" }
],
"which": "scope:product"
}
}
}'{code}
responds with
{code:java}
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","org.apache.solr.search.SyntaxError"],
"msg":"org.apache.solr.search.SyntaxError: Missing end to unquoted value
starting at 6 str='{!tag=color'",
"code":400}}{code}
Absolutely the same results I've got with
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"which": "scope:product",
"query": {
"bool": {
"must":"scope:sku",
"filter":[
"{!tag=color}color:black",
"{!tag=size}size:L"
]
}
}
}
}
}'{code}
and
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"which": "scope:product",
"query": {
"bool": {
"must":"scope:sku",
"filter":[
{ "#color" : "color:black" },
{ "#size" : "size:L" }
]
}
}
}
}
}'{code}
Finally [~mkhludnev]'s helped me to find a working version, which doesn't look
obvious for me.
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
"query" : {
"parent": {
"which": "scope:product",
"query": {
"bool": {
"must":"scope:sku",
"filter":[
{ "#color" : {lucene: { "query":"color:black"}
}},
{ "#size" : {lucene: { "query":"size:L"} }}
]
}
}
}
}
}'{code}
Let me express my huge thanks to [~mkhludnev] for his incredible creativity and
support.
> tag a query in JSON syntax
> --
>
> Key: SOLR-9685
> URL: https://issues.apache.org/jira/browse/SOLR-9685
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, JSON Request API
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-9685.patch, SOLR-9685.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> There should be a way to tag a query/filter in JSON syntax.
> Perhaps these two forms could be equivalent:
> {code}
> "{!tag=COLOR}color:blue"
> { tagged : { COLOR : "color:blue" }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9685) tag a query in JSON syntax
[
https://issues.apache.org/jira/browse/SOLR-9685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508275#comment-16508275
]
Dr Oleg Savrasov commented on SOLR-9685:
Is it supposed to work with BJQ?
I'm trying to replace
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
query : "{!parent tag=top filters=$child.fq which=scope:product
v=$childquery}",
filter : "{!tag=top}category:clothes",
params:{
childquery : "scope:sku",
child.fq : "{!tag=color}color:black"
}
}'
{code}
with
{code:java}
curl http://localhost:8983/solr/collection1/query -d 'json={
query : "{!parent tag=top filters=$child.fq which=scope:product
v=$childquery}",
filter : "{!tag=top}category:clothes",
params:{
childquery : "scope:sku",
child.fq : "{ "#color" : "color:black" }"
}
}'
{code}
and have an exception
{code:java}
"msg":"org.apache.solr.search.SyntaxError: Cannot parse '{ ': Encountered
\"\" at line 1, column 2.\nWas expecting one of:\n\"TO\" ...\n
...\n ...\n",
"code":400}}
{code}
> tag a query in JSON syntax
> --
>
> Key: SOLR-9685
> URL: https://issues.apache.org/jira/browse/SOLR-9685
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, JSON Request API
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-9685.patch, SOLR-9685.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> There should be a way to tag a query/filter in JSON syntax.
> Perhaps these two forms could be equivalent:
> {code}
> "{!tag=COLOR}color:blue"
> { tagged : { COLOR : "color:blue" }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5211) updating parent as childless makes old children orphans
[ https://issues.apache.org/jira/browse/SOLR-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499465#comment-16499465 ] Dr Oleg Savrasov commented on SOLR-5211: The patch is reworked to support checking _root_ in schema. New logic is covered with tests. DirectUpdateHandler2 is updated to delete children together with parent. > updating parent as childless makes old children orphans > --- > > Key: SOLR-5211 > URL: https://issues.apache.org/jira/browse/SOLR-5211 > Project: Solr > Issue Type: Sub-task > Components: update >Affects Versions: 4.5, 6.0 >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-5211.patch, SOLR-5211.patch > > > if I have parent with children in the index, I can send update omitting > children. as a result old children become orphaned. > I suppose separate \_root_ fields makes much trouble. I propose to extend > notion of uniqueKey, and let it spans across blocks that makes updates > unambiguous. > WDYT? Do you like to see a test proves this issue? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5211) updating parent as childless makes old children orphans
[ https://issues.apache.org/jira/browse/SOLR-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5211: --- Attachment: SOLR-5211.patch > updating parent as childless makes old children orphans > --- > > Key: SOLR-5211 > URL: https://issues.apache.org/jira/browse/SOLR-5211 > Project: Solr > Issue Type: Sub-task > Components: update >Affects Versions: 4.5, 6.0 >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-5211.patch, SOLR-5211.patch > > > if I have parent with children in the index, I can send update omitting > children. as a result old children become orphaned. > I suppose separate \_root_ fields makes much trouble. I propose to extend > notion of uniqueKey, and let it spans across blocks that makes updates > unambiguous. > WDYT? Do you like to see a test proves this issue? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5211) updating parent as childless makes old children orphans
[ https://issues.apache.org/jira/browse/SOLR-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16492079#comment-16492079 ] Dr Oleg Savrasov commented on SOLR-5211: [~mkhludnev] I've tried to implement proposed approach, i.e. 'let's just always copy $uniqueKey to _root_, span it to all children and use _root_:get($uniqueKey) as a delete term'. See the patch attached > updating parent as childless makes old children orphans > --- > > Key: SOLR-5211 > URL: https://issues.apache.org/jira/browse/SOLR-5211 > Project: Solr > Issue Type: Sub-task > Components: update >Affects Versions: 4.5, 6.0 >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-5211.patch > > > if I have parent with children in the index, I can send update omitting > children. as a result old children become orphaned. > I suppose separate \_root_ fields makes much trouble. I propose to extend > notion of uniqueKey, and let it spans across blocks that makes updates > unambiguous. > WDYT? Do you like to see a test proves this issue? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5211) updating parent as childless makes old children orphans
[ https://issues.apache.org/jira/browse/SOLR-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5211: --- Attachment: SOLR-5211.patch > updating parent as childless makes old children orphans > --- > > Key: SOLR-5211 > URL: https://issues.apache.org/jira/browse/SOLR-5211 > Project: Solr > Issue Type: Sub-task > Components: update >Affects Versions: 4.5, 6.0 >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-5211.patch > > > if I have parent with children in the index, I can send update omitting > children. as a result old children become orphaned. > I suppose separate \_root_ fields makes much trouble. I propose to extend > notion of uniqueKey, and let it spans across blocks that makes updates > unambiguous. > WDYT? Do you like to see a test proves this issue? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8998) JSON Facet API child roll-ups
[
https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16461033#comment-16461033
]
Dr Oleg Savrasov commented on SOLR-8998:
I've tried to document JSON Facet API changes, please review
[^SOLR-8998-api-doc.patch]
> JSON Facet API child roll-ups
> -
>
> Key: SOLR-8998
> URL: https://issues.apache.org/jira/browse/SOLR-8998
> Project: Solr
> Issue Type: New Feature
> Components: Facet Module
>Reporter: Yonik Seeley
>Priority: Major
> Attachments: SOLR-8998-api-doc.patch, SOLR-8998-doc.patch,
> SOLR-8998.patch, SOLR-8998.patch, SOLR-8998.patch, SOLR_8998.patch,
> SOLR_8998.patch, SOLR_8998.patch
>
>
> The JSON Facet API currently has the ability to map between parents and
> children ( see http://yonik.com/solr-nested-objects/ )
> This issue is about adding a true rollup ability where parents would take on
> derived values from their children. The most important part (and the most
> difficult part) will be the external API.
> [~mkhludnev] says
> {quote}
> The bottom line is to introduce {{uniqueBlock(\_root_)}} aggregation, which
> is supposed to be faster than {{unique(\_root_)}} and purposed for blocked
> index only. For now it it supports singlevalue string fields, docValues
> usually make sense.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8998) JSON Facet API child roll-ups
[
https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-8998:
---
Attachment: SOLR-8998-api-doc.patch
> JSON Facet API child roll-ups
> -
>
> Key: SOLR-8998
> URL: https://issues.apache.org/jira/browse/SOLR-8998
> Project: Solr
> Issue Type: New Feature
> Components: Facet Module
>Reporter: Yonik Seeley
>Priority: Major
> Attachments: SOLR-8998-api-doc.patch, SOLR-8998-doc.patch,
> SOLR-8998.patch, SOLR-8998.patch, SOLR-8998.patch, SOLR_8998.patch,
> SOLR_8998.patch, SOLR_8998.patch
>
>
> The JSON Facet API currently has the ability to map between parents and
> children ( see http://yonik.com/solr-nested-objects/ )
> This issue is about adding a true rollup ability where parents would take on
> derived values from their children. The most important part (and the most
> difficult part) will be the external API.
> [~mkhludnev] says
> {quote}
> The bottom line is to introduce {{uniqueBlock(\_root_)}} aggregation, which
> is supposed to be faster than {{unique(\_root_)}} and purposed for blocked
> index only. For now it it supports singlevalue string fields, docValues
> usually make sense.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-8998) JSON Facet API child roll-ups
[
https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459719#comment-16459719
]
Dr Oleg Savrasov edited comment on SOLR-8998 at 5/1/18 2:33 PM:
I like the idea. Usage of proposed {{uniqueBlock(_root_)}} aggregation looks
very similar to {{unique(_root_)}} workaround. But the implementation avoids
accumulating {{FixedBitSet}} for each slot. The only thing I worry about is
found hack with {{limit:-1}}. I believe {{uniqueBlock}} should perform much
better with it, but I don't see any way to use it by default without modifying
main facet machinery. Shall we somehow document that it's recommended to use
{{uniqueBlock}} aggregation with {{limit:-1}} facet parameter?
was (Author: osavrasov):
I like the idea. Usage of proposed uniqueBlock(_root_) aggregation looks very
similar to unique(_root_) workaround. But the implementation avoids
accumulating FixedBitSet for each slot. The only thing I worry about is found
hack with limit:-1. I believe uniqueBlock should perform much better with it,
but I don't see any way to use it by default without modifying main facet
machinery. Shall we somehow document that it's recommended to use uniqueBlock
aggregation with limit:-1 facet parameter?
> JSON Facet API child roll-ups
> -
>
> Key: SOLR-8998
> URL: https://issues.apache.org/jira/browse/SOLR-8998
> Project: Solr
> Issue Type: New Feature
> Components: Facet Module
>Reporter: Yonik Seeley
>Priority: Major
> Attachments: SOLR-8998.patch, SOLR-8998.patch, SOLR-8998.patch,
> SOLR_8998.patch, SOLR_8998.patch, SOLR_8998.patch
>
>
> The JSON Facet API currently has the ability to map between parents and
> children ( see http://yonik.com/solr-nested-objects/ )
> This issue is about adding a true rollup ability where parents would take on
> derived values from their children. The most important part (and the most
> difficult part) will be the external API.
> [~mkhludnev] says
> {quote}
> The bottom line is to introduce {{uniqueBlock(\_root_)}} aggregation, which
> is supposed to be faster than {{unique(\_root_)}} and purposed for blocked
> index only. For now it it supports singlevalue string fields, docValues
> usually make sense.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8998) JSON Facet API child roll-ups
[
https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459719#comment-16459719
]
Dr Oleg Savrasov commented on SOLR-8998:
I like the idea. Usage of proposed uniqueBlock(_root_) aggregation looks very
similar to unique(_root_) workaround. But the implementation avoids
accumulating FixedBitSet for each slot. The only thing I worry about is found
hack with limit:-1. I believe uniqueBlock should perform much better with it,
but I don't see any way to use it by default without modifying main facet
machinery. Shall we somehow document that it's recommended to use uniqueBlock
aggregation with limit:-1 facet parameter?
> JSON Facet API child roll-ups
> -
>
> Key: SOLR-8998
> URL: https://issues.apache.org/jira/browse/SOLR-8998
> Project: Solr
> Issue Type: New Feature
> Components: Facet Module
>Reporter: Yonik Seeley
>Priority: Major
> Attachments: SOLR-8998.patch, SOLR-8998.patch, SOLR-8998.patch,
> SOLR_8998.patch, SOLR_8998.patch, SOLR_8998.patch
>
>
> The JSON Facet API currently has the ability to map between parents and
> children ( see http://yonik.com/solr-nested-objects/ )
> This issue is about adding a true rollup ability where parents would take on
> derived values from their children. The most important part (and the most
> difficult part) will be the external API.
> [~mkhludnev] says
> {quote}
> The bottom line is to introduce {{uniqueBlock(\_root_)}} aggregation, which
> is supposed to be faster than {{unique(\_root_)}} and purposed for blocked
> index only. For now it it supports singlevalue string fields, docValues
> usually make sense.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394192#comment-16394192
]
Dr Oleg Savrasov edited comment on SOLR-9510 at 3/10/18 2:40 PM:
-
Latest version looks great. I have a couple of minor comments though:
* I would recommend to adjust comments in
TestJsonFacetsWithNestedObjects#testDomainFilterExclusionsInFilters. I think
it's better to explain the idea of excluding 'top' tag.
* -Changes made in QueryParsing seem to be deprecated, I think it's better to
revert them-
was (Author: osavrasov):
Latest version looks great. I have a couple of minor comments though:
* I would recommend to adjust comments in
TestJsonFacetsWithNestedObjects#testDomainFilterExclusionsInFilters. I think
it's better to explain the idea of excluding 'top' tag.
* Changes made in QueryParsing seem to be deprecated, I think it's better to
revert them
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394192#comment-16394192
]
Dr Oleg Savrasov commented on SOLR-9510:
Latest version looks great. I have a couple of minor comments though:
* I would recommend to adjust comments in
TestJsonFacetsWithNestedObjects#testDomainFilterExclusionsInFilters. I think
it's better to explain the idea of excluding 'top' tag.
* Changes made in QueryParsing seem to be deprecated, I think it's better to
revert them
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16378911#comment-16378911
]
Dr Oleg Savrasov commented on SOLR-9510:
All the remaining *TODO* items are elaborated
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-9510:
---
Comment: was deleted
(was: All the remaining *TODO* items are elaborated)
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-9510:
---
Attachment: SOLR_9510.patch
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-9510:
---
Attachment: SOLR_9510.patch
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-9510:
---
Attachment: (was: SOLR_9510.patch)
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8998) JSON Facet API child roll-ups
[ https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-8998: --- Attachment: SOLR_8998.patch > JSON Facet API child roll-ups > - > > Key: SOLR-8998 > URL: https://issues.apache.org/jira/browse/SOLR-8998 > Project: Solr > Issue Type: New Feature > Components: Facet Module >Reporter: Yonik Seeley >Priority: Major > Attachments: SOLR_8998.patch, SOLR_8998.patch, SOLR_8998.patch > > > The JSON Facet API currently has the ability to map between parents and > children ( see http://yonik.com/solr-nested-objects/ ) > This issue is about adding a true rollup ability where parents would take on > derived values from their children. The most important part (and the most > difficult part) will be the external API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8998) JSON Facet API child roll-ups
[ https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16378898#comment-16378898 ] Dr Oleg Savrasov commented on SOLR-8998: Latest patch is adjusted to use MultiAcc.[^SOLR_8998.patch] > JSON Facet API child roll-ups > - > > Key: SOLR-8998 > URL: https://issues.apache.org/jira/browse/SOLR-8998 > Project: Solr > Issue Type: New Feature > Components: Facet Module >Reporter: Yonik Seeley >Priority: Major > Attachments: SOLR_8998.patch, SOLR_8998.patch, SOLR_8998.patch > > > The JSON Facet API currently has the ability to map between parents and > children ( see http://yonik.com/solr-nested-objects/ ) > This issue is about adding a true rollup ability where parents would take on > derived values from their children. The most important part (and the most > difficult part) will be the external API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376847#comment-16376847
]
Dr Oleg Savrasov commented on SOLR-9510:
All the remaining *TODO* items are elaborated
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-9510:
---
Attachment: (was: SOLR_9510.patch)
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-9510:
---
Attachment: SOLR_9510.patch
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-9510:
---
Attachment: SOLR_9510.patch
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch,
> SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-9510:
---
Attachment: SOLR_9510.patch
Arpan Gupta has noticed that proposed approach differs from usual filter
exclusions. The problem is that eliminating child filters in facet queries
doesn't extend docSet of matched parent objects, it extends the set of child
objects in initial parent objects domain only. To extend parent objects domain,
we probably need to support similar definition, i.e.
domain: { blockChildren : type_s:book, excludeTags:stars }
This makes the syntax a bit more complicated, since same exclusions should be
mentioned both in domain (to extend docSet of matched parent objects) and facet
query (to extend the set of matched child objects).
The patch is reworked to support domain filter exclusions, please see
TestJsonFacetsWithNestedObjects.testDomainFilterExclusions test as a reference
to the new feature.
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
> Attachments: SOLR_9510.patch, SOLR_9510.patch, SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10634) Move calculation of some aggregations to collection phase
[
https://issues.apache.org/jira/browse/SOLR-10634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009749#comment-16009749
]
Dr Oleg Savrasov commented on SOLR-10634:
-
I think that proposed approach should significantly speed up lots of real
business cases when aggregations are calculated. Also I hope that it will
provide possibility to review and simplify design of
https://issues.apache.org/jira/browse/SOLR-8998. So I would ask to vote for
this JIRA and merge the feature into SOLR codebase.
> Move calculation of some aggregations to collection phase
> -
>
> Key: SOLR-10634
> URL: https://issues.apache.org/jira/browse/SOLR-10634
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module
>Reporter: Yonik Seeley
> Attachments: SOLR-10634.patch
>
>
> From http://markmail.org/message/pwgnt7iqxkzcnckh
> {quote}
> The current code is more optimized for finding the top K buckets from
> a total of N.
> When one asks to return the top 10 buckets when there are potentially
> millions of buckets, it makes sense to defer calculating other metrics
> for those buckets until we know which ones they are. After we
> identify the top 10 buckets, we calculate the domain for that bucket
> and use that to calculate the remaining metrics.
> The current method is obviously much slower when one is requesting
> *all* buckets. We might as well just calculate all metrics in the
> first pass rather than trying to defer them.
> {quote}
> So we should move aggregations from the second pass to the first pass under
> the following conditions:
> - no limit (or a high limit compared to the number of potential buckets?)
> - no sub-facets (or anything else) that will need the domain calculated anyway
> - aggregation is not really memory intensive per-slot (i.e. moving some
> calculations from per-bucket in the second phase, to all-buckets-in-parallel
> in the first phase could be really bad for peak memory usage)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-9510:
---
Attachment: SOLR_9510.patch
I created the previous patch using IDEA tool. Normally it works, but this time
it looks like it has issues with patch format. Will you try this one, please?
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
> Attachments: SOLR_9510.patch, SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10520) BlockJoinFacetComponet duplicate counts when rows>0
[
https://issues.apache.org/jira/browse/SOLR-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-10520:
Attachment: SOLR_10520.patch
I think the component should handle shard responses only in case of
ShardRequest.PURPOSE_GET_TOP_IDS, like for example FacetComponent. Appropriate
fix is attached. The test is modified to randomly choose either 0 or 10 value
for "rows" parameter. Random value of "distrib.singlePass" is also verified.
The test may be extended to check other parameters that potentially affect
shards purpose flow.
> BlockJoinFacetComponet duplicate counts when rows>0
> ---
>
> Key: SOLR-10520
> URL: https://issues.apache.org/jira/browse/SOLR-10520
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mikhail Khludnev
> Attachments: SOLR_10520.patch, SOLR-10520-test.patch
>
>
> when we have rows>0 every shard receive:
> bq. ids=2,3,6,7,9&isShard=true&shards.purpose=64&distrib=false
> this is ShardRequest.PURPOSE_GET_TOP_IDS
> In this case {{child_facet_fields}} are merged once again unnecessary.
> failure is
> {code}
> java.lang.AssertionError: don't({3=[1], s=[6], white=[0, 3, 5, 6, 7, 8, 9,
> 10], xl=[8], don't=[0, 1, 2, 5, 6, 8, 10], 6=[0, 2, 7], xml=[3, 6, 8],
> xxxl=[0, 7, 9], l=[8, 10], m=[5, 9, 10], xxl=[5], grey=[9]} COLOR_s:[don't
> (14), white (16), grey (2)] SIZE_s:[m (6), xxl (2), 6 (6), s (2), xml (6),
> xxxl (6), 3 (2), l (4), xl (2)]) expected:<7> but was:<14>
> at __
> org.apache.solr.search.join.BlockJoinFacetDistribTest.testBJQFacetComponent(BlockJoinFacetDistribTest.java:148)
> {code}
> see test patch attached
> I suppose it's worth to exclude not only PURPOSE_GET_TOP_IDS but any other
> too, and make sure that _reduce_ is done only once on distributed search.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9510) child level facet exclusions
[
https://issues.apache.org/jira/browse/SOLR-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-9510:
---
Attachment: SOLR_9510.patch
Initial implementation (without bitset processing optimizations). Comments are
welcome.
> child level facet exclusions
>
>
> Key: SOLR-9510
> URL: https://issues.apache.org/jira/browse/SOLR-9510
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module, faceting, query parsers
>Reporter: Mikhail Khludnev
> Attachments: SOLR_9510.patch
>
>
> h2. Challenge
> * Since SOLR-5743 achieved block join child level facets with counts roll-up
> to parents, there is a demand for filter exclusions.
> h2. Context
> * Then, it's worth to consider JSON Facets as an engine for this
> functionality rather than support a separate component.
> * During a discussion in SOLR-8998 [a solution for block join with child
> level
> exclusion|https://issues.apache.org/jira/browse/SOLR-8998?focusedCommentId=15487095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15487095]
> has been found.
>
> h2. Proposal
> It's proposed to provide a bit of syntax sugar to make it user friendly,
> believe it or not.
> h2. List of improvements
> * introducing a local parameter {{filters}} for {{\{!parent}} query parser
> referring to _multiple_ filters queries via parameter name: {{\{!parent
> filters=$child.fq ..}..&child.fq=color:Red&child.fq=size:XL}}
> these _filters_ are intersected with a child query supplied as a subordinate
> clause.
> * introducing {{\{!filters params=$child.fq excludeTags=color
> v=$subq}&subq=text:word&child.fq={!tag=color}color:Red&child.fq=size:XL}} it
> intersects a subordinate clause (here it's {{subq}} param, and the trick is
> to refer to the same query from {{\{!parent}}}), with multiple filters
> supplied via parameter name {{params=$child.fq}}, it also supports
> {{excludeTags}}.
> h2. Notes
> Regarding the latter parser, the alternative approach might be to move into
> {{domain:\{..}}} instruction of json facet. From the implementation
> perspective, it's desired to optimize with bitset processing, however I
> suppose it's might be deferred until some initial level of maturity.
> h2. Example
> {code}
> q={!parent which=type_s:book filters=$child.fq
> v=$childquery}&childquery=comment_t:good&child.fq={!tag=author}author_s:yonik&child.fq={!tag=stars}stars_i:(5
> 3)&wt=json&indent=on&json.facet={
> comments_for_author:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=author v=$childquery}",
> "//note":"author filter is excluded",
> domain:{
> blockChildren:"type_s:book",
> "//":"applying filters here might be more promising"
>}, facet:{
>authors:{
> type:terms,
> field:author_s,
> facet: {
> in_books: "unique(_root_)"
> }
> }
>}
> } ,
> comments_for_stars:{
> type:query,
> q:"{!filters params=$child.fq excludeTags=stars v=$childquery}",
> "//note":"stars_i filter is excluded",
> domain:{
> blockChildren:"type_s:book"
>}, facet:{
>stars:{
> type:terms,
> field:stars_i,
> facet: {
> in_books: "unique(_root_)"
> }
> }
> }
> }
> }
> {code}
> Votes? Opinions?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8998) JSON Facet API child roll-ups
[ https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-8998: --- Attachment: SOLR_8998.patch Previous patch is covered with random unit-test, which is quite similar to BlockJoinFacetRandomTest. The test proves that the patch is functionally correct, i.e. JSON Facet API behavior with proposed syntax change is equivalent to BlockJoinFacetComponent. The test helped to find a small defect in initial implementation, that has been fixed. > JSON Facet API child roll-ups > - > > Key: SOLR-8998 > URL: https://issues.apache.org/jira/browse/SOLR-8998 > Project: Solr > Issue Type: New Feature > Components: Facet Module >Reporter: Yonik Seeley > Attachments: SOLR_8998.patch, SOLR_8998.patch > > > The JSON Facet API currently has the ability to map between parents and > children ( see http://yonik.com/solr-nested-objects/ ) > This issue is about adding a true rollup ability where parents would take on > derived values from their children. The most important part (and the most > difficult part) will be the external API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8998) JSON Facet API child roll-ups
[
https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-8998:
---
Attachment: SOLR_8998.patch
Unfortunately definition like
facet: {
productsCount: "unique(_root_)"
}
assumes iterating one by one through all the facet values and creating
appropriate docset for each of them.
In order to leverage algorithm implemented in BlockJoinFieldFacetAccumulator, I
propose to introduce a new keyword for JSON facet:
rollup : .
The example of request should look like:
curl http://localhost:8983/solr/collection1/query -d 'q={!parent
which=type_s:parent}type_s:child&rows=0&
json.facet={
filter_by_children: {
type:query,
q:"type_s:child",
domain: { blockChildren : "type_s:parent" },
facet:{
colors:{
type : terms,
field : COLOR_s,
rollup : "type_s:parent"
},
sizes:{
type : terms,
field : SIZE_s,
rollup : "type_s:parent"
The patch for proposed solution is attached. For the above request it returns
facets=={ count:1,
filter_by_children:{count:2,
colors:{buckets:[ {val:Blue, count:1, rollup:1},
{val:Red, count:1, rollup:1} ]},
sizes:{buckets:[ {val:XL, count:2, rollup:1} ]
See TestFacetRollup.
> JSON Facet API child roll-ups
> -
>
> Key: SOLR-8998
> URL: https://issues.apache.org/jira/browse/SOLR-8998
> Project: Solr
> Issue Type: New Feature
> Components: Facet Module
>Reporter: Yonik Seeley
> Attachments: SOLR_8998.patch
>
>
> The JSON Facet API currently has the ability to map between parents and
> children ( see http://yonik.com/solr-nested-objects/ )
> This issue is about adding a true rollup ability where parents would take on
> derived values from their children. The most important part (and the most
> difficult part) will be the external API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8644) ArrayIndexOutOfBoundsException in BlockJoinFieldFacetAccumulator
[
https://issues.apache.org/jira/browse/SOLR-8644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dr Oleg Savrasov updated SOLR-8644:
---
Attachment: SOLR-8644.patch
The issue is reproduced with Unit test. It looks like BlockJoinFacetComponent
conflicts with calculating exclusions defined for usual facet.
> ArrayIndexOutOfBoundsException in BlockJoinFieldFacetAccumulator
>
>
> Key: SOLR-8644
> URL: https://issues.apache.org/jira/browse/SOLR-8644
> Project: Solr
> Issue Type: Sub-task
> Components: faceting
>Reporter: Ilya Kasnacheev
>Priority: Minor
> Fix For: 5.5, master
>
> Attachments: SOLR-8644.patch
>
>
> Not sure I can provide any minimal example, but possibly it's easier to fix
> than describe.
> {code}
> http://localhost:8983/solr/core0/bjqfacet?q={!parent+which%3Dtype_s:parent+v%3D$cq}&fq={!term+f%3DBRAND_s+tag%3Drbrand}Nike&facet=true&cq=type_s:child+AND+SIZE_s:XL&facet.field={!ex%3Drbrand}BRAND_s&child.facet.field=disp_clr
> {code}
> {code}
> java.lang.ArrayIndexOutOfBoundsException: 0
> at
> org.apache.solr.search.join.BlockJoinFieldFacetAccumulator$SortedIntsAggDocIterator.nextDoc(BlockJoinFieldFacetAccumulator.java:117)
> at
> org.apache.solr.search.join.BlockJoinFieldFacetAccumulator.updateCountsWithMatchedBlock(BlockJoinFieldFacetAccumulator.java:143)
> at
> org.apache.solr.search.join.BlockJoinFacetCollector.countFacets(BlockJoinFacetCollector.java:119)
> at
> org.apache.solr.search.join.BlockJoinFacetCollector.incrementFacets(BlockJoinFacetCollector.java:106)
> at
> org.apache.solr.search.join.BlockJoinFacetCollector.collect(BlockJoinFacetCollector.java:87)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java:1161)
> {code}
> Again it only shows up in BlockJoinFacetComponent, not in
> BlockJoinDocSetFacetComponent
> The error is at bottom of result:
> {code}
> java.lang.ArrayIndexOutOfBoundsException
> 500
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5743: --- Attachment: SOLR-5743.patch Fix for the distributed test failure > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature > Components: faceting >Reporter: abipc >Assignee: Mikhail Khludnev > Labels: features > Fix For: 5.5 > > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5375) ToChildBlockJoinQuery becomes crazy on wrong subquery
[ https://issues.apache.org/jira/browse/LUCENE-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated LUCENE-5375: - Attachment: LUCENE-5375.patch Validation message improvement > ToChildBlockJoinQuery becomes crazy on wrong subquery > - > > Key: LUCENE-5375 > URL: https://issues.apache.org/jira/browse/LUCENE-5375 > Project: Lucene - Core > Issue Type: Bug > Components: modules/join >Affects Versions: 4.6 >Reporter: Dr Oleg Savrasov > Labels: patch > Fix For: 4.6.1, Trunk > > Attachments: LUCENE-5375.patch, SOLR-5553-1.patch, > SOLR-5553-insufficient_assertions.patch, SOLR-5553.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably > throws IllegalStateException. > (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html > 'The child documents must be orthogonal to the parent documents: the wrapped > child query must never return a parent document.'). However > ToChildBlockJoinQuery just goes crazy silently. I want to provide simple > patch for ToChildBlockJoinQuery with if-throw clause and test. > See > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5743: --- Attachment: SOLR-5743.patch Unit test is extended to cover single value flow > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (LUCENE-5375) ToChildBlockJoinQuery becomes crazy on wrong subquery
[ https://issues.apache.org/jira/browse/LUCENE-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated LUCENE-5375: - Comment: was deleted (was: OK, на выхах буду глядеть, спасибо ) > ToChildBlockJoinQuery becomes crazy on wrong subquery > - > > Key: LUCENE-5375 > URL: https://issues.apache.org/jira/browse/LUCENE-5375 > Project: Lucene - Core > Issue Type: Bug > Components: modules/join >Affects Versions: 4.6 >Reporter: Dr Oleg Savrasov > Labels: patch > Fix For: 4.6.1, Trunk > > Attachments: SOLR-5553-1.patch, > SOLR-5553-insufficient_assertions.patch, SOLR-5553.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably > throws IllegalStateException. > (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html > 'The child documents must be orthogonal to the parent documents: the wrapped > child query must never return a parent document.'). However > ToChildBlockJoinQuery just goes crazy silently. I want to provide simple > patch for ToChildBlockJoinQuery with if-throw clause and test. > See > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5375) ToChildBlockJoinQuery becomes crazy on wrong subquery
[ https://issues.apache.org/jira/browse/LUCENE-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008600#comment-15008600 ] Dr Oleg Savrasov commented on LUCENE-5375: -- OK, на выхах буду глядеть, спасибо > ToChildBlockJoinQuery becomes crazy on wrong subquery > - > > Key: LUCENE-5375 > URL: https://issues.apache.org/jira/browse/LUCENE-5375 > Project: Lucene - Core > Issue Type: Bug > Components: modules/join >Affects Versions: 4.6 >Reporter: Dr Oleg Savrasov > Labels: patch > Fix For: 4.6.1, Trunk > > Attachments: SOLR-5553-1.patch, > SOLR-5553-insufficient_assertions.patch, SOLR-5553.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably > throws IllegalStateException. > (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html > 'The child documents must be orthogonal to the parent documents: the wrapped > child query must never return a parent document.'). However > ToChildBlockJoinQuery just goes crazy silently. I want to provide simple > patch for ToChildBlockJoinQuery with if-throw clause and test. > See > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5743: --- Attachment: SOLR-5743.patch Proposed component has been reworked to utilize algorithm described here https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-reverse-nested-aggregation.html. As a result code became more elegant and faster in about 2 times in comparison with the previous version. > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5743: --- Attachment: SOLR-5743.patch Performance improvement patch, which is prepared for lucene_solr_5_2 branch. On my local test data it makes proposed component faster in about 25 times. Please notice that it's recommended to apply patch SOLR-7730 as well, since it yields significant performance benefits too. > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5743: --- Comment: was deleted (was: We call this kind of requests which mix and match fields from different related entities a "deep search". To handle such requests we need to create a composition of Boolean query which will provide linguistic matching and Block Join query which will allow to return top level document when match happened on nested document. This topic worth its own JIRA (or few of them). Here, we are focusing on faceting rather than matching. ) > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14564401#comment-14564401 ] Dr Oleg Savrasov commented on SOLR-5743: We call this kind of requests which mix and match fields from different related entities a "deep search". To handle such requests we need to create a composition of Boolean query which will provide linguistic matching and Block Join query which will allow to return top level document when match happened on nested document. This topic worth its own JIRA (or few of them). Here, we are focusing on faceting rather than matching. > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14564400#comment-14564400 ] Dr Oleg Savrasov commented on SOLR-5743: We call this kind of requests which mix and match fields from different related entities a "deep search". To handle such requests we need to create a composition of Boolean query which will provide linguistic matching and Block Join query which will allow to return top level document when match happened on nested document. This topic worth its own JIRA (or few of them). Here, we are focusing on faceting rather than matching. > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support
[
https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376129#comment-14376129
]
Dr Oleg Savrasov commented on SOLR-5743:
I don't think that we need to introduce one more new special child.facet.query
parameter here.
It looks like that it's possible to achieve the same result by specifying
appropriate ToParentQuery in facet.query parameter.
For example, facet.query={!parent which=type_s:parent}price:[1 TO 100].
But please notice that in this case facet.query result could count child
documents which are not matched by search query.
For example, there could be a parent document with two children. One child has
COLOR_s:Red and price:200, while another one COLOR_s:Blue and price:50.
If you request q={!parent which=type_s:parent}COLOR_s:Red
and facet.query={!parent which=type_s:parent}price:[1 TO 100], this document is
going go be counted.
Sometimes it's OK, but if you want to eliminate this effect, you need to add
child documents filter from q to facet.query.
The best way to do it is introducing new http parameter, say qq=COLOR_s:Red and
referencing it both from q and facet.query, i.e.
q={!parent which=type_s:parent v=$qq}&facet.query={!parent
which=type_s:parent}+price:[1 TO 100] +{!v=$qq}&qq=type_s:child&facet=true
> Faceting with BlockJoin support
> ---
>
> Key: SOLR-5743
> URL: https://issues.apache.org/jira/browse/SOLR-5743
> Project: Solr
> Issue Type: New Feature
>Reporter: abipc
> Labels: features
> Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch,
> SOLR-5743.patch, SOLR-5743.patch
>
>
> For a sample inventory(note - nested documents) like this -
>
> 10
> parent
> Nike
>
> 11
> Red
> XL
>
>
> 12
> Blue
> XL
>
>
> Faceting results must contain -
> Red(1)
> XL(1)
> Blue(1)
> for a "q=*" query.
> PS : The inventory example has been taken from this blog -
> http://blog.griddynamics.com/2013/09/solr-block-join-support.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14360055#comment-14360055 ] Dr Oleg Savrasov commented on SOLR-5743: Performance improvements are still under investigation at the moment. I don't have too much time these days so I cannot promise that I'll come up with some solution soon. But we keep working on it. > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5743: --- Comment: was deleted (was: Performance improvements are still under investigation at the moment. I don't have too much time these days so I cannot promise that I'll come up with some solution soon. But we keep working on it.) > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14360056#comment-14360056 ] Dr Oleg Savrasov commented on SOLR-5743: Performance improvements are still under investigation at the moment. I don't have too much time these days so I cannot promise that I'll come up with some solution soon. But we keep working on it. > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343189#comment-14343189 ] Dr Oleg Savrasov commented on SOLR-5743: Video from the Lucene Revolution talk is available here http://www.youtube.com/watch?v=Su5SHc_uJw8 > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343137#comment-14343137 ] Dr Oleg Savrasov commented on SOLR-5743: Please checkout the latest patch. Solr Cloud support has been implemented here. Please notice that in order to make it working you should make some configuration changes. 1. If you have /select SearchHandler definition, you should add blockJoinFacet as a last component here, like . blockJoinFacet 2. If you don't have /select SearchHandler definition, you should configure your custom BlockJoinFacet search handler with shards.qt parameter, which should reference on search handler name. For example: blockJoinFacetRH blockJoinFacet > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5743: --- Attachment: SOLR-5743.patch Solr Cloud support has been implemented > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312419#comment-14312419 ] Dr Oleg Savrasov commented on SOLR-5743: After investigating it, I've found that float and int types work fine for multivalued fields, i.e. they should be configured like Unit test in the patch is extended to cover int and float types. I'll try to find out if it's possible to make it working for multiValued="false". > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5743: --- Attachment: SOLR-5743.patch > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, > SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support
[
https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298423#comment-14298423
]
Dr Oleg Savrasov commented on SOLR-5743:
In order to utilize proposed component, you need to configure it in
solrconfig.xml and introduce some search handler which uses it, for example
blockJoinFacet
Please notice that only string docValues fields could be used for faceting, int
type can be covered later, so you need to update appropriate fields
configuration in schema.xml file, for example
Then after indexing some set of hierarchical documents like
10
parent
Nike
11
child
Red
XL
12
child
Blue
XL
you need to pass required ToParentBlockJoinQuery to the configured request
handler, for example
http://localhost:8983/solr/collection1/blockJoinFacetRH?q={!parent+which%3D%22type_s%3Aparent%22}type_s%3Achild&wt=json&indent=true&facet=true&child.facet.field=COLOR_s&child.facet.field=SIZE_s
and it yields you the desired result
{
"responseHeader":{
"status":0,
"QTime":1},
"response":{"numFound":1,"start":0,"docs":[
{
"id":"10",
"type_s":"parent",
"BRAND_s":"Nike",
"_version_":1491642108914696192}]
},
"facet_counts":{
"facet_queries":{},
"facet_fields":{},
"facet_dates":{},
"facet_ranges":{},
"facet_intervals":{},
"facet_fields":[
"COLOR_s",[
"Blue",1,
"Red",1],
"SIZE_s",[
"XL",1]]}}
Please take the latest patch, it contains fix related to just found caching
issue.
> Faceting with BlockJoin support
> ---
>
> Key: SOLR-5743
> URL: https://issues.apache.org/jira/browse/SOLR-5743
> Project: Solr
> Issue Type: New Feature
>Reporter: abipc
> Labels: features
> Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch
>
>
> For a sample inventory(note - nested documents) like this -
>
> 10
> parent
> Nike
>
> 11
> Red
> XL
>
>
> 12
> Blue
> XL
>
>
> Faceting results must contain -
> Red(1)
> XL(1)
> Blue(1)
> for a "q=*" query.
> PS : The inventory example has been taken from this blog -
> http://blog.griddynamics.com/2013/09/solr-block-join-support.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5743: --- Attachment: SOLR-5743.patch > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293587#comment-14293587 ] Dr Oleg Savrasov commented on SOLR-5743: I created the files by copying and modifying existing configurations. It looks like my IDE processed changes incorrectly. Sorry about that. Please find updated patch attached. Should you have any issues, please notice me. > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5743: --- Attachment: SOLR-5743.patch > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch, SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5743: --- Attachment: SOLR-5743.patch Initial implementation which meets functionality requirements. There is new BlockJoinFacetComponent which expects ToParentBlockJoinQuery in search request. Facets are calculated for fields defined by child.facet.field parameter. Only DocValues fields are supported. > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > Attachments: SOLR-5743.patch > > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support
[ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016391#comment-14016391 ] Dr Oleg Savrasov commented on SOLR-5743: I'm preparing Lucene Revolution talk http://lucenerevolution.uservoice.com/forums/254256-internals-track/suggestions/5995621-faceting-with-lucene-blockjoinquery which addresses the feature. Your votes would be much appreciated. > Faceting with BlockJoin support > --- > > Key: SOLR-5743 > URL: https://issues.apache.org/jira/browse/SOLR-5743 > Project: Solr > Issue Type: New Feature >Reporter: abipc > Labels: features > > For a sample inventory(note - nested documents) like this - > > 10 > parent > Nike > > 11 > Red > XL > > > 12 > Blue > XL > > > Faceting results must contain - > Red(1) > XL(1) > Blue(1) > for a "q=*" query. > PS : The inventory example has been taken from this blog - > http://blog.griddynamics.com/2013/09/solr-block-join-support.html -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5052) bitset codec for off heap filters
[ https://issues.apache.org/jira/browse/LUCENE-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated LUCENE-5052: - Attachment: LUCENE-5052-1.patch Only DOCS_ONLY index option is supported. IllegalArgumentException is thrown for anything else. > bitset codec for off heap filters > - > > Key: LUCENE-5052 > URL: https://issues.apache.org/jira/browse/LUCENE-5052 > Project: Lucene - Core > Issue Type: New Feature > Components: core/codecs >Reporter: Mikhail Khludnev > Labels: features > Fix For: 5.0 > > Attachments: LUCENE-5052-1.patch, LUCENE-5052.patch, bitsetcodec.zip, > bitsetcodec.zip > > > Colleagues, > When we filter we don’t care any of scoring factors i.e. norms, positions, > tf, but it should be fast. The obvious way to handle this is to decode > postings list and cache it in heap (CachingWrappingFilter, Solr’s DocSet). > Both of consuming a heap and decoding as well are expensive. > Let’s write a posting list as a bitset, if df is greater than segment's > maxdocs/8 (what about skiplists? and overall performance?). > Beside of the codec implementation, the trickiest part to me is to design API > for this. How we can let the app know that a term query don’t need to be > cached in heap, but can be held as an mmaped bitset? > WDYT? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (LUCENE-5052) bitset codec for off heap filters
[ https://issues.apache.org/jira/browse/LUCENE-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated LUCENE-5052: - Comment: was deleted (was: Only DOCS_ONLY index option is supported. IllegalArgumentException is thrown otherwise.) > bitset codec for off heap filters > - > > Key: LUCENE-5052 > URL: https://issues.apache.org/jira/browse/LUCENE-5052 > Project: Lucene - Core > Issue Type: New Feature > Components: core/codecs >Reporter: Mikhail Khludnev > Labels: features > Fix For: 5.0 > > Attachments: LUCENE-5052.patch, bitsetcodec.zip, bitsetcodec.zip > > > Colleagues, > When we filter we don’t care any of scoring factors i.e. norms, positions, > tf, but it should be fast. The obvious way to handle this is to decode > postings list and cache it in heap (CachingWrappingFilter, Solr’s DocSet). > Both of consuming a heap and decoding as well are expensive. > Let’s write a posting list as a bitset, if df is greater than segment's > maxdocs/8 (what about skiplists? and overall performance?). > Beside of the codec implementation, the trickiest part to me is to design API > for this. How we can let the app know that a term query don’t need to be > cached in heap, but can be held as an mmaped bitset? > WDYT? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5052) bitset codec for off heap filters
[ https://issues.apache.org/jira/browse/LUCENE-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated LUCENE-5052: - Attachment: (was: LUCENE-5052-1.patch) > bitset codec for off heap filters > - > > Key: LUCENE-5052 > URL: https://issues.apache.org/jira/browse/LUCENE-5052 > Project: Lucene - Core > Issue Type: New Feature > Components: core/codecs >Reporter: Mikhail Khludnev > Labels: features > Fix For: 5.0 > > Attachments: LUCENE-5052.patch, bitsetcodec.zip, bitsetcodec.zip > > > Colleagues, > When we filter we don’t care any of scoring factors i.e. norms, positions, > tf, but it should be fast. The obvious way to handle this is to decode > postings list and cache it in heap (CachingWrappingFilter, Solr’s DocSet). > Both of consuming a heap and decoding as well are expensive. > Let’s write a posting list as a bitset, if df is greater than segment's > maxdocs/8 (what about skiplists? and overall performance?). > Beside of the codec implementation, the trickiest part to me is to design API > for this. How we can let the app know that a term query don’t need to be > cached in heap, but can be held as an mmaped bitset? > WDYT? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5052) bitset codec for off heap filters
[ https://issues.apache.org/jira/browse/LUCENE-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated LUCENE-5052: - Attachment: LUCENE-5052-1.patch Only DOCS_ONLY index option is supported. IllegalArgumentException is thrown otherwise. > bitset codec for off heap filters > - > > Key: LUCENE-5052 > URL: https://issues.apache.org/jira/browse/LUCENE-5052 > Project: Lucene - Core > Issue Type: New Feature > Components: core/codecs >Reporter: Mikhail Khludnev > Labels: features > Fix For: 5.0 > > Attachments: LUCENE-5052-1.patch, LUCENE-5052.patch, bitsetcodec.zip, > bitsetcodec.zip > > > Colleagues, > When we filter we don’t care any of scoring factors i.e. norms, positions, > tf, but it should be fast. The obvious way to handle this is to decode > postings list and cache it in heap (CachingWrappingFilter, Solr’s DocSet). > Both of consuming a heap and decoding as well are expensive. > Let’s write a posting list as a bitset, if df is greater than segment's > maxdocs/8 (what about skiplists? and overall performance?). > Beside of the codec implementation, the trickiest part to me is to design API > for this. How we can let the app know that a term query don’t need to be > cached in heap, but can be held as an mmaped bitset? > WDYT? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5052) bitset codec for off heap filters
[ https://issues.apache.org/jira/browse/LUCENE-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated LUCENE-5052: - Attachment: bitsetcodec.zip Methods nextDoc() and advance() have been implemented using Long.numberOfTrailingZeros() approach taken from FixedBitSetIterator > bitset codec for off heap filters > - > > Key: LUCENE-5052 > URL: https://issues.apache.org/jira/browse/LUCENE-5052 > Project: Lucene - Core > Issue Type: New Feature > Components: core/codecs >Reporter: Mikhail Khludnev > Labels: features > Fix For: 5.0 > > Attachments: LUCENE-5052.patch, bitsetcodec.zip, bitsetcodec.zip > > > Colleagues, > When we filter we don’t care any of scoring factors i.e. norms, positions, > tf, but it should be fast. The obvious way to handle this is to decode > postings list and cache it in heap (CachingWrappingFilter, Solr’s DocSet). > Both of consuming a heap and decoding as well are expensive. > Let’s write a posting list as a bitset, if df is greater than segment's > maxdocs/8 (what about skiplists? and overall performance?). > Beside of the codec implementation, the trickiest part to me is to design API > for this. How we can let the app know that a term query don’t need to be > cached in heap, but can be held as an mmaped bitset? > WDYT? -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5375) ToChildBlockJoinQuery becomes crazy on wrong subquery
[ https://issues.apache.org/jira/browse/LUCENE-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13872006#comment-13872006 ] Dr Oleg Savrasov commented on LUCENE-5375: -- Many thanks! > ToChildBlockJoinQuery becomes crazy on wrong subquery > - > > Key: LUCENE-5375 > URL: https://issues.apache.org/jira/browse/LUCENE-5375 > Project: Lucene - Core > Issue Type: Bug > Components: modules/join >Affects Versions: 4.6 >Reporter: Dr Oleg Savrasov > Labels: patch > Fix For: 5.0, 4.6.1 > > Attachments: SOLR-5553-1.patch, > SOLR-5553-insufficient_assertions.patch, SOLR-5553.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably > throws IllegalStateException. > (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html > 'The child documents must be orthogonal to the parent documents: the wrapped > child query must never return a parent document.'). However > ToChildBlockJoinQuery just goes crazy silently. I want to provide simple > patch for ToChildBlockJoinQuery with if-throw clause and test. > See > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.1.5#6160) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5375) ToChildBlockJoinQuery becomes crazy on wrong subquery
[ https://issues.apache.org/jira/browse/LUCENE-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13871724#comment-13871724 ] Dr Oleg Savrasov commented on LUCENE-5375: -- Hi Michael, Have you had a chance to look at it? Is there any update? Thank you, Oleg > ToChildBlockJoinQuery becomes crazy on wrong subquery > - > > Key: LUCENE-5375 > URL: https://issues.apache.org/jira/browse/LUCENE-5375 > Project: Lucene - Core > Issue Type: Bug > Components: modules/join >Affects Versions: 4.6 >Reporter: Dr Oleg Savrasov > Labels: patch > Attachments: SOLR-5553-1.patch, > SOLR-5553-insufficient_assertions.patch, SOLR-5553.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably > throws IllegalStateException. > (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html > 'The child documents must be orthogonal to the parent documents: the wrapped > child query must never return a parent document.'). However > ToChildBlockJoinQuery just goes crazy silently. I want to provide simple > patch for ToChildBlockJoinQuery with if-throw clause and test. > See > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.1.5#6160) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5375) ToChildBlockJoinQuery becomes crazy on wrong subquery
[ https://issues.apache.org/jira/browse/LUCENE-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated LUCENE-5375: - Attachment: SOLR-5553-insufficient_assertions.patch Test that shows insufficient assertions in ToChildBlockJoinScorer. Expects AssertionError and fails randomly, for example with -Dtests.seed=E8A0C61499EE8851:C5E7CB6721742C4F > ToChildBlockJoinQuery becomes crazy on wrong subquery > - > > Key: LUCENE-5375 > URL: https://issues.apache.org/jira/browse/LUCENE-5375 > Project: Lucene - Core > Issue Type: Bug > Components: modules/join >Affects Versions: 4.6 >Reporter: Dr Oleg Savrasov > Labels: patch > Attachments: SOLR-5553-1.patch, > SOLR-5553-insufficient_assertions.patch, SOLR-5553.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably > throws IllegalStateException. > (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html > 'The child documents must be orthogonal to the parent documents: the wrapped > child query must never return a parent document.'). However > ToChildBlockJoinQuery just goes crazy silently. I want to provide simple > patch for ToChildBlockJoinQuery with if-throw clause and test. > See > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.1.5#6160) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5375) ToChildBlockJoinQuery becomes crazy on wrong subquery
[ https://issues.apache.org/jira/browse/LUCENE-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated LUCENE-5375: - Attachment: SOLR-5553-1.patch Reworked patch > ToChildBlockJoinQuery becomes crazy on wrong subquery > - > > Key: LUCENE-5375 > URL: https://issues.apache.org/jira/browse/LUCENE-5375 > Project: Lucene - Core > Issue Type: Bug > Components: modules/join >Affects Versions: 4.6 >Reporter: Dr Oleg Savrasov > Labels: patch > Attachments: SOLR-5553-1.patch, > SOLR-5553-insufficient_assertions.patch, SOLR-5553.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably > throws IllegalStateException. > (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html > 'The child documents must be orthogonal to the parent documents: the wrapped > child query must never return a parent document.'). However > ToChildBlockJoinQuery just goes crazy silently. I want to provide simple > patch for ToChildBlockJoinQuery with if-throw clause and test. > See > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.1.5#6160) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5375) ToChildBlockJoinQuery becomes crazy on wrong subquery
[ https://issues.apache.org/jira/browse/LUCENE-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13865200#comment-13865200 ] Dr Oleg Savrasov commented on LUCENE-5375: -- After some investigations I came to conclusion that existing assertions in ToChildBlockJoinScorer are not sufficient to guarantee child and parent documents orthogonality. In order to prove that, I've created a test which expects appropriate AssertionError, please see attached SOLR-5553-insufficient_assertions.patch. The test fails for example with -Dtests.seed=E8A0C61499EE8851:C5E7CB6721742C4F. I have not found any way for correct validation other than checking parentBits. It costs retrieving appropriate bit from FixedBitSet, but it seems not too expensive. The test is reworked in order to be Lucene-level and cover the cases when existing assertions are not sufficient. Please see attached SOLR-5553-1.patch. > ToChildBlockJoinQuery becomes crazy on wrong subquery > - > > Key: LUCENE-5375 > URL: https://issues.apache.org/jira/browse/LUCENE-5375 > Project: Lucene - Core > Issue Type: Bug > Components: modules/join >Affects Versions: 4.6 >Reporter: Dr Oleg Savrasov > Labels: patch > Attachments: SOLR-5553.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably > throws IllegalStateException. > (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html > 'The child documents must be orthogonal to the parent documents: the wrapped > child query must never return a parent document.'). However > ToChildBlockJoinQuery just goes crazy silently. I want to provide simple > patch for ToChildBlockJoinQuery with if-throw clause and test. > See > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.1.5#6160) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5375) ToChildBlockJoinQuery becomes crazy on wrong subquery
[ https://issues.apache.org/jira/browse/LUCENE-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855493#comment-13855493 ] Dr Oleg Savrasov commented on LUCENE-5375: -- Hi Michael, Many thanks for reviewing the patch. I agree that it's rather Lucene issue and should be covered by appropriate tests. I see your point about adding cost for correct usage. If I enable assertions without adding the validateParents, testAdvanceValidationForToChildBjq always fail, which means that there could be another way for query validation. Let me investigate it. Thank you, Dr Oleg Savrasov, Community Coordinator, Grid Dynamics Search team > ToChildBlockJoinQuery becomes crazy on wrong subquery > - > > Key: LUCENE-5375 > URL: https://issues.apache.org/jira/browse/LUCENE-5375 > Project: Lucene - Core > Issue Type: Bug > Components: modules/join >Affects Versions: 4.6 >Reporter: Dr Oleg Savrasov > Labels: patch > Attachments: SOLR-5553.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably > throws IllegalStateException. > (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html > 'The child documents must be orthogonal to the parent documents: the wrapped > child query must never return a parent document.'). However > ToChildBlockJoinQuery just goes crazy silently. I want to provide simple > patch for ToChildBlockJoinQuery with if-throw clause and test. > See > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.1.5#6160) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5553) ToChildBlockJoinQuery becomes crazy on wrong subquery
[ https://issues.apache.org/jira/browse/SOLR-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13852751#comment-13852751 ] Dr Oleg Savrasov commented on SOLR-5553: Proposed patch is provided, see attach > ToChildBlockJoinQuery becomes crazy on wrong subquery > - > > Key: SOLR-5553 > URL: https://issues.apache.org/jira/browse/SOLR-5553 > Project: Solr > Issue Type: Bug > Components: query parsers >Affects Versions: 4.6 >Reporter: Dr Oleg Savrasov > Labels: patch > Attachments: SOLR-5553.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably > throws IllegalStateException. > (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html > 'The child documents must be orthogonal to the parent documents: the wrapped > child query must never return a parent document.'). However > ToChildBlockJoinQuery just goes crazy silently. I want to provide simple > patch for ToChildBlockJoinQuery with if-throw clause and test. > See > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.1.4#6159) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5553) ToChildBlockJoinQuery becomes crazy on wrong subquery
[ https://issues.apache.org/jira/browse/SOLR-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dr Oleg Savrasov updated SOLR-5553: --- Attachment: SOLR-5553.patch > ToChildBlockJoinQuery becomes crazy on wrong subquery > - > > Key: SOLR-5553 > URL: https://issues.apache.org/jira/browse/SOLR-5553 > Project: Solr > Issue Type: Bug > Components: query parsers >Affects Versions: 4.6 >Reporter: Dr Oleg Savrasov > Labels: patch > Attachments: SOLR-5553.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably > throws IllegalStateException. > (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html > 'The child documents must be orthogonal to the parent documents: the wrapped > child query must never return a parent document.'). However > ToChildBlockJoinQuery just goes crazy silently. I want to provide simple > patch for ToChildBlockJoinQuery with if-throw clause and test. > See > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.1.4#6159) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5553) ToChildBlockJoinQuery becomes crazy on wrong subquery
Dr Oleg Savrasov created SOLR-5553: -- Summary: ToChildBlockJoinQuery becomes crazy on wrong subquery Key: SOLR-5553 URL: https://issues.apache.org/jira/browse/SOLR-5553 Project: Solr Issue Type: Bug Components: query parsers Affects Versions: 4.6 Reporter: Dr Oleg Savrasov If user supplies wrong subquery to ToParentBlockJoinQuery it reasonably throws IllegalStateException. (http://lucene.apache.org/core/4_0_0/join/org/apache/lucene/search/join/ToParentBlockJoinQuery.html 'The child documents must be orthogonal to the parent documents: the wrapped child query must never return a parent document.'). However ToChildBlockJoinQuery just goes crazy silently. I want to provide simple patch for ToChildBlockJoinQuery with if-throw clause and test. See http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201311.mbox/%3cf415ce3a-ebe5-4d15-adf1-c5ead32a1...@sheffield.ac.uk%3E -- This message was sent by Atlassian JIRA (v6.1.4#6159) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
