Re: Duplicate facets when the handler configuration specifies facet fields
Should be fixed now, thanks to Hoss Man. I'll wait for Solr 4.10.4
Thanks,
Marius
On Wed, Feb 4, 2015 at 3:00 PM, Marius Dumitru Florea
wrote:
> I started having the same issue (facets being listed twice) after
> upgrading from Solr 4.8.1 to Solr 4.10.3.
> So it looks like a regression to me. I commented on SOLR-6780 as the
> Fix Version/s field is not correct. The fix was not merged correctly
> on the 4.10 branch (before the 4.10.3 release).
>
> I hope we can get this issue fixed on the next 4.10.x version.
>
> Thanks,
> Marius
>
>
> On Mon, Nov 24, 2014 at 1:29 PM, Burke, Brian wrote:
>> Good to hear you can reproduce it and thank you for entering that issue into
>> JIRA. FWIW, here is the full list of parameters from echoParams=all from my
>> test:
>>
>>
>> "params":{
>> "mlt.minwl":"2",
>> "mm":"5",
>> "facet":"true",
>> "mlt.boost":"true",
>> "mlt.fl":"title,summary,body",
>> "facet.mincount":"1",
>> "mlt.mintf":"2",
>> "f.typedef.facet.limit":"15",
>> "mlt.qf":"title^5 summary^3 body^1",
>> "mlt.maxqt":"8",
>> "f.subtype.facet.limit":"15",
>> "mlt.mindf":"3",
>> "echoParams":"all",
>> "df":"id",
>> "mlt.count":"3",
>> "facet.field":"primaryId",
>> "facet":"true",
>> "echoParams":"all",
>> "indent":"true",
>> "q":"*:*",
>> "facet.limit":"10",
>> "facet.field":"primaryId",
>> "wt":"json",
>> "rows":"0"}},
>>
>> Also, our testing is for an upgrade from solr 4.4 to solr 4.10.2. I found
>> that in our current production 4.4 instance, the params shows the duplicate
>> (facet.field listed twice), but the actual results only has one set of
>> facets. Guessing there must have been some code on the output side that was
>> removing the duplicates in that version of solr.
>>
>> Thanks,
>> Brian
>>
>>
>> On Nov 24, 2014, at 6:04 AM, Alexandre Rafalovitch
>> mailto:arafa...@gmail.com>> wrote:
>>
>> I can reproduce it.
>>
>> I added your parameters to the default section of the config and then
>> run the following:
>> curl
>> "http://localhost:8983/solr/schemaless/select?q=*:*&rows=0&wt=json&indent=true&facet=true&facet.field=primaryId2&facet.limit=10&echoParams=all";
>>
>> I get:
>> --
>> "params":{
>> "f.typedef.facet.limit":"15",
>> "facet.field":"primaryId2",
>> "df":"_text",
>> "f.subtype.facet.limit":"15",
>> "echoParams":"all",
>> "facet.mincount":"1",
>> "rows":"0",
>> "facet":"true",
>> "q":"*:*",
>> "facet.limit":"10",
>> "facet.field":"primaryId2",
>> "indent":"true",
>> "echoParams":"all",
>> "rows":"0",
>> "wt":"json",
>> "facet":"true"}},
>> --
>>
>> This is against Solr 5 build, but I think that bug is there all the
>> way to Solr 4.1. I think I traced the source of the bug too (parameter
>> and default names are just joined together but with the first
>> (override) value both times as shown above). Usually makes no
>> difference to anything, but it looks like faceting component iterates
>> over the elements, not just gets them, so it gets bitten twice.
>>
>> I've created a JIRA for this issue:
>> https://issues.apache.org/jira/browse/SOLR-6780
>>
>> Regards,
>> Alex.
>> Personal: http://www.outerthoughts.com/ and @arafalov
>> Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
>> Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
>>
>>
>> On 21 November 2014 at 18:29, Alexandre Rafalovitch
>> wrote:
>> Could you add echoParams=all to the query and see what comes back?
>> Currently, you echo the params you sent, would be good to see what
>> they look like after they combine with defaults.
>>
>> Regards,
>> Alex.
>> Personal: http://www.outerthoughts.com/ and @arafalov
>> Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
>> Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
>>
>>
>> On 21 November 2014 18:04, Tom Zimmermann wrote:
>> Brian and I are working together to diagnose this issue so I can chime in
>> quickly here as well. These values are defined as part of the the defaults
>> section of the config.
>>
Re: Duplicate facets when the handler configuration specifies facet fields
I started having the same issue (facets being listed twice) after
upgrading from Solr 4.8.1 to Solr 4.10.3.
So it looks like a regression to me. I commented on SOLR-6780 as the
Fix Version/s field is not correct. The fix was not merged correctly
on the 4.10 branch (before the 4.10.3 release).
I hope we can get this issue fixed on the next 4.10.x version.
Thanks,
Marius
On Mon, Nov 24, 2014 at 1:29 PM, Burke, Brian wrote:
> Good to hear you can reproduce it and thank you for entering that issue into
> JIRA. FWIW, here is the full list of parameters from echoParams=all from my
> test:
>
>
> "params":{
> "mlt.minwl":"2",
> "mm":"5",
> "facet":"true",
> "mlt.boost":"true",
> "mlt.fl":"title,summary,body",
> "facet.mincount":"1",
> "mlt.mintf":"2",
> "f.typedef.facet.limit":"15",
> "mlt.qf":"title^5 summary^3 body^1",
> "mlt.maxqt":"8",
> "f.subtype.facet.limit":"15",
> "mlt.mindf":"3",
> "echoParams":"all",
> "df":"id",
> "mlt.count":"3",
> "facet.field":"primaryId",
> "facet":"true",
> "echoParams":"all",
> "indent":"true",
> "q":"*:*",
> "facet.limit":"10",
> "facet.field":"primaryId",
> "wt":"json",
> "rows":"0"}},
>
> Also, our testing is for an upgrade from solr 4.4 to solr 4.10.2. I found
> that in our current production 4.4 instance, the params shows the duplicate
> (facet.field listed twice), but the actual results only has one set of
> facets. Guessing there must have been some code on the output side that was
> removing the duplicates in that version of solr.
>
> Thanks,
> Brian
>
>
> On Nov 24, 2014, at 6:04 AM, Alexandre Rafalovitch
> mailto:arafa...@gmail.com>> wrote:
>
> I can reproduce it.
>
> I added your parameters to the default section of the config and then
> run the following:
> curl
> "http://localhost:8983/solr/schemaless/select?q=*:*&rows=0&wt=json&indent=true&facet=true&facet.field=primaryId2&facet.limit=10&echoParams=all";
>
> I get:
> --
> "params":{
> "f.typedef.facet.limit":"15",
> "facet.field":"primaryId2",
> "df":"_text",
> "f.subtype.facet.limit":"15",
> "echoParams":"all",
> "facet.mincount":"1",
> "rows":"0",
> "facet":"true",
> "q":"*:*",
> "facet.limit":"10",
> "facet.field":"primaryId2",
> "indent":"true",
> "echoParams":"all",
> "rows":"0",
> "wt":"json",
> "facet":"true"}},
> --
>
> This is against Solr 5 build, but I think that bug is there all the
> way to Solr 4.1. I think I traced the source of the bug too (parameter
> and default names are just joined together but with the first
> (override) value both times as shown above). Usually makes no
> difference to anything, but it looks like faceting component iterates
> over the elements, not just gets them, so it gets bitten twice.
>
> I've created a JIRA for this issue:
> https://issues.apache.org/jira/browse/SOLR-6780
>
> Regards,
> Alex.
> Personal: http://www.outerthoughts.com/ and @arafalov
> Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
> Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
>
>
> On 21 November 2014 at 18:29, Alexandre Rafalovitch
> wrote:
> Could you add echoParams=all to the query and see what comes back?
> Currently, you echo the params you sent, would be good to see what
> they look like after they combine with defaults.
>
> Regards,
> Alex.
> Personal: http://www.outerthoughts.com/ and @arafalov
> Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
> Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
>
>
> On 21 November 2014 18:04, Tom Zimmermann wrote:
> Brian and I are working together to diagnose this issue so I can chime in
> quickly here as well. These values are defined as part of the the defaults
> section of the config.
>
Re: Duplicate facets when the handler configuration specifies facet fields
Good to hear you can reproduce it and thank you for entering that issue into
JIRA. FWIW, here is the full list of parameters from echoParams=all from my
test:
"params":{
"mlt.minwl":"2",
"mm":"5",
"facet":"true",
"mlt.boost":"true",
"mlt.fl":"title,summary,body",
"facet.mincount":"1",
"mlt.mintf":"2",
"f.typedef.facet.limit":"15",
"mlt.qf":"title^5 summary^3 body^1",
"mlt.maxqt":"8",
"f.subtype.facet.limit":"15",
"mlt.mindf":"3",
"echoParams":"all",
"df":"id",
"mlt.count":"3",
"facet.field":"primaryId",
"facet":"true",
"echoParams":"all",
"indent":"true",
"q":"*:*",
"facet.limit":"10",
"facet.field":"primaryId",
"wt":"json",
"rows":"0"}},
Also, our testing is for an upgrade from solr 4.4 to solr 4.10.2. I found that
in our current production 4.4 instance, the params shows the duplicate
(facet.field listed twice), but the actual results only has one set of facets.
Guessing there must have been some code on the output side that was removing
the duplicates in that version of solr.
Thanks,
Brian
On Nov 24, 2014, at 6:04 AM, Alexandre Rafalovitch
mailto:arafa...@gmail.com>> wrote:
I can reproduce it.
I added your parameters to the default section of the config and then
run the following:
curl
"http://localhost:8983/solr/schemaless/select?q=*:*&rows=0&wt=json&indent=true&facet=true&facet.field=primaryId2&facet.limit=10&echoParams=all";
I get:
--
"params":{
"f.typedef.facet.limit":"15",
"facet.field":"primaryId2",
"df":"_text",
"f.subtype.facet.limit":"15",
"echoParams":"all",
"facet.mincount":"1",
"rows":"0",
"facet":"true",
"q":"*:*",
"facet.limit":"10",
"facet.field":"primaryId2",
"indent":"true",
"echoParams":"all",
"rows":"0",
"wt":"json",
"facet":"true"}},
--
This is against Solr 5 build, but I think that bug is there all the
way to Solr 4.1. I think I traced the source of the bug too (parameter
and default names are just joined together but with the first
(override) value both times as shown above). Usually makes no
difference to anything, but it looks like faceting component iterates
over the elements, not just gets them, so it gets bitten twice.
I've created a JIRA for this issue:
https://issues.apache.org/jira/browse/SOLR-6780
Regards,
Alex.
Personal: http://www.outerthoughts.com/ and @arafalov
Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
On 21 November 2014 at 18:29, Alexandre Rafalovitch wrote:
Could you add echoParams=all to the query and see what comes back?
Currently, you echo the params you sent, would be good to see what
they look like after they combine with defaults.
Regards,
Alex.
Personal: http://www.outerthoughts.com/ and @arafalov
Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
On 21 November 2014 18:04, Tom Zimmermann wrote:
Brian and I are working together to diagnose this issue so I can chime in
quickly here as well. These values are defined as part of the the defaults
section of the config.
Re: Duplicate facets when the handler configuration specifies facet fields
I can reproduce it.
I added your parameters to the default section of the config and then
run the following:
curl
"http://localhost:8983/solr/schemaless/select?q=*:*&rows=0&wt=json&indent=true&facet=true&facet.field=primaryId2&facet.limit=10&echoParams=all";
I get:
--
"params":{
"f.typedef.facet.limit":"15",
"facet.field":"primaryId2",
"df":"_text",
"f.subtype.facet.limit":"15",
"echoParams":"all",
"facet.mincount":"1",
"rows":"0",
"facet":"true",
"q":"*:*",
"facet.limit":"10",
"facet.field":"primaryId2",
"indent":"true",
"echoParams":"all",
"rows":"0",
"wt":"json",
"facet":"true"}},
--
This is against Solr 5 build, but I think that bug is there all the
way to Solr 4.1. I think I traced the source of the bug too (parameter
and default names are just joined together but with the first
(override) value both times as shown above). Usually makes no
difference to anything, but it looks like faceting component iterates
over the elements, not just gets them, so it gets bitten twice.
I've created a JIRA for this issue:
https://issues.apache.org/jira/browse/SOLR-6780
Regards,
Alex.
Personal: http://www.outerthoughts.com/ and @arafalov
Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
On 21 November 2014 at 18:29, Alexandre Rafalovitch wrote:
> Could you add echoParams=all to the query and see what comes back?
> Currently, you echo the params you sent, would be good to see what
> they look like after they combine with defaults.
>
> Regards,
> Alex.
> Personal: http://www.outerthoughts.com/ and @arafalov
> Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
> Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
>
>
> On 21 November 2014 18:04, Tom Zimmermann wrote:
>> Brian and I are working together to diagnose this issue so I can chime in
>> quickly here as well. These values are defined as part of the the defaults
>> section of the config.
Re: Duplicate facets when the handler configuration specifies facet fields
Could you add echoParams=all to the query and see what comes back? Currently, you echo the params you sent, would be good to see what they look like after they combine with defaults. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 21 November 2014 18:04, Tom Zimmermann wrote: > Brian and I are working together to diagnose this issue so I can chime in > quickly here as well. These values are defined as part of the the defaults > section of the config.
Re: Duplicate facets when the handler configuration specifies facet fields
Brian and I are working together to diagnose this issue so I can chime in quickly here as well. These values are defined as part of the the defaults section of the config.
Re: Duplicate facets when the handler configuration specifies facet fields
What is the section's type where you define these: defaults, appends,
or invariants? You did not mention that but it might be important.
Regards,
Alex.
Personal: http://www.outerthoughts.com/ and @arafalov
Solr resources and newsletter: http://www.solr-start.com/ and @solrstart
Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
On 21 November 2014 11:54, Burke, Brian wrote:
> We’ve run into an issue during local testing of the 4.10.2 release, where if
> the search handler config in solrconfig.xml has facet.fields defined, and a
> different field is on the request, then the requested facets are included
> twice in the response. If the list of default facet fields is removed from
> the handler config, no dupes are returned.
>
> Is this configuration in solrconfig.xml not supported, or incorrect in some
> way?
>
> Here’s an excerpt from the solrconfig.xml (for the default search handler):
>
> ….
> true
> typedef
> 15
> subtype
> 15
> 1
> …
>
> And when requesting facets on another field with this request:
>
>
> Produces this result, with the facets listed twice:
> http://ctestserver/solr/core12/select?q=*:*&rows=0&wt=json&indent=true&facet=true&facet.field=primaryId&facet.limit=10
>
>
> {
> "responseHeader":{
> "status":0,
> "QTime":19,
> "params":{
> "facet":"true",
> "indent":"true",
> "q":"*:*",
> "facet.limit":"10",
> "facet.field":"primaryId",
> "wt":"json",
> "rows":"0"}},
> "response":{"numFound":3365954,"start":0,"docs":[]
> },
> "facet_counts":{
> "facet_queries":{},
> "facet_fields":{
> "primaryId":[
> "c7e512f03d300310VgnVCM100d01c80aRCRD",81047,
> "2f4e68c8f24f3310VgnVCM100d01c80aRCRD",34239,
> "cfe3d14b917e1210VgnVCM100d01c80aRCRD",16719,
> "0c88cd4fb27e1210VgnVCM100d01c80aRCRD",14179,
> "c4e39262b57e1210VgnVCM100d01c80aRCRD",10983,
> "98a13bc7f1c96310VgnVCM100d01c80aRCRD",10504,
> "69cb2c4c9c6e1210VgnVCM100d01c80aRCRD",10232,
> "435dd953c2772210VgnVCM100d01c80aRCRD",9250,
> "23a89ce3f561f110VgnVCM109d2916acRCRD",9024,
> "f4e6fc42247e1210VgnVCM100d01c80aRCRD",8022],
> "primaryId":[
> "c7e512f03d300310VgnVCM100d01c80aRCRD",81047,
> "2f4e68c8f24f3310VgnVCM100d01c80aRCRD",34239,
> "cfe3d14b917e1210VgnVCM100d01c80aRCRD",16719,
> "0c88cd4fb27e1210VgnVCM100d01c80aRCRD",14179,
> "c4e39262b57e1210VgnVCM100d01c80aRCRD",10983,
> "98a13bc7f1c96310VgnVCM100d01c80aRCRD",10504,
> "69cb2c4c9c6e1210VgnVCM100d01c80aRCRD",10232,
> "435dd953c2772210VgnVCM100d01c80aRCRD",9250,
> "23a89ce3f561f110VgnVCM109d2916acRCRD",9024,
> "f4e6fc42247e1210VgnVCM100d01c80aRCRD",8022]},
> "facet_dates":{},
> "facet_ranges":{},
> "facet_intervals":{}}}
>
>
> Any suggestions on how to eliminate these duplicates would be most
> appreciated.
> Thanks,
> Brian
>
>
>
Re: Duplicate facets when the handler configuration specifies facet fields
Hmmm, I just tried this with 4.10.2 and can't reproduce this at all.
If I define facets in the /select handler then specify any
facet.field on the URL, the URL completely overrides the defaults
in /select. I even tried specifying the facet.field twice and still only
a single section was returned in the response.
So more details, please. Are you running in cloud mode for instance?
If not, a bigger snippet of your solrconfig would help (or the whole
thing). And anything from your Solr log that looks interesting, including
the exact query received by Solr (from the log)...
Best,
Erick
On Fri, Nov 21, 2014 at 8:54 AM, Burke, Brian wrote:
> We’ve run into an issue during local testing of the 4.10.2 release, where if
> the search handler config in solrconfig.xml has facet.fields defined, and a
> different field is on the request, then the requested facets are included
> twice in the response. If the list of default facet fields is removed from
> the handler config, no dupes are returned.
>
> Is this configuration in solrconfig.xml not supported, or incorrect in some
> way?
>
> Here’s an excerpt from the solrconfig.xml (for the default search handler):
>
> ….
> true
> typedef
> 15
> subtype
> 15
> 1
> …
>
> And when requesting facets on another field with this request:
>
>
> Produces this result, with the facets listed twice:
> http://ctestserver/solr/core12/select?q=*:*&rows=0&wt=json&indent=true&facet=true&facet.field=primaryId&facet.limit=10
>
>
> {
> "responseHeader":{
> "status":0,
> "QTime":19,
> "params":{
> "facet":"true",
> "indent":"true",
> "q":"*:*",
> "facet.limit":"10",
> "facet.field":"primaryId",
> "wt":"json",
> "rows":"0"}},
> "response":{"numFound":3365954,"start":0,"docs":[]
> },
> "facet_counts":{
> "facet_queries":{},
> "facet_fields":{
> "primaryId":[
> "c7e512f03d300310VgnVCM100d01c80aRCRD",81047,
> "2f4e68c8f24f3310VgnVCM100d01c80aRCRD",34239,
> "cfe3d14b917e1210VgnVCM100d01c80aRCRD",16719,
> "0c88cd4fb27e1210VgnVCM100d01c80aRCRD",14179,
> "c4e39262b57e1210VgnVCM100d01c80aRCRD",10983,
> "98a13bc7f1c96310VgnVCM100d01c80aRCRD",10504,
> "69cb2c4c9c6e1210VgnVCM100d01c80aRCRD",10232,
> "435dd953c2772210VgnVCM100d01c80aRCRD",9250,
> "23a89ce3f561f110VgnVCM109d2916acRCRD",9024,
> "f4e6fc42247e1210VgnVCM100d01c80aRCRD",8022],
> "primaryId":[
> "c7e512f03d300310VgnVCM100d01c80aRCRD",81047,
> "2f4e68c8f24f3310VgnVCM100d01c80aRCRD",34239,
> "cfe3d14b917e1210VgnVCM100d01c80aRCRD",16719,
> "0c88cd4fb27e1210VgnVCM100d01c80aRCRD",14179,
> "c4e39262b57e1210VgnVCM100d01c80aRCRD",10983,
> "98a13bc7f1c96310VgnVCM100d01c80aRCRD",10504,
> "69cb2c4c9c6e1210VgnVCM100d01c80aRCRD",10232,
> "435dd953c2772210VgnVCM100d01c80aRCRD",9250,
> "23a89ce3f561f110VgnVCM109d2916acRCRD",9024,
> "f4e6fc42247e1210VgnVCM100d01c80aRCRD",8022]},
> "facet_dates":{},
> "facet_ranges":{},
> "facet_intervals":{}}}
>
>
> Any suggestions on how to eliminate these duplicates would be most
> appreciated.
> Thanks,
> Brian
>
>
>
Duplicate facets when the handler configuration specifies facet fields
We’ve run into an issue during local testing of the 4.10.2 release, where if
the search handler config in solrconfig.xml has facet.fields defined, and a
different field is on the request, then the requested facets are included twice
in the response. If the list of default facet fields is removed from the
handler config, no dupes are returned.
Is this configuration in solrconfig.xml not supported, or incorrect in some way?
Here’s an excerpt from the solrconfig.xml (for the default search handler):
….
true
typedef
15
subtype
15
1
…
And when requesting facets on another field with this request:
Produces this result, with the facets listed twice:
http://ctestserver/solr/core12/select?q=*:*&rows=0&wt=json&indent=true&facet=true&facet.field=primaryId&facet.limit=10
{
"responseHeader":{
"status":0,
"QTime":19,
"params":{
"facet":"true",
"indent":"true",
"q":"*:*",
"facet.limit":"10",
"facet.field":"primaryId",
"wt":"json",
"rows":"0"}},
"response":{"numFound":3365954,"start":0,"docs":[]
},
"facet_counts":{
"facet_queries":{},
"facet_fields":{
"primaryId":[
"c7e512f03d300310VgnVCM100d01c80aRCRD",81047,
"2f4e68c8f24f3310VgnVCM100d01c80aRCRD",34239,
"cfe3d14b917e1210VgnVCM100d01c80aRCRD",16719,
"0c88cd4fb27e1210VgnVCM100d01c80aRCRD",14179,
"c4e39262b57e1210VgnVCM100d01c80aRCRD",10983,
"98a13bc7f1c96310VgnVCM100d01c80aRCRD",10504,
"69cb2c4c9c6e1210VgnVCM100d01c80aRCRD",10232,
"435dd953c2772210VgnVCM100d01c80aRCRD",9250,
"23a89ce3f561f110VgnVCM109d2916acRCRD",9024,
"f4e6fc42247e1210VgnVCM100d01c80aRCRD",8022],
"primaryId":[
"c7e512f03d300310VgnVCM100d01c80aRCRD",81047,
"2f4e68c8f24f3310VgnVCM100d01c80aRCRD",34239,
"cfe3d14b917e1210VgnVCM100d01c80aRCRD",16719,
"0c88cd4fb27e1210VgnVCM100d01c80aRCRD",14179,
"c4e39262b57e1210VgnVCM100d01c80aRCRD",10983,
"98a13bc7f1c96310VgnVCM100d01c80aRCRD",10504,
"69cb2c4c9c6e1210VgnVCM100d01c80aRCRD",10232,
"435dd953c2772210VgnVCM100d01c80aRCRD",9250,
"23a89ce3f561f110VgnVCM109d2916acRCRD",9024,
"f4e6fc42247e1210VgnVCM100d01c80aRCRD",8022]},
"facet_dates":{},
"facet_ranges":{},
"facet_intervals":{}}}
Any suggestions on how to eliminate these duplicates would be most appreciated.
Thanks,
Brian
