[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16838607#comment-16838607
]
Jan Høydahl commented on SOLR-13126:
There will not be any further releases for 7.3 or 7.5. We’ll soon release 7.7.2
with this fix and you should consider upgrading to that version which is
backward compatible with all 7.x versions.
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.3, 7.4, 7.6, 7.7, 7.5.0, 7.7.1
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 7.7.2, 8.0
>
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711",
> // CORRECT, both "Netzteil" and "Sony" are included in the name
> "score":6.0},
> {
> "id":"someProducts/Online/taxTestingP
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16838310#comment-16838310
]
Thomas Zillinger commented on SOLR-13126:
-
[~janhoy] could this also be backported to Solr 7.5?
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.3, 7.4, 7.6, 7.7, 7.5.0, 7.7.1
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 7.7.2, 8.0
>
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711",
> // CORRECT, both "Netzteil" and "Sony" are included in the name
> "score":6.0},
> {
> "id":"someProducts/Online/taxTestingProductThree",
> "name_text_de":"Steuertestprodukt Zwei",
> "code_string":"taxTestingProductThree",
> // CORRECT
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16825983#comment-16825983
]
ASF subversion and git services commented on SOLR-13126:
Commit 53d48e21b499c321c4ebd2dc55b24565a72f6e0c in lucene-solr's branch
refs/heads/branch_7_7 from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=53d48e2 ]
SOLR-13126: Correctly combine multiplicative query boosts
(Cherry pick 03945a9f5932ee78bbaa46becb551f348bca509e and backport, fix CHANGES)
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.3, 7.4, 7.6, 7.7, 7.5.0, 7.7.1
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.0
>
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711"
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16825954#comment-16825954
]
Jan Høydahl commented on SOLR-13126:
I plan to back-port this to 7.7.2 (branch_7_7).
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.0
>
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711",
> // CORRECT, both "Netzteil" and "Sony" are included in the name
> "score":6.0},
> {
> "id":"someProducts/Online/taxTestingProductThree",
> "name_text_de":"Steuertestprodukt Zwei",
> "code_string":"taxTestingProductThree",
> // CORRECT, neither "Netzteil" nor "Sony" are included in t
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786733#comment-16786733
]
Andrzej Wislowski commented on SOLR-13126:
--
[~romseygeek] I have created: https://issues.apache.org/jira/browse/SOLR-13303
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.0
>
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711",
> // CORRECT, both "Netzteil" and "Sony" are included in the name
> "score":6.0},
> {
> "id":"someProducts/Online/taxTestingProductThree",
> "name_text_de":"Steuertestprodukt Zwei",
> "code_string":"taxTestingProductThree",
> // CORRECT, neit
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786635#comment-16786635
]
Alan Woodward commented on SOLR-13126:
--
> I wonder whether this fix (return true) should be added also on
>WrappedLongValuesSource
It should - can you open a separate issue for it? I don't think that's
actually used anywhere in Solr yet, but it will be needed in future.
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.0
>
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711",
> // CORRECT, both "Netzteil" and "Sony" are included in the name
> "score":6.0},
> {
> "id":"someProducts/Online/taxT
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786608#comment-16786608
]
Mikhail Khludnev commented on SOLR-13126:
-
ccing [~romseygeek]. WDYT about {{WrappedLongValuesSource}} ?
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.0
>
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711",
> // CORRECT, both "Netzteil" and "Sony" are included in the name
> "score":6.0},
> {
> "id":"someProducts/Online/taxTestingProductThree",
> "name_text_de":"Steuertestprodukt Zwei",
> "code_string":"taxTestingProductThree",
> // CORRECT, neither "Netzteil" nor
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785624#comment-16785624
]
Andrzej Wislowski commented on SOLR-13126:
--
I wonder whether this fix (return true) should be added also on
WrappedLongValuesSource
https://github.com/apache/lucene-solr/blob/5f91670a496275167d7699c24ee7df61d7e2f7e6/lucene/queries/src/java/org/apache/lucene/queries/function/ValueSource.java#L141
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.0
>
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711",
> // CORRECT, both "Netzteil" and "Sony" are included in the name
> "score":6.0},
> {
> "i
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16770063#comment-16770063
]
Lucene/Solr QA commented on SOLR-13126:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} SOLR-13126 does not apply to master. Rebase required? Wrong
Branch? See
https://wiki.apache.org/solr/HowToContribute#Creating_the_patch_file for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13126 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12958842/SOLR-13126.patch |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/304/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.0
>
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769382#comment-16769382
]
Tobias Ibounig commented on SOLR-13126:
---
Thanks [~mkhludnev] and [~romseygeek]!
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 8.0
>
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711",
> // CORRECT, both "Netzteil" and "Sony" are included in the name
> "score":6.0},
> {
> "id":"someProducts/Online/taxTestingProductThree",
> "name_text_de":"Steuertestprodukt Zwei",
> "code_string":"taxTestingProductThree",
> // CORRECT, neither "Netzteil" nor "Sony" are included in the
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769217#comment-16769217
]
ASF subversion and git services commented on SOLR-13126:
Commit 10acaf02f82f4fc9c08884a7d249658ca6a590c1 in lucene-solr's branch
refs/heads/master from Alan Woodward
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=10acaf0 ]
SOLR-13126: Correctly combine multiplicative query boosts
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711",
> // CORRECT, both "Netzteil" and "Sony" are included in the name
> "score":6.0},
> {
> "id":"someProducts/O
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769215#comment-16769215
]
ASF subversion and git services commented on SOLR-13126:
Commit 03945a9f5932ee78bbaa46becb551f348bca509e in lucene-solr's branch
refs/heads/branch_8_0 from Alan Woodward
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=03945a9 ]
SOLR-13126: Correctly combine multiplicative query boosts
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711",
> // CORRECT, both "Netzteil" and "Sony" are included in the name
> "score":6.0},
> {
> "id":"someProduc
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769216#comment-16769216
]
ASF subversion and git services commented on SOLR-13126:
Commit 249dfdae0179bd9b0df543b98f5bf02baa464afc in lucene-solr's branch
refs/heads/branch_8x from Alan Woodward
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=249dfda ]
SOLR-13126: Correctly combine multiplicative query boosts
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Alan Woodward
>Priority: Major
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> SOLR-13126.patch, debugQuery.json, image-2019-02-13-16-17-56-272.png,
> screenshot-1.png, solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_string":"test711",
> // CORRECT, both "Netzteil" and "Sony" are included in the name
> "score":6.0},
> {
> "id":"someProduct
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769109#comment-16769109
]
Alan Woodward commented on SOLR-13126:
--
I think the problem here is a mismatch between how ValueSource and
DoubleValuesSource access their underlying data. DVS exposes an iterator and
expects to jump over missing documents, while VS has an exists() method but
still returns a value even if exists() is false. The wrapper code to convert
VS to DVS checks whether or not the value exists, but instead it should always
return `true`.
I'll work on a patch and include the above tests.
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> debugQuery.json, image-2019-02-13-16-17-56-272.png, screenshot-1.png,
> solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0))
> {quote}
> When taking a look at the search result, among other the following products
> are included (see the JSON comments for an analysis of each result):
> {code:javascript}
> {
> "id":"someProducts/Online/test711",
> "name_text_de":"Original Sony Vaio Netzteil",
> "code_s
[jira] [Commented] (SOLR-13126) Multiplicative boost of isn't applied when one of the summed or multiplied queries doesn't match
[
https://issues.apache.org/jira/browse/SOLR-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768692#comment-16768692
]
Mikhail Khludnev commented on SOLR-13126:
-
I've done one more review.
# {{FunctionScoreQuery}} code does exactly what's expected in {{boostByValue}}
javadoc
bq. If the source has no value for a particular document, the score for that
document is preserved as-is.
# there's no assert for that case at
{{TestFunctionScoreQuery.testScoreModifyingSource()}}
# as I understand all Solr's multiplicative boosts expect boost score is
applied regardless it has values or not. (Don't ask me how it boosts when value
is absent, it does and people relies on it)
So, the proposal is
# keep {{FunctionScoreQuery.MultiplicativeBoostValuesSource}} as is.
# introduce _some static method_ like
{{FunctionScoreQuery.boostByValueRegardlessExistence(Query,
DoubleValuesSource)}}. Better names are welcome.
# move all Solr's boost on it
# don't try to catch 8.0 respin
> Multiplicative boost of isn't applied when one of the summed or multiplied
> queries doesn't match
> -
>
> Key: SOLR-13126
> URL: https://issues.apache.org/jira/browse/SOLR-13126
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search
>Affects Versions: 7.5.0
> Environment: Reproduced with macOS 10.14.1, a quick test with Windows
> 10 showed the same result.
>Reporter: Thomas Aglassinger
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments:
> 0001-use-deprecated-classes-to-fix-regression-introduced-.patch,
> 0002-SOLR-13126-Added-test-case.patch, 2019-02-14_1715.png, SOLR-13126.patch,
> debugQuery.json, image-2019-02-13-16-17-56-272.png, screenshot-1.png,
> solr_match_neither_nextteil_nor_sony.json,
> solr_match_neither_nextteil_nor_sony.txt, solr_match_netzteil_and_sony.json,
> solr_match_netzteil_and_sony.txt, solr_match_netzteil_only.json,
> solr_match_netzteil_only.txt
>
>
> Under certain circumstances search results from queries with multiple
> multiplicative boosts using the Solr functions {{product()}} and {{query()}}
> result in a score that is inconsistent with the one from the debugQuery
> information. Also only the debug score is correct while the actual search
> results show a wrong score.
> This seems somewhat similar to the behaviour described in
> https://issues.apache.org/jira/browse/LUCENE-7132, though this issue has been
> resolved a while ago.
> A little background: we are using Solr as a search platform for the
> e-commerce framework SAP Hybris. There the shop administrator can create
> multiplicative boost rules (see below for an example) where a value like 2.0
> means that an item gets boosted to 200%. This works fine in the demo shop
> distributed by SAP but breaks in our shop. We encountered the issue when
> Upgrading from Solr 7.2.1 / Hybris 6.7 to Solr 7.5 / Hybris 18.8.3 (which
> would have been named Hybris 6.8 but the version naming schema changed).
> We reduced the Solr query generated by Hybris to the relevant parts and could
> reproduce the issue in the Solr admin without any Hybris connection.
> I attached the JSON result of a test query but here's a description of the
> parts that seemed most relevant to me.
> The {{responseHeader.params}} reads (slightly rearranged):
> {code:java}
> "q":"{!boost b=$ymb}(+{!lucene v=$yq})",
> "ymb":"product(query({!v=\"name_text_de\\:Netzteil\\^=2.0\"},1),query({!v=\"name_text_de\\:Sony\\^=3.0\"},1))",
> "yq":"*:*",
> "sort":"score desc",
> "debugQuery":"true",
> // Added to keep the output small but probably unrelated to the actual issue
> "fl":"score,id,code_string,name_text_de",
> "fq":"catalogId:\"someProducts\"",
> "rows":"10",
> {code}
> This example boosts the German product name (field {{name_text_de}}) in case
> in contains certain terms:
> * "Netzteil" (power supply) is boosted to 200%
> * "Sony" is boosted to 300%
> Consequently a product containing both terms should be boosted to 600%.
> Also the query function has the value 1 specified as default in case the name
> does not contain the respective term resulting in a pseudo boost that
> preserves the score.
> According to the debug information the parser used is the LuceneQParser,
> which translates this to the following parsed query:
> {quote}FunctionScoreQuery(FunctionScoreQuery(+*:*, scored by
> boost(product(query((ConstantScore(name_text_de:netzteil))^2.0,def=1.0),query((ConstantScore(name_text_de:sony))^3.0,def=1.0)
> {quote}
> And the translated boost is:
> {quote}org.apache.lucene.queries.function.valuesource.ProductFloatFunction:product(query((ConstantScore(name_text_de:netzteil))^2.0,
