[
https://issues.apache.org/jira/browse/SOLR-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774195#comment-17774195
]
rajanimaski edited comment on SOLR-17018 at 11/30/23 5:02 PM:
--------------------------------------------------------------
ok figured it is lengthy queries "q" having many query terms. LTR feature
tries to compute tfidf of such a query and reports "The request took too long
to iterate over terms" in log file and on heap, the query parser tries to parse
the query fills it up causing full heap use and following by oom.
{noformat}
org.apache.solr.search.ReRankCollector @ 0x699161e600.00 MB70.81
MB1.16%org.apache.lucene.search.BooleanQuery @ 0x6991798000.00 MB0.03
MB0.00%java.lang.ThreadLocal$ThreadLocalMap @ 0x69440b3b00.00 MB0.03
MB0.00%org.apache.lucene.search.BooleanQuery @ 0x6b50f73f00.00 MB0.03
MB0.00%org.apache.lucene.queries.function.FunctionScoreQuery @ 0x699167f180.00
MB0.02 MB0.00%com.shutterstock.solr.parser.expression.Expression @
0x699161c200.00 MB0.01 MB0.00%org.apache.solr.response.SolrQueryResponse @
0x6991620780.00 MB0.00 MB0.00%java.util.ArrayList @ 0x6991882b00.00 MB0.00
MB0.00%org.apache.solr.handler.component.ResponseBuilder @ 0x699161fa80.00
MB0.00 MB0.00%java.lang.String @ 0x6b50fe8d0 {!edismax qf=description.en
pf2=description.en}((bang) OR (blast) OR (blow AND up) OR (breakout) OR (burst)
OR (explosion) OR (gale) OR (outbreak) OR (outburst) OR (esplosione)) AND
((dust) OR (filing) OR (powder) OR (polvere)) AND ((blanche) OR (white)
OR...0.00 MB0.00 MB0.00%java.lang.String @ 0x6b50fe3b0 ((bang) OR (blast) OR
(blow AND up) OR (breakout) OR (burst) OR (explosion) OR (gale) OR (outbreak)
OR (outburst) OR (esplosione)) AND ((dust) OR (filing) OR (powder) OR
(polvere)) AND ((blanche) OR (white) OR (white AND woman) OR (bianca)) AND
((confine) ...0.00 MB0.00 MB0.00%java.util.ArrayList @ 0x699161d780.00 MB0.00
MB0.00%org.apache.solr.search.ExtendedDismaxQParser$ExtendedDismaxConfiguration
@ 0x699161ca00.00 MB0.00 MB0.00%{noformat}
example query and feature defintion "\{!edismax qf=description.en
pf2=description.en}"
{noformat}
q=((bang) OR (blast) OR (blow AND up) OR (breakout) OR (burst) OR (explosion)
OR (gale) OR (outbreak) OR (outburst) OR (esplosione)) AND ((dust) OR (filing)
OR (powder) OR (polvere)) AND ((blanche) OR (white) OR (white AND woman) OR
(bianca)) AND ((confine) OR (cut AND off) OR (insular) OR (insulate) OR
(insulated) OR (insulating) OR (insulation) OR (insulator) OR (insulators) OR
(island) OR (islander) OR (islands) OR (isle) OR (isolable) OR (isolatable) OR
(isolate) OR (isolated) OR (isolates) OR (isolation) OR (isolations) OR
(isolete) OR (lonely) OR (remote) OR (seal) {noformat}
Thinking this is bug as it causes stop the world. This literally takes over
full heap, causes high gc activity (filling up tenured space), high cpu usage.
was (Author: rajanimaski):
ok figured it is lengthy queries "q" having many query terms. LTR feature
tries to compute tfidf of such a query and reports "The request took too long
to iterate over terms" in log file and on heap, the query parser tries to parse
the query fills it up causing full heap use and following by oom.
{noformat}
org.apache.solr.search.ReRankCollector @ 0x699161e600.00 MB70.81
MB1.16%org.apache.lucene.search.BooleanQuery @ 0x6991798000.00 MB0.03
MB0.00%java.lang.ThreadLocal$ThreadLocalMap @ 0x69440b3b00.00 MB0.03
MB0.00%org.apache.lucene.search.BooleanQuery @ 0x6b50f73f00.00 MB0.03
MB0.00%org.apache.lucene.queries.function.FunctionScoreQuery @ 0x699167f180.00
MB0.02 MB0.00%com.shutterstock.solr.parser.expression.Expression @
0x699161c200.00 MB0.01 MB0.00%org.apache.solr.response.SolrQueryResponse @
0x6991620780.00 MB0.00 MB0.00%java.util.ArrayList @ 0x6991882b00.00 MB0.00
MB0.00%org.apache.solr.handler.component.ResponseBuilder @ 0x699161fa80.00
MB0.00 MB0.00%java.lang.String @ 0x6b50fe8d0 {!edismax qf=description.en
pf2=description.en}((bang) OR (blast) OR (blow AND up) OR (breakout) OR (burst)
OR (explosion) OR (gale) OR (outbreak) OR (outburst) OR (esplosione)) AND
((dust) OR (filing) OR (powder) OR (polvere)) AND ((blanche) OR (white)
OR...0.00 MB0.00 MB0.00%java.lang.String @ 0x6b50fe3b0 ((bang) OR (blast) OR
(blow AND up) OR (breakout) OR (burst) OR (explosion) OR (gale) OR (outbreak)
OR (outburst) OR (esplosione)) AND ((dust) OR (filing) OR (powder) OR
(polvere)) AND ((blanche) OR (white) OR (white AND woman) OR (bianca)) AND
((confine) ...0.00 MB0.00 MB0.00%java.util.ArrayList @ 0x699161d780.00 MB0.00
MB0.00%org.apache.solr.search.ExtendedDismaxQParser$ExtendedDismaxConfiguration
@ 0x699161ca00.00 MB0.00 MB0.00%{noformat}
example query and feature defintion "\{!edismax qf=description.en
pf2=description.en}"
{noformat}
q=((bang) OR (blast) OR (blow AND up) OR (breakout) OR (burst) OR (explosion)
OR (gale) OR (outbreak) OR (outburst) OR (esplosione)) AND ((dust) OR (filing)
OR (powder) OR (polvere)) AND ((blanche) OR (white) OR (white AND woman) OR
(bianca)) AND ((confine) OR (cut AND off) OR (insular) OR (insulate) OR
(insulated) OR (insulating) OR (insulation) OR (insulator) OR (insulators) OR
(island) OR (islander) OR (islands) OR (isle) OR (isolable) OR (isolatable) OR
(isolate) OR (isolated) OR (isolates) OR (isolation) OR (isolations) OR
(isolete) OR (lonely) OR (remote) OR (seal) {noformat}
Handling such queries at client level should prevent it from happening again.
Thinking if this should be handled on solr side as well. This literally takes
over full heap, causes high gc activity (filling up tenured space), high cpu
uage and does not stop until query is complete, hence bringing solr nodes down.
> LTR queries producing out of memory issues
> ------------------------------------------
>
> Key: SOLR-17018
> URL: https://issues.apache.org/jira/browse/SOLR-17018
> Project: Solr
> Issue Type: Bug
> Components: contrib - LTR, ltr
> Affects Versions: 9.1.1
> Reporter: rajanimaski
> Priority: Major
> Labels: ltr
> Attachments: image-2023-10-11-14-38-43-351.png
>
>
> LTR queries are producing oom and looks like the issue is same as what is
> described on this SOLR-5986. Query that has 10 or more terms, and LTR trying
> to compute features for such queries lead to full heap usage, invoking full
> GC and stop the world.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
