Re: Stop solr query
Moshe, if you take a thread dump while a particular query stuck (via jstack of in SolrAdmin tab), it may explain where exactly it's stalled, just check the longest stack trace. FWIW, in 4.x timeallowed is checked only while documents are collected, and in 5 it's also checked during query expansion (see http://lucidworks.com/blog/solr-5-0/ now cut-offs requests https://issues.apache.org/jira/browse/SOLR-5986 during the query-expansion stage as well ). however I'm not sure it has place (long query expansion) with hon-synonyms. On Wed, Feb 25, 2015 at 3:21 PM, Moshe Recanati mos...@kmslh.com wrote: Hi Shawn, We checked this option and it didn't solve our problem. We're using https://github.com/healthonnet/hon-lucene-synonyms for query based synonyms. While running query with high number of words that have high number of synonyms the query got stuck and solr memory is exhausted. We tried to use this parameter suggested by you however it didn't stop the query and solve the issue. Please let me know if there is other option to tackle it. Today it might be high number of words that cause the issue and tomorrow it might be other something wrong. We can't rely only on user input check. Thank you in advance. Regards, Moshe Recanati SVP Engineering Office + 972-73-2617564 Mobile + 972-52-6194481 Skype: recanati More at: www.kmslh.com | LinkedIn | FB -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Monday, February 23, 2015 5:49 PM To: solr-user@lucene.apache.org Subject: Re: Stop solr query On 2/23/2015 7:23 AM, Moshe Recanati wrote: Recently there were some scenarios in which queries that user sent to solr got stuck and increased our solr heap. Is there any option to kill or timeout query that wasn't returned from solr by external command? The best thing you can do is examine all user input and stop such queries before they execute, especially if they are the kind of query that will cause your heap to grow out of control. The timeAllowed parameter can abort a query that takes too long in certain phases of the query. In recent months, Solr has been modified so that timeAllowed will take effect during more query phases. It is not a perfect solution, but it can be better than nothing. http://wiki.apache.org/solr/CommonQueryParameters#timeAllowed Be aware that sometimes legitimate queries will be slow, and using timeAllowed may cause those queries to fail. Thanks, Shawn -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Re: Stop solr query
On 2/25/2015 5:21 AM, Moshe Recanati wrote: We checked this option and it didn't solve our problem. We're using https://github.com/healthonnet/hon-lucene-synonyms for query based synonyms. While running query with high number of words that have high number of synonyms the query got stuck and solr memory is exhausted. We tried to use this parameter suggested by you however it didn't stop the query and solve the issue. Please let me know if there is other option to tackle it. Today it might be high number of words that cause the issue and tomorrow it might be other something wrong. We can't rely only on user input check. If legitimate queries use a lot of memory, you'll either need to increase the java heap so it can deal with the increased memory requirements, or you'll have to take steps to decrease memory usage. Those steps might include changes to your application code to detect problematic queries before they happen, and/or educating your users about how to properly use the search. Lucene and Solr are constantly making advances in memory efficiency, so making sure you're always on the latest version goes a long way towards keeping Solr efficient. Thanks, Shawn
RE: Stop solr query
Hi Shawn, We checked this option and it didn't solve our problem. We're using https://github.com/healthonnet/hon-lucene-synonyms for query based synonyms. While running query with high number of words that have high number of synonyms the query got stuck and solr memory is exhausted. We tried to use this parameter suggested by you however it didn't stop the query and solve the issue. Please let me know if there is other option to tackle it. Today it might be high number of words that cause the issue and tomorrow it might be other something wrong. We can't rely only on user input check. Thank you in advance. Regards, Moshe Recanati SVP Engineering Office + 972-73-2617564 Mobile + 972-52-6194481 Skype : recanati More at: www.kmslh.com | LinkedIn | FB -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Monday, February 23, 2015 5:49 PM To: solr-user@lucene.apache.org Subject: Re: Stop solr query On 2/23/2015 7:23 AM, Moshe Recanati wrote: Recently there were some scenarios in which queries that user sent to solr got stuck and increased our solr heap. Is there any option to kill or timeout query that wasn't returned from solr by external command? The best thing you can do is examine all user input and stop such queries before they execute, especially if they are the kind of query that will cause your heap to grow out of control. The timeAllowed parameter can abort a query that takes too long in certain phases of the query. In recent months, Solr has been modified so that timeAllowed will take effect during more query phases. It is not a perfect solution, but it can be better than nothing. http://wiki.apache.org/solr/CommonQueryParameters#timeAllowed Be aware that sometimes legitimate queries will be slow, and using timeAllowed may cause those queries to fail. Thanks, Shawn
RE: Stop solr query
HI Mikhail, We're using 4.7.1. This means I can't stop the search. I think this is mandatory feature. Regards, Moshe Recanati SVP Engineering Office + 972-73-2617564 Mobile + 972-52-6194481 Skype : recanati More at: www.kmslh.com | LinkedIn | FB -Original Message- From: Mikhail Khludnev [mailto:mkhlud...@griddynamics.com] Sent: Wednesday, February 25, 2015 3:42 PM To: solr-user Subject: Re: Stop solr query Moshe, if you take a thread dump while a particular query stuck (via jstack of in SolrAdmin tab), it may explain where exactly it's stalled, just check the longest stack trace. FWIW, in 4.x timeallowed is checked only while documents are collected, and in 5 it's also checked during query expansion (see http://lucidworks.com/blog/solr-5-0/ now cut-offs requests https://issues.apache.org/jira/browse/SOLR-5986 during the query-expansion stage as well ). however I'm not sure it has place (long query expansion) with hon-synonyms. On Wed, Feb 25, 2015 at 3:21 PM, Moshe Recanati mos...@kmslh.com wrote: Hi Shawn, We checked this option and it didn't solve our problem. We're using https://github.com/healthonnet/hon-lucene-synonyms for query based synonyms. While running query with high number of words that have high number of synonyms the query got stuck and solr memory is exhausted. We tried to use this parameter suggested by you however it didn't stop the query and solve the issue. Please let me know if there is other option to tackle it. Today it might be high number of words that cause the issue and tomorrow it might be other something wrong. We can't rely only on user input check. Thank you in advance. Regards, Moshe Recanati SVP Engineering Office + 972-73-2617564 Mobile + 972-52-6194481 Skype: recanati More at: www.kmslh.com | LinkedIn | FB -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Monday, February 23, 2015 5:49 PM To: solr-user@lucene.apache.org Subject: Re: Stop solr query On 2/23/2015 7:23 AM, Moshe Recanati wrote: Recently there were some scenarios in which queries that user sent to solr got stuck and increased our solr heap. Is there any option to kill or timeout query that wasn't returned from solr by external command? The best thing you can do is examine all user input and stop such queries before they execute, especially if they are the kind of query that will cause your heap to grow out of control. The timeAllowed parameter can abort a query that takes too long in certain phases of the query. In recent months, Solr has been modified so that timeAllowed will take effect during more query phases. It is not a perfect solution, but it can be better than nothing. http://wiki.apache.org/solr/CommonQueryParameters#timeAllowed Be aware that sometimes legitimate queries will be slow, and using timeAllowed may cause those queries to fail. Thanks, Shawn -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Re: Stop solr query
No. You can, but only search (collecting results) and not a query expansion. As I said, debugQuery=true, and the stacktrace or sampling can help to understand the reason. On Wed, Feb 25, 2015 at 5:45 PM, Moshe Recanati mos...@kmslh.com wrote: HI Mikhail, We're using 4.7.1. This means I can't stop the search. I think this is mandatory feature. Regards, Moshe Recanati SVP Engineering Office + 972-73-2617564 Mobile + 972-52-6194481 Skype: recanati More at: www.kmslh.com | LinkedIn | FB -Original Message- From: Mikhail Khludnev [mailto:mkhlud...@griddynamics.com] Sent: Wednesday, February 25, 2015 3:42 PM To: solr-user Subject: Re: Stop solr query Moshe, if you take a thread dump while a particular query stuck (via jstack of in SolrAdmin tab), it may explain where exactly it's stalled, just check the longest stack trace. FWIW, in 4.x timeallowed is checked only while documents are collected, and in 5 it's also checked during query expansion (see http://lucidworks.com/blog/solr-5-0/ now cut-offs requests https://issues.apache.org/jira/browse/SOLR-5986 during the query-expansion stage as well ). however I'm not sure it has place (long query expansion) with hon-synonyms. On Wed, Feb 25, 2015 at 3:21 PM, Moshe Recanati mos...@kmslh.com wrote: Hi Shawn, We checked this option and it didn't solve our problem. We're using https://github.com/healthonnet/hon-lucene-synonyms for query based synonyms. While running query with high number of words that have high number of synonyms the query got stuck and solr memory is exhausted. We tried to use this parameter suggested by you however it didn't stop the query and solve the issue. Please let me know if there is other option to tackle it. Today it might be high number of words that cause the issue and tomorrow it might be other something wrong. We can't rely only on user input check. Thank you in advance. Regards, Moshe Recanati SVP Engineering Office + 972-73-2617564 Mobile + 972-52-6194481 Skype: recanati More at: www.kmslh.com | LinkedIn | FB -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Monday, February 23, 2015 5:49 PM To: solr-user@lucene.apache.org Subject: Re: Stop solr query On 2/23/2015 7:23 AM, Moshe Recanati wrote: Recently there were some scenarios in which queries that user sent to solr got stuck and increased our solr heap. Is there any option to kill or timeout query that wasn't returned from solr by external command? The best thing you can do is examine all user input and stop such queries before they execute, especially if they are the kind of query that will cause your heap to grow out of control. The timeAllowed parameter can abort a query that takes too long in certain phases of the query. In recent months, Solr has been modified so that timeAllowed will take effect during more query phases. It is not a perfect solution, but it can be better than nothing. http://wiki.apache.org/solr/CommonQueryParameters#timeAllowed Be aware that sometimes legitimate queries will be slow, and using timeAllowed may cause those queries to fail. Thanks, Shawn -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Stop solr query
Hi, Recently there were some scenarios in which queries that user sent to solr got stuck and increased our solr heap. Is there any option to kill or timeout query that wasn't returned from solr by external command? Thank you, Regards, Moshe Recanati SVP Engineering Office + 972-73-2617564 Mobile + 972-52-6194481 Skype: recanati [KMS2]http://finance.yahoo.com/news/kms-lighthouse-named-gartner-cool-121000184.html More at: www.kmslh.comhttp://www.kmslh.com/ | LinkedInhttp://www.linkedin.com/company/kms-lighthouse | FBhttps://www.facebook.com/pages/KMS-lighthouse/123774257810917
Re: Stop solr query
On 2/23/2015 7:23 AM, Moshe Recanati wrote: Recently there were some scenarios in which queries that user sent to solr got stuck and increased our solr heap. Is there any option to kill or timeout query that wasn't returned from solr by external command? The best thing you can do is examine all user input and stop such queries before they execute, especially if they are the kind of query that will cause your heap to grow out of control. The timeAllowed parameter can abort a query that takes too long in certain phases of the query. In recent months, Solr has been modified so that timeAllowed will take effect during more query phases. It is not a perfect solution, but it can be better than nothing. http://wiki.apache.org/solr/CommonQueryParameters#timeAllowed Be aware that sometimes legitimate queries will be slow, and using timeAllowed may cause those queries to fail. Thanks, Shawn
