Re: Monitoring Solr for currently running queries
Hello Ufuk, You can log slow queries [1]. If you would want to see currently running queries you would have to extend SearchHandler and build the custom logic yourself. Watch out for SolrCloud because the main query as well as the per-shard queries can pass through that same SearchHandler. You can distinguish between then reading the shard=true parameter. Regards, Markus [1] https://lucene.apache.org/solr/guide/6_6/configuring-logging.html Op di 29 dec. 2020 om 16:49 schreef ufuk yılmaz : > Hello All, > > Is there a way to see currently executing queries in a SolrCloud? Or a > general strategy to detect a query using absurd amount or resources? > > We are using Solr for not only simple querying, but running complex > streaming expressions, facets with large data etc. Sometimes, randomly, CPU > usage gets so high that it starts to respond very slowly to even simple > queries, or don’t respond at all. I’m trying to determine if it’s a result > of simple overloading of the system by many “normal” queries, or someone > sends Solr an unreasonably compute-heavy request. > > A few days ago when this occured, I stopped every service that can send > Solr a query. After that, for about an hour, nodes were reading from the > disk at 1GB/s which is the maximum of our disks. Then everything went back > to the normal as I started the other services. > > One (bad) idea I had is to build a proxy service which proxies every > request to our SolrCloud and monitors current running requests, but scaling > this to the size of SolrCloud may be reinventing the wheel. > > For now all I can detect is that Solr is struggling, but I have no idea > what causes that and when. > > -Chees and happy new year >
Upgrade httpclient version due to CVE-2020-13956?
CVE-2020-13956: Apache HttpClient incorrect handling of malformed authority component in request URIs Severity: Medium Vendor: The Apache Software Foundation Versions Affected: Apache HttpClient 4.5.12 and prior Apache HttpClient 5.0.2 and prior Description: Apache HttpClient versions prior to version 4.5.13 and 5.0.3 can misinterpret malformed authority component in request URIs passed to the library as java.net.URI object and pick the wrong target host for request execution. Mitigation: As of release 4.5.13 and 5.0.3 HttpClient will reject URIs with ambiguous malformed authority component as invalid. Users of HttpClient are advised to upgrade to version 4.5.13 or 5.0.3 and sanitize request URIs when using java.net.URI as input. Credit: This issue was discovered and reported by Priyank Nigam
Monitoring Solr for currently running queries
Hello All, Is there a way to see currently executing queries in a SolrCloud? Or a general strategy to detect a query using absurd amount or resources? We are using Solr for not only simple querying, but running complex streaming expressions, facets with large data etc. Sometimes, randomly, CPU usage gets so high that it starts to respond very slowly to even simple queries, or don’t respond at all. I’m trying to determine if it’s a result of simple overloading of the system by many “normal” queries, or someone sends Solr an unreasonably compute-heavy request. A few days ago when this occured, I stopped every service that can send Solr a query. After that, for about an hour, nodes were reading from the disk at 1GB/s which is the maximum of our disks. Then everything went back to the normal as I started the other services. One (bad) idea I had is to build a proxy service which proxies every request to our SolrCloud and monitors current running requests, but scaling this to the size of SolrCloud may be reinventing the wheel. For now all I can detect is that Solr is struggling, but I have no idea what causes that and when. -Chees and happy new year
Re: Why do I get different results for the same query with two Solr versions?
Can you post the managed schema and solrconfig content here ? Do try the solr admin analysis screen once as well to see the behaviour for this field. https://lucene.apache.org/solr/guide/7_6/index.html On Sun, 27 Dec, 2020, 6:54 pm nettadalet, wrote: > Thank you, that was helpful! > > For Solr 4.6 I get > "parsedquery": "PhraseQuery(TITLE_ItemCode_t:\"ki 7\")" > > For Solr 7.5 I get > "parsedquery":"+(+(TITLE_ItemCode_t:ki7 (+TITLE_ItemCode_t:ki > +TITLE_ItemCode_t:7)))" > > So this is the cause of the difference in the search result, but I still > don't know why the parsedquery is different between the two versions. > Any idea/guess? > Is it some internal implementation that changed sometime between 4.6 and > 7.5? > > > > -- > Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
Re: Re:Re: Why do I get different results for the same query with two Solr versions?
Hi, thank for the comment, but I tried to use both "sow=false" and "saw=true" and I still get the same result. For query (TITLE_ItemCode_t:KI_7) I still see: Solr 4.6: "parsedquery": "PhraseQuery(TITLE_ItemCode_t:\"ki 7\")" Solr 7.5: "parsedquery":"+(+(TITLE_ItemCode_t:ki7 (+TITLE_ItemCode_t:ki +TITLE_ItemCode_t:7)))" Tulsi wrote > Hi , > Yes this look like related to sow (split on whitespace) param default > behaviour change in solr 7. > > The sow parameter (short for "Split on Whitespace") now defaults to > false, which allows support for multi-word synonyms out of the box. > This parameter is used with the eDismax and standard/"lucene" query > parsers. If this parameter is not explicitly specified as true, query > text will not be split on whitespace before analysis. > > https://lucene.apache.org/solr/guide/7_0/major-changes-in-solr-7.html > > > On Sun, 27 Dec, 2020, 8:25 pm nettadalet, > nsteinberg@ > wrote: > >> I added "defType=lucene" to both searches to make sure I use the same >> query >> parser, but it didn't change the results. >> >> >> >> -- >> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html >> -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Atomic Update Failures with Nested Schema and Lazy Field Loading
Hi, I've encountered another issue that might be related to nested schema. Not always, but many times atomic updates fails for some shards with the message "TransactionLog doesn't know how to serialize class org.apache.lucene.document.LazyDocument$LazyField". I checked both options: 1. Set false. 2. Set true but removed child documents. In both cases atomic update worked without any errors. This might suggest that there is an issue with this combination. Thanks in advance, Ronen. This electronic message may contain proprietary and confidential information of Verint Systems Inc., its affiliates and/or subsidiaries. The information is intended to be for the use of the individual(s) or entity(ies) named above. If you are not the intended recipient (or authorized to receive this e-mail for the intended recipient), you may not use, copy, disclose or distribute to anyone this message or any information contained in this message. If you have received this electronic message in error, please notify us by replying to this e-mail.
