Overseer & Backups - Questions

2020-03-09 Thread Ashwin Ramesh
Hi everybody, Quick Specs: - Solr 7.4 Solr Cloud - 30gb index on 8 shards Tlog/Pull We run daily backups on our 30gb index and noticed that the overseer does not process other jobs on it's task list while the backup is being taken. They remain on the pending list (in ZK). Is this expected? Also

Re: Possible performance bug - JSON facet - numBuckets:true

2020-03-09 Thread Mikhail Khludnev
Hello, Artur. Thanks for your interest. Perhaps, we can amend doc mentioning this effect. In long term it can be optimized by adding a proper condition. Both patches are welcome. On Wed, Feb 12, 2020 at 10:48 PM Rudenko, Artur wrote: > Hello everyone, > I'm am currently investigating a performa

RE: OutOfMemory error solr 8.4.1

2020-03-09 Thread Srinivas Kashyap
Hi Erick, Yes you were right, in my custom jar I'm using HttpSolrClient as below: HttpSolrClient client = new HttpSolrClient.Builder("http://"; + server + ":" + port + "/" + webapp + "/").build(); try { client.request(new QueryRequest(params),coreName);

Re: multivalue faceting term optimization

2020-03-09 Thread Jörn Franke
hll stands for https://en.wikipedia.org/wiki/HyperLogLog You will not get the exact distinct count, but a distinct count very close to the real number. It is very fast and memory efficient for large number of distinct values. > Am 10.03.2020 um 00:25 schrieb Nicolas Paris : > >  > Erick Erick

Re: multivalue faceting term optimization

2020-03-09 Thread Nicolas Paris
Erick Erickson writes: > Have you looked at the HyperLogLog stuff? Here’s at least a mention of > it: https://lucene.apache.org/solr/guide/8_4/the-stats-component.html I am used to hll in the context of count distinct values -- cardinality. I have to admit that section https://lucene.apache.o

Re: multivalue faceting term optimization

2020-03-09 Thread Nicolas Paris
Toke Eskildsen writes: > JSON faceting allows you to skip the fine counting with the parameter > refine: I also tried the facet.refine parameter, but didn't notice any improvement. >> I am wondering how I could filter the documents to get approximate >> facets ? > > Clunky idea: Introduce a

Re: Repeatable search term bug in Solr 8?

2020-03-09 Thread Jan Høydahl
You must fetch logs from the solr server and paste here for us to be able to tell what’s wrong. Also provide your schema and config so it is possible to reproduce. Jan Høydahl > 9. mar. 2020 kl. 18:17 skrev Staley, Phil R - DCF : > > All, > > We recently upgraded to our Drupal 8 sites to SOL

Does LTRQueryParser accept local variables?

2020-03-09 Thread David White
Hi all, Consider the following edismax query parser usage: {!edismax qf="field1 field2" v=$query} A local variable, in this case, query, is used in the parser, through use of the dollar sign operator. Does the LTR query parser have the same capability? Thanks, David White This message an

Repeatable search term bug in Solr 8?

2020-03-09 Thread Staley, Phil R - DCF
All, We recently upgraded to our Drupal 8 sites to SOLR 8.3.1. We are now getting reports of certain patterns of search terms resulting in an error that reads, "The website encountered an unexpected error. Please try again later." Below is a list of example terms that repeatably result in this

Re: OutOfMemory error solr 8.4.1

2020-03-09 Thread Erick Erickson
I’m 99% certain that something in your custom jar is the culprit, otherwise we’d have seen a _lot_ of these. TIMED_WAITING is usually just a listener thread, but they shouldn’t be generated when SOlr is just sitting there. The first thing I’d do is dummy out my custom code or remove it complete

Re: Problem with Solr 7.7.2 after OOM

2020-03-09 Thread Erick Erickson
Glad you found the problem. The “core.properties” file, when found, assumes a core and tries to load it. Problem is that it must correspond to certain information in ZooKeeper and they can get out of sync. But there should be a message somewhere in the logs. I’ll give it a spin sometime and

Re: multivalue faceting term optimization

2020-03-09 Thread Erick Erickson
Have you looked at the HyperLogLog stuff? Here’s at least a mention of it: https://lucene.apache.org/solr/guide/8_4/the-stats-component.html Best, Erick > On Mar 9, 2020, at 02:39, Nicolas Paris wrote: > > Hello, > > > Environment: > - SolrCloud 8.4.1 > - 4 shards with xmx = 120GO and ssd

Re: "SolrCore Initialization Failures" in the Solr's current UI, but not in the original UI

2020-03-09 Thread Ryan W
I am using v6.6.6, which is the most recent release in the v6 branch, and is the branch commonly used with my app. I have tried on Chrome, Firefox and Internet Explorer, and I see the same problem in all three. Albeit it seems to be a minor problem. On Thu, Mar 5, 2020 at 4:34 PM Jan Høydahl wr

Re: multivalue faceting term optimization

2020-03-09 Thread Toke Eskildsen
On Mon, 2020-03-09 at 10:39 +0100, Nicolas Paris wrote: > I want to provide terms facet on a string multivalue field. > ... > How to improve brute performances ? It might help to have everything in a single shard, to avoid the secondary fine count. But your index is rather large for single-shard s

multivalue faceting term optimization

2020-03-09 Thread Nicolas Paris
Hello, Environment: - SolrCloud 8.4.1 - 4 shards with xmx = 120GO and ssd disks - 50M documents / 40GO physical per shard - mainly large texts fields and also, one multivalue/docvalue/indexed string list of 15 values per document Goal: I want to provide terms facet on a string multivalue field.

RE: OutOfMemory error solr 8.4.1

2020-03-09 Thread Srinivas Kashyap
Hi Erick, I recompiled my custom code with 8.4.1 jars and placed back my jar in the lib folder. Under Solr admin console/Thread Dump, I'm seeing a lot of below threads which are in TIMED_WAITING stage. Connection evictor (999) java.lang.Thread.sleep​(Native Method) org.apache.h

AW: Problem with Solr 7.7.2 after OOM

2020-03-09 Thread Bunde Torsten
Hello Erick, no there are no more OOMs and there were no errors in the logs. But the problem is solved now. The root cause seemed to be a duplicate core (two cores with the same name) because someone did a backup of an existing one ... Thank you for your support! - Torsten -Ursprüngliche N