Re: Solr 8.5.1 startup error - lengthTag=109, too big.

2020-05-27 Thread Zheng Lin Edwin Yeo
Hi Mike, Thanks for your reply. Yes, I have SSL enabled in 8.2.1 as well. The error is there even it I use the same certificate for 8.2.1, which was working fine there. I have also generated the certificate for both 8.2.1 and 8.5.1 by the same method. Is there any changes between these 2 version

Re: TimestampUpdateProcessorFactory updates the field even if the value if present

2020-05-27 Thread Erick Erickson
When is “NOW” ;) ?. The process for updating a doc in SolrCloud is: 1> the doc is received by some solr node. 2> the doc is forwarded to the shard leader if necessary. 3> the doc is distributed from the shard leader to all replicas of that shard. 4> the doc is indexed on each replica. So just

Re: SolrCloud upgrade concern

2020-05-27 Thread Erick Erickson
The biggest issue with CDCR is it’s rather fragile and requires monitoring, it’s not a “fire and forget” type of functionality. For instance, the use of the tlogs as a queueing mechanism means that if, for any reason, the communications between DCs is broken, the tlogs will grow forever until the

Re: SolrCloud upgrade concern

2020-05-27 Thread gnandre
Thanks, Jason. This is very helpful. I should clarify though that I am not using CDCR currently with my existing master-slave architecture. What I meant to say earlier was that we will be relying heavily on the CDCR feature if we migrate from solr master-slave architecture to solrcloud architectur

Re: TimestampUpdateProcessorFactory updates the field even if the value if present

2020-05-27 Thread gnandre
Thanks for the detailed response, Chris. I am aware of the partial (atomic) updates. Thanks for clarifying the confusion about input document vs indexed document. I was thinking that TimestampUpdateProcessorFactory checks if the value exists in the field inside indexed document before updating it b

Re: Solr multi core query too slow

2020-05-27 Thread Erick Erickson
First of all, asking for that many rows will spend a lot of time gathering the document fields. Assuming you have stored fields, each doc requires 1> the aggregator node getting the candidate 10 docs from each shard 2> The aggregator node sorting those 10 docs from each shard into the true

Re: search in solrcloud on replicas

2020-05-27 Thread Erick Erickson
The base algorithm for searches picks out one replica from each shard in a round-robin fashion, without regard to whether it’s on the same machine or not. You can alter this behavior, see: https://lucene.apache.org/solr/guide/8_1/distributed-requests.html When you say “the exact same search”, i

search in solrcloud on replicas

2020-05-27 Thread Odysci
Hi, I have a question regarding solrcloud searches on both replicas of an index. I have a solrcloud setup with 2 physical machines (let's call them A and B), and my index is divided into 2 shards, and 2 replicas, such that each machine has a full copy of the index. My Zookeeper setup uses 3 instan

Solr multi core query too slow

2020-05-27 Thread Anshuman Singh
I have a Solr cloud setup (Solr 7.4) with a collection "test" having two shards on two different nodes. There are 4M records equally distributed across the shards. If I query the collection like below, it is slow. http://localhost:8983/solr/*test*/select?q=*:*&rows=10 QTime: 6930 If I query a

Re: SolrCloud upgrade concern

2020-05-27 Thread Jason Gerlowski
Hi Arnold, >From what I saw in the community, CDCR saw an initial burst of development around when it was contributed, but hasn't seen much attention or improvement since. So while it's been around for a few years, I'm not sure it's improved much in terms of stability or compatibility with other

Re: unified highlighter performance in solr 8.5.1

2020-05-27 Thread David Smiley
try setting hl.fragsizeIsMinimum=true I did some benchmarking and found that this helps quite a bit BTW I used the highlights.alg benchmark file, with some changes to make it more reflective of your scenario -- offsets in postings, and used "enwiki" (english wikipedia) docs which are larger than