Hi Eric,
Thanks for your reply!
I have one more question which I think you missed in my previous email.
*"When our core size becomes ~100 G, indexing becomes really slow. Why is
this happening? Do we need to put a limit on how large each core can grow?"*
This question is unrelated to segments. I
New segments are created when
1> the RAMBufferSizeMB is exceeded
or
2> a commit happens.
The maximum segment size defaults to 5G, but TieredMergePolicy can be
configured in solrconfig.xml to have larger max sizes by setting
maxMergedSegmentMB
Depending on your indexing rate, requiring commits
Hi Eric,
We are looking into TLOG/PULL replicas. But I have some doubts regarding
segments. Can you explain what causes creation of a new segment and how
large it can grow?
And this is my index config:
maxMergeAtOnce - 20
segmentsPerTier - 20
ramBufferSizeMB - 512 MB
Can I configure these
Have you considered TLOG/PULL replicas rather than NRT replicas?
That way, all the indexing happens on a single machine and you can
use shards.preference to confine the searches happen on the PULL replicas,
see: https://lucene.apache.org/solr/guide/7_7/distributed-requests.html
No, you can’t
Hi Nicolas,
Commit happens automatically at 100k documents. We don't commit explicitly.
We didn't limit the number of segments. There are 35+ segments in each core.
But unrelated to the question, I would like to know if we can limit the
number of segments in the core. I tried it in the past but
The real questions are:
* how much often do you commit (either explicitly or automatically)?
* how much segments do you allow? If you only allow 1 segment,
then that whole segment is recreated using the old documents and the updates.
And yes, that requires reading the old segment.
It is
I noticed that while indexing, when commit happens, there is high disk read
by Solr. The problem is that it is impacting search performance when the
index is loaded from the disk with respect to the query, as the disk read
speed is not quite good and the whole index is not cached in RAM.
When no
