David:
Some of this still matters even with 7.5+. Prior to 7.5, you could easily have
50% of your index consist of deleted docs. With 7.5, this ceiling is reduced.
expungeDeletes will reduce the size to no more than 10% while still respecting
the default max segment size of 5G. Optimizing and
I use the same algorithm and for me, initialMaxSegments is always the number of
segments currently in the index (seen, e.g, in the SOLR admin UI).
finalMaxSegments depends on what kind of updates have happened. If I know that
"older" documents are untouched, then I'll usually use -60% or even
Thanks @Michael Joyner, how did you decide initialmax segment to 256 ? Or it
is some random number i can use for my case ? Can you guuide me how to
decide the initial & final max segments ?
Michael Joyner wrote
> That is the way we do it here - also helps a lot with not needing x2 or
> x3
> On Jun 7, 2019, at 7:53 AM, David Santamauro
> wrote:
>
> So is this new optimize maxSegments / commit expungeDeletes behavior in 7.5?
> My experience, and I watch the my optimize process very closely, is that
> using maxSgements does not touch every segment with a deleted document.
>
So is this new optimize maxSegments / commit expungeDeletes behavior in 7.5? My
experience, and I watch the my optimize process very closely, is that using
maxSgements does not touch every segment with a deleted document.
expungeDeletes merges all segments that have deleted documents that have
Optimizing guarantees that there will be _no_ deleted documents in an index
when done. If a segment has even one deleted document, it’s merged, no matter
what you specify for maxSegments.
Segments are write-once, so to remove deleted data from a segment it must be at
least rewritten into a
Thanks @Erick for the suggestions. That looks so bad, yes your assumptions
are right, we have lot of delete & index documents as well.
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Thanks Shawn for suggestions. Interesting to know deleteByQuery has some
impact, will try to change it as you have suggested. Thabks
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
/clarification/ ... expungeDeletes will merge every segment *touched by the
current commit* that has a deleted document.
On 6/7/19, 10:07 AM, "David Santamauro" wrote:
Erick, on 6.0.1, optimize with maxSegments only merges down to the
specified number. E.g., given an index with 75
Erick, on 6.0.1, optimize with maxSegments only merges down to the specified
number. E.g., given an index with 75 segments, optimize with maxSegments=74
will only merge 2 segments leaving 74 segments. It will choose a segment to
merge that has deleted documents, but does not merge every segment
That is the way we do it here - also helps a lot with not needing x2 or
x3 disk space to handle the merge:
public void solrOptimize() {
int initialMaxSegments = 256;
int finalMaxSegments = 4;
if (isShowSegmentCounter()) {
log.info("Optimizing ...");
}
This isn’t quite right. Solr will rewrite _all_ segments that have _any_
deleted documents in them when optimizing, even one. Given your description,
I’d guess that all your segments will have deleted documents, so even if you do
specify maxSegments on the optimize command, the entire index
On 6/6/2019 11:27 PM, jena wrote:
Because of heavy indexing & deletion, we optimise solr instance everyday,
because of that our solr cloud getting unstable , every solr instance go on
recovery mode & our search is getting affected & very slow because of that.
Optimisation takes around 1hr
Thanks @Nicolas Franck for reply, i don't see any any segment info for 4.4
version. Is there any API i can use to get my segment information ? Will try
to use maxSegments and see if it can help us during optimization.
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
In that case, hard optimisation like that is out the question.
Resort to automatic merge policies, specifying a maximum
amount of segments. Solr is created with multiple segments
in mind. Hard optimisation seems like not worth the problem.
The problem is this: the less segments you specify during
Hello guys,
We have 4 solr(version 4.4) instance on production environment, which are
linked/associated with zookeeper for replication. We do heavy deleted & add
operations. We have around 26million records and the index size is around
70GB. We serve 100k+ requests per day.
Because of heavy
16 matches
Mail list logo
