That is the way we do it here - also helps a lot with not needing x2 or
x3 disk space to handle the merge:
public void solrOptimize() {
int initialMaxSegments = 256;
int finalMaxSegments = 4;
if (isShowSegmentCounter()) {
log.info("Optimizing ...");
}
try (SolrClient solrServerInstance = getSolrClientInstance()) {
for (int segments = initialMaxSegments; segments >=
finalMaxSegments; segments--) {
if (isShowSegmentCounter()) {
System.out.println("Optimizing to a max of " +
segments + " segments.");
}
try {
solrServerInstance.optimize(true, true, segments);
} catch (RemoteSolrException | SolrServerException |
IOException e) {
log.severe(e.getMessage());
}
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
On 6/7/19 4:56 AM, Nicolas Franck wrote:
In that case, hard optimisation like that is out the question.
Resort to automatic merge policies, specifying a maximum
amount of segments. Solr is created with multiple segments
in mind. Hard optimisation seems like not worth the problem.
The problem is this: the less segments you specify during
during an optimisation, the longer it will take, because it has to read
all of these segments to be merged, and redo the sorting. And a cluster
has a lot of housekeeping on top of it.
If you really want to issue a optimisation, then you can
also do it in steps (max segments parameter)
10 -> 9 -> 8 -> 7 .. -> 1
that way less segments need to be merged in one go.
testing your index will show you what a good maximum
amount of segments is for your index.
On 7 Jun 2019, at 07:27, jena <sthita2...@gmail.com> wrote:
Hello guys,
We have 4 solr(version 4.4) instance on production environment, which are
linked/associated with zookeeper for replication. We do heavy deleted & add
operations. We have around 26million records and the index size is around
70GB. We serve 100k+ requests per day.
Because of heavy indexing & deletion, we optimise solr instance everyday,
because of that our solr cloud getting unstable , every solr instance go on
recovery mode & our search is getting affected & very slow because of that.
Optimisation takes around 1hr 30minutes.
We are not able fix this issue, please help.
Thanks & Regards
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
