A forced merge might improve speed 20%. Going from spinning disk to SSD will improve speed 20X or more. Don’t waste your time even thinking about forced merges.
You need to get SSDs. The even bigger speedup is to get enough RAM that the OS can keep the Solr index files in file system buffers. Check how much space is used by your indexes, then make sure that there is that much available RAM that is not used by the OS or Solr JVM. Some people make the mistake of giving a huge heap to the JVM, thinking this will improve caching. This almost always makes things worse, by using RAM that could be use for caching files. 8GB of heap is usually enough. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Feb 21, 2021, at 11:52 PM, Danilo Tomasoni <tomas...@cosbi.eu> wrote: > > Hello all, > we are running a solr instance with around 41 MLN documents on a SATA class > 10 disk with around 10.000 rpm. > We are experiencing very slow query responses (in the order of hours..) with > an average of 205 segments. > We made a test with a normal pc and an SSD disk, and there the same solr > instance with the same data and the same number of segments was around 45 > times faster. > Force optimize was also tried to improve the performances, but it was very > slow, so we abandoned it. > > Since we still don't have enterprise server ssd disks, we are now wondering > if in the meanwhile defragmenting the solrdata folder can help. > The idea is that due to many updates, each segment file is fragmented across > different phisical blocks. > Put in another way, each segment file is non-contiguous on disk, and this can > slow-down the solr response. > > What do you suggest? > Is this somewhat equivalent to force-optimize or it can be faster? > > Thank you. > Danilo > > Danilo Tomasoni > > Fondazione The Microsoft Research - University of Trento Centre for > Computational and Systems Biology (COSBI) > Piazza Manifattura 1, 38068 Rovereto (TN), Italy > tomas...@cosbi.eu<https://webmail.cosbi.eu/owa/redir.aspx?C=VNXi3_8-qSZTBi-FPvMwmwSB3IhCOjY8nuCBIfcNIs_5SgD-zNPWCA..&URL=mailto%3acalabro%40cosbi.eu> > http://www.cosbi.eu<https://webmail.cosbi.eu/owa/redir.aspx?C=CkilyF54_imtLHzZqF1gCGvmYXjsnf4bzGynd8OXm__5SgD-zNPWCA..&URL=http%3a%2f%2fwww.cosbi.eu%2f> > > As for the European General Data Protection Regulation 2016/679 on the > protection of natural persons with regard to the processing of personal data, > we inform you that all the data we possess are object of treatment in the > respect of the normative provided for by the cited GDPR. > It is your right to be informed on which of your data are used and how; you > may ask for their correction, cancellation or you may oppose to their use by > written request sent by recorded delivery to The Microsoft Research – > University of Trento Centre for Computational and Systems Biology Scarl, > Piazza Manifattura 1, 38068 Rovereto (TN), Italy. > P Please don't print this e-mail unless you really need to
