Agreed, and since it takes three times the space is part of the reason it takes so long, so that 190gb index ends up writing another 380 gb until it compresses down and deletes the two left over files. its a pretty hefty operation
On Thu, Mar 2, 2017 at 10:13 AM, Alexandre Rafalovitch <arafa...@gmail.com> wrote: > Optimize operation is no longer recommended for Solr, as the > background merges got a lot smarter. > > It is an extremely expensive operation that can require up to 3-times > amount of disk during the processing. > > This is not to say yours is a valid question, which I am leaving to > others to respond. > > Regards, > Alex. > ---- > http://www.solr-start.com/ - Resources for Solr users, new and experienced > > > On 2 March 2017 at 10:04, Caruana, Matthew <mcaru...@icij.org> wrote: > > I’m currently performing an optimise operation on a ~190GB index with > about 4 million documents. The process has been running for hours. > > > > This is surprising, because the machine is an EC2 r4.xlarge with four > cores and 30GB of RAM, 24GB of which is allocated to the JVM. > > > > The load average has been steady at about 1.3. Memory usage is 25% or > less the whole time. iostat reports ~6% util. > > > > What gives? > > > > Running Solr 6.4.1. >
