Thank you @Shawn Heisey<mailto:apa...@elyograg.org> , you help me many times.
My -xms=1G When restart solr, I can see the progress of memory increasing (from 1G to 9G, took near 10s). I have a guess: maybe solr is loading some needed files into heap memory, e.g. *.tip : term index file. What's your thoughts? thanks. ________________________________ From: Shawn Heisey <apa...@elyograg.org> Sent: Thursday, November 14, 2019 1:15 To: solr-user@lucene.apache.org <solr-user@lucene.apache.org> Subject: Re: Question about startup memory usage On 11/13/2019 2:03 AM, Hongxu Ma wrote: > I have a solr-cloud cluster with a big collection, after startup (no any > search/index operations), its jvm memory usage is 9GB (via top: RES). > > Cluster and collection info: > each host: total 64G mem, two solr nodes with -xmx=15G > collection: total 9B billion docs (but each doc is very small: only some > bytes), total size 3TB. > > My question is: > Is the 9G mem usage after startup normal? If so, I am worried that the follow > up index/search operations will cause an OOM error. > And how can I reduce the memory usage? Maybe I should introduce more host > with nodes, but besides this, is there any other solution? With the "-Xmx=15G" option, you've told Java that it can use up to 15GB for heap. It's total resident memory usage is eventually going to reach a little over 15GB and probably never go down. This is how Java works. The amount of memory that Java allocates immediately on program startup is related to the -Xms setting. Normally Solr uses the same number for both -Xms and -Xmx, but that can be changed if you desire. We recommend using the same number. If -Xms is smaller than -Xmx, Java may allocate less memory as soon as it starts, then Solr is going to run through its startup procedure. We will not know exactly how much memory allocation is going to occur when that happens ... but with billions of documents, it's not going to be small. Thanks, Shawn
