I have not spent a lot of time researching it, but one would expect that the OS RAM requirement for optimization of an index to be minimal.
My understanding is that during optimization an essentially new index is built. Once complete it switches out the indexes and will throw away the old one. (In Windows it may not throw away the old one until the next Commit). JRJ -----Original Message----- From: Sujatha Arun [mailto:suja.a...@gmail.com] Sent: Friday, October 21, 2011 12:10 AM To: solr-user@lucene.apache.org Subject: Re: Optimization /Commit memory Just one more thing ,when we are talking about Optimization , we are referring to HD free space for replicating the index (2 or 3 times the index size ) .what is role of RAM (OS) here? Regards Suajtha On Fri, Oct 21, 2011 at 10:12 AM, Sujatha Arun <suja.a...@gmail.com> wrote: > Thanks that helps. > > Regards > Sujatha > > > On Thu, Oct 20, 2011 at 6:23 PM, Jaeger, Jay - DOT > <jay.jae...@dot.wi.gov>wrote: > >> Well, since the OS RAM includes the JVM RAM, that is part of your >> requirement, yes? Aside from the JVM and normal OS requirements, all you >> need OS RAM for is file caching. Thus, for updates, the OS RAM is not a >> major factor. For searches, you want sufficient OS RAM to cache enough of >> the index to get the query performance you need, and to cache queries inside >> the JVM if you get a lot of repeat queries (see solrconfig.xml for the >> various caches: we have not played with them much). So, the amount of RAM >> necessary for that is very much dependent upon the size of your index, so I >> cannot give you a simple number. >> >> You seem to believe that you have to have sufficient memory to have the >> entire index in memory. Except where extremely high performance is >> required, I have not found that to be the case. >> >> This is just one of those "your mileage may vary" things. There is not a >> single answer or formula that fits every situation. >> >> JRJ >> >> -----Original Message----- >> From: Sujatha Arun [mailto:suja.a...@gmail.com] >> Sent: Wednesday, October 19, 2011 11:58 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Optimization /Commit memory >> >> Thanks Jay , >> >> I was trying to compute the *OS RAM requirement* *not JVM RAM* for a 14 >> GB >> Index [cumulative Index size of all Instances].And I put it thus - >> >> Requirement of Operating System RAM for an Index of 14GB is - Index >> Size >> + 3 Times the maximum Index Size of Individual Instance for Optimize . >> >> That is to say ,I have several Instances ,combined Index Size is 14GB >> .Maximum Individual Index Size is 2.5GB .so My requirement for OS RAM is >> 14GB +3 * 2.5 GB ~ = 22GB. >> >> Correct? >> >> Regards >> Sujatha >> >> >> >> On Thu, Oct 20, 2011 at 3:45 AM, Jaeger, Jay - DOT <jay.jae...@dot.wi.gov >> >wrote: >> >> > Commit does not particularly spike disk or memory usage, unless you are >> > adding a very large number of documents between commits. A commit can >> cause >> > a need to merge indexes, which can increase disk space temporarily. An >> > optimize is *likely* to merge indexes, which will usually increase disk >> > space temporarily. >> > >> > How much disk space depends very much upon how big your index is in the >> > first place. A 2 to 3 times factor of the sum of your peak index file >> size >> > seems safe, to me. >> > >> > Solr uses only modest amounts of memory for the JVM for this stuff. >> > >> > JRJ >> > >> > -----Original Message----- >> > From: Sujatha Arun [mailto:suja.a...@gmail.com] >> > Sent: Wednesday, October 19, 2011 4:04 AM >> > To: solr-user@lucene.apache.org >> > Subject: Optimization /Commit memory >> > >> > Do we require 2 or 3 Times OS RAM memory or Hard Disk Space while >> > performing Commit or Optimize or Both? >> > >> > what is the requirement in terms of size of RAM and HD for commit and >> > Optimize >> > >> > Regards >> > Sujatha >> > >> > >
