Interesting. Which version of Solr is this? What happens if you do a commit?
On Sat, Jul 21, 2012 at 8:01 AM, geetha anjali <anjaliprabh...@gmail.com> wrote: > Hi uwe, > Great to know. We have files indexing 10000/min. After 30 mins I see all > my physical memory say its 100 percentage used(windows). On deep > investigation found that mmap is not releasing os files handles. Do you > find this behaviour? > > Thanks > > On 20 Jul 2012 14:04, "Uwe Schindler" <u...@thetaphi.de> wrote: > > Hi Bill, > > MMapDirectory uses the file system cache of your operating system, which has > following consequences: In Linux, top & free should normally report only > *few* free memory, because the O/S uses all memory not allocated by > applications to cache disk I/O (and shows it as allocated, so having 0% free > memory is just normal on Linux and also Windows). If you have other > applications or Lucene/Solr itself that allocate lot's of heap space or > malloc() a lot, then you are reducing free physical memory, so reducing fs > cache. This depends also on your swappiness parameter (if swappiness is > higher, inactive processes are swapped out easier, default is 60% on linux - > freeing more space for FS cache - the backside is of course that maybe > in-memory structures of Lucene and other applications get pages out). > > You will only see no paging at all if all memory allocated all applications > + all mmapped files fit into memory. But paging in/out the mmapped Lucene > index is muuuuuch cheaper than using SimpleFSDirectory or NIOFSDirectory. If > you use SimpleFS or NIO and your index is not in FS cache, it will also read > it from physical disk again, so where is the difference. Paging is actually > cheaper as no syscalls are involved. > > If you want as much as possible of your index in physical RAM, copy it to > /dev/null regularily and buy more RUM :-) > > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: uwe@thetaphi... > >> From: Bill Bell [mailto:billnb...@gmail.com] >> Sent: Friday, July 20, 2012 5:17 AM >> Subject: Re: ... >> s=op using it? The least used memory will be removed from the OS >> automaticall=? Isee some paging. Wouldn't paging slow down the querying? > >> >> My index is 10gb and every 8 hours we get most of it in shared memory. The >> m=mory is 99 percent used, and that does not leave any room for other > apps. = > >> Other implications? >> >> Sent from my mobile device >> 720-256-8076 >> >> On Jul 19, 2012, at 9:49 A... >> H=ap space or free system RAM: > >> > >> > http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.htm >> > l >> > >> > Uwe >> >... >> >> use i= since you might run out of memory on large indexes right? > >> >> >> >> Here is how I got iSimpleFSDirectoryFactory to work. Just set - >> >> Dsolr.directoryFactor... >> >> set it=all up with a helper in solrconfig.xml... > >> >> >> >> if (Constants.WINDOWS) { >> >> if (MMapDirectory.UNMAP_SUPPORTED && Constants.JRE_IS_64... -- Lance Norskog goks...@gmail.com
