Hi, Thank you for sharing the blog.I am using FSDirectory.open() in my program.So, I guess I am using MMapDirectory. It takes about 3 minutes when I search for a key(which is actually present in 80% of total data) in all the fields(1000) in this 1 million documents.
Best Regards, Sreedevi S On Thu, Feb 5, 2015 at 3:20 PM, Uwe Schindler <u...@thetaphi.de> wrote: > Hi, > > If you use FSDirectory.open() it will automatically choose MMapDirectory > on 64 bit systems. Please note, virtual memory is != physical RAM. A 64 bit > machine has *always* >1 Terabyte of virtual address space available, this > is unrelated to physical memory (a common misunderstanding about mmap): > http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html > > The speed difference depends on use case: In general MMapDirectory is much > faster in multi-threaded environments, because no concurrency problems. If > you use SimpleFSDirectory this is the largest bootleneck. NIOFSDirectory > does not have concurrency problems, but it is still slower because it does > a lot of extra copying of data between kernel space and user space for > buffering. MMapDirectory is muuuuuuuch faster if you sort by docvalues > fields, because it supports random access without any buffering overhead. > > So please: Use MMapDirectory where possible - this is completely unrelated > to how much RAM you have available! > > Uwe > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > -----Original Message----- > > From: sreedevi s [mailto:sreedevi.payik...@gmail.com] > > Sent: Thursday, February 05, 2015 10:13 AM > > To: java-user@lucene.apache.org > > Subject: MMapDirectory or FSDirectory > > > > Hi, > > I am doing some performance analysis with lucene. I have 1 million > resources > > with 1000 attributes. > > According to how I index, I will have 1 million documents with 1000 > fields. > > For me the total data was about 100 GB and while using FSDirectory to > store > > my indices, index size was almost 6 GB. > > I have virtual memory available of almost 8 GB. Is it advised to use > > MMapDirectory for increased performance? > > Many blogs suggest it doesnt bring out much performance difference. > > > > > > Best Regards, > > Sreedevi S > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
