I have roughly read the codes of RAMDirectory. it use a list of 1024 byte arrays and many overheads. But as far as I know, using MMapDirectory, I can't prevent the page faults. OS will swap less frequent pages out. Even if I allocate enough memory for JVM, I can guarantee all the files in the directory are in memory. am I understanding right? if it is, then some less frequent queries will be slow. How can I let them always in memory?
On Fri, Jun 8, 2012 at 5:53 PM, Lance Norskog <goks...@gmail.com> wrote: > Yes, use MMapDirectory. It is faster and uses memory more efficiently > than RAMDirectory. This sounds wrong, but it is true. With > RAMDirectory, Java has to work harder doing garbage collection. > > On Fri, Jun 8, 2012 at 1:30 AM, Li Li <fancye...@gmail.com> wrote: >> hi all >> I want to use lucene 3.6 providing searching service. my data is >> not very large, raw data is less that 1GB and I want to use load all >> indexes into memory. also I need save all indexes into disk >> persistently. >> I originally want to use RAMDirectory. But when I read its javadoc. >> >> Warning: This class is not intended to work with huge indexes. >> Everything beyond several hundred megabytes >> will waste resources (GC cycles), because it uses an internal buffer >> size of 1024 bytes, producing millions of byte >> [1024] arrays. This class is optimized for small memory-resident >> indexes. It also has bad concurrency on >> multithreaded environments. >> It is recommended to materialize large indexes on disk and use >> MMapDirectory, which is a high-performance >> directory implementation working directly on the file system cache of >> the operating system, so copying data to >> Java heap space is not useful. >> >> should I use MMapDirectory? it seems another contrib instantiated. >> anyone test it with RAMDirectory? > > > > -- > Lance Norskog > goks...@gmail.com
