Hi,- it would be nice to create a Lucene index in files and then effectively load it into memory once (since i use in read-only mode). I am looking into if this is doable in Lucene.
i wish there were an option to load whole Lucene index into memory: Both of below urls have links to the blog url where i quoted a very nice section: https://lucene.apache.org/core/8_5_0/core/org/apache/lucene/store/MMapDirectory.html https://lucene.apache.org/core/8_5_2/core/org/apache/lucene/store/MMapDirectory.html This following blog mentions about such option to run in the memory: (see the underlined sentence below) https://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html?m=1 MMapDirectory will not load the whole index into physical memory. Why should it do this? We just ask the operating system to map the file into address space for easy access, by no means we are requesting more. Java and the O/S optionally provide the option to try loading the whole file into RAM (if enough is available), but Lucene does not use that option (we may add this possibility in a later version). My question is: is there such an option? is the method setPreLoad for this purpose: to load all Lucene lndex into memory? I would like to use MMapDirectory and set my JVM heap to 16G or a bit less (since my index is around this much). The Lucene 8.5.2 (8.5.0 as well) javadocs say: public void setPreload(boolean preload) Set to true to ask mapped pages to be loaded into physical memory on init. The behavior is best-effort and operating system dependent. For example Lucene 4.0.0 does not have setPreLoad method. https://lucene.apache.org/core/4_0_0/core/org/apache/lucene/store/MMapDirectory.html Happy Holidays Best regards Ps. i know there is also BytesBuffersDirectory class for in memory Lucene but this requires creating Lucene Index on the fly. This is great for only such kind of Lucene indexes that can be created quickly on the fly. Ekaterina has a nice article on this BytesBuffersDirectory class: https://medium.com/@ekaterinamihailova/in-memory-search-and-autocomplete-with-lucene-8-5-f2df1bc71c36
