Hi,
Lucene provides different storage options for in-memory indexes. I
found three structures that would qualify for the task:
* RamDirectory (which I currently use for prototyping, but wonder if it
is the ideal choice for my task)
* MemoryIndex, which claims to have better performance and resource use
for small documents
* MMapDirectory which should outperform RamDirectory for huge indices
(what is "huge?")
My plan is to periodically index some properties (string codes, longs,
lat/lng points) of a larger database content with Lucene for quicker
lookups (compared to slow SQL queries).
What would be the most efficient (or intended) storage option for such
an index in terms of lookup speed and CPU/memory use? Below [1] is a
brief summary of the index contents and I hope these figures are
sufficient to get a recommendation. But I am also happy to study more
detailed documentation on the matter.
- Matthias
[1]: Summary of index contents and intended use
* Total documents: 500.000 - 1.000.000, may grow to 10.000.000 records
in mid future.
* Document fields (all of them single value fields):
* String (9x), usually 1-10 characters long, mostly recurring
values (5% distinct)
* LongPoint (4x), two fields contain mostly distinct values, one
lostly recurring values (5-10% distinct), one field acts as a primary
key
* LatLonPoint (1x), 30% distinct
* Refresh interval: 1..5 minutes (I currently create a fresh index
instance on each update and discard the old one)
* Most queries are range queries and exact matches on several
properties, sometimes I need to retrieve the property fields of a
single document based on a primary key value.
