On Thu, 2013-09-12 at 14:48 +0200, Per Steffensen wrote: > Actually some months back I made PoC of a FieldCache that could expand > beyond the heap. Basically imagine a FieldCache with room for > "unlimited" data-arrays, that just behind the scenes goes to > memory-mapped files when there is no more room on heap.
That sounds a lot like disk-based DocValues. [...] > But that solution will also have the "running out of swap space"-problems. Not really. Memory mapping works like the disk cache: There is no requirement that a certain amount of physical memory needs to be available, it just takes what it can get. If there are not a lot of physical memory, it will require a lot of storage access, but it will not over-allocate swap space. It seems that different setups vary quite a lot in this area and some systems are prone to aggressive use of the swap file, which can severely harm responsiveness of applications with out-swapped data. However, this should still not result in any OOM's, as the system can always discard some of the memory mapped data if it needs more physical memory. - Toke Eskildsen, State and University Library, Denmark
