On Mon, Nov 9, 2015 at 12:06 PM, Alexandre Rafalovitch <arafa...@gmail.com> wrote: > Thank you Yonik. > > So I would probably advise then to "keep your indexed=true" and think > about _adding_ docValues when there is a memory pressure or when there > is clear performance issue for the ...specific... uses. > > But if we are keeping the indexed=true, then docValues=true will STILL > use at least as much memory however efficient docValues are > themselves, right? Or will something that is normally loaded and use > memory will stay unloaded in this combination scenario?
Think about it this way: for something like sorting, we need a column for fast docid->value lookup. Enabling docValues means building this column at index time. At search time, it gets memory mapped, just like most other parts of the index. The required memory is off-heap... the OS needs to keep the file in it's buffer cache for good performance. If docValues aren't enabled, this means that we need to build the column on-the-fly on-heap (i.e. FieldCache entry is built from un-inverting the indexed values). An indexed field by itself only takes up disk space, just like docValues. Of course for searches to be fast, off-heap RAM (in the form of OS buffer cache / disk cache) is still needed. -Yonik
