mark harwood wrote:
Thanks, I have a heap dump now from a run with reduced JVM memory
(in order to speed up a failure point) and am working through it
offline with VisualVm.
This test induced a proper OOM as opposed to one of those "timed out
waiting for GC " type OOMs so may be misleading.
Hmm -- it's not good that the problem changed on you ;) You may
simply be not giving the app enough memory now?
The main culprit in this particular dump looks to be
FreqProxTermsWriter$PostingsList but the number of instances is in
line
with the volumes of terms I would have expected at that stage of
indexing.
I'll report back on my findings as I discover more.
Yeah this is expected to be a top user of RAM when you have mostly
unique terms.
I have another run soldiering on with the -XX:-UseGCOverheadLimit
setting to avoid GC -related timeouts and this has not hit OOM but
is slowing to a crawl.
I'll try capturing InfoStream too if it doesn't generate terabytes
of data.
Be careful, though: if your root cause is "GC takes too long to
run" (that "timed out waiting for GC" exception), then running with
this setting changes the game.
Ie, it's still not clear if you are running out of memory vs hitting
some weird "it's too hard for GC to deal" kind of massive heap
fragmentation situation or something. It reminds me of the special
("I cannot be played on record player X") record (your application)
that cannot be played on a given record player X (your JRE) in Gödel,
Escher, Bach ;)
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
