Hi Mike, thanks for looking into it...
I am now positive, it was definitely a problem for OS to map() large continuous chunk of process memory... if I use this machine for a while as a desktop, eclipse,... I get the same problem again... but after cold restart, mapping succeeds. The problem never happened with 2.4, but I guess this was a pure coincidence. Cheers, eks ----- Original Message ---- > From: Michael McCandless <luc...@mikemccandless.com> > To: java-user@lucene.apache.org > Sent: Monday, 13 July, 2009 11:52:31 > Subject: Re: OOM with 2.9 > > Ahh good point. I agree it makes sense to make MMapDir's chunking > user-controllable. Can you open an issue? > > Mike > > On Mon, Jul 13, 2009 at 5:49 AM, Uwe Schindlerwrote: > >> On Sun, Jul 12, 2009 at 10:51 AM, eks devwrote: > >> > >> > MMapDirectory has support for chunking (Ineteger.MAX_VALUE) anyhow... > >> maybe for such cases this threshold can become user settable. I will try > >> to experiment with it (I am talking about MMapDirectory -> private final > >> int MAX_BBUF = Integer.MAX_VALUE;) > >> > >> This chunking doesn't reduce virtual memory usage, because the buffers > >> are all mapped up front on opening the IndexInput. > >> > >> It's only done because of a limitation of ByteBuffer (it uses int not > >> long to address the values). > > > > The idea of chunking is because of address space fragmentation. If the Java > > VM allocates or frees memory (e.g. when expanding the heap from Xms to Xmx), > > the address space of the process gets fragmented. If the index is very > > large, the byte buffer must fit in complete as one fragment into the address > > space, which may easily fail when you only have 32 bit. > > > > If the file is mapped in different chunks, each chunk can be somewhere else > > in address space (e.g., not only one 1 Gig junk is needed, there can be ten > > 100 Meg junks at different places). > > > > Uwe > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
