Bill: I'm not sure I follow. why would raising the JVM memory to 4GB ever cause a crash in python? Our server has 48GB.
thanks Marcus On Fri, Apr 15, 2011 at 7:33 AM, Bill Janssen <[email protected]> wrote: > Marcus <[email protected]> wrote: > > > --bcaec53043296dfbfd04a0ece1ac > > Content-Type: text/plain; charset=ISO-8859-1 > > > > we're currently using 4GB max heap. > > We recently moved from 2GB to 4GB when we discovered it prevented a crash > > with a certain set of docs. > > Marcus > > I've tried the same workaround with the heap in the past, and I found it > caused NoMemory crashes in the Python side of the house, because the > Python VM couldn't get enough memory to operate. So, be careful. > > > On Thu, Apr 14, 2011 at 5:01 PM, Andi Vajda <[email protected]> wrote: > > > > > > > > On Thu, 14 Apr 2011, Marcus wrote: > > > > > > thanks. > > >> > > >> I have documents that will consistently cause this upon writing them > to > > >> the > > >> index. let me see if I can reduce them down to the crux of the crash. > > >> granted, these are docs are very large, unruly "bad" data, that should > > >> have > > >> never gotten this stage in our pipeline, but I was hoping for a java > or > > >> lucene exception. > > >> > > >> I also get "Java GC overhead" exceptions passed into my code from time > to > > >> time, but those manageable, and not crashes. > > >> > > >> Are there known memory constraint scenarios that force a c++ > exception, > > >> whereas in a normal Java environment, you would get a memory error? > > >> > > > > > > Not sure. > > > > > > > > > and just confirming, do "java.lang.OutOfMemoryError" errors pass into > > >> python, or force a crash? > > >> > > > > > > Not sure, I've never seen these as I make sure I've got enough memory. > > > initVM() is the place where you can configure the memory for your JVM. > > > > > > Andi.. > > > > > > > > > > > >> thanks again > > >> Marcus > > >> > > >> On Thu, Apr 14, 2011 at 2:07 PM, Andi Vajda <[email protected]> wrote: > > >> > > >> > > >>> On Thu, 14 Apr 2011, Marcus wrote: > > >>> > > >>> in certain cases when a java/pylucene exception occurs, it gets > passed > > >>> up > > >>> > > >>>> in my code, and I'm able to analyze the situation. > > >>>> sometimes though, the python process just crashes, and if I happen > to > > >>>> be > > >>>> in > > >>>> top (linux top that is), I see a JCC exception flash up in the top > > >>>> console. > > >>>> where can I go to look for this exception, or is it just lost? > > >>>> I looked in the locations where a java crash would be located, but > > >>>> didn't > > >>>> find anything. > > >>>> > > >>>> > > >>> If you're hitting a crash because of an unhandled C++ exception, > running > > >>> a > > >>> debug build with symbols under gdb will help greatly in tracking it > down. > > >>> > > >>> An unhandled C++ exception would be a PyLucene/JCC bug. If you have a > > >>> simple way to reproduce this failure, send it to this list. > > >>> > > >>> Andi.. > > >>> > > >>> > > >> > > > > --bcaec53043296dfbfd04a0ece1ac-- >
