Just found the memory leak in another library we were using, GATE. No memory leak issues in opennlp.
Thanks. On 30 October 2013 15:26, Jörn Kottmann <[email protected]> wrote: > Looks good to me, its needs quite a bit of memory to hold all these > statistical models. > > Does your memory usage grows after processing millions of sentences? > > Jörn > > > On 10/25/2013 08:36 AM, Samarth Seksaria wrote: > >> Following is an extract from my analysis using MAT: >> >> The object of type chunking.Parser has the following memory consumption: >> >> Class Name | Shallow Heap | >> Retained Heap >> ------------------------------**------------------------------** >> ----------------------------- >> cParser opennlp.tools.parser.chunking.**Parser @ 0x79604bb08| >> 104 | >> *27,41,83,328* >> >> ------------------------------**------------------------------** >> ----------------------------- >> >> Within this object, following are the objects that have the majority of >> the >> memory accumulation: >> >> Class Name | Shallow Heap | Retained >> Heap >> ------------------------------**------------------------------** >> ------------------- >> buildModel opennlp.maxent.GISModel @ 0x79249e968| 32 | * >> 17,02,70,504* >> >> ------------------------------**------------------------------** >> ------------------- >> Class Name | Shallow Heap | >> Retained Heap >> ------------------------------**------------------------------** >> ------------------------ >> chunker opennlp.tools.chunker.**ChunkerME @ 0x79604c850| 24 | >> * >> 4,62,08,192* >> >> ------------------------------**------------------------------** >> ------------------------ >> Class Name | Shallow Heap | Retained >> Heap >> ------------------------------**------------------------------** >> ------------------- >> checkModel opennlp.maxent.GISModel @ 0x788ee2810| 32 | * >> 3,69,60,416* >> >> ------------------------------**------------------------------** >> ------------------- >> Class Name | Shallow Heap | >> Retained Heap >> ------------------------------**------------------------------** >> ------------------------ >> tagger opennlp.tools.postag.**POSTaggerME @ 0x79604c370| 48 | >> * >> 2,05,61,544* >> >> ------------------------------**------------------------------** >> ------------------------ >> >> >> >> On 25 October 2013 11:56, Samarth Seksaria >> <samarth.seksaria@germinait.**com <[email protected]>>wrote: >> >> Hi, >>> >>> I am using opennlp for generating parse trees. After quite an amount of >>> analysis using MAT ( memory analyzer tool ) on eclipse, I suspect the >>> accumulation of objects is in an instance of the chunking.Parser class. >>> >>> I may be completely wrong, and the issue may lie somewhere in my >>> application code. I am still trying to find other possibilities. Still, >>> would like to get an heads up from someone who is aware of, if any, >>> memory >>> leak issues in opennlp. >>> >>> -- >>> Thanks & Regards, >>> Samarth Seksaria >>> >>> >> >> > -- Thanks & Regards, Samarth Seksaria
