Note that a couple of memory issues are fixed in Lucene 2.9.3. Leaking when indexing big docs, and indolent reclamation of space from the FieldCache.
Bill Arijit Mukherjee <ariji...@gmail.com> wrote: > I've a similar problem. Although I'm not going out of memory yet, I can see > the heap constantly growing, and JProfiler says most of it is due to the > Lucene indexing. And even if I do the commit after every X transactions, > once the population is finished, the final commit is done, and the graph db > closed - the heap stays like that - almost full. An explicit gc will clean > up some part, but not fully. > > Arijit > > On 9 July 2010 17:00, Mattias Persson <matt...@neotechnology.com> wrote: > > > 2010/7/9 Marko Rodriguez <okramma...@gmail.com> > > > > > Hi, > > > > > > > Would it actually be worth something to be able to begin a transaction > > > which > > > > auto-committs stuff every X write operation, like a batch inserter mode > > > > which can be used in normal EmbeddedGraphDatabase? Kind of like: > > > > > > > > graphDb.beginTx( Mode.BATCH_INSERT ) > > > > > > > > ...so that you can start such a transaction and then just insert data > > > > without having to care about restarting it now and then? > > > > > > Thats cool! Does that already exist? In my code (like others on the list > > it > > > seems) I have a counter++ that every 20,000 inserts (some made up number > > > that is not going to throw an OutOfMemory) commits and the reopens a new > > > transaction. Sorta sux. > > > > > > > No it doesn't, I just wrote stuff which I though someone could think of as > > useful. A cool thing with just telling it to do a batch insert mode > > transaction (not the actual commit interval) is that it could look at how > > much memory it had to play around with and commit whenever it would be the > > most efficient, even having the ability to change the limit on the fly if > > the memory suddenly ran out. > > > > > > > Thanks, > > > Marko. > > > > > > _______________________________________________ > > > Neo4j mailing list > > > User@lists.neo4j.org > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > > -- > > Mattias Persson, [matt...@neotechnology.com] > > Hacker, Neo Technology > > www.neotechnology.com > > _______________________________________________ > > Neo4j mailing list > > User@lists.neo4j.org > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > -- > "And when the night is cloudy, > There is still a light that shines on me, > Shine on until tomorrow, let it be." > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
