Thanks. Big transactions were indeed problematic. Splitting them down into smaller chunks did the trick.
I'm still disappointed by the on-disk size of a minimal node without any relationships or attributes. For 500K nodes, it is taking 80MB space (160 byes/node) and for 1M objects it is consuming 160MB (again 160 byes/node). Is this normal? 4.0K active_tx_log 12K lucene 12K lucene-fulltext 4.0K neostore 4.0K neostore.id 4.4M neostore.nodestore.db 4.0K neostore.nodestore.db.id 12M neostore.propertystore.db 4.0K neostore.propertystore.db.arrays 4.0K neostore.propertystore.db.arrays.id 4.0K neostore.propertystore.db.id 4.0K neostore.propertystore.db.index 4.0K neostore.propertystore.db.index.id 4.0K neostore.propertystore.db.index.keys 4.0K neostore.propertystore.db.index.keys.id 64M neostore.propertystore.db.strings 4.0K neostore.propertystore.db.strings.id 4.0K neostore.relationshipstore.db 4.0K neostore.relationshipstore.db.id 4.0K neostore.relationshiptypestore.db 4.0K neostore.relationshiptypestore.db.id 4.0K neostore.relationshiptypestore.db.names 4.0K neostore.relationshiptypestore.db.names.id 4.0K nioneo_logical.log.active 4.0K tm_tx_log.1 80M total On Wed, Jun 2, 2010 at 12:17 AM, Mattias Persson <matt...@neotechnology.com>wrote: > Exactly, the problem is most likely that you try to insert all your > stuff in one transaction. All data for a transaction is kept in memory > until committed so for really big transactions it can fill your entire > heap. Try to group 10k operations or so for big insertions or use the > batch inserter. > > Links: > http://wiki.neo4j.org/content/Transactions#Big_transactions > http://wiki.neo4j.org/content/Batch_Insert > > 2010/6/2, Laurent Laborde <kerdez...@gmail.com>: > > On Wed, Jun 2, 2010 at 3:50 AM, Biren Gandhi <biren.gan...@gmail.com> > wrote: > >> > >> Is there any limit on number of nodes that can be created in a neo4j > >> instance? Any other tips? > > > > I created hundreds of millions of nodes without problems, but it was > > splitted into many transaction. > > > > -- > > Laurent "ker2x" Laborde > > Sysadmin & DBA at http://www.over-blog.com/ > > _______________________________________________ > > Neo4j mailing list > > User@lists.neo4j.org > > https://lists.neo4j.org/mailman/listinfo/user > > > > > -- > Mattias Persson, [matt...@neotechnology.com] > Hacker, Neo Technology > www.neotechnology.com > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
