Neo4j unfortunately keeps all transaction state on the heap, so for large ingestions it is recommended to either do it in batches (but that makes cross-batch relationships harder), or actually have a sufficiently large heap (but you might not have the hardware), or use the BatchInserter (but then you have to write Java code to make it go, and take the server offline when you run it).
In 2.1.0-M1, we added PERIODIC COMMIT as a band-aid to this problem, when using the new LOAD CSV statement, but of course that doesn't help you if you want to use a production-ready release. If you go for the large heap, then I recommend saving enough memory for the operating system such that the JVM process will never need to use swap memory. -- Chris Vest System Engineer, Neo Technology [ skype: mr.chrisvest, twitter: chvest ] On 25 Feb 2014, at 08:23, Nigel Small <ni...@nigelsmall.com> wrote: > It may be that you're hitting a memory issue - I've certainly never thrown > anything that big at it. Have you tried increasing the size of your JVM? > > > On 25 February 2014 00:49, <t.kluse...@gmail.com> wrote: > Hey guys, I'm trying to import a rather large dataset in the Geoff format > into Neo4j. The file containing the graph data is about 1.3 GB and defines > more than 4 million nodes and 8 million relationships. I'm able to import > files that are about 1 MB in size using load2neo. However, for the large > file, the POST request to the load2neo server extension produces a 500 server > error. As far as I know, I can't split the file into smaller chunks because > relationships and the nodes comprising them have to be defined in the same > file for load2neo. I would appreciate your suggestions on dealing with this > issue. > > Thank you. > > -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to neo4j+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to neo4j+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
