Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-30 Thread 'Curtis Mosters' via Neo4j
Well the data is there. I does not matter what is in there. And no I have no index or anything, just testing the row mass insertion. Ussing a good SSD. Well the RAM could be a reason, will test this soon. Am Freitag, 29. August 2014 23:57:12 UTC+2 schrieb Michael Hunger: > > Cypher won't be fast

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-29 Thread Michael Hunger
Cypher won't be faster than the Java API, only the batch-inserter is faster than the java API Cypher is just more convenient. Do you have any long property strings? Most of the time is spent in the label store and in the index updates, if you run the same without labels and indexes it will be f

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-29 Thread 'Curtis Mosters' via Neo4j
neo4j-sh (?)$ USING PERIODIC COMMIT 1 > LOAD CSV WITH HEADERS FROM "file:///C:/test/tls206_part01.txt" AS csvLine WITH csvLine LIMIT 10 > CREATE (p:Person { person_id: toInt(csvLine.person_id), doc_std_name_id: csvLi ne.doc_std_name_id , person_name: csvLine.person_name }); +

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-29 Thread 'Curtis Mosters' via Neo4j
Ohh didn't know that this was a command. Sorry. Could you maybe let it be code next time. So it's much easier too see. Will test it this evening. Thanks. Am Donnerstag, 28. August 2014 22:15:55 UTC+2 schrieb Michael Hunger: > > did you see the _WITH csvLine_ before the limit? > > Am 28.08.2014 u

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread Michael Hunger
did you see the _WITH csvLine_ before the limit? Am 28.08.2014 um 19:03 schrieb 'Curtis Mosters' via Neo4j : > C:\Program Files\Neo4j Community\neo4j-community-2.1.3\bin>Neo4jShell.bat > start > Welcome to the Neo4j Shell! Enter 'help' for a list of commands > NOTE: Remote Neo4j graph database

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread 'Curtis Mosters' via Neo4j
C:\Program Files\Neo4j Community\neo4j-community-2.1.3\bin>Neo4jShell.bat start Welcome to the Neo4j Shell! Enter 'help' for a list of commands NOTE: Remote Neo4j graph database service 'shell' at port 1337 neo4j-sh (?)$ USING PERIODIC COMMIT 1 > LOAD CSV WITH HEADERS FROM "file:///C:/test/tl

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread Michael Hunger
What happen in your shell? Do you have the index? can you try >> USING PERIODIC COMMIT 1 >> LOAD CSV WITH HEADERS FROM "file:C:/test/tls206_part01.txt" AS csvLine WITH csvLine LIMIT 10 >> CREATE (p:Person { person_id: toInt(csvLine.person_id), doc_std_name_id: >> csvLine.doc_std_name_id , pe

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread 'Curtis Mosters' via Neo4j
Already tried this out. Does not work in my Shell. Am Donnerstag, 28. August 2014 17:50:24 UTC+2 schrieb david fauth: > > No semi-colon after the first line: > > > USING PERIODIC COMMIT 1 > LOAD CSV WITH HEADERS FROM "file:C:/test/tls206_part01.txt" AS csvLine > CREATE (p:Person { person_id: t

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread david fauth
No semi-colon after the first line: USING PERIODIC COMMIT 1 LOAD CSV WITH HEADERS FROM "file:C:/test/tls206_part01.txt" AS csvLine CREATE (p:Person { person_id: toInt(csvLine.person_id), doc_std_name_id: csvLine.doc_std_name_id , person_name: csvLine.person_name}); On Thursday, August 28, 2

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread 'Curtis Mosters' via Neo4j
As you see in all lines above there is actually already a comma? What do you mean with comma after CREATE, Michael? Am Donnerstag, 28. August 2014 16:11:13 UTC+2 schrieb Michael Hunger: > > Nope no semicolon after 1 > > Only after the create > > Sent from mobile device > > Am 28.08.2014 um 16

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread Michael Hunger
Nope no semicolon after 1 Only after the create Sent from mobile device Am 28.08.2014 um 16:01 schrieb "'Curtis Mosters' via Neo4j" : > That was it. Now I have: > > USING PERIODIC COMMIT 1; > LOAD CSV WITH HEADERS FROM "file:C:/test/tls206_part01.txt" AS csvLine > CREATE (p:Person { p

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread 'Curtis Mosters' via Neo4j
That was it. Now I have: USING PERIODIC COMMIT 1; LOAD CSV WITH HEADERS FROM "file:C:/test/tls206_part01.txt" AS csvLine CREATE (p:Person { person_id: toInt(csvLine.person_id), doc_std_name_id: csvLine.doc_std_name_id , person_name: csvLine.person_name }); Well it seems to do something. My C

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread Rik Van Bruggen
Did you forget a semi-colon at the end? In the shell all commands need to end with ; ... Rik On Thursday, August 28, 2014 2:04:22 PM UTC+1, Curtis Mosters wrote: > > Well it works under Browser Cypher. So it should be fin. But I changed it: > > USING PERIODIC COMMIT 1 > LOAD CSV WITH HEADERS

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread 'Curtis Mosters' via Neo4j
Well it works under Browser Cypher. So it should be fin. But I changed it: USING PERIODIC COMMIT 1 LOAD CSV WITH HEADERS FROM "file:///C:/test/tls206_part01.txt" AS csvLine CREATE (p:Person { person_id: toInt(csvLine.person_id), doc_std_name_id: csvLine.doc_std_name_id , person_name: csvLine.

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread Aru Sahni
Your file URI seems to be malformed. Here's the correct syntax for Windows: http://blogs.msdn.com/b/ie/archive/2006/12/06/file-uris-in-windows.aspx ~A On Thu, Aug 28, 2014 at 8:34 AM, 'Curtis Mosters' via Neo4j < neo4j@googlegroups.com> wrote: > Well I will definitily test this. But I don't kno

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread 'Curtis Mosters' via Neo4j
Well I will definitily test this. But I don't know how to start this command with the shell: http://i.imgur.com/uiYbq0O.png Any tips for windows user? Nothing happening. Am Donnerstag, 28. August 2014 11:39:59 UTC+2 schrieb Michael Hunger: > > Hi Curtis, > > if you do this: > > USING PERIODIC C

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread Michael Hunger
Hi Curtis, if you do this: USING PERIODIC COMMIT 1 LOAD CSV WITH HEADERS FROM "file:C:/test.txt" AS csvLine CREATE (:Person { person_id: toInt(csvLine.person_id), name: csvLine.name }); It should do between 10k-30k nodes per second. It will be slower if you have a unique constraint in place.

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-28 Thread 'Curtis Mosters' via Neo4j
Yesterday evening I was running it 52 minutes. And then I got an "Unknown Error". So now I tested it now with the "USING PERIODIC COMMIT 1" and now its about 60 minutes. Then the same error. I now looked into the graph.db folder and its overall 1,75 GB big. The propertystore file ist 370 m

Re: [Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-27 Thread Chris Vest
All transaction state is currently kept in memory on the java heap, and 20+ mio. changes is too much to fit in a 4 GB heap. When you have too much stuff on the heap, it will manifest with those "GC overhead limit exceeded" and the database will run slow, though there are other things that can pr

[Neo4j] Speed up CSV import Cypher process as best as possible

2014-08-27 Thread 'Curtis Mosters' via Neo4j
Let's say I have: LOAD CSV WITH HEADERS FROM "file:C:/test.txt" AS csvLine CREATE (p:Person { person_id: toInt(csvLine.person_id), name: csvLine.name }) I run this query in the browser. I know that it's not the fastest way and I should think about using the batch importer. But I really like tha