Can you check neo4j.com/download? Von meinem iPhone gesendet
> Am 22.09.2014 um 06:48 schrieb 'Curtis Mosters' via Neo4j > <[email protected]>: > > Sure Michael! I'll be available then. > > Well but there are more things broken. And I still have the 2.1.3 version > because I'm not able to download the 2.1.4 via > http://www.neo4j.org/download/other_versions > I always get the download page but not any download. > > So could you please fix the download and then I can try the separeted import. > Maybe that's it already! Would be great. > > Am Sonntag, 21. September 2014 21:22:46 UTC+2 schrieb Michael Hunger: >> >> Ok, it's the two create statements in a row if you split it up it will not >> happen: >> >> // !! Eager Loading kills your import, everything is pulled from the file >> into memory after the first create! >> >> profile >> LOAD CSV >> WITH HEADERS FROM >> "https://gist.githubusercontent.com/jexp/909e92c65e715ea91062/raw/cbe617cdf551ef51111fff349f1b1826a7a52d3d/appln.txt"; >> AS csvLine >> WITH csvLine >> LIMIT 1 >> MATCH (appln:Appln) >> WHERE appln.ID = toInt(csvLine.appln_id) >> CREATE (title:Title { title:csvLine.appln_title }) >> CREATE (appln)-[:HAS_TITLE]->(title); >> >> +-------------+------+--------+---------------------+-------------------------------------------------------+ >> | Operator | Rows | DbHits | Identifiers | >> Other | >> +-------------+------+--------+---------------------+-------------------------------------------------------+ >> | EmptyResult | 0 | 0 | | >> | >> | UpdateGraph | 0 | 0 | title, UNNAMED304 | >> CreateNode; CreateRelationship | >> | Eager | 0 | 0 | | >> | >> | SchemaIndex | 0 | 1 | appln, appln | >> ToIntFunction(Property(csvLine,appln_id)); :Appln(ID) | >> | Slice | 1 | 0 | | >> { AUTOINT0} | >> | LoadCSV | 1 | 0 | csvLine | >> | >> +-------------+------+--------+---------------------+-------------------------------------------------------+ >> >> >> // Do it like this: >> >> CREATE INDEX ON :Appln(ID); >> CREATE INDEX ON :Title(title); >> >> profile >> LOAD CSV >> WITH HEADERS FROM >> "https://gist.githubusercontent.com/jexp/909e92c65e715ea91062/raw/cbe617cdf551ef51111fff349f1b1826a7a52d3d/appln.txt"; >> AS csvLine >> WITH csvLine >> LIMIT 1 >> CREATE (title:Title { title:csvLine.appln_title }); >> >> +-------------+------+--------+-------------+--------------+ >> | Operator | Rows | DbHits | Identifiers | Other | >> +-------------+------+--------+-------------+--------------+ >> | EmptyResult | 0 | 0 | | | >> | UpdateGraph | 1 | 5 | title | CreateNode | >> | Slice | 1 | 0 | | { AUTOINT0} | >> | LoadCSV | 1 | 0 | csvLine | | >> +-------------+------+--------+-------------+--------------+ >> >> profile >> LOAD CSV >> WITH HEADERS FROM >> "https://gist.githubusercontent.com/jexp/909e92c65e715ea91062/raw/cbe617cdf551ef51111fff349f1b1826a7a52d3d/appln.txt"; >> AS csvLine >> WITH csvLine >> LIMIT 1 >> MATCH (appln:Appln {ID: csvLine.appln_id}) >> MATCH (title:Title { title:csvLine.appln_title }) >> CREATE (appln)-[:HAS_TITLE]->(title); >> >> +----------------+------+--------+--------------+----------------------------------------------+ >> | Operator | Rows | DbHits | Identifiers | >> Other | >> +----------------+------+--------+--------------+----------------------------------------------+ >> | EmptyResult | 0 | 0 | | >> | >> | UpdateGraph | 0 | 0 | UNNAMED285 | >> CreateRelationship | >> | SchemaIndex(0) | 0 | 0 | title, title | >> Property(csvLine,appln_title); :Title(title) | >> | SchemaIndex(1) | 0 | 1 | appln, appln | >> Property(csvLine,appln_id); :Appln(ID) | >> | Slice | 1 | 0 | | >> { AUTOINT0} | >> | LoadCSV | 1 | 0 | csvLine | >> | >> +----------------+------+--------+--------------+----------------------------------------------+ >> >> >>> On Sun, Sep 21, 2014 at 9:03 PM, Michael Hunger >>> <[email protected]> wrote: >>> Ah, cool, then we can do a call this week, perhaps tomorrow or on Thu. >>> >>> That's really weird. It shouldn't build up memory in 2.1.4 as the team >>> changed the eagerness requirement for queries where we determine >>> independence. >>> >>> Can you do a a limit 10 and prefix your query with profile? There should be >>> no "Eager"-pipe in between the match and merge. >>> >>> >>>> On Sun, Sep 21, 2014 at 8:31 PM, 'Curtis Mosters' via Neo4j >>>> <[email protected]> wrote: >>>> Sure y. And testing with LIMIT 1000000 works and needs 1 min or so. >>>> >>>> So this is just a Heap Stack thingi. >>>> >>>> I'm from Germany like you ;) >>>> >>>> Am Sonntag, 21. September 2014 19:25:15 UTC+2 schrieb Michael Hunger: >>>>> >>>>> In what Timezone are you located? Perhaps someone can help you and have a >>>>> look at your setup? via Skype >>>>> >>>>> But you have an index or constraint on the thing you match? >>>>> >>>>>> CREATE INDEX ON :Appln(ID); >>>>> >>>>> >>>>>> On Sun, Sep 21, 2014 at 4:08 PM, 'Curtis Mosters' via Neo4j >>>>>> <[email protected]> wrote: >>>>>> Jup, same issue with -Xmx512M >>>>>> >>>>>> So you have any other tips? Observing the IO values, it seems to idle =/ >>>>>> >>>>>> Am Samstag, 20. September 2014 12:58:33 UTC+2 schrieb Curtis Mosters: >>>>>>> Well I tried it with >>>>>>> >>>>>>> -Xms4096m -Xmx15000m >>>>>>> >>>>>>> but the import did not finish in now 16 hours. >>>>>>> >>>>>>> Now I try your >>>>>>> >>>>>>> -Xmx512M >>>>>>> >>>>>>> Am Freitag, 19. September 2014 16:08:42 UTC+2 schrieb Michael Hunger: >>>>>>>> >>>>>>>> If you specifiy it I'd keep it to a few MB (e.g. -Xmx512M) as the >>>>>>>> shell against a running server is just a terminal. >>>>>>>> It's different if when you run the shell against a on-disk directory >>>>>>>> then it fires up the database itself. >>>>>>>> >>>>>>>> Your memory config looks ok otherwise, I'd only swap mapped memory >>>>>>>> settings for nodes (to 50MB) and rels (to 500MB) >>>>>>>> >>>>>>>> I think there was one memory issue in 2.1.3 could you try it with >>>>>>>> 2.1.4 ? >>>>>>>> >>>>>>>> Michael >>>>>>>> >>>>>>>> >>>>>>>>> On Fri, Sep 19, 2014 at 4:00 PM, 'Curtis Mosters' via Neo4j >>>>>>>>> <[email protected]> wrote: >>>>>>>>> Here is the complete file, well I still don't know which settings are >>>>>>>>> the best in my case. >>>>>>>>> >>>>>>>>> Should I delete the Xmx thing? Is it better to not specify it? >>>>>>>>> >>>>>>>>> Am Donnerstag, 18. September 2014 23:14:09 UTC+2 schrieb Michael >>>>>>>>> Hunger: >>>>>>>>>> >>>>>>>>>> If you connect to the server it uses the servers RAM >>>>>>>>>> The shell then needs much less like 512m >>>>>>>>>> >>>>>>>>>> Can you check the server's graph.db/messages.log for the root cause >>>>>>>>>> error? >>>>>>>>>> >>>>>>>>>> Which version? >>>>>>>>>> >>>>>>>>>> Sent from mobile device >>>>>>>>>> >>>>>>>>>>> Am 18.09.2014 um 14:28 schrieb "'Curtis Mosters' via Neo4j" >>>>>>>>>>> <[email protected]>: >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Well I was thinking that the Xmx is the heap and it should be not >>>>>>>>>>> above 4 GB, so far I have learned. >>>>>>>>>>> >>>>>>>>>>> I currently let the import run with "COMMIT 100" so every 100 >>>>>>>>>>> lines, and it's still running. >>>>>>>>>>> >>>>>>>>>>> But looking on the IO it seems to idle... >>>>>>>>>>> >>>>>>>>>>> I don't know, but could you tell me exactly where to set which >>>>>>>>>>> setting? >>>>>>>>>>> >>>>>>>>>>> As I said I have 30 GB RAM. To avoid more complications let's maybe >>>>>>>>>>> set the Heap 4 GB and the maximum RAM to 20 GB. >>>>>>>>>>> >>>>>>>>>>> And again I use the Neo4jShell for the import. It's loading a file >>>>>>>>>>> containing LOAD CSV. >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> Am Donnerstag, 18. September 2014 13:42:34 UTC+2 schrieb Clark >>>>>>>>>>> Richey: >>>>>>>>>>>> >>>>>>>>>>>> According to the screenshot you don't give it 30G of ram. You only >>>>>>>>>>>> provide 3G. Is that your intent? >>>>>>>>>>>> >>>>>>>>>>>> Sent from my iPhone >>>>>>>>>>>> >>>>>>>>>>>>> On Sep 18, 2014, at 02:51, 'Curtis Mosters' via Neo4j >>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Well somehow the behaviour of the Shell is odd. >>>>>>>>>>>>> >>>>>>>>>>>>> I imported 76 Mio lines without any problems. But now I have >>>>>>>>>>>>> started a new import over night and now I watched the process and >>>>>>>>>>>>> it said Java Heap Space Error. >>>>>>>>>>>>> >>>>>>>>>>>>> But that cannot be. I have set the Neo4jShell.bat like this: >>>>>>>>>>>>> >>>>>>>>>>>>> %JAVACMD% -Xmx3G -Dstorage.diskCache.bufferSize=20000 %JAVA_OPTS% >>>>>>>>>>>>> %EXTRA_JVM_ARGUMENTS% -classpath %CLASSPATH_PREFIX%;%CLASSPATH% >>>>>>>>>>>>> -Dapp.name="neo4j-shell" -Dapp.repo="%REPO%" >>>>>>>>>>>>> -Dbasedir="%BASEDIR%" org.neo4j.shell.StartClient %CMD_LINE_ARGS% >>>>>>>>>>>>> >>>>>>>>>>>>> That is a great ratio of heap. But yeah why did I get this error? >>>>>>>>>>>>> I have 30 GB RAM. This happens so random that a productive import >>>>>>>>>>>>> is nearly impossible. >>>>>>>>>>>>> >>>>>>>>>>>>> So my question is, what have I done wrong? Where do I need to set >>>>>>>>>>>>> the Java Settings correctly? Were my edited settings active? >>>>>>>>>>>>> >>>>>>>>>>>>> Here the first file or let's say script i'm loading in and where >>>>>>>>>>>>> this error happens: >>>>>>>>>>>>> >>>>>>>>>>>>> USING PERIODIC COMMIT 2000 >>>>>>>>>>>>> LOAD CSV WITH HEADERS FROM "file:///C:/data/tls202_part01.txt" AS >>>>>>>>>>>>> csvLine WITH csvLine LIMIT 1000 >>>>>>>>>>>>> MATCH (appln:Appln) >>>>>>>>>>>>> WHERE appln.ID = toInt(csvLine.appln_id) >>>>>>>>>>>>> CREATE (title:Title { title:csvLine.appln_title }) >>>>>>>>>>>>> CREATE (appln)-[:HAS_TITLE]->(title); >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "Neo4j" group. >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>>>>> send an email to [email protected]. >>>>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>>> Groups "Neo4j" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>>> send an email to [email protected]. >>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Neo4j" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Neo4j" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "Neo4j" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
