Re: [Neo4j] OSMImporter: Is there a way to do incremental imports?

2011-12-06 Thread Craig Taverner
There was only a method ending in 'WithCheck', or something like that, lying unused in the code from last year. Nothing more than that. Except for thinking about it, which is why I wrote the previous mail. On Dec 2, 2011 12:50 PM, Peter Neubauer pe...@neubauer.se wrote: Not sure, Craig, do you

Re: [Neo4j] OSMImporter: Is there a way to do incremental imports?

2011-12-02 Thread Peter Neubauer
Not sure, Craig, do you have the code somewhere? /peter On Tue, Nov 22, 2011 at 4:17 PM, grimace macegh...@gmail.com wrote: thanks for the response(s)!  The hardware I'm testing on is not the best and only 4G of ram so I'm limited, but this seems the best opportunity for me to learn

Re: [Neo4j] OSMImporter: Is there a way to do incremental imports?

2011-11-22 Thread Peter Neubauer
Gregory, incremental loads (and thus, restarts of OSM imports) are a feature we want to add later on, but it's not in there yet. This would also mean we could stitch in other areas on demand, and support submitting changesets back to OSM or at least capture them, so you as an OSM based app can

Re: [Neo4j] OSMImporter: Is there a way to do incremental imports?

2011-11-22 Thread Craig Taverner
I did some initial work on incremental imports back in 2010, but stopped due to some complications: - We needed to mix lucene reads and writes during the import (read to check if the node already exists, so we don't import twice) and this performs very badly in the batch inserter. We

Re: [Neo4j] OSMImporter: Is there a way to do incremental imports?

2011-11-22 Thread grimace
thanks for the response(s)! The hardware I'm testing on is not the best and only 4G of ram so I'm limited, but this seems the best opportunity for me to learn this...that being said... For incremental imports, stitching osm files together, we re-activate the old code that tests the lucene

[Neo4j] OSMImporter: Is there a way to do incremental imports?

2011-11-21 Thread grimace
I've been playing with OSMImporter; tried batch and native java. I've had mixed success trying to import the planet, but since it's of considerable size, the job usually blows up or grinds to a halt about half way. I think the most I've made it to is 651M nodes and that's not even the ways or