Re: [OSM-dev] XAPI and other solutions

2011-04-28 Thread Peter Körner
Am 28.04.2011 16:11, schrieb Oliver Tonnhofer: On 28.04.2011, at 16:00, Martijn van Exel wrote: Interesting, this totally uproots my firm conviction that SAX parsing is always more time- and memory-efficient than tree/DOM-based parsing. From my experience the expat-sax immplementation is fast

Re: [OSM-dev] XAPI and other solutions

2011-04-28 Thread Oliver Tonnhofer
On 28.04.2011, at 16:00, Martijn van Exel wrote: > Interesting, this totally uproots my firm conviction that SAX parsing is > always more time- and memory-efficient than tree/DOM-based parsing. Sure. DOM based parser need to keep the whole tree in memory which doesn't work well with large XML f

Re: [OSM-dev] XAPI and other solutions

2011-04-28 Thread Martijn van Exel
On 4/28/11 2:58 PM, Oliver Tonnhofer wrote: On 28.04.2011, at 14:46, Serge Wroclawski wrote: On Thu, Apr 28, 2011 at 8:30 AM, Oliver Tonnhofer wrote: If you are using Python, you should use iterparse from the ElementTree API, it is much nicer and faster. Faster than SAX? I'm not so sure.

Re: [OSM-dev] XAPI and other solutions

2011-04-28 Thread Oliver Tonnhofer
On 28.04.2011, at 14:46, Serge Wroclawski wrote: > On Thu, Apr 28, 2011 at 8:30 AM, Oliver Tonnhofer wrote: > >> If you are using Python, you should use iterparse from the ElementTree API, >> it is much nicer and faster. > > Faster than SAX? I'm not so sure. SAX calls the C directly. Here i

Re: [OSM-dev] XAPI and other solutions

2011-04-28 Thread Serge Wroclawski
On Thu, Apr 28, 2011 at 8:30 AM, Oliver Tonnhofer wrote: > If you are using Python, you should use iterparse from the ElementTree API, > it is much nicer and faster. Faster than SAX? I'm not so sure. SAX calls the C directly. The technique used in iterparse is the same as in SAX parsing, and t

Re: [OSM-dev] XAPI and other solutions

2011-04-28 Thread Oliver Tonnhofer
On 28.04.2011, at 13:57, Pierre-Alain Dorange wrote: >> What kind of processing were you doing? I'm assuming some kind of XML >> (unless you were using the protobuffs). If thats the case, you can use >> a simple SAX parser. If you need an example, I'll send you one. > > Yes i preprocess XML to ex

Re: [OSM-dev] XAPI and other solutions

2011-04-28 Thread Pierre-Alain Dorange
Serge Wroclawski wrote: > > Yes, but they are huge... > > My original place=* for france is 50 MB (not compressed). > > 50 meg doesn't seem so big. Yes 50MB is fine. 2 GB (compressed) is huge. > > France extract if 2 GB compressed, i don't think my python script was > > able to handle such a h

[Potlatch-dev] [OpenStreetMap] #3727: Cannot change role of a member of a route relation in simple mode

2011-04-28 Thread OpenStreetMap
#3727: Cannot change role of a member of a route relation in simple mode ---+ Reporter: monxton| Owner: potlatch-dev@… Type: defect | Status: new

Re: [OSM-dev] osm2pgsql slow on update import

2011-04-28 Thread Sven Geggus
Markus Wagner wrote: > After vacuuming and fiddling with those values I am much closer to > realtime. Currently at approx ~120% of realtime. So I think, there is > hope, once I get faster disks. I'm reopening this thread because I have a very simular Problem and I suspect that this could be an i