On Thu, Apr 28, 2011 at 06:03:29PM +0200, Peter Körner wrote:
Am 28.04.2011 16:11, schrieb Oliver Tonnhofer:
On 28.04.2011, at 16:00, Martijn van Exel wrote:
Interesting, this totally uproots my firm conviction that SAX parsing is
always more time- and memory-efficient than tree/DOM-based
Serge Wroclawski emac...@gmail.com wrote:
Yes, but they are huge...
My original place=* for france is 50 MB (not compressed).
50 meg doesn't seem so big.
Yes 50MB is fine. 2 GB (compressed) is huge.
France extract if 2 GB compressed, i don't think my python script was
able to handle
On 28.04.2011, at 13:57, Pierre-Alain Dorange wrote:
What kind of processing were you doing? I'm assuming some kind of XML
(unless you were using the protobuffs). If thats the case, you can use
a simple SAX parser. If you need an example, I'll send you one.
Yes i preprocess XML to extract
On 28.04.2011, at 14:46, Serge Wroclawski wrote:
On Thu, Apr 28, 2011 at 8:30 AM, Oliver Tonnhofer o...@omniscale.de wrote:
If you are using Python, you should use iterparse from the ElementTree API,
it is much nicer and faster.
Faster than SAX? I'm not so sure. SAX calls the C
On 28.04.2011, at 16:00, Martijn van Exel wrote:
Interesting, this totally uproots my firm conviction that SAX parsing is
always more time- and memory-efficient than tree/DOM-based parsing.
Sure. DOM based parser need to keep the whole tree in memory which doesn't work
well with large XML
Am 28.04.2011 16:11, schrieb Oliver Tonnhofer:
On 28.04.2011, at 16:00, Martijn van Exel wrote:
Interesting, this totally uproots my firm conviction that SAX parsing is always
more time- and memory-efficient than tree/DOM-based parsing.
From my experience the expat-sax immplementation is
Hi,
Pierre-Alain Dorange wrote:
I finally build my proto and it works fine, but i'm disappointed : it's
slow...
curl http://download.geofabrik.de/osm/europe/france.osm.pbf; \
-o ./temp/france.osm.pbf
osmosis --rb ./temp/france.osm.pbf \
--tf accept-nodes man_made=surveillance
On 26. 04. 11 22:02, Pierre-Alain Dorange wrote:
Pierre-Alain Dorangepdora...@mac.com wrote:
./osmosis --rb france.osm.pbf -tf accept-nodes place=* --wx
fr_places.osm
You're missing the second - before tf, making Osmosis think that
-tf was a second argument to --rb.
Oh my god, shame on me.
Hi,
On 26.04.2011 22:29, Frederik Ramm wrote:
You could probably save some time by doing something like this
(shortened by dropping arguments):
curl ... -o - | osmosis --rb /dev/stdin --tee 3 --tf --tf -tf --wx --tf
--tf --un --wx --tf --un --uw --wx
that way you would start processing while
Hi,
Igor Podolskiy wrote:
osmosis --rb /dev/stdin --tee 3 --b --b --b --tf --tf -tf --wx --tf --tf
--un --wx --tf --un --uw --merge -merge --wx
(Though if I'd be doing this for real, I'd spell out the at least the
out/in-pipe names,
You would have to. The intermediate --wx in your command
Toby Murray toby.mur...@gmail.com wrote:
osmosis --read-pbf france.osm.pbf \
--tf accept-ways place=* \
--tf accept-nodes place=*
--tf accept-relations place=*
--write-xml france_place.osm
After some tests i cant run osmosis...
My config
Hi,
Pierre-Alain Dorange wrote:
./osmosis --rb france.osm.pbf -tf accept-nodes place=* --wx
fr_places.osm
You're missing the second - before tf, making Osmosis think that
-tf was a second argument to --rb.
Bye
Frederik
--
Frederik Ramm ## eMail frede...@remote.org ## N49°00'09
Frederik Ramm frede...@remote.org wrote:
./osmosis --rb france.osm.pbf -tf accept-nodes place=* --wx
fr_places.osm
You're missing the second - before tf, making Osmosis think that
-tf was a second argument to --rb.
Oh my god, shame on me.
Thanks.
--
Pierre-Alain Dorange
OSM
On Sunday 24 April 2011 02:40:52 Nakor Osm wrote:
Download the planet.pbf and extract from it.
From a pbf planet file on a quad-core/8Gb RAM machine it takes me roughly
1.5 h to extract all ways marked natural=coastline.
If you need updates more often than the weekly PBF, it takes about 2
Cartinus carti...@xs4all.nl wrote:
If you need updates more often than the weekly PBF, it takes about 2 hours
on the same machine as above to apply the daily diffs with Osmosis and if
you do not need a daily update just merge several daily diffs together
before updating your planet file
On Sun, Apr 24, 2011 at 6:43 AM, Pierre-Alain Dorange pdora...@mac.com wrote:
He is talking about France only. So if he used a country extract [1] in stead
of the whole planet, then his processing times would be much shorter for both
the methods mentioned bij Toby and Nakor.
[1]
On 24.04.2011, at 12:43, Pierre-Alain Dorange wrote:
Yes, but they are huge...
My original place=* for france is 50 MB (not compressed).
France extract if 2 GB compressed, i don't think my python script was
able to handle such a huge data.
You should have a look at imposm.parser, it's a
On Sun, Apr 24, 2011 at 5:43 AM, Pierre-Alain Dorange pdora...@mac.com wrote:
Yes, but they are huge...
My original place=* for france is 50 MB (not compressed).
France extract if 2 GB compressed, i don't think my python script was
able to handle such a huge data.
That's why you use osmosis
Toby Murray toby.mur...@gmail.com wrote:
France extract if 2 GB compressed, i don't think my python script was
able to handle such a huge data.
That's why you use osmosis to trim it down to what you want.
The following osmosis command gives me a 34 MB XML file when run on
the France
Hi Pierre,
As far as XAPI servers go I have given up for now, although MapQuest is
running a more reliable instance[1] that may be useful, although they employ
Ian's Java implementation as well so the same bbox limitations likely apply.
I am, in fact, working on a similar project in the
Hi all,
I already replied Pierre-Alain on the french list, but it maybe useful
for others too.
For an extract of cross-country trails at [1], I use a world planet file
updated by osmosis as a file with daily diffs whenever I feel like too.
Find the script attached.
Then Osmfilter [2] do the
Hi, i'm a french contributor and i look for advice for a project.
I'm processing free french city official data (from insee, official stat
agency) and i have to match those city with osm city (place=*) to
geolocalize it on a map.
The project is to present cctv surveillance in the country (see
You can set up your own jXAPI server and disable the bounding box
limit. But this takes about 500 GB of drive space and several days of
processing.
Another option is to use osmosis to filter objects out the planet file:
Download the planet.pbf and extract from it.
From a pbf planet file on a quad-core/8Gb RAM machine it takes me roughly
1.5 h to extract all ways marked natural=coastline.
If you need updates more often than the weekly PBF, it takes about 2 hours
on the same machine as above to apply the daily
24 matches
Mail list logo
