Le 04/06/2015 11:04, Andrew Byrd a écrit :
Hello Andrew,
Can anyone provide anecdotes of use cases where multi-threaded PBF
reading significantly speeds up processing? Generally I would expect PBF
reading to be IO bound rather than processor bound, but I still need to
make more accurate measurements.
Here are simple figures:
cp planet.osm.pbf /dev/null: 205s
planet.osm.pbf parsing(*): 629s
(*)With some optimization (memory allocations), without: 799s
If I/O capacity is not an issue, using two threads may improve things
but from my experience if you're doing something other than a simple
dump with filtering like putting OSM data into a database there will be
none to little gain (and you will take useful resources).
Of course actually processing the OSM data once the PBF is decoded can
be quite CPU intensive, but that would imply buffering decoded data and
parallelizing geometric operations for example, not the reading.
Some processing may require to wait for a specific object so the locking
between reader threads could be rather complex. At least if it's only
for buffering it's OK but then some memory will be needed...
Best regards
_______________________________________________
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
