Hi Patrick,
Just wanted to ping you off-list. I just had a question about your
update process....how often if at all do you update your PBF data, and
how do you manage the process? We're thinking of just setting a cron job
to download and extract the new PBF every weekend to a temporary folder,
stop the server, move the new data to the correct location and start the
server again.
Kind regards,
Kieran Caplice
On 02/03/16 17:01, Patrick Niklaus wrote:
Hey Kieran,
there have been a lot of structural changes (e.g. moving code from
osrm-prepare into osrm-extract) that probably invalidate that numbers.
Also we support 64bit OSM ids now, which sadly uses a lot more disk
space. I think stxxl need like 200GB. I think on our setup we have a
turn-around of 6 hours for the planet dataset on an SSD setup (car
profile, any other profile needs significantly longer). You should
probably think about updating your hard drives as this is IO bound. At
your current read/write speed it will already take more than an hour
to just write 200GB of data once. We scan it at least twice just for
pre-processing.
Cheers,
Patrick
On Wed, Mar 2, 2016 at 5:51 PM, Kieran Caplice
<kieran.capl...@temetra.com> wrote:
Hello,
I'm currently extracting the planet PBF (~31 GB), and it's been running for
hours. I notice in the "Running OSRM" wiki page, it says " On a Core i7 with
8GB RAM and (slow) 5400 RPM Samsung SATA hard disks it took about 65 minutes
to do so from a PBF formatted planet", which is making me wonder why it's
taking so long on our server. Below are some example output messages:
[info] Parsing finished after 3584.35 seconds
[extractor] Erasing duplicate nodes ... ok, after 319.091s
[extractor] Sorting all nodes ... ok, after 3632.87s
[extractor] Building node id map ... ok, after 2025.29s
[extractor] Confirming/Writing used nodes ... ok, after 1096.24s
[extractor] Sorting edges by start ... ok, after 2000.08s
Some stxxl errors were outputted as I set the disk size to 100GB thinking it
was enough - but I didn't think it would cause such slowdowns as this,
considering extracting the Europe PBF takes hours also without the stxxl
errors.
Server specs:
Ubuntu 14.04
Intel Xeon CPU E5-1650 v3 @ 3.50GHz (hex-core with HT)
64 GB RAM @ 2133 MHz
2 TB Western Digital Enterprise 7200 RPM hard drive
At the moment, disk IO is averaging around 35-40 MB/s R/W (~90%).
Anyone have any ideas as to what might be going on? Or is it normal to take
this long without an SSD?
Thanks in advance.
Kind regards,
Kieran Caplice
_______________________________________________
OSRM-talk mailing list
OSRM-talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/osrm-talk
_______________________________________________
OSRM-talk mailing list
OSRM-talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/osrm-talk
_______________________________________________
OSRM-talk mailing list
OSRM-talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/osrm-talk
