[OSRM-talk] osrm performance with multiple threads is confusing me

[Peter Becker]

I'm a little bit confused.

i have load the map in memory with osrm-datastore und run one instance with

"osrm-route --shared-memory=yes -t 1 -p 5000"

with only one thread, one cpu-core raise up tzo 90% and i get ~500
routes per second

if i set thread count to more then 2 or more

"osrm-route --shared-memory=yes -t 2 -p 5000"

3 cpu-cores are raise up to 30-50% usage and i only get ~333 routes per second.

so i also try run 2 instances with one core:

"osrm-route --shared-memory=yes -t 1 -p 5000"
"osrm-route --shared-memory=yes -t 1 -p 5001"

and use ngnix as load-balancer .. it dosn't make a difference to one
instance with 2 or more threads

what is wrong? with "-t 2" or 2 instances i expect that 2 cpu-cores
are at 100% and i get more routes per second as with "-t 1".

___
OSRM-talk mailing list
OSRM-talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/osrm-talk

Re: [OSRM-talk] How to disable all ferry routes?

[Julien Coupey]

Hi

As I'm curious about this thread, I gave the proposed solution a try. On 
the same osm.pbf data for France, I compared the default car profile 
with the same profile modified with mode_ferry = 0 on line 151.


The modified profile does prevent routing through some ways tagged as 
"route": "ferry" (e.g. http://www.osm.org/way/148185791 and 
http://www.osm.org/way/30154495). But some are still used for routing, 
like for example http://www.osm.org/way/307774731.


Yet, I would expect the same behavior for all above examples in the 
ferries handling part inside way_function in the profile:

https://github.com/Project-OSRM/osrm-backend/blob/develop/profiles/car.lua#L257-L269

Could this be related to other tags, or am I missing something here?

Thanks,
Julien


Le 12/11/2015 04:24, Richard Marsden a écrit :

thanks - I was still at the LUA  stage (and quite a bit can be worked
out from just this level).

The explanation below explains 0 and 1, but I couldn't see any
explanation of the other values - despite some c++ code that set default
TravelMode values of 4 in a couple of places. No documentation in
TravelMode.hpp

The lack of documentation is an issue, and I was planning to write a few
notes about the LUA profiles for others on my blog. They're bound to be
incomplete, but they would be a start and could be updated/corrected as
a living document.

Richard

On Nov 11, 2015, at 6:50 PM, Daniel Hofmann > wrote:


Let me show you how you can find out more about specific variables
like mode_ferry:

your best bet is to search the code base, for example, see this
initial search for mode_ferry:

https://github.com/Project-OSRM/osrm-backend/search?utf8=%E2%9C%93=mode_ferry=Code

you can see how we set result.forward_mode and result.backward_mode to
mode_ferry there, so let's search for forward_mode:

https://github.com/Project-OSRM/osrm-backend/search?utf8=%E2%9C%93=forward_mode=Code

this gives us some lua profiles, which we ignore, since we want to see
how the OSRM implementation uses forward_mode and backward_mode; the
last search hit is scripting_environment.cpp:

https://github.com/Project-OSRM/osrm-backend/blob/8f6fc0146ba76d34d20c5b7a87b75249bbb12b82/extractor/scripting_environment.cpp#L121-L124

this is where we expose the ExtractionWay's set_forward_mode and
set_backward_mode member functions, aliasing them to the lua
properties forward_mode and backward_mode --- so let's go on searching
for the ExtractionWay type:

https://github.com/Project-OSRM/osrm-backend/blob/8f6fc0146ba76d34d20c5b7a87b75249bbb12b82/extractor/extraction_way.hpp#L112-L115

in those member functions, the forward_travel_mode and
backward_travel_mode are set accordingly. If you check their types a
few lines below in the member attribute declarations, you will see
they are of type TravelMode. And here it is:

https://github.com/Project-OSRM/osrm-backend/blob/8f6fc0146ba76d34d20c5b7a87b75249bbb12b82/data_structures/travel_mode.hpp#L34-L35

a quick search for TRAVEL_MODE_INACCESSIBLE reveals the following line
in the extractor:

https://github.com/Project-OSRM/osrm-backend/blob/9ef1f8cba31ec8323b357d233f1c552b1c7c9e09/extractor/extractor_callbacks.cpp#L95-L99

where ways are discarded, if they are inaccessible! Tada!


As you can see, even though it is quite a bit of effort tracing back
specific variables, it can be done in a few minutes with only the most
basic tools (I did this entirely using the Github search functionality
--- of course you can use grep or your code browser of choice, too).

Hope that helps,
Daniel J H

On Wed, Nov 11, 2015 at 7:06 PM, Richard Marsden > wrote:

By coincidence I was working through the lua scripts trying to
understand them.

So what is the significance of the 1,2,3?  Just unique
identifiers. As long as they're non-zero, they will be enabled?

Richard


On Nov 11, 2015, at 9:23 AM, Daniel Hofmann > wrote:


If you take a look at the car profile, you will see a ferry_mode
variable, that sets the travel mode:


https://github.com/Project-OSRM/osrm-backend/blob/8f6fc0146ba76d34d20c5b7a87b75249bbb12b82/profiles/car.lua#L151

If you set this to 0 (i.e. 'inaccessible") as defined here:


https://github.com/Project-OSRM/osrm-backend/blob/8f6fc0146ba76d34d20c5b7a87b75249bbb12b82/data_structures/travel_mode.hpp#L34-L35

then the extractor discards ferry routes because you marked them
inaccessible.

On Wed, Nov 11, 2015 at 9:03 AM, Peter Becker
> wrote:

Hello, i dont want any ferries in my routes. Is this possible?

___
OSRM-talk mailing list
OSRM-talk@openstreetmap.org 
https://lists.openstreetmap.org/listinfo/osrm-talk

Re: [OSRM-talk] osrm performance with multiple threads is confusing me

[Patrick Niklaus]

Query performance is also limited by disk performance as part of the
RTree that is used for the nearest neighbor lookup is on disk. You
might want to check your IO stats as well.

On Thu, Nov 12, 2015 at 3:55 PM, Daniel Patterson  wrote:
> Hi Peter,
>
>   How are you performing the tests?  From the same machine, or from another 
> machine over the network?
>
>   OSRM responses are usually around 5-20ms.  If you're doing some analysis of 
> the results, on the same machine,
>   it's possible that your tests themselves are CPU limited.  Can you give 
> more info on your testing setup?
>
> daniel
>
>> On Nov 12, 2015, at 5:59 AM, Peter Becker  wrote:
>>
>> I'm a little bit confused.
>>
>> i have load the map in memory with osrm-datastore und run one instance with
>>
>> "osrm-route --shared-memory=yes -t 1 -p 5000"
>>
>> with only one thread, one cpu-core raise up tzo 90% and i get ~500
>> routes per second
>>
>> if i set thread count to more then 2 or more
>>
>> "osrm-route --shared-memory=yes -t 2 -p 5000"
>>
>> 3 cpu-cores are raise up to 30-50% usage and i only get ~333 routes per 
>> second.
>>
>> so i also try run 2 instances with one core:
>>
>> "osrm-route --shared-memory=yes -t 1 -p 5000"
>> "osrm-route --shared-memory=yes -t 1 -p 5001"
>>
>> and use ngnix as load-balancer .. it dosn't make a difference to one
>> instance with 2 or more threads
>>
>> what is wrong? with "-t 2" or 2 instances i expect that 2 cpu-cores
>> are at 100% and i get more routes per second as with "-t 1".
>>
>> ___
>> OSRM-talk mailing list
>> OSRM-talk@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/osrm-talk
>
>
> ___
> OSRM-talk mailing list
> OSRM-talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk

___
OSRM-talk mailing list
OSRM-talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/osrm-talk

Re: [OSRM-talk] osrm performance with multiple threads is confusing me

[Daniel Patterson]

Hi Peter,

  How are you performing the tests?  From the same machine, or from another 
machine over the network?

  OSRM responses are usually around 5-20ms.  If you're doing some analysis of 
the results, on the same machine,
  it's possible that your tests themselves are CPU limited.  Can you give more 
info on your testing setup?

daniel

> On Nov 12, 2015, at 5:59 AM, Peter Becker  wrote:
> 
> I'm a little bit confused.
> 
> i have load the map in memory with osrm-datastore und run one instance with
> 
> "osrm-route --shared-memory=yes -t 1 -p 5000"
> 
> with only one thread, one cpu-core raise up tzo 90% and i get ~500
> routes per second
> 
> if i set thread count to more then 2 or more
> 
> "osrm-route --shared-memory=yes -t 2 -p 5000"
> 
> 3 cpu-cores are raise up to 30-50% usage and i only get ~333 routes per 
> second.
> 
> so i also try run 2 instances with one core:
> 
> "osrm-route --shared-memory=yes -t 1 -p 5000"
> "osrm-route --shared-memory=yes -t 1 -p 5001"
> 
> and use ngnix as load-balancer .. it dosn't make a difference to one
> instance with 2 or more threads
> 
> what is wrong? with "-t 2" or 2 instances i expect that 2 cpu-cores
> are at 100% and i get more routes per second as with "-t 1".
> 
> ___
> OSRM-talk mailing list
> OSRM-talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk


___
OSRM-talk mailing list
OSRM-talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/osrm-talk

Re: [OSRM-talk] osrm performance with multiple threads is confusing me

[Peter Becker]

iotop: 0%

2015-11-12 16:53 GMT+01:00 Peter Becker :
> Client: different machine over network, Windows 8.1, .Net App with 4
> Cores (Ivy Bridge), Tested with 4, 8, 20 Threads parallel
> Server: Hyper-V-Instance, Haswell 4 Core's, 8 GB RAM, Ubuntu 14.04.
> LTS, Samsung 512 GB Pro SSD; + german map file
>
> german.maps.osrm is placed on ssd and load with datastore in shared memory
>
> $ osrm-route --shared-memory=yes -t 1 -p 5000
> CPU-Usage:
>   Core 1: 0%
>   Core 2: ~90%;
>   Core 3: 0%;
>   Core 4: ~5%;
> Routes (2 Points) per Second: ~500
>
> --
>
> $ osrm-route --shared-memory=yes -t 2 -p 5000
> CPU-Usage:
>   Core 1: ~33%
>   Core 2: ~33%;
>   Core 3: ~54%;
>   Core 4: ~5%;
> Routes (2 Points) per Second: ~333
>
> --
>
> $ osrm-route --shared-memory=yes -t 1 -p 5000
> $ osrm-route --shared-memory=yes -t 1 -p 5001
> CPU-Usage:
>   Core 1: ~33%
>   Core 2: ~33%;
>   Core 3: ~54%;
>   Core 4: ~5%;
> Routes (2 Points) per Second: ~333
> (same result)

___
OSRM-talk mailing list
OSRM-talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/osrm-talk