Re: [OSRM-talk] Segfault with 5.4.0 osrm-extract and foot.lua
Hi, so all OSRM releases up to 5.2.7 work ok on my machine and with my input file and the foot profile; all releases from 5.3.0 upwards have the segfault. I'll continue trying to identify exactly where it was introduced. (It could of course still be a funny problem with 16.04 or with my input file that somehow triggers the segfault, I haven't tried other environments yet.) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ OSRM-talk mailing list OSRM-talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/osrm-talk
Re: [OSRM-talk] Per-Direction Node Restrictions
Come to think of it, it would probably be simpler to just make each node aware of the "highest" order of highway. Knowing that, it would work to read that info for each constituent way and determine priority based on if the way's order matches the node's order.. On Thu, Oct 6, 2016 at 1:29 AM, Daniel Hofmann wrote: > This is not planned at the moment, since all the profiles know about are > tags. All additional processing has to happen internally in the C++ code, > and that's exactly why priority roads / give ways / stop signs are a bit > ugly to implement. Sorry. > > On Thu, Oct 6, 2016 at 12:19 AM, Spencer Gardner > wrote: > >> I started working on this in https://github.com/Project-OSR >>> M/osrm-backend/pull/3009 but it won't be a quick change in the >>> profiles, for sure. >>> >>> Cheers, >>> Daniel J H >>> >>> >> Thanks. Will your changes also allow me to read the tags of other roads >> approaching the same node? That's the important piece for my purposes. A >> priority_road tag is likely to be unreliable in the US since it has no >> legal status, so I need to make some assumptions about priority based on >> the various highway tags of the incoming ways. >> >> ___ >> OSRM-talk mailing list >> OSRM-talk@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/osrm-talk >> >> > > ___ > OSRM-talk mailing list > OSRM-talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/osrm-talk > > ___ OSRM-talk mailing list OSRM-talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/osrm-talk
Re: [OSRM-talk] Segfault with 5.4.0 osrm-extract and foot.lua
Another thing to try is using the pre-build binaries we publish to npm for node-osrm; you can get them via npm install then the osrm-* binaries Travis built should be at node_modules/osrm/lib/binding/ On Thu, Oct 6, 2016 at 1:45 PM, Frederik Ramm wrote: > Hi, > >I've re-run with debug enabled but that wasn't much better: > > === Backtrace: = > /lib/x86_64-linux-gnu/libc.so.6(+0x77725)[0x7fd9e4134725] > /lib/x86_64-linux-gnu/libc.so.6(+0x7ff4a)[0x7fd9e413cf4a] > /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fd9e4140abc] > osrm-extract(_ZN9__gnu_cxx13new_allocatorISt10_List_ > nodeIyEE10deallocateEPS2_m+0xc)[0x988c24] > osrm-extract(_ZNSt7__cxx1110_List_baseIySaIyEE11_M_put_ > nodeEPSt10_List_nodeIyE+0xe)[0x988c34] > osrm-extract(_ZNSt7__cxx1110_List_baseIySaIyEE8_M_clearEv+0x1d)[0x988c53] > osrm-extract(_ZNSt7__cxx1110_List_baseIySaIyEED1Ev+0x9)[0x988c67] > osrm-extract(_ZNSt7__cxx114listIySaIyEED1Ev+0x9)[0x988c73] > osrm-extract(_ZN5stxxl9lru_pagerILj8EED1Ev+0x1d)[0x988c93] > osrm-extract(_ZN5stxxl6vectorIjLj4ENS_9lru_pagerILj8EEELj2097152ENS_ > 2RCEyED1Ev+0xe8)[0x9ad4bc] > osrm-extract(_ZN4osrm9extractor20ExtractionContainersD1Ev+0x3c)[0x9ae67c] > osrm-extract(_ZN4osrm9extractor9Extractor3runERNS0_ > 20ScriptingEnvironmentE+0xbc2)[0x9bd03a] > osrm-extract(main+0x22c)[0x9123cb] > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fd9e40dd830] > osrm-extract(_start+0x29)[0x905799] > > On 10/06/2016 11:05 AM, Daniel Hofmann wrote: > > I think we need more details here: first of all, it seems like Ubuntu > > 16.04 ships Lua5.3 with apt, for which we saw an immense increase in > > memory usage. > > I didn't get any out-of-memory errors. Also I didn't use Lua5.3, I only > have 5.1 and 5.2: > > % dpkg -l |grep lua > ii liblua5.1-0:amd64 5.1.5-8ubuntu1 > amd64Shared library for the Lua interpreter version 5.1 > ii liblua5.1-0-dev:amd64 5.1.5-8ubuntu1 > amd64Development files for the Lua language version 5.1 > ii liblua5.2-0:amd64 5.2.4-1ubuntu1 > amd64Shared library for the Lua interpreter version 5.2 > ii liblua5.2-dev:amd645.2.4-1ubuntu1 > amd64Development files for the Lua language version 5.2 > ii libluabind-dev 0.9.1+dfsg-10 > amd64luabind c++ binding for lua: static library and > headers > ii libluabind0.9.1v5 0.9.1+dfsg-10 > amd64luabind c++ binding for lua: runtime library > ii lua5.1 5.1.5-8ubuntu1 > amd64Simple, extensible, embeddable programming > language > ii lua5.2 5.2.4-1ubuntu1 > amd64Simple, extensible, embeddable programming > language > > and > > % ldd /usr/local/bin/osrm-extract |grep lua > libluabind.so.0.9.1 => /usr/lib/libluabind.so.0.9.1 > (0x7f2207e1f000) > liblua5.2.so.0 => /usr/lib/x86_64-linux-gnu/liblua5.2.so.0 > (0x7f2207bed000) > > > Then, can you tell us if you're using the default profile or did you > > make any adjustments to it? > > None, using the standard foot.lua supplied with 5.4.0. > > I'll try (a) on a 14.04 Ubuntu, (b) with an older OSRM, (c) with a > differently-produced Europe file to see what happens. Any other ideas > are welcome. > > Bye > Frederik > > -- > Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" > > ___ > OSRM-talk mailing list > OSRM-talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/osrm-talk > ___ OSRM-talk mailing list OSRM-talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/osrm-talk
Re: [OSRM-talk] Segfault with 5.4.0 osrm-extract and foot.lua
Hi, I've re-run with debug enabled but that wasn't much better: === Backtrace: = /lib/x86_64-linux-gnu/libc.so.6(+0x77725)[0x7fd9e4134725] /lib/x86_64-linux-gnu/libc.so.6(+0x7ff4a)[0x7fd9e413cf4a] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fd9e4140abc] osrm-extract(_ZN9__gnu_cxx13new_allocatorISt10_List_nodeIyEE10deallocateEPS2_m+0xc)[0x988c24] osrm-extract(_ZNSt7__cxx1110_List_baseIySaIyEE11_M_put_nodeEPSt10_List_nodeIyE+0xe)[0x988c34] osrm-extract(_ZNSt7__cxx1110_List_baseIySaIyEE8_M_clearEv+0x1d)[0x988c53] osrm-extract(_ZNSt7__cxx1110_List_baseIySaIyEED1Ev+0x9)[0x988c67] osrm-extract(_ZNSt7__cxx114listIySaIyEED1Ev+0x9)[0x988c73] osrm-extract(_ZN5stxxl9lru_pagerILj8EED1Ev+0x1d)[0x988c93] osrm-extract(_ZN5stxxl6vectorIjLj4ENS_9lru_pagerILj8EEELj2097152ENS_2RCEyED1Ev+0xe8)[0x9ad4bc] osrm-extract(_ZN4osrm9extractor20ExtractionContainersD1Ev+0x3c)[0x9ae67c] osrm-extract(_ZN4osrm9extractor9Extractor3runERNS0_20ScriptingEnvironmentE+0xbc2)[0x9bd03a] osrm-extract(main+0x22c)[0x9123cb] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fd9e40dd830] osrm-extract(_start+0x29)[0x905799] On 10/06/2016 11:05 AM, Daniel Hofmann wrote: > I think we need more details here: first of all, it seems like Ubuntu > 16.04 ships Lua5.3 with apt, for which we saw an immense increase in > memory usage. I didn't get any out-of-memory errors. Also I didn't use Lua5.3, I only have 5.1 and 5.2: % dpkg -l |grep lua ii liblua5.1-0:amd64 5.1.5-8ubuntu1 amd64Shared library for the Lua interpreter version 5.1 ii liblua5.1-0-dev:amd64 5.1.5-8ubuntu1 amd64Development files for the Lua language version 5.1 ii liblua5.2-0:amd64 5.2.4-1ubuntu1 amd64Shared library for the Lua interpreter version 5.2 ii liblua5.2-dev:amd645.2.4-1ubuntu1 amd64Development files for the Lua language version 5.2 ii libluabind-dev 0.9.1+dfsg-10 amd64luabind c++ binding for lua: static library and headers ii libluabind0.9.1v5 0.9.1+dfsg-10 amd64luabind c++ binding for lua: runtime library ii lua5.1 5.1.5-8ubuntu1 amd64Simple, extensible, embeddable programming language ii lua5.2 5.2.4-1ubuntu1 amd64Simple, extensible, embeddable programming language and % ldd /usr/local/bin/osrm-extract |grep lua libluabind.so.0.9.1 => /usr/lib/libluabind.so.0.9.1 (0x7f2207e1f000) liblua5.2.so.0 => /usr/lib/x86_64-linux-gnu/liblua5.2.so.0 (0x7f2207bed000) > Then, can you tell us if you're using the default profile or did you > make any adjustments to it? None, using the standard foot.lua supplied with 5.4.0. I'll try (a) on a 14.04 Ubuntu, (b) with an older OSRM, (c) with a differently-produced Europe file to see what happens. Any other ideas are welcome. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ OSRM-talk mailing list OSRM-talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/osrm-talk
Re: [OSRM-talk] Segfault with 5.4.0 osrm-extract and foot.lua
Hey Frederik, the 5.4 release and its three profiles (car, foot, bike) work fine for us on all our deployments, with the latest planet. I think we need more details here: first of all, it seems like Ubuntu 16.04 ships Lua5.3 with apt, for which we saw an immense increase in memory usage. This seems to come from changes in their garbage collection, and is the reason we're only allowing Lua51 and Lua52 in master (the check for Lua53 unfortunately didn't make it for 5.4). Please make sure to read the public ticket over at: https://github.com/Project-OSRM/osrm-backend/issues/2926 Then, can you tell us if you're using the default profile or did you make any adjustments to it? Cheers, Daniel J H On Thu, Oct 6, 2016 at 8:19 AM, Frederik Ramm wrote: > Hi, > >trying to run > > osrm-extract europe-latest.osm.pbf -p osrm-backend/profiles/foot.lua > > from the 5.4.0 release terminates with a segfault: > > [info] Using script osrm-backend/profiles/foot.lua > [info] Input file: europe-latest.osm.pbf > [info] Profile: foot.lua > [info] Threads: 12 > [STXXL-MSG] STXXL v1.4.1 (prerelease/Debug) > [STXXL-MSG] Disk 'none' is allocated, space: 20 MiB, I/O > implementation: memory queue=0 devid=0 > [info] Parsing in progress.. > [info] input file generated by Osmium > (http://wiki.openstreetmap.org/wiki/Osmium) > [info] timestamp: 2016-10-04T19:29:02Z > [info] Parsing finished after 1083.06 seconds > [info] Raw input contains 1779030664 nodes, 217734659 ways, and 3212466 > relations > [extractor] Sorting used nodes... ok, after 8.96221s > [extractor] Erasing duplicate nodes ... ok, after 12.9021s > [extractor] Sorting all nodes ... ok, after 357.738s > [extractor] Building node id map ... ok, after 79.1193s > [extractor] setting number of nodes ... ok > [extractor] Confirming/Writing used nodes ... ok, after 54.7668s > [info] Processed 383867427 nodes > [extractor] Sorting edges by start... ok, after 112.587s > [extractor] Setting start coords ... ok, after 103.99s > [extractor] Sorting edges by target ... ok, after 112.214s > [extractor] Computing edge weights... ok, after 152.099s > [extractor] Sorting edges by renumbered start ... ok, after 107.876s > [extractor] Writing used edges ... ok, after 24.1231s > [extractor] setting number of edges ... ok > [info] Processed 403687206 edges > [extractor] Sorting used ways ... ok, after 16.2582s > [extractor] Sorting 0 restriction. by from... ok, after 3e-06s > [extractor] Fixing restriction starts ... ok, after 0s > [extractor] Sorting restrictions. by to ... ok, after 0s > [extractor] Fixing restriction ends ... ok, after 0s > [info] usable restrictions: 0 > [extractor] writing street name index ... ok, after 0.453898s > [info] extraction finished after 2273.88s > *** Error in `osrm-extract': double free or corruption (fasttop): > 0x00913f60 *** > === Backtrace: = > /lib/x86_64-linux-gnu/libc.so.6(+0x77725)[0x7f651750c725] > /lib/x86_64-linux-gnu/libc.so.6(+0x7ff4a)[0x7f6517514f4a] > /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f6517518abc] > osrm-extract[0x500c18] > osrm-extract[0x5044dc] > osrm-extract[0x506e94] > osrm-extract(_ZN4osrm9extractor9Extractor3runERNS0_ > 20ScriptingEnvironmentE+0x1f80)[0x4ad8c0] > osrm-extract(main+0x1626)[0x440b16] > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f65174b5830] > osrm-extract(_start+0x29)[0x442b49] > > Running it on a smaller extract (germany) works ok. Using the "car" > profile works ok too. > > Running it on the whole planet seems to run into a different issue, it > runs for three days and doesn't complete (Europe takes just a few > hours). Machine has 256G of RAM and is Ubuntu 16.04. > > I'm building a debug executable to find out more about the problem. I > wonder if there's a "last known good version" - which is the last > version that somebody successfully used the "foot" profile with, on an > extract the size of Europe or larger? > > Bye > Frederik > > -- > Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" > > ___ > OSRM-talk mailing list > OSRM-talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/osrm-talk > ___ OSRM-talk mailing list OSRM-talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/osrm-talk
