[mkgmap-dev] Commit: r3093: Fix a possible assertion when building the street index.
Version mkgmap-r3093 was committed by steve on Sat, 08 Mar 2014 Fix a possible assertion when building the street index. All cases where sorted names are compared with equals() need to be changed to use Collator.compare() with the SrtCollator from Sort. This patch fixes just a few cases to fix an observed assertion error. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
What have you done to solve this Bernd, should I just wait for another extract tomorrow? I have seen this error last week, too. I had to recreate my dach extract. Bernd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Hi Gerd, I always mix the osm data with address nodes for Vlaanderen and the Netherlands with osmconvert. This part of my batch file looks like this: :osmconvert %DIR%\Tools\osmconvert --drop-version %DIR%\%date%\Splitter\%extract% %DIR%\resources\adres.o5m %DIR%\resources\vlaanderen-adres.o5m -B=%DIR%\resources\bnl_01.poly -o=%DIR%\%date%\Splitter\benelux.o5m %extract% is europe.osm.pbf from today NL adres.o5m: lon min: 3.3590433 lon max: 7.2259690 lat min: 50.7510383 lat max: 53.4972667 nodes: 8551053 ways: 0 relations: 0 node id min: 1000 node id max: 18551052 keyval pairs max: 2 keyval pairs max object: node 1000 Vlaanderen vlaanderen-adres.o5m lon min: 2.3066895 lon max: 5.8980601 lat min: 49.2933351 lat max: 51.5047683 nodes: 2449726 ways: 0 relations: 0 node id min: 2001 node id max: 22449726 keyval pairs max: 4 keyval pairs max object: node 2001 After this I split the resulting file as follows :Splitter java -Xmx8000m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -jar %SPLITTER% --output-dir=%date%\Splitter --max-threads=8 --problem-file=%DIR%\resources\problem_areas.txt --precomp-sea=%seafile% --keep-complete --overlap=0 --mapid=10010001 --polygon-file=%DIR%\resources\benelux.poly --max-nodes=175 --write-kml=%areas%.kml --output=o5m --geonames-file=%DIR%\resources\cities5000.txt --description=OFM_BNL %DIR%\%date%\Splitter\benelux.o5m splitter2.log 2err2.log Normally it all goes well until this morning. I've tried the latest splitter version too but it didnt help. I also tried to make the first polygon bnl_01.poly much bigger ( it is a little bigger than the second benelux.poly in the splitter step 2). This also didnt help. Now if I skip the first step and split it without merging the address nodes it goes well without problems. Could it be that the splitter creates fake ID's that overlap my address data nodes? Or is there an overlap in osm id's with the fake address nodes? ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
BTW Here are the links to the files in case you need them: https://www.dropbox.com/sh/d3ssenipw2vx6qs/ucrj2cNiFM/adres.o5m https://www.dropbox.com/sh/d3ssenipw2vx6qs/bgWpOCpfwK/vlaanderen-adres.o5m https://www.dropbox.com/sh/d3ssenipw2vx6qs/zqzksFWZcp/bnl_01.poly https://www.dropbox.com/sh/d3ssenipw2vx6qs/2N8CdpLXQc/benelux.poly ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Hi Minko, ligfietser wrote Now if I skip the first step and split it without merging the address nodes it goes well without problems. Could it be that the splitter creates fake ID's that overlap my address data nodes? Or is there an overlap in osm id's with the fake address nodes? I have no idea. Download of Europe will take a while... Gerd -- View this message in context: http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799084.html Sent from the Mkgmap Development mailing list archive at Nabble.com. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
I see this very often as well, when I use osmupdate on an earlier download. The only solution I have found is to download a new pbf, then it works again. I suspect therefore that osmupdate or one of the underlying tools is corrupting the data. Colin On 8 March 2014 10:44:16 CET, Minko ligfiet...@online.nl wrote: Hi, I was trying to update my Benelux map but the splitter refused to finish the process. Normally, splitter is finished in 2 minutes, now it gets in an infinite loop with Full GC. What can be the cause for this, a corrupt download of the europe extract? ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Colin, thanks for the hint. No matter what other tools do: infinite loop in splitter is the worst case as a reaction on errors ;-) Gerd Colin Smale wrote I see this very often as well, when I use osmupdate on an earlier download. The only solution I have found is to download a new pbf, then it works again. I suspect therefore that osmupdate or one of the underlying tools is corrupting the data. Colin On 8 March 2014 10:44:16 CET, Minko lt; ligfietser@ gt; wrote: Hi, I was trying to update my Benelux map but the splitter refused to finish the process. Normally, splitter is finished in 2 minutes, now it gets in an infinite loop with Full GC. What can be the cause for this, a corrupt download of the europe extract? ___ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- View this message in context: http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799086.html Sent from the Mkgmap Development mailing list archive at Nabble.com. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Gerd, If I skip merging the Vlaanderen address data then splitting goes well. I have uploaded my 'corrupt' Benelux data which goes wrong (incl the Vlaanderen addresses) here: http://mijndev.openstreetmap.nl/~ligfietser/test/benelux.o5m ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Hi Minko, is there a reason for merging the files before splitting? Did you try to just pass all the files to splitter? I'm doing so with osm and srtm-data. Don't know, if this helps in this case, but in general I think it should be much faster. Henning ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Hi Henning, I didnt know that you could feed multiple files to the splitter? How do you do this? Hi Minko, is there a reason for merging the files before splitting? Did you try to just pass all the files to splitter? I'm doing so with osm and srtm-data. Don't know, if this helps in this case, but in general I think it should be much faster. Henning ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Hi Minko, thanks, I can reproduce the problem with this file. Looking for a fix now. Gerd ligfietser wrote Gerd, If I skip merging the Vlaanderen address data then splitting goes well. I have uploaded my 'corrupt' Benelux data which goes wrong (incl the Vlaanderen addresses) here: http://mijndev.openstreetmap.nl/~ligfietser/test/benelux.o5m ___ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- View this message in context: http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799099.html Sent from the Mkgmap Development mailing list archive at Nabble.com. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Hi Minko, I've committed r320 to fix the loop problem. You can specify multiplie input files, but you have to make sure that each file contains ids which are higher than all previous ones. I think this is the case with your address files. The reason for the loop was a way in a mp-relation for which no nodes were found. I guess this is caused by using osmconvert without the parameters --complete-ways --complex-ways I am not sure what's faster: Using osmconvert with polygon first or only splitter with multiple input files. When all files are in o5m format, single run with splitter should be quite fast. Gerd -- View this message in context: http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799103.html Sent from the Mkgmap Development mailing list archive at Nabble.com. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Gerd wrote: The reason for the loop was a way in a mp-relation for which no nodes were found. I guess this is caused by using osmconvert without the parameters --complete-ways --complex-ways If I use --complete-ways or --complex-ways with osmconvert, I didnt get any output at all so I always skip this. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Hi Minko, the order of input files would be java ... splitter.jar [options] europe.o5m adres.o5m vlaanderen-adres.o5m Your approach with osmconvert is probably faster because you have to use osmconvert anyway to convert to o5m. Gerd ligfietser wrote Hi Gerd, Thanks for the fix, I will test it out. I tried Hennings approach but splitting seemed to last much longer because it analyses the whole EU extract and I only need the Benelux plus a small area of France and Germany. Splitting wasn't completed and threw out this error: warning: --keep-complete is only used for the first input file. New way id 195314 is not higher than last id 264955883 Maybe the IDs are not sorted. This is not supported with keep-complete=true or --problem-list I'll try now r320. I've committed r320 to fix the loop problem. You can specify multiplie input files, but you have to make sure that each file contains ids which are higher than all previous ones. I think this is the case with your address files. The reason for the loop was a way in a mp-relation for which no nodes were found. I guess this is caused by using osmconvert without the parameters --complete-ways --complex-ways I am not sure what's faster: Using osmconvert with polygon first or only splitter with multiple input files. When all files are in o5m format, single run with splitter should be quite fast. Gerd ___ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- View this message in context: http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799107.html Sent from the Mkgmap Development mailing list archive at Nabble.com. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Gerd wrote: the order of input files would be java ... splitter.jar [options] europe.o5m adres.o5m vlaanderen-adres.o5m Yes I did that too. Anyway, with r320 it seems to work, mkgmap is running now thanks! ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Hi Minko, fine:-) I think the only good reason to use splitter with an input file like europe.o5m or planet.o5m in combination with a poly file that extracts a small part is that it is able to keep ferry lines complete. osmconvert --complete-ways --complex-ways -B=bnl_01.poly europe.o5m -o=benelux.o5m fails for me as well. I'll ask the author about it. Gerd ligfietser wrote Gerd wrote: the order of input files would be java ... splitter.jar [options] europe.o5m adres.o5m vlaanderen-adres.o5m Yes I did that too. Anyway, with r320 it seems to work, mkgmap is running now thanks! ___ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- View this message in context: http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799112.html Sent from the Mkgmap Development mailing list archive at Nabble.com. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] new branch NOD127
Hi Gerd, For both trunk and branch, I see a big detour depending on the start point, so something is weird with road 6. I'm afraid it could be quite difficult to get good routing in Poland with default style. especially in BaseCamp. That's because our road network miss highways and Garmin programs need class 4 roads, when calculating long route. I have tested my maps, which I know best, but maybe you could find similar behavior in other area? If not, I can send you my style for road map. I will try investigate a bit, maybe I will be able to isolate this problem. -- Best regards, Andrzej ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Infinite loop splitter
Hi all, I've contacted Markus Weber, the author of osmconvert and osmfilter. It seems that the problem with complex-ways occurs only in the Windows version, maybe because something goes wrong with files 4GB. Markus doesn't have access to a Win64 system. I use a mingw-64 installation to compile my own binary for 64 bit, maybe an update will help, but I doubt it, and unfortunately I forgot how the installation works, I just remember it wasn't simple. So, if you use osmconvert/osmfilter to update your files it is probably better to use a 64 bit Linux installation. Gerd GerdP wrote Hi Minko, fine:-) I think the only good reason to use splitter with an input file like europe.o5m or planet.o5m in combination with a poly file that extracts a small part is that it is able to keep ferry lines complete. osmconvert --complete-ways --complex-ways -B=bnl_01.poly europe.o5m -o=benelux.o5m fails for me as well. I'll ask the author about it. Gerd ligfietser wrote Gerd wrote: the order of input files would be java ... splitter.jar [options] europe.o5m adres.o5m vlaanderen-adres.o5m Yes I did that too. Anyway, with r320 it seems to work, mkgmap is running now thanks! ___ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- View this message in context: http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799142.html Sent from the Mkgmap Development mailing list archive at Nabble.com. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
