[mkgmap-dev] Commit r4106: Better detect errors in action command lists.
Version mkgmap-r4106 was committed by steve on Wed, 07 Feb 2018 Better detect errors in action command lists. It is no longer required to end set, add, setaccess and addaccess commands with a semicolon. http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4106 ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] max-jobs patch
I don't know, if Nordrhein-Westfalen is that kind of large tiles you meant. Splitter ended up with 43 files and i tried to make mapsource maps with and without --max-jobs option and mkgmap r4000. My computer does the job with 43 files about 12minutes faster than without this option. My laptop is a core i5 with 8Gb of memory. The command i use is like this line : java -Xmx7500m -XX:-UseGCOverheadLimit -ea -jar -Dlog.config=logging.properties C:\PROGRA~2\KartenTools\mkgmap\mkgmap.jar --location-autofill=is_in,nearest --mapname=70060001 --family-id=7006 .\styles\mystyle.txt --series-name="STYLE TEST" --family-name="OpenStreetmap" --country-name=STYLETEST --country-abbr=STY --area-name=STY --overview-mapname="overview" --latin1 --style-file=.\styles --style=myland --max-jobs --keep-going --check-roundabouts --drive-on=detect,right --output-dir=.\out\MYSTYLE --index --bounds=mybounds --route --name-tag-list=name,place_name,loc_name --housenumbers --x-split-name-index --add-pois-to-areas --precomp-sea=.\input\sea.zip --tdbfile --draw-priority=10 --gmapsupp -c .\data\MYSTYLE\template.args --description=MYSTYLE --remove-ovm-work-files=true Hope this gave you some useful hints Am 07.02.2018 um 20:06 schrieb Gerd Petermann: Hi Mike, I did not yet try it. My understanding is that the Garbage Collection will try hard to avoid an out of heap exception. In other words: A user might see better throughput with fewer parallel jobs. What are your experiences? Did you try with large maps, say > 40 large tiles ? Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Mittwoch, 7. Februar 2018 01:58 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] max-jobs patch Hi Gerd, please find attached v2 of the max-jobs patch. This version additionally catches out of heap memory exceptions and displays a message suggesting either the use of the Java -Xmx option to increase the available heap memory or the mkgmap --max-jobs option with a smaller value to reduce the memory requirement (the latter is suggested only if using more than one thread). How does that seem? Regards, Mike -Original Message- From: Gerd Petermann [mailto:gpetermann_muenc...@hotmail.com] Sent: 05 February 2018 07:28 To: Mike Baggaley ; 'Development list for mkgmap' Subject: Re: [mkgmap-dev] max-jobs patch Hi Mike, thought about this again. Maybe this change is too simple. With multiple jobs one also needs more heap (-Xmx JRE option). If you create rather large tiles with splitter (max-nodes=160) you need 0.5 - 1 GB for each job. Not sure what happens when a user creates a map with 10 tiles on an 8-core machine without any -Xmx option. I fear this will result in OutOfMemory exception, so better check the available heap as well. Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Sonntag, 4. Februar 2018 15:14 An: 'Development list for mkgmap' Betreff: [mkgmap-dev] max-jobs patch Hi Gerd, Please find attached a patch that amends the default behaviour if the --max-jobs option is not specified, to using a value equal to the number of CPU cores, as suggested in a previous post. The documentation is also amended to reflect the change. This halves the execution time of mkgmap for building a map of Staffordshire on my 8-core PC when --max-jobs is not specified (I didn't know about this option previously and was unaware the performance could have been improved). Cheers, Mike ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- Mit freundlichen Grüßen # Manfred Haiduk, Zum Fischbach 9, 52393 Hürtgenwald e-mail mhai...@t-online.de # ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] max-jobs patch
Hi Mike, I did not yet try it. My understanding is that the Garbage Collection will try hard to avoid an out of heap exception. In other words: A user might see better throughput with fewer parallel jobs. What are your experiences? Did you try with large maps, say > 40 large tiles ? Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Mittwoch, 7. Februar 2018 01:58 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] max-jobs patch Hi Gerd, please find attached v2 of the max-jobs patch. This version additionally catches out of heap memory exceptions and displays a message suggesting either the use of the Java -Xmx option to increase the available heap memory or the mkgmap --max-jobs option with a smaller value to reduce the memory requirement (the latter is suggested only if using more than one thread). How does that seem? Regards, Mike -Original Message- From: Gerd Petermann [mailto:gpetermann_muenc...@hotmail.com] Sent: 05 February 2018 07:28 To: Mike Baggaley ; 'Development list for mkgmap' Subject: Re: [mkgmap-dev] max-jobs patch Hi Mike, thought about this again. Maybe this change is too simple. With multiple jobs one also needs more heap (-Xmx JRE option). If you create rather large tiles with splitter (max-nodes=160) you need 0.5 - 1 GB for each job. Not sure what happens when a user creates a map with 10 tiles on an 8-core machine without any -Xmx option. I fear this will result in OutOfMemory exception, so better check the available heap as well. Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Sonntag, 4. Februar 2018 15:14 An: 'Development list for mkgmap' Betreff: [mkgmap-dev] max-jobs patch Hi Gerd, Please find attached a patch that amends the default behaviour if the --max-jobs option is not specified, to using a value equal to the number of CPU cores, as suggested in a previous post. The documentation is also amended to reflect the change. This halves the execution time of mkgmap for building a map of Staffordshire on my 8-core PC when --max-jobs is not specified (I didn't know about this option previously and was unaware the performance could have been improved). Cheers, Mike ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
[mkgmap-dev] patch for crash in MapSource
Hi all, attached is a patch that seems to help in many cases where --dem-poly causes crashes in MapSource "Show Profile..." A binary is here: http://files.mkgmap.org.uk/download/414/mkgmap.jar It changes the TRE file and the *.tdb file. I still see a few crashes but maybe those are caused by other problems. Please try and let me know your results. I found no bad impact so far, long distance routing still seems to work, but I've not tested much. @Steve: Maybe we have to make sure that the areas in *.tdb do not overlap. In this case this patch is not yet okay. Gerd profile-crash.patch Description: profile-crash.patch ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Garmin uses overlapping tiles
Hi experts, this is quite complex. My current understanding is this: 1) Garmin maps can have overlapping areas in TRE, boundaries are stored with 24 bit resolution rounded to either multiples of 2 or 4. Overlaps are also either 2 or 4. In current mkgmap maps there is no overlap. 2) The *.tdb file in Garmin maps describes areas that do not overlap. The boundaries are stored in 32 bit resolution, probably not rounded. 3) The DEM areas in Garmin maps overlap even the TRE areas and only the position of the upper left corner is explicitly stored in 32 bit resolution, and is always a multiple of the dem-dist. The position of the lower right corner could be calculated but I assume that it is not important. Changing the boundaries in TRE has an effect reg. the crash. Changing the position of the upper left corner boundaries in DEM has an effect reg. the crash. Changing the position of the lower right corner and therefore the overlap has an effect reg. the crash, probably the current code in mkgmap is off by one in both directions (too small). The current code in mkgmap calculates the *.tdb file boundaries from the TRE boundaries. This might cause problems when I change the code to write TRE boundaries that overlap. Maybe we have to align tiles to certain multiples, maybe it is sufficient to align the TRE boundaries to certain values. I still did not find a working rule. The TRE bounds may depend on the DEM dist value or on certain prefered values, maybe there is a flag in the DEM or TRE header that has to match. Maybe the DEM bounds depend on the TRE bounds, maybe the *.tdb file also is important here. Please let me know if you find different results in routable Garmin maps with DEM. Gerd Von: mkgmap-dev im Auftrag von Gerd Petermann Gesendet: Dienstag, 6. Februar 2018 16:51 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Garmin uses overlapping tiles Hi Andrzej, changing the TRE bounds helps sometimes, but not always :-( @Frank: All Garmin TRE files that I looked at where aligned to multiples of 4 map units. I tried this as well as aligning to multiples of 2, nothing worked without crashes so far. Anyhow, it seems to be the right thing to look at. Gerd Von: mkgmap-dev im Auftrag von Andrzej Popowski Gesendet: Dienstag, 6. Februar 2018 15:49 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Garmin uses overlapping tiles Hi Gerd, I guess basemap is overview map. 0x4A objects are derived form backgrounds of detailed tiles, but with resolution of overview map, it would be difficult to guess, if background overlap or not. I hope extending values written to TRE will work. -- Best regards, Andrzej ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] splitter r590,wrong handling of relations
Thank you :-). Yesterday I had send a message to the Author Javier Sanchez . thomas -- Von: "Gerd Petermann" Datum: Mittwoch, 7. Februar 2018 06:24 An: "Development list for mkgmap" Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations Hi Thomas, I've just fixed the relation [1], so please try again with a fresh download tomorrow. Gerd [1] https://www.openstreetmap.org/changeset/56139020 Von: mkgmap-dev im Auftrag von Gerd Petermann Gesendet: Dienstag, 6. Februar 2018 14:47 An: Thomas Morgenstern; Development list for mkgmap Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations Hi Thomas, the relation is broken. Use JOSM validator and it will report 2 errors. I think this way https://www.openstreetmap.org/way/556302821 should not belong to it and a few nodes should be merged to repiar it: node 4298102908 node 4298102907 node 655141348 Gerd Von: mkgmap-dev im Auftrag von Thomas Morgenstern Gesendet: Dienstag, 6. Februar 2018 14:35 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations Hi Gerd, i have now made a different split, so that 1 tile contains the whole isle. In this case it looks good. I assume, that the relation is okay. No idea what should be changed in data. Also i have looked before with JOSM in the old split and confirm, that in the left tile the relation also exists, but not rendered from mkgmap. I can't judge, if the relation is broken or not. Therefore the problem is in mkgmap ? I thomas -- Von: "Gerd Petermann" Datum: Dienstag, 6. Februar 2018 12:15 An: "Thomas Morgenstern" ; "Development list for mkgmap" Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations Hi Thomas, I think the problem is that the relation is broken. Splitter writes it to both tiles, but mkgmap seems to have problems with it. Anyway, the problem is in the data and should be fixed in OSM. Gerd Von: mkgmap-dev im Auftrag von Thomas Morgenstern Gesendet: Dienstag, 6. Februar 2018 12:50 An: Development list for mkgmap Betreff: [mkgmap-dev] splitter r590,wrong handling of relations Hi Gerd, i found out, that splitter r590 can not proper handle the relation 2.691.138. I can reproduce the problem. It makes no different, if I splitt the canary-islands-latest.osm.pbf with the default-option or with my own splitt.list. If I move the tileborder by changing the nord-south border, also the problem is moved. See picture at http://img2ms.de/OSM/ErrorInSplitter-r590.jpg. At the tile-boundary splitter creates only for the rightsite tile the polygon for tag boundary=protected_area , respektive leisure=nature_reserve. In the leftsite-tile this tags are missing. maybee you can have a look at that ? regards thomas ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
