Re: [mkgmap-dev] Error writing overviewmap - continue at failure
Don't you have sea in your map? If yes, what do you need coastline for? El 28/2/21 a las 17:47, Mike Baggaley escribió: I have coastline in my overview of the UK down to the lowest level. At the lowest level, my overview contains just coastline, country boundaries, major cities and motorways. For an island, I would suggest the coastline is the most important thing to have in the overview. Cheers, Mike -Original Message- From: Ticker Berkin [mailto:rwb-mkg...@jagit.co.uk] Sent: 28 February 2021 10:51 To: Development list for mkgmap Subject: Re: [mkgmap-dev] Error writing overviewmap - continue at failure Hi The default style has 5 levels of overview map! This seems excessive. I'd have thought 2 or 3 would be acceptable and save a lot of space The only contents is larger and larger cities, fast trunk roads and motorways, some boundaries, sea and, at 3 of the 5 levels, coastline. I don't think coastline adds anything useful to a typical map, certainly not an overview, so dropping this would save space. Ticker ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error writing overviewmap - continue at failure
I have coastline in my overview of the UK down to the lowest level. At the lowest level, my overview contains just coastline, country boundaries, major cities and motorways. For an island, I would suggest the coastline is the most important thing to have in the overview. Cheers, Mike -Original Message- From: Ticker Berkin [mailto:rwb-mkg...@jagit.co.uk] Sent: 28 February 2021 10:51 To: Development list for mkgmap Subject: Re: [mkgmap-dev] Error writing overviewmap - continue at failure Hi The default style has 5 levels of overview map! This seems excessive. I'd have thought 2 or 3 would be acceptable and save a lot of space The only contents is larger and larger cities, fast trunk roads and motorways, some boundaries, sea and, at 3 of the 5 levels, coastline. I don't think coastline adds anything useful to a typical map, certainly not an overview, so dropping this would save space. Ticker ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Overview Too Big
Hi Gerd, Please find attached an updated patch. I'm not sure whether what I've done is exactly what you intended - it now includes the img file name and the MDR/RGN/TRE subfile that has exceeded the size limit in the error message. I couldn't see a clean easy way to get at the subfile without amending every sub file handler, so have resorted to examining the stack trace. I've also changed the keep going handling to what works for me and seems more logical. Let me know if you think it is not quite right. Cheers, Mike -Original Message- From: Mike Baggaley [mailto:m...@tvage.co.uk] Sent: 24 February 2021 13:59 To: 'Development list for mkgmap' Subject: RE: [mkgmap-dev] Overview Too Big Hi Gerd, I'll have a look at that. The patch catches the low level error and re-throws in at a higher level in the case of the overview map. It will need to do the same for the main map. I notice that when the map is too big, processing does not stop, even though I have not got --keep-going set. Looking at the code, it seems that the keep going check is in the Throwable catcher, rather than the MapFailedException catcher. Is this intended or a mistake? It seems to me that you'd be unlikely to want to continue from some random error, but might want to if the problem was known to be related to just one map. Cheers, Mike -Original Message- From: Gerd Petermann [mailto:gpetermann_muenc...@hotmail.com] Sent: 24 February 2021 09:30 To: Development list for mkgmap Subject: Re: [mkgmap-dev] Overview Too Big Hi Mike, I agree that the message should be improved. In fact I am always unsure what kind of limit this is. I've just created a working single tile map of Berlin including NOD which has a filesize of > 21MB. Your patch reports a number which probably counts bytes (I miss that info), but I think it is not about tile/map size. I assume it is about max. sub file size. So, maybe show that sub file name also? Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Mittwoch, 24. Februar 2021 00:16 An: 'Development list for mkgmap' Betreff: [mkgmap-dev] Overview Too Big Hi Gerd, The error message that mkgmap outputs when the overview map is too big is not helpful. Please find attached a patch that improves this. Cheers, Mike OverviewTooBig2.patch Description: Binary data ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error writing overviewmap - continue at failure
Hi The default style has 5 levels of overview map! This seems excessive. I'd have thought 2 or 3 would be acceptable and save a lot of space The only contents is larger and larger cities, fast trunk roads and motorways, some boundaries, sea and, at 3 of the 5 levels, coastline. I don't think coastline adds anything useful to a typical map, certainly not an overview, so dropping this would save space. Ticker ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error writing overviewmap - continue at failure
Hi I can't remember all the details of this message, but what has happened is a tile needs 3-byte references to a city index and this will force the full MDR for all tiles to use 3-byte city references, so growing the index structure a lot. So, probably not relevant for overview. Ticker On Sun, 2021-02-28 at 10:23 +, Gerd Petermann wrote: > Hi Joris, > > the message is printed when the LBL file contains so many citie > names. Those names are are extracted from mkgmap:city (maybe more), > so maybe addr:city is a problem. Probably a mapping error in that > area. > > Gerd > > > Von: mkgmap-dev im Auftrag > von Joris Bo > Gesendet: Sonntag, 28. Februar 2021 11:11 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Error writing overviewmap - continue at > failure > > Hi > > When reading my error log again there was another error. Too many > cities... Could it have caused my overview map too fail? > > SEVERE (MapBuilder): C:\Garmin\Temp\8004 - JBM - Planet\Output > Splitter\80041630.osm.pbf: Map contains 94484 Cities - more than > 65535 might cause indexing problems and excess size. Suggest splitter > with lower --max-nodes > > I Investigated the area (which is more or less a dessert in Jemen :-) > Overpass does not count a lot place = city. Could somebody help to > explain? > Could this halfway-error also be a reason to fail the overview map > being created at the end? > > Kind regards > Joris > > KML - file of the area > > > http://www.opengis.net/kml/2.2";>; > > >> >1.5 >> > > > > 80041630 > #transWhitePoly > > > > > > > > 43.637695,13.666992 > 43.637695,16.962891 > 45.571289,16.962891 > 45.571289,13.666992 > 43.637695,13.666992 > > > > > > > > > > > > > Met vriendelijke groeten, > > Joris Bo > jori...@hotmail.com > > -Oorspronkelijk bericht- > Van: mkgmap-dev Namens Joris > Bo > Verzonden: zaterdag 27 februari 2021 18:40 > Aan: Development list for mkgmap > Onderwerp: Re: [mkgmap-dev] Error writing overviewmap - continue at > failure > > Hi Mike & Gerd , > > My europe map (Europe-latest.osm.pbf split of 24Gb) which works > fine, has these img- files > > 80010001.img to 80011554.img (1554 map files) Osmmap.img of 7Mb > Osmmap_mdr.img of 760Mb > > I assumed the 7Mb file is the overview map? Not the Osmmap_mdr.img > file? > For planet.osm (56Gb in stead of 24Gb) would then make an overview > map of lets say approximately 20Mb My planet ovm work files together > are 1.2Gb 😊 this is as pointed out, not a reference So I assume th > e real problem is that the overview map is loaded with too many > objects. > > I will study Gerd's proposal first (for now I get the concept but > don't understand the exact procedure) But with a good backup not much > can go wrong. > > Thank you for the solutions and the effort off course, Kind regards > Joris > > > Met vriendelijke groeten, > > Joris Bo > jori...@hotmail.com > > -Oorspronkelijk bericht- > Van: mkgmap-dev Namens Gerd > Petermann > Verzonden: zaterdag 27 februari 2021 17:58 > Aan: Development list for mkgmap > Onderwerp: Re: [mkgmap-dev] Error writing overviewmap - continue at > failure > > Hi Mike and Joris, > > as Ticker pointer out before the overall sum of file sizes for the > ovm*.img files is typically much larger than the resultiing overview > map. Reason is that the ovm* files contain the full LBL section (all > Strings that appear in the detailed map), only a few of them are used > in the overview map. > > Gerd > > > Von: mkgmap-dev im Auftrag > von Mike Baggaley > Gesendet: Samstag, 27. Februar 2021 17:38 > An: 'Development list for mkgmap' > Betreff: Re: [mkgmap-dev] Error writing overviewmap - continue at > failure > > Hi Joris, > > I will soon be passing an updated patch to Gerd to improve this, as I > had also been experiencing similar problems. The overview map is not > built until all the tiles have been created (assuming the error is > not in an overview tile), and I don’t think it uses the style file at > that point, just merges all the ovm data, so using a new style will > not help without restarting your run. The message produced by the > patch when the overview file is too big is “The overview map is too > big. The maximum size is 16777215. Try reducing the highest overview > resolution or reducing the amount of information included in the > overview.” If you build your own version of mkgmap, you can use the > version of the patch I posted a few days ago. For the whole world, I > expect the maximum resolution of overview to be pretty low. If I > build a single UK c00ff >normal >
Re: [mkgmap-dev] Error writing overviewmap - continue at failure
Hi Joris, the message is printed when the LBL file contains so many citie names. Those names are are extracted from mkgmap:city (maybe more), so maybe addr:city is a problem. Probably a mapping error in that area. Gerd Von: mkgmap-dev im Auftrag von Joris Bo Gesendet: Sonntag, 28. Februar 2021 11:11 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error writing overviewmap - continue at failure Hi When reading my error log again there was another error. Too many cities... Could it have caused my overview map too fail? SEVERE (MapBuilder): C:\Garmin\Temp\8004 - JBM - Planet\Output Splitter\80041630.osm.pbf: Map contains 94484 Cities - more than 65535 might cause indexing problems and excess size. Suggest splitter with lower --max-nodes I Investigated the area (which is more or less a dessert in Jemen :-) Overpass does not count a lot place = city. Could somebody help to explain? Could this halfway-error also be a reason to fail the overview map being created at the end? Kind regards Joris KML - file of the area http://www.opengis.net/kml/2.2";>1.5 80041630 #transWhitePoly 43.637695,13.666992 43.637695,16.962891 45.571289,16.962891 45.571289,13.666992 43.637695,13.666992 Met vriendelijke groeten, Joris Bo jori...@hotmail.com -Oorspronkelijk bericht- Van: mkgmap-dev Namens Joris Bo Verzonden: zaterdag 27 februari 2021 18:40 Aan: Development list for mkgmap Onderwerp: Re: [mkgmap-dev] Error writing overviewmap - continue at failure Hi Mike & Gerd , My europe map (Europe-latest.osm.pbf split of 24Gb) which works fine, has these img- files 80010001.img to 80011554.img (1554 map files) Osmmap.img of 7Mb Osmmap_mdr.img of 760Mb I assumed the 7Mb file is the overview map? Not the Osmmap_mdr.img file? For planet.osm (56Gb in stead of 24Gb) would then make an overview map of lets say approximately 20Mb My planet ovm work files together are 1.2Gb 😊 this is as pointed out, not a reference So I assume the real problem is that the overview map is loaded with too many objects. I will study Gerd's proposal first (for now I get the concept but don't understand the exact procedure) But with a good backup not much can go wrong. Thank you for the solutions and the effort off course, Kind regards Joris Met vriendelijke groeten, Joris Bo jori...@hotmail.com -Oorspronkelijk bericht- Van: mkgmap-dev Namens Gerd Petermann Verzonden: zaterdag 27 februari 2021 17:58 Aan: Development list for mkgmap Onderwerp: Re: [mkgmap-dev] Error writing overviewmap - continue at failure Hi Mike and Joris, as Ticker pointer out before the overall sum of file sizes for the ovm*.img files is typically much larger than the resultiing overview map. Reason is that the ovm* files contain the full LBL section (all Strings that appear in the detailed map), only a few of them are used in the overview map. Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Samstag, 27. Februar 2021 17:38 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] Error writing overviewmap - continue at failure Hi Joris, I will soon be passing an updated patch to Gerd to improve this, as I had also been experiencing similar problems. The overview map is not built until all the tiles have been created (assuming the error is not in an overview tile), and I don’t think it uses the style file at that point, just merges all the ovm data, so using a new style will not help without restarting your run. The message produced by the patch when the overview file is too big is “The overview map is too big. The maximum size is 16777215. Try reducing the highest overview resolution or reducing the amount of information included in the overview.” If you build your own version of mkgmap, you can use the version of the patch I posted a few days ago. For the whole world, I expect the maximum resolution of overview to be pretty low. If I build a single UK county, I can have an overview at resolution 22, but only 20 for the UK. I haven’t tried anything bigger. You may get some idea how close you are to the limit by totalling the sizes of the ovm files. The maximum size may not necessarily be the max for the whole file, it may be the limit of a sub file though. That is certainly the case for ordinary map tiles, but I’m not sure on the overview. I also have some code that I’m working on that adds support for a mkgmap:nooverview tag in the style file, so that if you have overlapping main and overview resolutions, you can direct your style to omit some of the information from the overv 00ff normal
Re: [mkgmap-dev] Error writing overviewmap - continue at failure
Hi When reading my error log again there was another error. Too many cities... Could it have caused my overview map too fail? SEVERE (MapBuilder): C:\Garmin\Temp\8004 - JBM - Planet\Output Splitter\80041630.osm.pbf: Map contains 94484 Cities - more than 65535 might cause indexing problems and excess size. Suggest splitter with lower --max-nodes I Investigated the area (which is more or less a dessert in Jemen :-) Overpass does not count a lot place = city. Could somebody help to explain? Could this halfway-error also be a reason to fail the overview map being created at the end? Kind regards Joris KML - file of the area http://www.opengis.net/kml/2.2";>1.5 80041630 #transWhitePoly 43.637695,13.666992 43.637695,16.962891 45.571289,16.962891 45.571289,13.666992 43.637695,13.666992 Met vriendelijke groeten, Joris Bo jori...@hotmail.com -Oorspronkelijk bericht- Van: mkgmap-dev Namens Joris Bo Verzonden: zaterdag 27 februari 2021 18:40 Aan: Development list for mkgmap Onderwerp: Re: [mkgmap-dev] Error writing overviewmap - continue at failure Hi Mike & Gerd , My europe map (Europe-latest.osm.pbf split of 24Gb) which works fine, has these img- files 80010001.img to 80011554.img (1554 map files) Osmmap.img of 7Mb Osmmap_mdr.img of 760Mb I assumed the 7Mb file is the overview map? Not the Osmmap_mdr.img file? For planet.osm (56Gb in stead of 24Gb) would then make an overview map of lets say approximately 20Mb My planet ovm work files together are 1.2Gb 😊 this is as pointed out, not a reference So I assume the real problem is that the overview map is loaded with too many objects. I will study Gerd's proposal first (for now I get the concept but don't understand the exact procedure) But with a good backup not much can go wrong. Thank you for the solutions and the effort off course, Kind regards Joris Met vriendelijke groeten, Joris Bo jori...@hotmail.com -Oorspronkelijk bericht- Van: mkgmap-dev Namens Gerd Petermann Verzonden: zaterdag 27 februari 2021 17:58 Aan: Development list for mkgmap Onderwerp: Re: [mkgmap-dev] Error writing overviewmap - continue at failure Hi Mike and Joris, as Ticker pointer out before the overall sum of file sizes for the ovm*.img files is typically much larger than the resultiing overview map. Reason is that the ovm* files contain the full LBL section (all Strings that appear in the detailed map), only a few of them are used in the overview map. Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Samstag, 27. Februar 2021 17:38 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] Error writing overviewmap - continue at failure Hi Joris, I will soon be passing an updated patch to Gerd to improve this, as I had also been experiencing similar problems. The overview map is not built until all the tiles have been created (assuming the error is not in an overview tile), and I don’t think it uses the style file at that point, just merges all the ovm data, so using a new style will not help without restarting your run. The message produced by the patch when the overview file is too big is “The overview map is too big. The maximum size is 16777215. Try reducing the highest overview resolution or reducing the amount of information included in the overview.” If you build your own version of mkgmap, you can use the version of the patch I posted a few days ago. For the whole world, I expect the maximum resolution of overview to be pretty low. If I build a single UK county, I can have an overview at resolution 22, but only 20 for the UK. I haven’t tried anything bigger. You may get some idea how close you are to the limit by totalling the sizes of the ovm files. The maximum size may not necessarily be the max for the whole file, it may be the limit of a sub file though. That is certainly the case for ordinary map tiles, but I’m not sure on the overview. I also have some code that I’m working on that adds support for a mkgmap:nooverview tag in the style file, so that if you have overlapping main and overview resolutions, you can direct your style to omit some of the information from the overview. Will be passing this patch over shortly as well. I don’t think allowing a continuation after a failure in the final overview map would be helpful, as there is very little left to do at that point. There are some issues with the keep going handling and I’m also looking at that. Cheers, Mike From: Joris Bo [mailto:jori...@hotmail.com] Sent: 27 February 2021 14:49 To: Development list for mkgmap Subject: [mkgmap-dev] Error writing overviewmap 00ff normal
