Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, I think self-intersection is not the (only) root cause for the problems of empty sea areas. At least the Garmin software doesn't seem to have problem with that. I learned that we really should avoid to split in PolygonSplitterFilter with resolution < 24. The split process can create small fragments which collaps to a single point or line when rounding and thus are not added to the map. Maybe the code could be changed to merge the small fragments again when the goal of the splitting is the reduction of points. That also means that my idea of "merging all shapes before splitting into subdivs" doesn't work well when this means that the merged polygon is split again. I'd also prefer to merge only those shapes which share at least one segment but the cutting of multipolygons often produces two shapes which only seem to have a common segment but in fact do not, probably a possible common node is removed when converting to java area. I've not yet found out why this happens so often, I would have expected that this is rare. My suspect is the CutPoint.compareTo() method. There are various ways to merge n shapes even when you only merge when shapes have a common segment, so I think that it is OK to get different results as the algorithmn doesn't even try to avoid self-intersections. Whenever shapes are merged so that they enclose a hole you have self-intersection. It could be avoided to do such a merge, I'll try if this helps. The current code just tries to merge the longer sequence of common segments. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Dienstag, 25. Mai 2021 10:11 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I think we need to step back and consider the various stages towards a good overview map which I see as: 0/ MultiPolygons spliting (just added this after I wrote the rest) 1/ subDiv shapeMerge 2/ filter chain into the ovm_ img 3/ loading all the ovm_ img into the combiner 4/ any special global merging (per overview level?) 5/ standard-ish map generation into the overview map ie subdivs as necessary, shape allocation into them with possible splitting, subdiv shapeMerge, filtering & output Just considering subDiv shapeMerge, because, while this is producing self-intersection and/or other errors, looking to solve problems in the following stages is futile. For stress-testing, it should be allowed to merge without a point limit at res24; ShapeSpitter will deal it or report errors. Verification should be done by looking at the ovm_ img rather than the final overview map. It must be wrong that its behaviour depends on the processing order. You say it just uses identity of coord points to make all decisions; ie it doesn't try and test if a point in one shape is on a line in another shape. Assuming this: To merge 2 shapes, they must have a LineSegment in common (ie each has 2 identical adjacent points in the correct direction (determinable from the sign of the area). From finding a common LineSegment, adjacent common LineSegments are included. Then the 2 shapes are joined without these LineSegments (after any necessary direction correction). Any other common LineSegments between these polygons must be ignored, otherwise holes will get closed up. The process is repeated until all shapes in the candidate pool have been considered. Sharing a single point is not good enough to consider a merge - I see figure-of-8 problems emerging from trying this. Ticker ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd I think we need to step back and consider the various stages towards a good overview map which I see as: 0/ MultiPolygons spliting (just added this after I wrote the rest) 1/ subDiv shapeMerge 2/ filter chain into the ovm_ img 3/ loading all the ovm_ img into the combiner 4/ any special global merging (per overview level?) 5/ standard-ish map generation into the overview map ie subdivs as necessary, shape allocation into them with possible splitting, subdiv shapeMerge, filtering & output Just considering subDiv shapeMerge, because, while this is producing self-intersection and/or other errors, looking to solve problems in the following stages is futile. For stress-testing, it should be allowed to merge without a point limit at res24; ShapeSpitter will deal it or report errors. Verification should be done by looking at the ovm_ img rather than the final overview map. It must be wrong that its behaviour depends on the processing order. You say it just uses identity of coord points to make all decisions; ie it doesn't try and test if a point in one shape is on a line in another shape. Assuming this: To merge 2 shapes, they must have a LineSegment in common (ie each has 2 identical adjacent points in the correct direction (determinable from the sign of the area). From finding a common LineSegment, adjacent common LineSegments are included. Then the 2 shapes are joined without these LineSegments (after any necessary direction correction). Any other common LineSegments between these polygons must be ignored, otherwise holes will get closed up. The process is repeated until all shapes in the candidate pool have been considered. Sharing a single point is not good enough to consider a merge - I see figure-of-8 problems emerging from trying this. Ticker ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, that's an interesting idea. Did you try that? My results are even worse, but maybe I changed the wrong places. For sure the ovm* files are much larger. I guess there will also be problems with memory when you try to combine many tiles. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Montag, 24. Mai 2021 18:00 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd This seems to be false when combining the ovm_, rather than when generating them! It suppresses RoundCoords but maybe shouldn't. Not sure why RoundCoords should ever be suppressed when res < 24? It doesn't suppress removeObsolete, which removes points in any straight line. I suppose the simpler way to view this is that unique points per element on the tile boundary should be kept; making the job of the overview combiner mergeShape possible. Ticker On Mon, 2021-05-24 at 15:35 +, Gerd Petermann wrote: > Hi Ticker, > > yes, it is. See usage of enableLineCleanFilters > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Montag, 24. Mai 2021 17:33 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > Not quite related, but if merging the shapes from all ovm_ imgs, then > maybe some types of filtering should be turned off when the contents > for these is being generated; esp DP, and probably straigh-line > removal > in removeObsolete > > Ticker > > On Mon, 2021-05-24 at 15:21 +, Gerd Petermann wrote: > > Hi Ticker, > > > > sorry, I can't follow. The old code only preserved points on the > > boundary of the merged shape, but the outer ring can be far from > > rectangular. > > This way some important points on outer ring were not preserved. > > > > There's still a possible problem with large white area in sea which > > seems to depend on the order in which shapes are merged. Sill > > trying > > to understand what goes, but at least I can reproduce it... > > > > Gerd > > > > > > ____________________ > > Von: mkgmap-dev im Auftrag > > von Ticker Berkin > > Gesendet: Montag, 24. Mai 2021 17:11 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > > > Hi Gerd > > > > Re. last commit: preserve all horizontal and vertical lines in > > shapes. > > > > Do the cut lines go right across the outer polygon or just in from > > one > > side to the hole? If all the way across, might this change inhibit > > the > > merging to one side of the hole, or, if divided for some other > > reason > > in the other direction, merging across the now unneeded cut line. > > > > Ticker > > > > ___ > > mkgmap-dev mailing list > > mkgmap-dev@lists.mkgmap.org.uk > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > > ___ > > mkgmap-dev mailing list > > mkgmap-dev@lists.mkgmap.org.uk > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd This seems to be false when combining the ovm_, rather than when generating them! It suppresses RoundCoords but maybe shouldn't. Not sure why RoundCoords should ever be suppressed when res < 24? It doesn't suppress removeObsolete, which removes points in any straight line. I suppose the simpler way to view this is that unique points per element on the tile boundary should be kept; making the job of the overview combiner mergeShape possible. Ticker On Mon, 2021-05-24 at 15:35 +, Gerd Petermann wrote: > Hi Ticker, > > yes, it is. See usage of enableLineCleanFilters > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Montag, 24. Mai 2021 17:33 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > Not quite related, but if merging the shapes from all ovm_ imgs, then > maybe some types of filtering should be turned off when the contents > for these is being generated; esp DP, and probably straigh-line > removal > in removeObsolete > > Ticker > > On Mon, 2021-05-24 at 15:21 +, Gerd Petermann wrote: > > Hi Ticker, > > > > sorry, I can't follow. The old code only preserved points on the > > boundary of the merged shape, but the outer ring can be far from > > rectangular. > > This way some important points on outer ring were not preserved. > > > > There's still a possible problem with large white area in sea which > > seems to depend on the order in which shapes are merged. Sill > > trying > > to understand what goes, but at least I can reproduce it... > > > > Gerd > > > > > > ____________________ > > Von: mkgmap-dev im Auftrag > > von Ticker Berkin > > Gesendet: Montag, 24. Mai 2021 17:11 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > > > Hi Gerd > > > > Re. last commit: preserve all horizontal and vertical lines in > > shapes. > > > > Do the cut lines go right across the outer polygon or just in from > > one > > side to the hole? If all the way across, might this change inhibit > > the > > merging to one side of the hole, or, if divided for some other > > reason > > in the other direction, merging across the now unneeded cut line. > > > > Ticker > > > > ___ > > mkgmap-dev mailing list > > mkgmap-dev@lists.mkgmap.org.uk > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > > ___ > > mkgmap-dev mailing list > > mkgmap-dev@lists.mkgmap.org.uk > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, yes, it is. See usage of enableLineCleanFilters Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Montag, 24. Mai 2021 17:33 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd Not quite related, but if merging the shapes from all ovm_ imgs, then maybe some types of filtering should be turned off when the contents for these is being generated; esp DP, and probably straigh-line removal in removeObsolete Ticker On Mon, 2021-05-24 at 15:21 +, Gerd Petermann wrote: > Hi Ticker, > > sorry, I can't follow. The old code only preserved points on the > boundary of the merged shape, but the outer ring can be far from > rectangular. > This way some important points on outer ring were not preserved. > > There's still a possible problem with large white area in sea which > seems to depend on the order in which shapes are merged. Sill trying > to understand what goes, but at least I can reproduce it... > > Gerd > > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Montag, 24. Mai 2021 17:11 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > Re. last commit: preserve all horizontal and vertical lines in > shapes. > > Do the cut lines go right across the outer polygon or just in from > one > side to the hole? If all the way across, might this change inhibit > the > merging to one side of the hole, or, if divided for some other reason > in the other direction, merging across the now unneeded cut line. > > Ticker > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd Not quite related, but if merging the shapes from all ovm_ imgs, then maybe some types of filtering should be turned off when the contents for these is being generated; esp DP, and probably straigh-line removal in removeObsolete Ticker On Mon, 2021-05-24 at 15:21 +, Gerd Petermann wrote: > Hi Ticker, > > sorry, I can't follow. The old code only preserved points on the > boundary of the merged shape, but the outer ring can be far from > rectangular. > This way some important points on outer ring were not preserved. > > There's still a possible problem with large white area in sea which > seems to depend on the order in which shapes are merged. Sill trying > to understand what goes, but at least I can reproduce it... > > Gerd > > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Montag, 24. Mai 2021 17:11 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > Re. last commit: preserve all horizontal and vertical lines in > shapes. > > Do the cut lines go right across the outer polygon or just in from > one > side to the hole? If all the way across, might this change inhibit > the > merging to one side of the hole, or, if divided for some other reason > in the other direction, merging across the now unneeded cut line. > > Ticker > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, sorry, I can't follow. The old code only preserved points on the boundary of the merged shape, but the outer ring can be far from rectangular. This way some important points on outer ring were not preserved. There's still a possible problem with large white area in sea which seems to depend on the order in which shapes are merged. Sill trying to understand what goes, but at least I can reproduce it... Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Montag, 24. Mai 2021 17:11 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd Re. last commit: preserve all horizontal and vertical lines in shapes. Do the cut lines go right across the outer polygon or just in from one side to the hole? If all the way across, might this change inhibit the merging to one side of the hole, or, if divided for some other reason in the other direction, merging across the now unneeded cut line. Ticker ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd Re. last commit: preserve all horizontal and vertical lines in shapes. Do the cut lines go right across the outer polygon or just in from one side to the hole? If all the way across, might this change inhibit the merging to one side of the hole, or, if divided for some other reason in the other direction, merging across the now unneeded cut line. Ticker ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Ah When I scanned this area a while ago I guessed, without looking carefully or thinking, that mergeShapesFirst took all the shapes from all the ovm_ imgs and attempted to merge them. I see now that it doesn't. I don't know what problem it can solve better than the standard subdivision merge. It might have been there because, a long time ago, outer zoom subdivs (esp. overview) were far too small when there was anything (sea/coastline) because the content #points was calculated at res24. So, yes, I'd try without. Ticker On Mon, 2021-05-24 at 14:19 +, Gerd Petermann wrote: > Oops, I meant > should I remove mergeShapesFirst() completely? > > Gerd > > > Von: mkgmap-dev im Auftrag > von Gerd Petermann > Gesendet: Montag, 24. Mai 2021 16:07 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Ticker, > > you mean I should mergeShapesFirst() completely? > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Montag, 24. Mai 2021 15:57 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > Not sure about this. It is a very drastic change and might break a > lot > of the element>subdiv code (esp --order-by). Many element well be too > big and have to be split again. > > It ends up as a merge attempt at all lines/shapes in the tile, so it > might as well be resolution independent! > > Ticker > On Mon, 2021-05-24 at 13:33 +, Gerd Petermann wrote: > > Hi Ticker, > > > > I think I found my mistake. The code to re-calc the preserved > > points > > was executed for each sub division instead of once per level. > > > > The attached patch might already solve this. > > > > Gerd > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd That would be the conclusion - but I don't think you should do it. I don't see that making the problem bigger will help! Ticker On Mon, 2021-05-24 at 14:07 +, Gerd Petermann wrote: > Hi Ticker, > > you mean I should mergeShapesFirst() completely? > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Montag, 24. Mai 2021 15:57 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > Not sure about this. It is a very drastic change and might break a > lot > of the element>subdiv code (esp --order-by). Many element well be too > big and have to be split again. > > It ends up as a merge attempt at all lines/shapes in the tile, so it > might as well be resolution independent! > > Ticker > On Mon, 2021-05-24 at 13:33 +, Gerd Petermann wrote: > > Hi Ticker, > > > > I think I found my mistake. The code to re-calc the preserved > > points > > was executed for each sub division instead of once per level. > > > > The attached patch might already solve this. > > > > Gerd > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Oops, I meant should I remove mergeShapesFirst() completely? Gerd Von: mkgmap-dev im Auftrag von Gerd Petermann Gesendet: Montag, 24. Mai 2021 16:07 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Ticker, you mean I should mergeShapesFirst() completely? Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Montag, 24. Mai 2021 15:57 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd Not sure about this. It is a very drastic change and might break a lot of the element>subdiv code (esp --order-by). Many element well be too big and have to be split again. It ends up as a merge attempt at all lines/shapes in the tile, so it might as well be resolution independent! Ticker On Mon, 2021-05-24 at 13:33 +, Gerd Petermann wrote: > Hi Ticker, > > I think I found my mistake. The code to re-calc the preserved points > was executed for each sub division instead of once per level. > > The attached patch might already solve this. > > Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, you mean I should mergeShapesFirst() completely? Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Montag, 24. Mai 2021 15:57 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd Not sure about this. It is a very drastic change and might break a lot of the element>subdiv code (esp --order-by). Many element well be too big and have to be split again. It ends up as a merge attempt at all lines/shapes in the tile, so it might as well be resolution independent! Ticker On Mon, 2021-05-24 at 13:33 +, Gerd Petermann wrote: > Hi Ticker, > > I think I found my mistake. The code to re-calc the preserved points > was executed for each sub division instead of once per level. > > The attached patch might already solve this. > > Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd Not sure about this. It is a very drastic change and might break a lot of the element>subdiv code (esp --order-by). Many element well be too big and have to be split again. It ends up as a merge attempt at all lines/shapes in the tile, so it might as well be resolution independent! Ticker On Mon, 2021-05-24 at 13:33 +, Gerd Petermann wrote: > Hi Ticker, > > I think I found my mistake. The code to re-calc the preserved points > was executed for each sub division instead of once per level. > > The attached patch might already solve this. > > Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, I think I found my mistake. The code to re-calc the preserved points was executed for each sub division instead of once per level. The attached patch might already solve this. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Montag, 24. Mai 2021 15:03 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd See other thread about DP / removeObsolete ordering. RemoveObsolete tidies up the remains after DP has done things it shouldn't For non-road lines re. first and last - I don't think any filters will cause there not to be a point at the res-coord of the start or end, so I don't see that there would be a gap for different types. ShapeSplitter will notice if the dividing line has inconsistent (non -alternating) direction crossings (probably caused by self -intersection). It can't deal with this, just issues a warning. It won't notice self-intersection elsewhere and allocates "outside" before the first line crossing, "inside" next and alternates. I think it is possible to come up with shapes where this is the opposite what you'd expect. Maybe we need a proper intersecting segments checker. I've investigated some algos. Not sure what to do with self-intersection when we find it. Lots are simply output in RGN and devices might have some defined way of rendering. It's only the cutting and the overview merging that don't really have a solution. Ticker On Mon, 2021-05-24 at 12:20 +, Gerd Petermann wrote: > Hi Ticker, > > the removeObsolete filter sometimes ignores the preserved status, and > probably it doesn't make sure to keep or copy that status. Maybe that > explains why a following DP gives other results? I think in the > current order there is no problem with this but I'll double check > because I think somewhere mkgmap removes points which should be > preserved. > > The first/last point of lines is preserved to avoid gaps between > connected lines of different types, e.g. when a stream gets a river. > > The shape merger produces self intersecting shapes. I think this is > not a problem as long as ShapeSplitter can handle this. Can it? > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Montag, 24. Mai 2021 12:54 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > The meaning of the preserved flag + handling of road/line/polygon etc > to different filters seem to be getting a bit mixed up. > > I'm currently seeing some problems because of first/last points in > line > being preserved (which some filters respect and others don't). I can > understand this for Roads, but not other lines. Is there a reason for > this. > > preserveHorizonalAndVertical preserving multiple use points might be > good initially but could hinder removeObsolete spike removal from > removing the cuts to a collapsed hole. > > Still haven't got to grips with shapeMergeFilter, but the main tile / > ovm_ generation is producing self-intersecting polygons. > > Ticker > > On Mon, 2021-05-24 at 06:52 +, Gerd Petermann wrote: > > Hi Ticker / all, > > > > it seems I've looked at the wrong test files. I've tested with a > > few > > more input files and now I see large white triangles with the > > branch > > version 4736 while trunk 4735 doesn't show them. > > Looks like a problem with preserving points or maybe special cases > > with ShapeSplitter... > > > > Gerd > > > > > > > > > > Von: mkgmap-dev im Auftrag > > von Ticker Berkin > > Gesendet: Samstag, 22. Mai 2021 13:30 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > > > Hi Gerd > > > > I've been doing experiments on tiles with much more sea and islands > > (tests before only had a bit of sea and lots of city). > > > > With my re-ordering I get a very slightly smaller main tile (4096 > > bytes > > in 5614K) and larger osmmap.img (34304 vs. 25088) > > > > In both cases I'm getting quite a few shapeSplitter messages where > > it > > finds inconsistencies in the line directions across the split-line; > > normally due to self-intersection. These are happening in both the > > main > > tile generation and the overview combining. > > > > Once shapeSplitter has generated this type of warning it is > > indeterminate which is shape and which is hole, so I'd expect gaps &g
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd See other thread about DP / removeObsolete ordering. RemoveObsolete tidies up the remains after DP has done things it shouldn't For non-road lines re. first and last - I don't think any filters will cause there not to be a point at the res-coord of the start or end, so I don't see that there would be a gap for different types. ShapeSplitter will notice if the dividing line has inconsistent (non -alternating) direction crossings (probably caused by self -intersection). It can't deal with this, just issues a warning. It won't notice self-intersection elsewhere and allocates "outside" before the first line crossing, "inside" next and alternates. I think it is possible to come up with shapes where this is the opposite what you'd expect. Maybe we need a proper intersecting segments checker. I've investigated some algos. Not sure what to do with self-intersection when we find it. Lots are simply output in RGN and devices might have some defined way of rendering. It's only the cutting and the overview merging that don't really have a solution. Ticker On Mon, 2021-05-24 at 12:20 +, Gerd Petermann wrote: > Hi Ticker, > > the removeObsolete filter sometimes ignores the preserved status, and > probably it doesn't make sure to keep or copy that status. Maybe that > explains why a following DP gives other results? I think in the > current order there is no problem with this but I'll double check > because I think somewhere mkgmap removes points which should be > preserved. > > The first/last point of lines is preserved to avoid gaps between > connected lines of different types, e.g. when a stream gets a river. > > The shape merger produces self intersecting shapes. I think this is > not a problem as long as ShapeSplitter can handle this. Can it? > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Montag, 24. Mai 2021 12:54 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > The meaning of the preserved flag + handling of road/line/polygon etc > to different filters seem to be getting a bit mixed up. > > I'm currently seeing some problems because of first/last points in > line > being preserved (which some filters respect and others don't). I can > understand this for Roads, but not other lines. Is there a reason for > this. > > preserveHorizonalAndVertical preserving multiple use points might be > good initially but could hinder removeObsolete spike removal from > removing the cuts to a collapsed hole. > > Still haven't got to grips with shapeMergeFilter, but the main tile / > ovm_ generation is producing self-intersecting polygons. > > Ticker > > On Mon, 2021-05-24 at 06:52 +, Gerd Petermann wrote: > > Hi Ticker / all, > > > > it seems I've looked at the wrong test files. I've tested with a > > few > > more input files and now I see large white triangles with the > > branch > > version 4736 while trunk 4735 doesn't show them. > > Looks like a problem with preserving points or maybe special cases > > with ShapeSplitter... > > > > Gerd > > > > > > > > > > Von: mkgmap-dev im Auftrag > > von Ticker Berkin > > Gesendet: Samstag, 22. Mai 2021 13:30 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > > > Hi Gerd > > > > I've been doing experiments on tiles with much more sea and islands > > (tests before only had a bit of sea and lots of city). > > > > With my re-ordering I get a very slightly smaller main tile (4096 > > bytes > > in 5614K) and larger osmmap.img (34304 vs. 25088) > > > > In both cases I'm getting quite a few shapeSplitter messages where > > it > > finds inconsistencies in the line directions across the split-line; > > normally due to self-intersection. These are happening in both the > > main > > tile generation and the overview combining. > > > > Once shapeSplitter has generated this type of warning it is > > indeterminate which is shape and which is hole, so I'd expect gaps > > in > > the rendering. > > > > I think trunk had eradicated split/self-intersection problems in > > the > > tile processing, including the ovm_ file generation. > > > > I'm not sure about the overview combiner as, in my standard map > > -build, > > I don't merge shapes. > > > > Ticker > > > > On
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, the removeObsolete filter sometimes ignores the preserved status, and probably it doesn't make sure to keep or copy that status. Maybe that explains why a following DP gives other results? I think in the current order there is no problem with this but I'll double check because I think somewhere mkgmap removes points which should be preserved. The first/last point of lines is preserved to avoid gaps between connected lines of different types, e.g. when a stream gets a river. The shape merger produces self intersecting shapes. I think this is not a problem as long as ShapeSplitter can handle this. Can it? Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Montag, 24. Mai 2021 12:54 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd The meaning of the preserved flag + handling of road/line/polygon etc to different filters seem to be getting a bit mixed up. I'm currently seeing some problems because of first/last points in line being preserved (which some filters respect and others don't). I can understand this for Roads, but not other lines. Is there a reason for this. preserveHorizonalAndVertical preserving multiple use points might be good initially but could hinder removeObsolete spike removal from removing the cuts to a collapsed hole. Still haven't got to grips with shapeMergeFilter, but the main tile / ovm_ generation is producing self-intersecting polygons. Ticker On Mon, 2021-05-24 at 06:52 +, Gerd Petermann wrote: > Hi Ticker / all, > > it seems I've looked at the wrong test files. I've tested with a few > more input files and now I see large white triangles with the branch > version 4736 while trunk 4735 doesn't show them. > Looks like a problem with preserving points or maybe special cases > with ShapeSplitter... > > Gerd > > > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Samstag, 22. Mai 2021 13:30 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I've been doing experiments on tiles with much more sea and islands > (tests before only had a bit of sea and lots of city). > > With my re-ordering I get a very slightly smaller main tile (4096 > bytes > in 5614K) and larger osmmap.img (34304 vs. 25088) > > In both cases I'm getting quite a few shapeSplitter messages where it > finds inconsistencies in the line directions across the split-line; > normally due to self-intersection. These are happening in both the > main > tile generation and the overview combining. > > Once shapeSplitter has generated this type of warning it is > indeterminate which is shape and which is hole, so I'd expect gaps in > the rendering. > > I think trunk had eradicated split/self-intersection problems in the > tile processing, including the ovm_ file generation. > > I'm not sure about the overview combiner as, in my standard map > -build, > I don't merge shapes. > > Ticker > > On Fri, 2021-05-21 at 16:41 +, Gerd Petermann wrote: > > Hi Ticker, > > > > I tried this patch with r4734 and I see no improvements. Maps are > > larger, more white rectangles in sea. > > > > Maybe my recent changes don't work well with yours? Do you have an > > example where it improves something with r4734? > > > > Gerd > > > > > > Von: mkgmap-dev im Auftrag > > von Ticker Berkin > > Gesendet: Freitag, 21. Mai 2021 18:10 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > > > Hi Gerd > > > > I'd been doing some investigation of filters ordering (based on > > trunk). > > I'd also done the pre-filtering of lines & shapes by minRes. > > > > My conclusions are: > > > > Shapes: > > It is better to run SizeFilter after RemoveObsoleteFilter. > > It is more efficient to run DP filter after both of these. > > > > Lines: > > LineSplitterFilter should be run after everything that can remove > > points. > > It is more efficient to run DP after RemoveObsoleteFilter. > > > > I've adapted my changes into a patch for the low-res-opt branch, > > along > > with removal of some resolution tests that are now redundant. > > > > For the contourFilters, I've left DP as the first filter but moved > > LineSplitter as for the normalFilters > > > > Ticker > > > > > > ___ > > mkgm
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd The meaning of the preserved flag + handling of road/line/polygon etc to different filters seem to be getting a bit mixed up. I'm currently seeing some problems because of first/last points in line being preserved (which some filters respect and others don't). I can understand this for Roads, but not other lines. Is there a reason for this. preserveHorizonalAndVertical preserving multiple use points might be good initially but could hinder removeObsolete spike removal from removing the cuts to a collapsed hole. Still haven't got to grips with shapeMergeFilter, but the main tile / ovm_ generation is producing self-intersecting polygons. Ticker On Mon, 2021-05-24 at 06:52 +, Gerd Petermann wrote: > Hi Ticker / all, > > it seems I've looked at the wrong test files. I've tested with a few > more input files and now I see large white triangles with the branch > version 4736 while trunk 4735 doesn't show them. > Looks like a problem with preserving points or maybe special cases > with ShapeSplitter... > > Gerd > > > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Samstag, 22. Mai 2021 13:30 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I've been doing experiments on tiles with much more sea and islands > (tests before only had a bit of sea and lots of city). > > With my re-ordering I get a very slightly smaller main tile (4096 > bytes > in 5614K) and larger osmmap.img (34304 vs. 25088) > > In both cases I'm getting quite a few shapeSplitter messages where it > finds inconsistencies in the line directions across the split-line; > normally due to self-intersection. These are happening in both the > main > tile generation and the overview combining. > > Once shapeSplitter has generated this type of warning it is > indeterminate which is shape and which is hole, so I'd expect gaps in > the rendering. > > I think trunk had eradicated split/self-intersection problems in the > tile processing, including the ovm_ file generation. > > I'm not sure about the overview combiner as, in my standard map > -build, > I don't merge shapes. > > Ticker > > On Fri, 2021-05-21 at 16:41 +, Gerd Petermann wrote: > > Hi Ticker, > > > > I tried this patch with r4734 and I see no improvements. Maps are > > larger, more white rectangles in sea. > > > > Maybe my recent changes don't work well with yours? Do you have an > > example where it improves something with r4734? > > > > Gerd > > > > > > Von: mkgmap-dev im Auftrag > > von Ticker Berkin > > Gesendet: Freitag, 21. Mai 2021 18:10 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > > > Hi Gerd > > > > I'd been doing some investigation of filters ordering (based on > > trunk). > > I'd also done the pre-filtering of lines & shapes by minRes. > > > > My conclusions are: > > > > Shapes: > > It is better to run SizeFilter after RemoveObsoleteFilter. > > It is more efficient to run DP filter after both of these. > > > > Lines: > > LineSplitterFilter should be run after everything that can remove > > points. > > It is more efficient to run DP after RemoveObsoleteFilter. > > > > I've adapted my changes into a patch for the low-res-opt branch, > > along > > with removal of some resolution tests that are now redundant. > > > > For the contourFilters, I've left DP as the first filter but moved > > LineSplitter as for the normalFilters > > > > Ticker > > > > > > ___ > > mkgmap-dev mailing list > > mkgmap-dev@lists.mkgmap.org.uk > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
yes I know - some version last week or so was horrible in this regard. I thought this happened because of trying to make the overview map smaller - so I just wanted to note that the size here is less important. On Mon, 24 May 2021 at 11:29, Gerd Petermann < gpetermann_muenc...@hotmail.com> wrote: > Hi Felix, > > white triangles in the sea are obvious errors and therefore I want to fix > this. The number of points is not the interesting point, it is about > keeping the important points and removing unnecessary ones. This doesn't > work well. The same problem occurs with other complex multipolygons that > have many inners, but you don't see them often in the alps ;) > > Gerd > > > Von: mkgmap-dev im Auftrag von > Felix Hartmann > Gesendet: Montag, 24. Mai 2021 11:07 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > I feel that there is not too much need to get the size of the overview map > down. Even for Asia continent map there is enough room. So it is better to > make sure it looks nice - than smallest size. > Also for resolution higher than overview map - it is usually not too > important. Most people are navigating on land - so sea does not have to be > rendered as often on your GPS device compared to land and roads. Meaning > for >90% of Garmin mkgmap created map users it is not too often that you > have a complex sea in your map. Even if you live on a mid sized island - > say Cyprus or so - most of the time you do not have much sea on your GPS > device. > The recent work on reducing road complexity was much more important in > real world use than what can be achieved by reducing sea complexity. Just > my opinion. Of course nice if there is an improvement - but compared to > land features for most users for sea it is not too important to reduce the > points. The changes recently made did make quite a big difference in speed > when zooming out / panning the map on my Oregon or etrex devices. For sea > there is much less to be gained. Of course it is never bad to see some > improvements - but it is less important. On PC / notebook - usually the > processor should be plenty fast for drawing the map (plus the cache of > Basecamp).. > > On Mon, 24 May 2021 at 08:52, Gerd Petermann < > gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com>> > wrote: > Hi Ticker / all, > > it seems I've looked at the wrong test files. I've tested with a few more > input files and now I see large white triangles with the branch version > 4736 while trunk 4735 doesn't show them. > Looks like a problem with preserving points or maybe special cases with > ShapeSplitter... > > Gerd > > > > > Von: mkgmap-dev mkgmap-dev-boun...@lists.mkgmap.org.uk>> im Auftrag von Ticker Berkin < > rwb-mkg...@jagit.co.uk<mailto:rwb-mkg...@jagit.co.uk>> > Gesendet: Samstag, 22. Mai 2021 13:30 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I've been doing experiments on tiles with much more sea and islands > (tests before only had a bit of sea and lots of city). > > With my re-ordering I get a very slightly smaller main tile (4096 bytes > in 5614K) and larger osmmap.img (34304 vs. 25088) > > In both cases I'm getting quite a few shapeSplitter messages where it > finds inconsistencies in the line directions across the split-line; > normally due to self-intersection. These are happening in both the main > tile generation and the overview combining. > > Once shapeSplitter has generated this type of warning it is > indeterminate which is shape and which is hole, so I'd expect gaps in > the rendering. > > I think trunk had eradicated split/self-intersection problems in the > tile processing, including the ovm_ file generation. > > I'm not sure about the overview combiner as, in my standard map-build, > I don't merge shapes. > > Ticker > > On Fri, 2021-05-21 at 16:41 +, Gerd Petermann wrote: > > Hi Ticker, > > > > I tried this patch with r4734 and I see no improvements. Maps are > > larger, more white rectangles in sea. > > > > Maybe my recent changes don't work well with yours? Do you have an > > example where it improves something with r4734? > > > > Gerd > > > > > > Von: mkgmap-dev mkgmap-dev-boun...@lists.mkgmap.org.uk>> im Auftrag > > von Ticker Berkin mailto:rwb-mkg...@jagit.co.uk > >> > > Gesendet: Freitag, 21. Mai 2021 18:10 > > An: Develop
Re: [mkgmap-dev] Problems with sea in overview map
Hi Felix, white triangles in the sea are obvious errors and therefore I want to fix this. The number of points is not the interesting point, it is about keeping the important points and removing unnecessary ones. This doesn't work well. The same problem occurs with other complex multipolygons that have many inners, but you don't see them often in the alps ;) Gerd Von: mkgmap-dev im Auftrag von Felix Hartmann Gesendet: Montag, 24. Mai 2021 11:07 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map I feel that there is not too much need to get the size of the overview map down. Even for Asia continent map there is enough room. So it is better to make sure it looks nice - than smallest size. Also for resolution higher than overview map - it is usually not too important. Most people are navigating on land - so sea does not have to be rendered as often on your GPS device compared to land and roads. Meaning for >90% of Garmin mkgmap created map users it is not too often that you have a complex sea in your map. Even if you live on a mid sized island - say Cyprus or so - most of the time you do not have much sea on your GPS device. The recent work on reducing road complexity was much more important in real world use than what can be achieved by reducing sea complexity. Just my opinion. Of course nice if there is an improvement - but compared to land features for most users for sea it is not too important to reduce the points. The changes recently made did make quite a big difference in speed when zooming out / panning the map on my Oregon or etrex devices. For sea there is much less to be gained. Of course it is never bad to see some improvements - but it is less important. On PC / notebook - usually the processor should be plenty fast for drawing the map (plus the cache of Basecamp).. On Mon, 24 May 2021 at 08:52, Gerd Petermann mailto:gpetermann_muenc...@hotmail.com>> wrote: Hi Ticker / all, it seems I've looked at the wrong test files. I've tested with a few more input files and now I see large white triangles with the branch version 4736 while trunk 4735 doesn't show them. Looks like a problem with preserving points or maybe special cases with ShapeSplitter... Gerd Von: mkgmap-dev mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>> im Auftrag von Ticker Berkin mailto:rwb-mkg...@jagit.co.uk>> Gesendet: Samstag, 22. Mai 2021 13:30 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I've been doing experiments on tiles with much more sea and islands (tests before only had a bit of sea and lots of city). With my re-ordering I get a very slightly smaller main tile (4096 bytes in 5614K) and larger osmmap.img (34304 vs. 25088) In both cases I'm getting quite a few shapeSplitter messages where it finds inconsistencies in the line directions across the split-line; normally due to self-intersection. These are happening in both the main tile generation and the overview combining. Once shapeSplitter has generated this type of warning it is indeterminate which is shape and which is hole, so I'd expect gaps in the rendering. I think trunk had eradicated split/self-intersection problems in the tile processing, including the ovm_ file generation. I'm not sure about the overview combiner as, in my standard map-build, I don't merge shapes. Ticker On Fri, 2021-05-21 at 16:41 +, Gerd Petermann wrote: > Hi Ticker, > > I tried this patch with r4734 and I see no improvements. Maps are > larger, more white rectangles in sea. > > Maybe my recent changes don't work well with yours? Do you have an > example where it improves something with r4734? > > Gerd > > > Von: mkgmap-dev > mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>> > im Auftrag > von Ticker Berkin mailto:rwb-mkg...@jagit.co.uk>> > Gesendet: Freitag, 21. Mai 2021 18:10 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I'd been doing some investigation of filters ordering (based on > trunk). > I'd also done the pre-filtering of lines & shapes by minRes. > > My conclusions are: > > Shapes: > It is better to run SizeFilter after RemoveObsoleteFilter. > It is more efficient to run DP filter after both of these. > > Lines: > LineSplitterFilter should be run after everything that can remove > points. > It is more efficient to run DP after RemoveObsoleteFilter. > > I've adapted my changes into a patch for the low-res-opt branch, > along > with removal of some resolution tests that are now redundant. > > For the contourFilters, I've left
Re: [mkgmap-dev] Problems with sea in overview map
I feel that there is not too much need to get the size of the overview map down. Even for Asia continent map there is enough room. So it is better to make sure it looks nice - than smallest size. Also for resolution higher than overview map - it is usually not too important. Most people are navigating on land - so sea does not have to be rendered as often on your GPS device compared to land and roads. Meaning for >90% of Garmin mkgmap created map users it is not too often that you have a complex sea in your map. Even if you live on a mid sized island - say Cyprus or so - most of the time you do not have much sea on your GPS device. The recent work on reducing road complexity was much more important in real world use than what can be achieved by reducing sea complexity. Just my opinion. Of course nice if there is an improvement - but compared to land features for most users for sea it is not too important to reduce the points. The changes recently made did make quite a big difference in speed when zooming out / panning the map on my Oregon or etrex devices. For sea there is much less to be gained. Of course it is never bad to see some improvements - but it is less important. On PC / notebook - usually the processor should be plenty fast for drawing the map (plus the cache of Basecamp).. On Mon, 24 May 2021 at 08:52, Gerd Petermann < gpetermann_muenc...@hotmail.com> wrote: > Hi Ticker / all, > > it seems I've looked at the wrong test files. I've tested with a few more > input files and now I see large white triangles with the branch version > 4736 while trunk 4735 doesn't show them. > Looks like a problem with preserving points or maybe special cases with > ShapeSplitter... > > Gerd > > > > > Von: mkgmap-dev im Auftrag von > Ticker Berkin > Gesendet: Samstag, 22. Mai 2021 13:30 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I've been doing experiments on tiles with much more sea and islands > (tests before only had a bit of sea and lots of city). > > With my re-ordering I get a very slightly smaller main tile (4096 bytes > in 5614K) and larger osmmap.img (34304 vs. 25088) > > In both cases I'm getting quite a few shapeSplitter messages where it > finds inconsistencies in the line directions across the split-line; > normally due to self-intersection. These are happening in both the main > tile generation and the overview combining. > > Once shapeSplitter has generated this type of warning it is > indeterminate which is shape and which is hole, so I'd expect gaps in > the rendering. > > I think trunk had eradicated split/self-intersection problems in the > tile processing, including the ovm_ file generation. > > I'm not sure about the overview combiner as, in my standard map-build, > I don't merge shapes. > > Ticker > > On Fri, 2021-05-21 at 16:41 +, Gerd Petermann wrote: > > Hi Ticker, > > > > I tried this patch with r4734 and I see no improvements. Maps are > > larger, more white rectangles in sea. > > > > Maybe my recent changes don't work well with yours? Do you have an > > example where it improves something with r4734? > > > > Gerd > > > > > > Von: mkgmap-dev im Auftrag > > von Ticker Berkin > > Gesendet: Freitag, 21. Mai 2021 18:10 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > > > Hi Gerd > > > > I'd been doing some investigation of filters ordering (based on > > trunk). > > I'd also done the pre-filtering of lines & shapes by minRes. > > > > My conclusions are: > > > > Shapes: > > It is better to run SizeFilter after RemoveObsoleteFilter. > > It is more efficient to run DP filter after both of these. > > > > Lines: > > LineSplitterFilter should be run after everything that can remove > > points. > > It is more efficient to run DP after RemoveObsoleteFilter. > > > > I've adapted my changes into a patch for the low-res-opt branch, > > along > > with removal of some resolution tests that are now redundant. > > > > For the contourFilters, I've left DP as the first filter but moved > > LineSplitter as for the normalFilters > > > > Ticker > > > > > > ___ > > mkgmap-dev mailing list > > mkgmap-dev@lists.mkgmap.org.uk > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > -- Felix Hartman - Openmtbmap.org & VeloMap.org ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker / all, it seems I've looked at the wrong test files. I've tested with a few more input files and now I see large white triangles with the branch version 4736 while trunk 4735 doesn't show them. Looks like a problem with preserving points or maybe special cases with ShapeSplitter... Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Samstag, 22. Mai 2021 13:30 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I've been doing experiments on tiles with much more sea and islands (tests before only had a bit of sea and lots of city). With my re-ordering I get a very slightly smaller main tile (4096 bytes in 5614K) and larger osmmap.img (34304 vs. 25088) In both cases I'm getting quite a few shapeSplitter messages where it finds inconsistencies in the line directions across the split-line; normally due to self-intersection. These are happening in both the main tile generation and the overview combining. Once shapeSplitter has generated this type of warning it is indeterminate which is shape and which is hole, so I'd expect gaps in the rendering. I think trunk had eradicated split/self-intersection problems in the tile processing, including the ovm_ file generation. I'm not sure about the overview combiner as, in my standard map-build, I don't merge shapes. Ticker On Fri, 2021-05-21 at 16:41 +, Gerd Petermann wrote: > Hi Ticker, > > I tried this patch with r4734 and I see no improvements. Maps are > larger, more white rectangles in sea. > > Maybe my recent changes don't work well with yours? Do you have an > example where it improves something with r4734? > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Freitag, 21. Mai 2021 18:10 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I'd been doing some investigation of filters ordering (based on > trunk). > I'd also done the pre-filtering of lines & shapes by minRes. > > My conclusions are: > > Shapes: > It is better to run SizeFilter after RemoveObsoleteFilter. > It is more efficient to run DP filter after both of these. > > Lines: > LineSplitterFilter should be run after everything that can remove > points. > It is more efficient to run DP after RemoveObsoleteFilter. > > I've adapted my changes into a patch for the low-res-opt branch, > along > with removal of some resolution tests that are now redundant. > > For the contourFilters, I've left DP as the first filter but moved > LineSplitter as for the normalFilters > > Ticker > > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd I've been doing experiments on tiles with much more sea and islands (tests before only had a bit of sea and lots of city). With my re-ordering I get a very slightly smaller main tile (4096 bytes in 5614K) and larger osmmap.img (34304 vs. 25088) In both cases I'm getting quite a few shapeSplitter messages where it finds inconsistencies in the line directions across the split-line; normally due to self-intersection. These are happening in both the main tile generation and the overview combining. Once shapeSplitter has generated this type of warning it is indeterminate which is shape and which is hole, so I'd expect gaps in the rendering. I think trunk had eradicated split/self-intersection problems in the tile processing, including the ovm_ file generation. I'm not sure about the overview combiner as, in my standard map-build, I don't merge shapes. Ticker On Fri, 2021-05-21 at 16:41 +, Gerd Petermann wrote: > Hi Ticker, > > I tried this patch with r4734 and I see no improvements. Maps are > larger, more white rectangles in sea. > > Maybe my recent changes don't work well with yours? Do you have an > example where it improves something with r4734? > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Freitag, 21. Mai 2021 18:10 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I'd been doing some investigation of filters ordering (based on > trunk). > I'd also done the pre-filtering of lines & shapes by minRes. > > My conclusions are: > > Shapes: > It is better to run SizeFilter after RemoveObsoleteFilter. > It is more efficient to run DP filter after both of these. > > Lines: > LineSplitterFilter should be run after everything that can remove > points. > It is more efficient to run DP after RemoveObsoleteFilter. > > I've adapted my changes into a patch for the low-res-opt branch, > along > with removal of some resolution tests that are now redundant. > > For the contourFilters, I've left DP as the first filter but moved > LineSplitter as for the normalFilters > > Ticker > > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, I tried this patch with r4734 and I see no improvements. Maps are larger, more white rectangles in sea. Maybe my recent changes don't work well with yours? Do you have an example where it improves something with r4734? Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Freitag, 21. Mai 2021 18:10 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I'd been doing some investigation of filters ordering (based on trunk). I'd also done the pre-filtering of lines & shapes by minRes. My conclusions are: Shapes: It is better to run SizeFilter after RemoveObsoleteFilter. It is more efficient to run DP filter after both of these. Lines: LineSplitterFilter should be run after everything that can remove points. It is more efficient to run DP after RemoveObsoleteFilter. I've adapted my changes into a patch for the low-res-opt branch, along with removal of some resolution tests that are now redundant. For the contourFilters, I've left DP as the first filter but moved LineSplitter as for the normalFilters Ticker ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd
I'd been doing some investigation of filters ordering (based on trunk).
I'd also done the pre-filtering of lines & shapes by minRes.
My conclusions are:
Shapes:
It is better to run SizeFilter after RemoveObsoleteFilter.
It is more efficient to run DP filter after both of these.
Lines:
LineSplitterFilter should be run after everything that can remove
points.
It is more efficient to run DP after RemoveObsoleteFilter.
I've adapted my changes into a patch for the low-res-opt branch, along
with removal of some resolution tests that are now redundant.
For the contourFilters, I've left DP as the first filter but moved
LineSplitter as for the normalFilters
Ticker
Index: src/uk/me/parabola/mkgmap/build/MapBuilder.java
===
--- src/uk/me/parabola/mkgmap/build/MapBuilder.java (revision 4731)
+++ src/uk/me/parabola/mkgmap/build/MapBuilder.java (working copy)
@@ -915,12 +915,15 @@
processPoints(map, div, points);
final int res = z.getResolution();
- lines = lines.stream().filter(l -> l.getMinResolution() <= res).collect(Collectors.toList());
- shapes = shapes.stream().filter(s -> s.getMinResolution() <= res).collect(Collectors.toList());
-
+ final int level = z.getLevel();
+ if (level != 0) {
+ lines = lines.stream().filter(l -> l.getMinResolution() <= res).collect(Collectors.toList());
+ shapes = shapes.stream().filter(s -> s.getMinResolution() <= res).collect(Collectors.toList());
+ }
+
if (mergeLines) {
- LineMergeFilter merger = new LineMergeFilter();
- lines = merger.merge(lines, res, !hasNet, allowReverseMerge);
+ LineMergeFilter lineMergeFilter = new LineMergeFilter();
+ lines = lineMergeFilter.merge(lines, res, !hasNet, allowReverseMerge);
}
if (mergeShapes) {
@@ -930,7 +933,7 @@
// recalculate preserved status for all points in lines and shapes
shapes.forEach(e -> e.getPoints().forEach(p -> p.preserved(false)));
- if (z.getLevel() == 0 && hasNet) {
+ if (level == 0 && hasNet) {
lines.forEach(e -> e.getPoints().forEach(p -> p.preserved(p.isNumberNode(;
} else {
lines.forEach(e -> e.getPoints().forEach(p -> p.preserved(false)));
@@ -1218,12 +1221,14 @@
* Note that the location and resolution of map elements is relative to the
* subdivision that they occur in.
*
- * @param map The map to add points to.
+ * @param map The map to add lines to.
* @param div The subdivision that the lines belong to.
* @param lines The lines to be added.
*/
private void processLines(Map map, Subdivision div, List lines) {
div.startLines(); // Signal that we are beginning to draw the lines.
+ if (lines.size() == 0)
+ return;
int res = div.getResolution();
@@ -1234,6 +1239,7 @@
LayerFilterChain normalFilters = new LayerFilterChain(config);
LayerFilterChain contourFilters = new LayerFilterChain(config);
+ DouglasPeuckerFilter dp = null;
if (enableLineCleanFilters && (res < 24)) {
MapFilter rounder = new RoundCoordsFilter();
MapFilter sizeFilter = new SizeFilter(MIN_SIZE_LINE);
@@ -1240,32 +1246,33 @@
normalFilters.addFilter(rounder);
normalFilters.addFilter(sizeFilter);
double errorForRes = dpFilterLineLevelMap.ceilingEntry(res).getValue();
- if(errorForRes > 0) {
-DouglasPeuckerFilter dp = new DouglasPeuckerFilter(errorForRes);
-normalFilters.addFilter(dp);
+ if (errorForRes > 0) {
+dp = new DouglasPeuckerFilter(errorForRes);
contourFilters.addFilter(dp);
+// normalFilter dp is run after removeObsolete. Should be more efficient
}
contourFilters.addFilter(rounder);
contourFilters.addFilter(sizeFilter);
}
+ RemoveObsoletePointsFilter removeObsolete = new RemoveObsoletePointsFilter();
+ normalFilters.addFilter(removeObsolete);
+ contourFilters.addFilter(removeObsolete);
+ if (dp != null)
+ normalFilters.addFilter(dp);
for (MapFilter filter : Arrays.asList(
-new LineSplitterFilter(),
new RemoveEmpty(),
-new RemoveObsoletePointsFilter(),
-new LinePreparerFilter(div),
+new LineSplitterFilter(),
+new LinePreparerFilter(div),
new LineAddFilter(div, map))) {
normalFilters.addFilter(filter);
contourFilters.addFilter(filter);
}
-
- for (MapLine line : lines) {
- if (line.getMinResolution() <= res) {
-if (GType.isContourLine(line))
- contourFilters.startFilter(line);
-else
- normalFilters.startFilter(line);
- }
- }
+
+ for (MapLine line : lines)
+ if (GType.isContourLine(line))
+contourFilters.startFilter(line);
+ else
+normalFilters.startFilter(line);
}
/**
@@ -1281,6 +1288,8 @@
*/
private void processShapes(Map map, Subdivision div, List shapes) {
div.startShapes(); // Signal that we are beginning to draw the shapes.
+ if (shapes.size() == 0)
+ return;
int res = div.getResolution();
@@ -1288,36 +1297,34 @@
config.setResolution(res);
config.setLevel(div.getZoom().getLevel());
Re: [mkgmap-dev] Problems with sea in overview map
Hi all, I think r4729 in the low-res-opt branch is a nice improvement reg. sea, probably also other complex (multi)polygons. For details see https://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4729 Gerd Von: mkgmap-dev im Auftrag von Gerd Petermann Gesendet: Freitag, 21. Mai 2021 11:33 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Ticker, yes, I think it should be enough to replace the boundary nodes, the others are already unique. It might even be a good idea to merge all sea (and if needed also land) in SeaGenerator. I'm about to commit a big change which implements full merge and remove small holes before MapSplitter is called, but that is done again for each level. I see nice improvements with complex sea. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Freitag, 21. Mai 2021 11:26 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd When loading precomp sea tiles, shouldn't SeaGenerator should replace all nodes on the edges of all the precomp tiles used with nodes from a shared coordPool. Maybe it needs to replace even more to cover the cut lines. Ticker On Thu, 2021-05-20 at 14:09 +, Gerd Petermann wrote: > Hi Ticker, > > reg. Also thinking that shapeMerge might need to operate at > highPrecision : > ShapeMerger doesn't compare positions, it checks for identical > Coords. > > For the overview map the ovm_* data is read and > MapBuilder.prepShapesForMerge() is used to make equal Coord instances > identical. This is not 100% safe, no matter what precision we use, > when the original data has touching rings with equal, but different > Coords the OverviewBuilder can see identical ones. > > Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, yes, I think it should be enough to replace the boundary nodes, the others are already unique. It might even be a good idea to merge all sea (and if needed also land) in SeaGenerator. I'm about to commit a big change which implements full merge and remove small holes before MapSplitter is called, but that is done again for each level. I see nice improvements with complex sea. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Freitag, 21. Mai 2021 11:26 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd When loading precomp sea tiles, shouldn't SeaGenerator should replace all nodes on the edges of all the precomp tiles used with nodes from a shared coordPool. Maybe it needs to replace even more to cover the cut lines. Ticker On Thu, 2021-05-20 at 14:09 +, Gerd Petermann wrote: > Hi Ticker, > > reg. Also thinking that shapeMerge might need to operate at > highPrecision : > ShapeMerger doesn't compare positions, it checks for identical > Coords. > > For the overview map the ovm_* data is read and > MapBuilder.prepShapesForMerge() is used to make equal Coord instances > identical. This is not 100% safe, no matter what precision we use, > when the original data has touching rings with equal, but different > Coords the OverviewBuilder can see identical ones. > > Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd When loading precomp sea tiles, shouldn't SeaGenerator should replace all nodes on the edges of all the precomp tiles used with nodes from a shared coordPool. Maybe it needs to replace even more to cover the cut lines. Ticker On Thu, 2021-05-20 at 14:09 +, Gerd Petermann wrote: > Hi Ticker, > > reg. Also thinking that shapeMerge might need to operate at > highPrecision : > ShapeMerger doesn't compare positions, it checks for identical > Coords. > > For the overview map the ovm_* data is read and > MapBuilder.prepShapesForMerge() is used to make equal Coord instances > identical. This is not 100% safe, no matter what precision we use, > when the original data has touching rings with equal, but different > Coords the OverviewBuilder can see identical ones. > > Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, reg. Also thinking that shapeMerge might need to operate at highPrecision : ShapeMerger doesn't compare positions, it checks for identical Coords. For the overview map the ovm_* data is read and MapBuilder.prepShapesForMerge() is used to make equal Coord instances identical. This is not 100% safe, no matter what precision we use, when the original data has touching rings with equal, but different Coords the OverviewBuilder can see identical ones. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Donnerstag, 20. Mai 2021 11:25 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I'm getting worried by rounding-to-resolution and how often it creates self-intersection. When doing the shapeSplitter.splitShape, we found that, unless operating with high-precision points (ie there were problems at 24bit precision), it could fail because of self-intersection. The polygon points limit filters got around this by tricky backtracking. This option isn't available when the overview map is being generated as the problem points have been frozen at very low res. Also thinking that shapeMerge might need to operate at highPrecision. I'm just trying do work out if the different levels have their own copy of shapes pre-filtering and how this effects what shapeMerge does as makeMapAreas steps through the levels. Ticker On Thu, 2021-05-20 at 09:07 +, Gerd Petermann wrote: > Hi Ticker, > > I've attached a test case that shows how the order is important. > Depending on the order of shapes to be merged the resulting shape > sometimes is self-intersecting, sometimes it isn't. > I've not yet understood what the trigger is. > > Gerd > > > > Von: mkgmap-dev im Auftrag > von Gerd Petermann > Gesendet: Mittwoch, 19. Mai 2021 18:14 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Ticker, > > yes, I think ShapeMerger can always merge. There seems to be no need > to look at the number of points. Maybe 24 is an exception. > > Sorting has an effect, but not always to the better. > > The merger is already used with rounded coord when the shapes for the > overview map are merged. > > Attached is a patch that shows what I am playing with. Will continue > tomorrow... > > Gerd > > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Mittwoch, 19. Mai 2021 17:41 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I still haven't deciphered the complicated bits of ShapeMergeFilter. > > Could it do the testing of mergeable candidates with rounded-to > -resolution points, then, if merging, convert all the points. > > Regardless, the limit checks against MAX_POINT_IN_ELEMENT could use > PredictFilterPoints(). It isn't a major problem if this limit is > exceeded because filtering might take it below again and, if it > doesn't, PolygonSplitter will deal with it. > > Regarding ordering, maybe if merge succeeds for some potential > shapes, > they should all be done. > > Intersecting polygons from rounding of concave shapes with a long > diagonal is slightly insoluble. Maybe something like WrongAngleFixer > is > going to be needed. Multipolygon cuts that are vertical or horizontal > might be less of a problem. > > I'm considering the effect of different orders of filters, such that > if > a small hole becomes a point and cuts to it, after merging, become a > series of spikes going into the shape, removeObsoletePoints removes > it. > > Ticker > > On Wed, 2021-05-19 at 14:12 +, Gerd Petermann wrote: > > Hi Ticker, > > > > I think ShapeMergeFilter could be part of the problem. It sometimes > > finds strange ways to merge shapes, esp. when many small parts can > > be > > merged. Maybe this can be improved by sorting so that it merges > > from > > top to bottom. > > > > Gerd > > > > > > Von: mkgmap-dev im Auftrag > > von Ticker Berkin > > Gesendet: Mittwoch, 19. Mai 2021 14:14 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > > > Hi Gerd > > > > I've just been experimenting and looking at shapes and rounding-to > > -low > > -resolution can easily cause self-intersections. > > > > It isn't just a problem with sea, or even with multipolygon hole > > cutting. GPSMapEdit reports over 1000 in my test area
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd I'm getting worried by rounding-to-resolution and how often it creates self-intersection. When doing the shapeSplitter.splitShape, we found that, unless operating with high-precision points (ie there were problems at 24bit precision), it could fail because of self-intersection. The polygon points limit filters got around this by tricky backtracking. This option isn't available when the overview map is being generated as the problem points have been frozen at very low res. Also thinking that shapeMerge might need to operate at highPrecision. I'm just trying do work out if the different levels have their own copy of shapes pre-filtering and how this effects what shapeMerge does as makeMapAreas steps through the levels. Ticker On Thu, 2021-05-20 at 09:07 +, Gerd Petermann wrote: > Hi Ticker, > > I've attached a test case that shows how the order is important. > Depending on the order of shapes to be merged the resulting shape > sometimes is self-intersecting, sometimes it isn't. > I've not yet understood what the trigger is. > > Gerd > > > > Von: mkgmap-dev im Auftrag > von Gerd Petermann > Gesendet: Mittwoch, 19. Mai 2021 18:14 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Ticker, > > yes, I think ShapeMerger can always merge. There seems to be no need > to look at the number of points. Maybe 24 is an exception. > > Sorting has an effect, but not always to the better. > > The merger is already used with rounded coord when the shapes for the > overview map are merged. > > Attached is a patch that shows what I am playing with. Will continue > tomorrow... > > Gerd > > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Mittwoch, 19. Mai 2021 17:41 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I still haven't deciphered the complicated bits of ShapeMergeFilter. > > Could it do the testing of mergeable candidates with rounded-to > -resolution points, then, if merging, convert all the points. > > Regardless, the limit checks against MAX_POINT_IN_ELEMENT could use > PredictFilterPoints(). It isn't a major problem if this limit is > exceeded because filtering might take it below again and, if it > doesn't, PolygonSplitter will deal with it. > > Regarding ordering, maybe if merge succeeds for some potential > shapes, > they should all be done. > > Intersecting polygons from rounding of concave shapes with a long > diagonal is slightly insoluble. Maybe something like WrongAngleFixer > is > going to be needed. Multipolygon cuts that are vertical or horizontal > might be less of a problem. > > I'm considering the effect of different orders of filters, such that > if > a small hole becomes a point and cuts to it, after merging, become a > series of spikes going into the shape, removeObsoletePoints removes > it. > > Ticker > > On Wed, 2021-05-19 at 14:12 +, Gerd Petermann wrote: > > Hi Ticker, > > > > I think ShapeMergeFilter could be part of the problem. It sometimes > > finds strange ways to merge shapes, esp. when many small parts can > > be > > merged. Maybe this can be improved by sorting so that it merges > > from > > top to bottom. > > > > Gerd > > > > > > Von: mkgmap-dev im Auftrag > > von Ticker Berkin > > Gesendet: Mittwoch, 19. Mai 2021 14:14 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > > > Hi Gerd > > > > I've just been experimenting and looking at shapes and rounding-to > > -low > > -resolution can easily cause self-intersections. > > > > It isn't just a problem with sea, or even with multipolygon hole > > cutting. GPSMapEdit reports over 1000 in my test area. This number > > reduces only slightly when I disable DP for polygons. Generally > > these > > are not noticeable - a little bit of wood missing when zoomed out > > isn't > > obvious, but same is not true of sea. > > > > Ticker > > > > On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote: > > > Hi Ticker, > > > > > > no need to guess, I use GpxCreator in various places to visualize > > > input and output of filters. I load the gpx files into JOSM and > > > maybe > > > convert to Data layer to understand what happens, e.g. where the > > > s
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, I've attached a test case that shows how the order is important. Depending on the order of shapes to be merged the resulting shape sometimes is self-intersecting, sometimes it isn't. I've not yet understood what the trigger is. Gerd Von: mkgmap-dev im Auftrag von Gerd Petermann Gesendet: Mittwoch, 19. Mai 2021 18:14 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Ticker, yes, I think ShapeMerger can always merge. There seems to be no need to look at the number of points. Maybe 24 is an exception. Sorting has an effect, but not always to the better. The merger is already used with rounded coord when the shapes for the overview map are merged. Attached is a patch that shows what I am playing with. Will continue tomorrow... Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Mittwoch, 19. Mai 2021 17:41 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I still haven't deciphered the complicated bits of ShapeMergeFilter. Could it do the testing of mergeable candidates with rounded-to -resolution points, then, if merging, convert all the points. Regardless, the limit checks against MAX_POINT_IN_ELEMENT could use PredictFilterPoints(). It isn't a major problem if this limit is exceeded because filtering might take it below again and, if it doesn't, PolygonSplitter will deal with it. Regarding ordering, maybe if merge succeeds for some potential shapes, they should all be done. Intersecting polygons from rounding of concave shapes with a long diagonal is slightly insoluble. Maybe something like WrongAngleFixer is going to be needed. Multipolygon cuts that are vertical or horizontal might be less of a problem. I'm considering the effect of different orders of filters, such that if a small hole becomes a point and cuts to it, after merging, become a series of spikes going into the shape, removeObsoletePoints removes it. Ticker On Wed, 2021-05-19 at 14:12 +, Gerd Petermann wrote: > Hi Ticker, > > I think ShapeMergeFilter could be part of the problem. It sometimes > finds strange ways to merge shapes, esp. when many small parts can be > merged. Maybe this can be improved by sorting so that it merges from > top to bottom. > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Mittwoch, 19. Mai 2021 14:14 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I've just been experimenting and looking at shapes and rounding-to > -low > -resolution can easily cause self-intersections. > > It isn't just a problem with sea, or even with multipolygon hole > cutting. GPSMapEdit reports over 1000 in my test area. This number > reduces only slightly when I disable DP for polygons. Generally these > are not noticeable - a little bit of wood missing when zoomed out > isn't > obvious, but same is not true of sea. > > Ticker > > On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote: > > Hi Ticker, > > > > no need to guess, I use GpxCreator in various places to visualize > > input and output of filters. I load the gpx files into JOSM and > > maybe > > convert to Data layer to understand what happens, e.g. where the > > start /end point is. > > Reg. self-intersection: > > Think of a merged sea shape with several islands in it. The > > rounding > > may result in only three or four different nodes, visitied in a > > more > > or less random order while the correct shape would be a triangle or > > rectangle. > > Not sure if this is only a problem with sea, probably not. > > > > Feel free to try other orders of the filters, or maybe different > > algos to set the preserved flag. There is definitely room for > > improvements. I've played with this very often and sometimes one > > area > > is improved and another is worsened. > > > > Gerd > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev testMergeOrder.osm Description: testMergeOrder.osm ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, yes, I think ShapeMerger can always merge. There seems to be no need to look at the number of points. Maybe 24 is an exception. Sorting has an effect, but not always to the better. The merger is already used with rounded coord when the shapes for the overview map are merged. Attached is a patch that shows what I am playing with. Will continue tomorrow... Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Mittwoch, 19. Mai 2021 17:41 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I still haven't deciphered the complicated bits of ShapeMergeFilter. Could it do the testing of mergeable candidates with rounded-to -resolution points, then, if merging, convert all the points. Regardless, the limit checks against MAX_POINT_IN_ELEMENT could use PredictFilterPoints(). It isn't a major problem if this limit is exceeded because filtering might take it below again and, if it doesn't, PolygonSplitter will deal with it. Regarding ordering, maybe if merge succeeds for some potential shapes, they should all be done. Intersecting polygons from rounding of concave shapes with a long diagonal is slightly insoluble. Maybe something like WrongAngleFixer is going to be needed. Multipolygon cuts that are vertical or horizontal might be less of a problem. I'm considering the effect of different orders of filters, such that if a small hole becomes a point and cuts to it, after merging, become a series of spikes going into the shape, removeObsoletePoints removes it. Ticker On Wed, 2021-05-19 at 14:12 +, Gerd Petermann wrote: > Hi Ticker, > > I think ShapeMergeFilter could be part of the problem. It sometimes > finds strange ways to merge shapes, esp. when many small parts can be > merged. Maybe this can be improved by sorting so that it merges from > top to bottom. > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Mittwoch, 19. Mai 2021 14:14 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I've just been experimenting and looking at shapes and rounding-to > -low > -resolution can easily cause self-intersections. > > It isn't just a problem with sea, or even with multipolygon hole > cutting. GPSMapEdit reports over 1000 in my test area. This number > reduces only slightly when I disable DP for polygons. Generally these > are not noticeable - a little bit of wood missing when zoomed out > isn't > obvious, but same is not true of sea. > > Ticker > > On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote: > > Hi Ticker, > > > > no need to guess, I use GpxCreator in various places to visualize > > input and output of filters. I load the gpx files into JOSM and > > maybe > > convert to Data layer to understand what happens, e.g. where the > > start /end point is. > > Reg. self-intersection: > > Think of a merged sea shape with several islands in it. The > > rounding > > may result in only three or four different nodes, visitied in a > > more > > or less random order while the correct shape would be a triangle or > > rectangle. > > Not sure if this is only a problem with sea, probably not. > > > > Feel free to try other orders of the filters, or maybe different > > algos to set the preserved flag. There is definitely room for > > improvements. I've played with this very often and sometimes one > > area > > is improved and another is worsened. > > > > Gerd > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev improve-shape-merge-alpha.patch Description: improve-shape-merge-alpha.patch ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd I still haven't deciphered the complicated bits of ShapeMergeFilter. Could it do the testing of mergeable candidates with rounded-to -resolution points, then, if merging, convert all the points. Regardless, the limit checks against MAX_POINT_IN_ELEMENT could use PredictFilterPoints(). It isn't a major problem if this limit is exceeded because filtering might take it below again and, if it doesn't, PolygonSplitter will deal with it. Regarding ordering, maybe if merge succeeds for some potential shapes, they should all be done. Intersecting polygons from rounding of concave shapes with a long diagonal is slightly insoluble. Maybe something like WrongAngleFixer is going to be needed. Multipolygon cuts that are vertical or horizontal might be less of a problem. I'm considering the effect of different orders of filters, such that if a small hole becomes a point and cuts to it, after merging, become a series of spikes going into the shape, removeObsoletePoints removes it. Ticker On Wed, 2021-05-19 at 14:12 +, Gerd Petermann wrote: > Hi Ticker, > > I think ShapeMergeFilter could be part of the problem. It sometimes > finds strange ways to merge shapes, esp. when many small parts can be > merged. Maybe this can be improved by sorting so that it merges from > top to bottom. > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Mittwoch, 19. Mai 2021 14:14 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I've just been experimenting and looking at shapes and rounding-to > -low > -resolution can easily cause self-intersections. > > It isn't just a problem with sea, or even with multipolygon hole > cutting. GPSMapEdit reports over 1000 in my test area. This number > reduces only slightly when I disable DP for polygons. Generally these > are not noticeable - a little bit of wood missing when zoomed out > isn't > obvious, but same is not true of sea. > > Ticker > > On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote: > > Hi Ticker, > > > > no need to guess, I use GpxCreator in various places to visualize > > input and output of filters. I load the gpx files into JOSM and > > maybe > > convert to Data layer to understand what happens, e.g. where the > > start /end point is. > > Reg. self-intersection: > > Think of a merged sea shape with several islands in it. The > > rounding > > may result in only three or four different nodes, visitied in a > > more > > or less random order while the correct shape would be a triangle or > > rectangle. > > Not sure if this is only a problem with sea, probably not. > > > > Feel free to try other orders of the filters, or maybe different > > algos to set the preserved flag. There is definitely room for > > improvements. I've played with this very often and sometimes one > > area > > is improved and another is worsened. > > > > Gerd > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, I think ShapeMergeFilter could be part of the problem. It sometimes finds strange ways to merge shapes, esp. when many small parts can be merged. Maybe this can be improved by sorting so that it merges from top to bottom. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Mittwoch, 19. Mai 2021 14:14 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I've just been experimenting and looking at shapes and rounding-to-low -resolution can easily cause self-intersections. It isn't just a problem with sea, or even with multipolygon hole cutting. GPSMapEdit reports over 1000 in my test area. This number reduces only slightly when I disable DP for polygons. Generally these are not noticeable - a little bit of wood missing when zoomed out isn't obvious, but same is not true of sea. Ticker On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote: > Hi Ticker, > > no need to guess, I use GpxCreator in various places to visualize > input and output of filters. I load the gpx files into JOSM and maybe > convert to Data layer to understand what happens, e.g. where the > start /end point is. > Reg. self-intersection: > Think of a merged sea shape with several islands in it. The rounding > may result in only three or four different nodes, visitied in a more > or less random order while the correct shape would be a triangle or > rectangle. > Not sure if this is only a problem with sea, probably not. > > Feel free to try other orders of the filters, or maybe different > algos to set the preserved flag. There is definitely room for > improvements. I've played with this very often and sometimes one area > is improved and another is worsened. > > Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd I've just been experimenting and looking at shapes and rounding-to-low -resolution can easily cause self-intersections. It isn't just a problem with sea, or even with multipolygon hole cutting. GPSMapEdit reports over 1000 in my test area. This number reduces only slightly when I disable DP for polygons. Generally these are not noticeable - a little bit of wood missing when zoomed out isn't obvious, but same is not true of sea. Ticker On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote: > Hi Ticker, > > no need to guess, I use GpxCreator in various places to visualize > input and output of filters. I load the gpx files into JOSM and maybe > convert to Data layer to understand what happens, e.g. where the > start /end point is. > Reg. self-intersection: > Think of a merged sea shape with several islands in it. The rounding > may result in only three or four different nodes, visitied in a more > or less random order while the correct shape would be a triangle or > rectangle. > Not sure if this is only a problem with sea, probably not. > > Feel free to try other orders of the filters, or maybe different > algos to set the preserved flag. There is definitely room for > improvements. I've played with this very often and sometimes one area > is improved and another is worsened. > > Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, no need to guess, I use GpxCreator in various places to visualize input and output of filters. I load the gpx files into JOSM and maybe convert to Data layer to understand what happens, e.g. where the start /end point is. Reg. self-intersection: Think of a merged sea shape with several islands in it. The rounding may result in only three or four different nodes, visitied in a more or less random order while the correct shape would be a triangle or rectangle. Not sure if this is only a problem with sea, probably not. Feel free to try other orders of the filters, or maybe different algos to set the preserved flag. There is definitely room for improvements. I've played with this very often and sometimes one area is improved and another is worsened. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Mittwoch, 19. Mai 2021 13:14 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I don't see that rounding can cause self-intersecting polygons - it might produce sides that touch each other. I imagine that DP can cause self-intersection and this is more likely on shapes typical of the result of a hole-cut. I wasn't presuming it would fix them. RemoveObsoletePointsFilter should be able to remove the cut to a hole that has become a point. However this is run after DP. Maybe swapping the order would improve matters. Ticker On Wed, 2021-05-19 at 10:56 +, Gerd Petermann wrote: > Hi Ticker, > > I think the self-intersections are a result of the rounding of > polygons with holes. DP just cannot fix those. > > I don't want to "delay hole cutting per resolution". I want to remove > the small holes. It doesn't matter if precomp-sea is used or not, it > works for both. > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Mittwoch, 19. Mai 2021 12:46 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I often see other polygons becoming self-intersecting at lower > resolutions (or at least GPSMapEdit does). If the DP filter is doing > this then maybe this should be tacked first, with an alternate > version > for polygons - I have no idea how this would be implemented. > > Should shapeMergeFilter somehow consider each polygon as if its > points > are rounded to resolution, with adjacent duplicates removed. This > should allow more merging and will be much less likely to be > inhibited > by the 256 point limit. Doing this, small holes/islands are likely to > disappear. > > I don't like the idea of trying to delay hole cutting per resolution, > and I don't see quite how it would work for pre-comp-sea > > Ticker > > On Wed, 2021-05-19 at 07:06 +, Gerd Petermann wrote: > > Hi all, > > > > I've identified a few problems with the handling of natural=sea in > > areas with lots of islands, e.g. coast of Norway. Thouands of > > rather > > small polygons are created, esp. when also natual=land is rendered. > > 1. Typical tiles are rather large, they span several precompiled > > sea > > tiles if --precomp-sea=sea.zip is used. As of now, the polygons > > from > > different sea-tiles are never merged. This might be improved to > > reduce file size and possibly also improve rendering > > 2. Even at resolution 12 the curent code distributes the thousands > > of > > tiles into multiple sub divisions before ShapeMerger is executed > > for > > each subdiv. Shapes (polygons) are only merged when the number of > > points for one shape stays inside the IMG limit of 256 points, so > > the > > merger cannot do much. > > 3. Douglas-Peucker (DP) filter is used for the merged shapes which > > contain holes. This gives weird results because the invisible lines > > which connect the holes are causing heavy self-intersections at low > > resolutions. I think this is one reason for white triangles in the > > sea. > > > > I am experimenting with these ideas: > > - At low resolutions the ShapeMerger could merge more so that > > shapes > > with more than > 256 points are produced, assuming that DP filter > > will remove many of them. Not sure if this works with shapes that > > have holes. > > - At low resolutions it is likely/wanted that small islands > > disappear, so we should remove those first, together with the lines > > that connect them with the outer ring. > > - The simplified sea polygons without holes should be merged as > > much > > as possible before using any filter, maybe DP should be execut
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd I don't see that rounding can cause self-intersecting polygons - it might produce sides that touch each other. I imagine that DP can cause self-intersection and this is more likely on shapes typical of the result of a hole-cut. I wasn't presuming it would fix them. RemoveObsoletePointsFilter should be able to remove the cut to a hole that has become a point. However this is run after DP. Maybe swapping the order would improve matters. Ticker On Wed, 2021-05-19 at 10:56 +, Gerd Petermann wrote: > Hi Ticker, > > I think the self-intersections are a result of the rounding of > polygons with holes. DP just cannot fix those. > > I don't want to "delay hole cutting per resolution". I want to remove > the small holes. It doesn't matter if precomp-sea is used or not, it > works for both. > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Mittwoch, 19. Mai 2021 12:46 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I often see other polygons becoming self-intersecting at lower > resolutions (or at least GPSMapEdit does). If the DP filter is doing > this then maybe this should be tacked first, with an alternate > version > for polygons - I have no idea how this would be implemented. > > Should shapeMergeFilter somehow consider each polygon as if its > points > are rounded to resolution, with adjacent duplicates removed. This > should allow more merging and will be much less likely to be > inhibited > by the 256 point limit. Doing this, small holes/islands are likely to > disappear. > > I don't like the idea of trying to delay hole cutting per resolution, > and I don't see quite how it would work for pre-comp-sea > > Ticker > > On Wed, 2021-05-19 at 07:06 +, Gerd Petermann wrote: > > Hi all, > > > > I've identified a few problems with the handling of natural=sea in > > areas with lots of islands, e.g. coast of Norway. Thouands of > > rather > > small polygons are created, esp. when also natual=land is rendered. > > 1. Typical tiles are rather large, they span several precompiled > > sea > > tiles if --precomp-sea=sea.zip is used. As of now, the polygons > > from > > different sea-tiles are never merged. This might be improved to > > reduce file size and possibly also improve rendering > > 2. Even at resolution 12 the curent code distributes the thousands > > of > > tiles into multiple sub divisions before ShapeMerger is executed > > for > > each subdiv. Shapes (polygons) are only merged when the number of > > points for one shape stays inside the IMG limit of 256 points, so > > the > > merger cannot do much. > > 3. Douglas-Peucker (DP) filter is used for the merged shapes which > > contain holes. This gives weird results because the invisible lines > > which connect the holes are causing heavy self-intersections at low > > resolutions. I think this is one reason for white triangles in the > > sea. > > > > I am experimenting with these ideas: > > - At low resolutions the ShapeMerger could merge more so that > > shapes > > with more than > 256 points are produced, assuming that DP filter > > will remove many of them. Not sure if this works with shapes that > > have holes. > > - At low resolutions it is likely/wanted that small islands > > disappear, so we should remove those first, together with the lines > > that connect them with the outer ring. > > - The simplified sea polygons without holes should be merged as > > much > > as possible before using any filter, maybe DP should be executed > > first on the merged shapes so that fever subdivs are created. > > > > Each method has its own effects and I'm searching for the best > > combination and order. > > > > A lot of this is only possible because we now have Tickers code > > that > > allows multipolygon splitting without using > > java.awt.geom.Area.intersect() . Tickers code in ShapeSplitter > > seems > > to be able to handle the holes properly. > > > > Gerd > > > > > > ___ > > mkgmap-dev mailing list > > mkgmap-dev@lists.mkgmap.org.uk > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, I think the self-intersections are a result of the rounding of polygons with holes. DP just cannot fix those. I don't want to "delay hole cutting per resolution". I want to remove the small holes. It doesn't matter if precomp-sea is used or not, it works for both. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Mittwoch, 19. Mai 2021 12:46 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I often see other polygons becoming self-intersecting at lower resolutions (or at least GPSMapEdit does). If the DP filter is doing this then maybe this should be tacked first, with an alternate version for polygons - I have no idea how this would be implemented. Should shapeMergeFilter somehow consider each polygon as if its points are rounded to resolution, with adjacent duplicates removed. This should allow more merging and will be much less likely to be inhibited by the 256 point limit. Doing this, small holes/islands are likely to disappear. I don't like the idea of trying to delay hole cutting per resolution, and I don't see quite how it would work for pre-comp-sea Ticker On Wed, 2021-05-19 at 07:06 +, Gerd Petermann wrote: > Hi all, > > I've identified a few problems with the handling of natural=sea in > areas with lots of islands, e.g. coast of Norway. Thouands of rather > small polygons are created, esp. when also natual=land is rendered. > 1. Typical tiles are rather large, they span several precompiled sea > tiles if --precomp-sea=sea.zip is used. As of now, the polygons from > different sea-tiles are never merged. This might be improved to > reduce file size and possibly also improve rendering > 2. Even at resolution 12 the curent code distributes the thousands of > tiles into multiple sub divisions before ShapeMerger is executed for > each subdiv. Shapes (polygons) are only merged when the number of > points for one shape stays inside the IMG limit of 256 points, so the > merger cannot do much. > 3. Douglas-Peucker (DP) filter is used for the merged shapes which > contain holes. This gives weird results because the invisible lines > which connect the holes are causing heavy self-intersections at low > resolutions. I think this is one reason for white triangles in the > sea. > > I am experimenting with these ideas: > - At low resolutions the ShapeMerger could merge more so that shapes > with more than > 256 points are produced, assuming that DP filter > will remove many of them. Not sure if this works with shapes that > have holes. > - At low resolutions it is likely/wanted that small islands > disappear, so we should remove those first, together with the lines > that connect them with the outer ring. > - The simplified sea polygons without holes should be merged as much > as possible before using any filter, maybe DP should be executed > first on the merged shapes so that fever subdivs are created. > > Each method has its own effects and I'm searching for the best > combination and order. > > A lot of this is only possible because we now have Tickers code that > allows multipolygon splitting without using > java.awt.geom.Area.intersect() . Tickers code in ShapeSplitter seems > to be able to handle the holes properly. > > Gerd > > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd I often see other polygons becoming self-intersecting at lower resolutions (or at least GPSMapEdit does). If the DP filter is doing this then maybe this should be tacked first, with an alternate version for polygons - I have no idea how this would be implemented. Should shapeMergeFilter somehow consider each polygon as if its points are rounded to resolution, with adjacent duplicates removed. This should allow more merging and will be much less likely to be inhibited by the 256 point limit. Doing this, small holes/islands are likely to disappear. I don't like the idea of trying to delay hole cutting per resolution, and I don't see quite how it would work for pre-comp-sea Ticker On Wed, 2021-05-19 at 07:06 +, Gerd Petermann wrote: > Hi all, > > I've identified a few problems with the handling of natural=sea in > areas with lots of islands, e.g. coast of Norway. Thouands of rather > small polygons are created, esp. when also natual=land is rendered. > 1. Typical tiles are rather large, they span several precompiled sea > tiles if --precomp-sea=sea.zip is used. As of now, the polygons from > different sea-tiles are never merged. This might be improved to > reduce file size and possibly also improve rendering > 2. Even at resolution 12 the curent code distributes the thousands of > tiles into multiple sub divisions before ShapeMerger is executed for > each subdiv. Shapes (polygons) are only merged when the number of > points for one shape stays inside the IMG limit of 256 points, so the > merger cannot do much. > 3. Douglas-Peucker (DP) filter is used for the merged shapes which > contain holes. This gives weird results because the invisible lines > which connect the holes are causing heavy self-intersections at low > resolutions. I think this is one reason for white triangles in the > sea. > > I am experimenting with these ideas: > - At low resolutions the ShapeMerger could merge more so that shapes > with more than > 256 points are produced, assuming that DP filter > will remove many of them. Not sure if this works with shapes that > have holes. > - At low resolutions it is likely/wanted that small islands > disappear, so we should remove those first, together with the lines > that connect them with the outer ring. > - The simplified sea polygons without holes should be merged as much > as possible before using any filter, maybe DP should be executed > first on the merged shapes so that fever subdivs are created. > > Each method has its own effects and I'm searching for the best > combination and order. > > A lot of this is only possible because we now have Tickers code that > allows multipolygon splitting without using > java.awt.geom.Area.intersect() . Tickers code in ShapeSplitter seems > to be able to handle the holes properly. > > Gerd > > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Felix,
reg. --polygon-size-limits:
The default style has this rule
natural=sea {add mkgmap:skipSizeFilter=true; set mkgmap:drawLevel=2} [0x32
resolution 10]
and thus the size filter has no effect. This is needed to avoid small empty
areas in the sea, at least with the current code.
Gerd
Von: mkgmap-dev im Auftrag von Felix
Hartmann
Gesendet: Mittwoch, 19. Mai 2021 09:35
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Problems with sea in overview map
I think this is worse with too low polygon-size-limits... The default of 8 is
pretty little. I think also for sea/islands a better approach is to only
consider resolution 24 and 23 to be detailed without much dropping /
generalisation. And then drop more. So something like
--polygon-size-limits=24:8,23:10,22:12,21:14,20 or bigger 20 makes more sense
as long as this does not empty the sea :-)
But yes of course, I am sure there are better ways to merge more than just
higher polygon size limits. That one is the dumb aproach...
In general maybe there is a better data source than OSM for the overview map?
Garmin is not using OSM data for their OSM based maps for the overview map.
Likely they decided it is too much work to get OSM filtered down nicely.
However I do not know which data would be compatible (to mkgmap and more
importantly in licensing). Aside from specilised things - like ICN/NCN
cycleroutes in a map for cyclists, the overview map should include:
Sea/Land (generalized)
Country borders
Very big cities (but relative to the population of that country) to at some
point only major capitals.
Very big/long rivers (rivers moved to relations - but they are often lacking)
Maybe major railways
Major highways.
For resolution 17 and 16 maybe huge forests. but not much in terms of
polygons...
I likely missed 2-3 things but no more. And those things will be hard again
with OSM database. Would be nice to show the ice on the poles or maybe green
for the worlds largest jungles/forests - but that is soo hard with OSM.
Likely with 3000-4000 hours manual work one could create a OSM derived basemap
database and then all those filters and considerations would be much less
important.
The full OSM dataset is just a burden for the overview map, I really appreciate
that you spend so much effort and time to improve mkgmap to work around the
limitations.
On Wed, 19 May 2021 at 15:06, Gerd Petermann
mailto:gpetermann_muenc...@hotmail.com>> wrote:
Hi all,
I've identified a few problems with the handling of natural=sea in areas with
lots of islands, e.g. coast of Norway. Thouands of rather small polygons are
created, esp. when also natual=land is rendered.
1. Typical tiles are rather large, they span several precompiled sea tiles if
--precomp-sea=sea.zip is used. As of now, the polygons from different sea-tiles
are never merged. This might be improved to reduce file size and possibly also
improve rendering
2. Even at resolution 12 the curent code distributes the thousands of tiles
into multiple sub divisions before ShapeMerger is executed for each subdiv.
Shapes (polygons) are only merged when the number of points for one shape stays
inside the IMG limit of 256 points, so the merger cannot do much.
3. Douglas-Peucker (DP) filter is used for the merged shapes which contain
holes. This gives weird results because the invisible lines which connect the
holes are causing heavy self-intersections at low resolutions. I think this is
one reason for white triangles in the sea.
I am experimenting with these ideas:
- At low resolutions the ShapeMerger could merge more so that shapes with more
than > 256 points are produced, assuming that DP filter will remove many of
them. Not sure if this works with shapes that have holes.
- At low resolutions it is likely/wanted that small islands disappear, so we
should remove those first, together with the lines that connect them with the
outer ring.
- The simplified sea polygons without holes should be merged as much as
possible before using any filter, maybe DP should be executed first on the
merged shapes so that fever subdivs are created.
Each method has its own effects and I'm searching for the best combination and
order.
A lot of this is only possible because we now have Tickers code that allows
multipolygon splitting without using java.awt.geom.Area.intersect() . Tickers
code in ShapeSplitter seems to be able to handle the holes properly.
Gerd
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
--
Felix Hartman - Openmtbmap.org & VeloMap.org
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
I think this is worse with too low polygon-size-limits... The default of 8 is pretty little. I think also for sea/islands a better approach is to only consider resolution 24 and 23 to be detailed without much dropping / generalisation. And then drop more. So something like --polygon-size-limits=24:8,23:10,22:12,21:14,20 or bigger 20 makes more sense as long as this does not empty the sea :-) But yes of course, I am sure there are better ways to merge more than just higher polygon size limits. That one is the dumb aproach... In general maybe there is a better data source than OSM for the overview map? Garmin is not using OSM data for their OSM based maps for the overview map. Likely they decided it is too much work to get OSM filtered down nicely. However I do not know which data would be compatible (to mkgmap and more importantly in licensing). Aside from specilised things - like ICN/NCN cycleroutes in a map for cyclists, the overview map should include: Sea/Land (generalized) Country borders Very big cities (but relative to the population of that country) to at some point only major capitals. Very big/long rivers (rivers moved to relations - but they are often lacking) Maybe major railways Major highways. For resolution 17 and 16 maybe huge forests. but not much in terms of polygons... I likely missed 2-3 things but no more. And those things will be hard again with OSM database. Would be nice to show the ice on the poles or maybe green for the worlds largest jungles/forests - but that is soo hard with OSM. Likely with 3000-4000 hours manual work one could create a OSM derived basemap database and then all those filters and considerations would be much less important. The full OSM dataset is just a burden for the overview map, I really appreciate that you spend so much effort and time to improve mkgmap to work around the limitations. On Wed, 19 May 2021 at 15:06, Gerd Petermann < gpetermann_muenc...@hotmail.com> wrote: > Hi all, > > I've identified a few problems with the handling of natural=sea in areas > with lots of islands, e.g. coast of Norway. Thouands of rather small > polygons are created, esp. when also natual=land is rendered. > 1. Typical tiles are rather large, they span several precompiled sea tiles > if --precomp-sea=sea.zip is used. As of now, the polygons from different > sea-tiles are never merged. This might be improved to reduce file size and > possibly also improve rendering > 2. Even at resolution 12 the curent code distributes the thousands of > tiles into multiple sub divisions before ShapeMerger is executed for each > subdiv. Shapes (polygons) are only merged when the number of points for one > shape stays inside the IMG limit of 256 points, so the merger cannot do > much. > 3. Douglas-Peucker (DP) filter is used for the merged shapes which contain > holes. This gives weird results because the invisible lines which connect > the holes are causing heavy self-intersections at low resolutions. I think > this is one reason for white triangles in the sea. > > I am experimenting with these ideas: > - At low resolutions the ShapeMerger could merge more so that shapes with > more than > 256 points are produced, assuming that DP filter will remove > many of them. Not sure if this works with shapes that have holes. > - At low resolutions it is likely/wanted that small islands disappear, so > we should remove those first, together with the lines that connect them > with the outer ring. > - The simplified sea polygons without holes should be merged as much as > possible before using any filter, maybe DP should be executed first on the > merged shapes so that fever subdivs are created. > > Each method has its own effects and I'm searching for the best combination > and order. > > A lot of this is only possible because we now have Tickers code that > allows multipolygon splitting without using java.awt.geom.Area.intersect() > . Tickers code in ShapeSplitter seems to be able to handle the holes > properly. > > Gerd > > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > -- Felix Hartman - Openmtbmap.org & VeloMap.org ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
[mkgmap-dev] Problems with sea in overview map
Hi all, I've identified a few problems with the handling of natural=sea in areas with lots of islands, e.g. coast of Norway. Thouands of rather small polygons are created, esp. when also natual=land is rendered. 1. Typical tiles are rather large, they span several precompiled sea tiles if --precomp-sea=sea.zip is used. As of now, the polygons from different sea-tiles are never merged. This might be improved to reduce file size and possibly also improve rendering 2. Even at resolution 12 the curent code distributes the thousands of tiles into multiple sub divisions before ShapeMerger is executed for each subdiv. Shapes (polygons) are only merged when the number of points for one shape stays inside the IMG limit of 256 points, so the merger cannot do much. 3. Douglas-Peucker (DP) filter is used for the merged shapes which contain holes. This gives weird results because the invisible lines which connect the holes are causing heavy self-intersections at low resolutions. I think this is one reason for white triangles in the sea. I am experimenting with these ideas: - At low resolutions the ShapeMerger could merge more so that shapes with more than > 256 points are produced, assuming that DP filter will remove many of them. Not sure if this works with shapes that have holes. - At low resolutions it is likely/wanted that small islands disappear, so we should remove those first, together with the lines that connect them with the outer ring. - The simplified sea polygons without holes should be merged as much as possible before using any filter, maybe DP should be executed first on the merged shapes so that fever subdivs are created. Each method has its own effects and I'm searching for the best combination and order. A lot of this is only possible because we now have Tickers code that allows multipolygon splitting without using java.awt.geom.Area.intersect() . Tickers code in ShapeSplitter seems to be able to handle the holes properly. Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
