Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, reg. Also thinking that shapeMerge might need to operate at highPrecision : ShapeMerger doesn't compare positions, it checks for identical Coords. For the overview map the ovm_* data is read and MapBuilder.prepShapesForMerge() is used to make equal Coord instances identical. This is not 100% safe, no matter what precision we use, when the original data has touching rings with equal, but different Coords the OverviewBuilder can see identical ones. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Donnerstag, 20. Mai 2021 11:25 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I'm getting worried by rounding-to-resolution and how often it creates self-intersection. When doing the shapeSplitter.splitShape, we found that, unless operating with high-precision points (ie there were problems at 24bit precision), it could fail because of self-intersection. The polygon points limit filters got around this by tricky backtracking. This option isn't available when the overview map is being generated as the problem points have been frozen at very low res. Also thinking that shapeMerge might need to operate at highPrecision. I'm just trying do work out if the different levels have their own copy of shapes pre-filtering and how this effects what shapeMerge does as makeMapAreas steps through the levels. Ticker On Thu, 2021-05-20 at 09:07 +, Gerd Petermann wrote: > Hi Ticker, > > I've attached a test case that shows how the order is important. > Depending on the order of shapes to be merged the resulting shape > sometimes is self-intersecting, sometimes it isn't. > I've not yet understood what the trigger is. > > Gerd > > > > Von: mkgmap-dev im Auftrag > von Gerd Petermann > Gesendet: Mittwoch, 19. Mai 2021 18:14 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Ticker, > > yes, I think ShapeMerger can always merge. There seems to be no need > to look at the number of points. Maybe 24 is an exception. > > Sorting has an effect, but not always to the better. > > The merger is already used with rounded coord when the shapes for the > overview map are merged. > > Attached is a patch that shows what I am playing with. Will continue > tomorrow... > > Gerd > > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Mittwoch, 19. Mai 2021 17:41 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I still haven't deciphered the complicated bits of ShapeMergeFilter. > > Could it do the testing of mergeable candidates with rounded-to > -resolution points, then, if merging, convert all the points. > > Regardless, the limit checks against MAX_POINT_IN_ELEMENT could use > PredictFilterPoints(). It isn't a major problem if this limit is > exceeded because filtering might take it below again and, if it > doesn't, PolygonSplitter will deal with it. > > Regarding ordering, maybe if merge succeeds for some potential > shapes, > they should all be done. > > Intersecting polygons from rounding of concave shapes with a long > diagonal is slightly insoluble. Maybe something like WrongAngleFixer > is > going to be needed. Multipolygon cuts that are vertical or horizontal > might be less of a problem. > > I'm considering the effect of different orders of filters, such that > if > a small hole becomes a point and cuts to it, after merging, become a > series of spikes going into the shape, removeObsoletePoints removes > it. > > Ticker > > On Wed, 2021-05-19 at 14:12 +, Gerd Petermann wrote: > > Hi Ticker, > > > > I think ShapeMergeFilter could be part of the problem. It sometimes > > finds strange ways to merge shapes, esp. when many small parts can > > be > > merged. Maybe this can be improved by sorting so that it merges > > from > > top to bottom. > > > > Gerd > > > > > > Von: mkgmap-dev im Auftrag > > von Ticker Berkin > > Gesendet: Mittwoch, 19. Mai 2021 14:14 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > > > Hi Gerd > > > > I've just been experimenting and looking at shapes and rounding-to > > -low > > -resolution can easily cause self-intersections. > > > > It isn't just a problem with sea, or even with multipolygon hole > > cutting. GPSMapEdit reports over 1000 in my test area. This number > > reduces only slightly when I disable DP for polygons. Generally > > these > > are not noticeable - a little bit of wood missing when zoomed out > > isn't > > obvious, but same is not true of sea. > > > > Ticker > > > > On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote: > > > Hi Ticker, > > > > > > no need to guess, I use GpxCreator in various places to visualize > > > input and output of filters. I load the
Re: [mkgmap-dev] logging improvement
Hi Mike, sorry, that slipped my mind. I've not tried it yet but I think the deprecated --check-roundabouts option should be interpreted as --fix-roundabout-direction so that the IMG file is the same as before. The message "... reversing it " is logged when --check-roundabouts is used but the way is not reversed? Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Donnerstag, 20. Mai 2021 13:13 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] logging improvement Hi Gerd, I have already sent you the roundabout checks patch which covers both of the requests. It may have got missed as I know you are heavily involved in other issues at the moment. I've attached a copy. Cheers, Mike -Original Message- From: Gerd Petermann [mailto:gpetermann_muenc...@hotmail.com] Sent: 20 May 2021 10:20 To: Development list for mkgmap Subject: Re: [mkgmap-dev] logging improvement Hi Mike, I've hoped for a patch reg. the messages about roundabouts for users without a logging.config. I think we need a way to enable the functionality of reversing roundabouts (done in StyledConverter) without also getting all the check messages from RoadNetwork. Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Mittwoch, 19. Mai 2021 12:18 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] logging improvement Hi Gerd, Please find attached the updated patch. In most cases where ways are duplicated, the duplicate ways are given a faked id. However, there are 3 places in StyledConverter where the id is copied rather than using a faked id - in addRoad, splitWayAt and makeCycleWay. I am unsure as to whether this is intentional or an oversight, so have left the id as it is currently (I have had to introduce new Way copy constructors to handle this). If these should also use a faked id, I can update the patch. I have followed your suggestion of amending the similar arcs code to get rid of duplicated ways, and using my style file have reduced the number of similar arcs warnings from about 21000 to about 600 which also results in a reduction in size of the gmapsupp file. I will be posting this patch shortly. Cheers, Mike -Original Message- From: Mike Baggaley [mailto:m...@tvage.co.uk] Sent: 16 May 2021 08:48 To: 'Development list for mkgmap' Subject: RE: [mkgmap-dev] logging improvement Hi Gerd, There are lots of pedestrian areas that are relations. Some of these overlap with adjacent pedestrian areas and produce a warning like: Similar arcs (generated way) and (generated way) found at http://www.openstreetmap.org/?mlat=50.728438=-1.143728=17 However, don't commit it yet - I have realised that although this fixes most of the messages, there are some still being displayed as generated way, presumably because the relation is copied prior to being used. I'm looking at this now. The above example after the patch displays: Similar arcs (generated way) and (http://www.openstreetmap.org/relation/9381074) found at http://www.openstreetmap.org/?mlat=50.728438=-1.143728=17 Cheers, Mike -Original Message- From: Gerd Petermann [mailto:gpetermann_muenc...@hotmail.com] Sent: 16 May 2021 07:32 To: Development list for mkgmap Subject: Re: [mkgmap-dev] logging improvement Hi Mike, please help me: In what situation do we create roads from ways which were created from relations? Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Samstag, 15. Mai 2021 14:31 An: 'Development list for mkgmap' Betreff: [mkgmap-dev] logging improvement Hi Gerd, Please find attached a small patch that changes the logging of RoadDef when the original element is a relation. Currently it just says "generated way"; with the patch it gives the URL of the relation in the same way as it does for ways. Cheers, Mike ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] logging improvement
Hi Gerd, I have already sent you the roundabout checks patch which covers both of the requests. It may have got missed as I know you are heavily involved in other issues at the moment. I've attached a copy. Cheers, Mike -Original Message- From: Gerd Petermann [mailto:gpetermann_muenc...@hotmail.com] Sent: 20 May 2021 10:20 To: Development list for mkgmap Subject: Re: [mkgmap-dev] logging improvement Hi Mike, I've hoped for a patch reg. the messages about roundabouts for users without a logging.config. I think we need a way to enable the functionality of reversing roundabouts (done in StyledConverter) without also getting all the check messages from RoadNetwork. Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Mittwoch, 19. Mai 2021 12:18 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] logging improvement Hi Gerd, Please find attached the updated patch. In most cases where ways are duplicated, the duplicate ways are given a faked id. However, there are 3 places in StyledConverter where the id is copied rather than using a faked id - in addRoad, splitWayAt and makeCycleWay. I am unsure as to whether this is intentional or an oversight, so have left the id as it is currently (I have had to introduce new Way copy constructors to handle this). If these should also use a faked id, I can update the patch. I have followed your suggestion of amending the similar arcs code to get rid of duplicated ways, and using my style file have reduced the number of similar arcs warnings from about 21000 to about 600 which also results in a reduction in size of the gmapsupp file. I will be posting this patch shortly. Cheers, Mike -Original Message- From: Mike Baggaley [mailto:m...@tvage.co.uk] Sent: 16 May 2021 08:48 To: 'Development list for mkgmap' Subject: RE: [mkgmap-dev] logging improvement Hi Gerd, There are lots of pedestrian areas that are relations. Some of these overlap with adjacent pedestrian areas and produce a warning like: Similar arcs (generated way) and (generated way) found at http://www.openstreetmap.org/?mlat=50.728438=-1.143728=17 However, don't commit it yet - I have realised that although this fixes most of the messages, there are some still being displayed as generated way, presumably because the relation is copied prior to being used. I'm looking at this now. The above example after the patch displays: Similar arcs (generated way) and (http://www.openstreetmap.org/relation/9381074) found at http://www.openstreetmap.org/?mlat=50.728438=-1.143728=17 Cheers, Mike -Original Message- From: Gerd Petermann [mailto:gpetermann_muenc...@hotmail.com] Sent: 16 May 2021 07:32 To: Development list for mkgmap Subject: Re: [mkgmap-dev] logging improvement Hi Mike, please help me: In what situation do we create roads from ways which were created from relations? Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Samstag, 15. Mai 2021 14:31 An: 'Development list for mkgmap' Betreff: [mkgmap-dev] logging improvement Hi Gerd, Please find attached a small patch that changes the logging of RoadDef when the original element is a relation. Currently it just says "generated way"; with the patch it gives the URL of the relation in the same way as it does for ways. Cheers, Mike --- Begin Message --- Hi Gerd, Please find attached the previously discussed patch that splits checking and fixing roundabouts and flares and provides enhanced flare analysis. I note that some checks are performed when the style is being processed with others being performed later. I have not attempted to combine these - is there a technical reason for the separation, or is it just that the checks were separately developed? Thanks, Mike roundaboutchecks.patch Description: Binary data --- End Message --- ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Gerd I'm getting worried by rounding-to-resolution and how often it creates self-intersection. When doing the shapeSplitter.splitShape, we found that, unless operating with high-precision points (ie there were problems at 24bit precision), it could fail because of self-intersection. The polygon points limit filters got around this by tricky backtracking. This option isn't available when the overview map is being generated as the problem points have been frozen at very low res. Also thinking that shapeMerge might need to operate at highPrecision. I'm just trying do work out if the different levels have their own copy of shapes pre-filtering and how this effects what shapeMerge does as makeMapAreas steps through the levels. Ticker On Thu, 2021-05-20 at 09:07 +, Gerd Petermann wrote: > Hi Ticker, > > I've attached a test case that shows how the order is important. > Depending on the order of shapes to be merged the resulting shape > sometimes is self-intersecting, sometimes it isn't. > I've not yet understood what the trigger is. > > Gerd > > > > Von: mkgmap-dev im Auftrag > von Gerd Petermann > Gesendet: Mittwoch, 19. Mai 2021 18:14 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Ticker, > > yes, I think ShapeMerger can always merge. There seems to be no need > to look at the number of points. Maybe 24 is an exception. > > Sorting has an effect, but not always to the better. > > The merger is already used with rounded coord when the shapes for the > overview map are merged. > > Attached is a patch that shows what I am playing with. Will continue > tomorrow... > > Gerd > > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Mittwoch, 19. Mai 2021 17:41 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I still haven't deciphered the complicated bits of ShapeMergeFilter. > > Could it do the testing of mergeable candidates with rounded-to > -resolution points, then, if merging, convert all the points. > > Regardless, the limit checks against MAX_POINT_IN_ELEMENT could use > PredictFilterPoints(). It isn't a major problem if this limit is > exceeded because filtering might take it below again and, if it > doesn't, PolygonSplitter will deal with it. > > Regarding ordering, maybe if merge succeeds for some potential > shapes, > they should all be done. > > Intersecting polygons from rounding of concave shapes with a long > diagonal is slightly insoluble. Maybe something like WrongAngleFixer > is > going to be needed. Multipolygon cuts that are vertical or horizontal > might be less of a problem. > > I'm considering the effect of different orders of filters, such that > if > a small hole becomes a point and cuts to it, after merging, become a > series of spikes going into the shape, removeObsoletePoints removes > it. > > Ticker > > On Wed, 2021-05-19 at 14:12 +, Gerd Petermann wrote: > > Hi Ticker, > > > > I think ShapeMergeFilter could be part of the problem. It sometimes > > finds strange ways to merge shapes, esp. when many small parts can > > be > > merged. Maybe this can be improved by sorting so that it merges > > from > > top to bottom. > > > > Gerd > > > > > > Von: mkgmap-dev im Auftrag > > von Ticker Berkin > > Gesendet: Mittwoch, 19. Mai 2021 14:14 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > > > Hi Gerd > > > > I've just been experimenting and looking at shapes and rounding-to > > -low > > -resolution can easily cause self-intersections. > > > > It isn't just a problem with sea, or even with multipolygon hole > > cutting. GPSMapEdit reports over 1000 in my test area. This number > > reduces only slightly when I disable DP for polygons. Generally > > these > > are not noticeable - a little bit of wood missing when zoomed out > > isn't > > obvious, but same is not true of sea. > > > > Ticker > > > > On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote: > > > Hi Ticker, > > > > > > no need to guess, I use GpxCreator in various places to visualize > > > input and output of filters. I load the gpx files into JOSM and > > > maybe > > > convert to Data layer to understand what happens, e.g. where the > > > start /end point is. > > > Reg. self-intersection: > > > Think of a merged sea shape with several islands in it. The > > > rounding > > > may result in only three or four different nodes, visitied in a > > > more > > > or less random order while the correct shape would be a triangle > > > or > > > rectangle. > > > Not sure if this is only a problem with sea, probably not. > > > > > > Feel free to try other orders of the filters, or maybe different > > > algos to set the preserved flag. There is definitely room for > > > improvements. I've played
Re: [mkgmap-dev] logging improvement
Hi Mike, I've hoped for a patch reg. the messages about roundabouts for users without a logging.config. I think we need a way to enable the functionality of reversing roundabouts (done in StyledConverter) without also getting all the check messages from RoadNetwork. Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Mittwoch, 19. Mai 2021 12:18 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] logging improvement Hi Gerd, Please find attached the updated patch. In most cases where ways are duplicated, the duplicate ways are given a faked id. However, there are 3 places in StyledConverter where the id is copied rather than using a faked id - in addRoad, splitWayAt and makeCycleWay. I am unsure as to whether this is intentional or an oversight, so have left the id as it is currently (I have had to introduce new Way copy constructors to handle this). If these should also use a faked id, I can update the patch. I have followed your suggestion of amending the similar arcs code to get rid of duplicated ways, and using my style file have reduced the number of similar arcs warnings from about 21000 to about 600 which also results in a reduction in size of the gmapsupp file. I will be posting this patch shortly. Cheers, Mike -Original Message- From: Mike Baggaley [mailto:m...@tvage.co.uk] Sent: 16 May 2021 08:48 To: 'Development list for mkgmap' Subject: RE: [mkgmap-dev] logging improvement Hi Gerd, There are lots of pedestrian areas that are relations. Some of these overlap with adjacent pedestrian areas and produce a warning like: Similar arcs (generated way) and (generated way) found at http://www.openstreetmap.org/?mlat=50.728438=-1.143728=17 However, don't commit it yet - I have realised that although this fixes most of the messages, there are some still being displayed as generated way, presumably because the relation is copied prior to being used. I'm looking at this now. The above example after the patch displays: Similar arcs (generated way) and (http://www.openstreetmap.org/relation/9381074) found at http://www.openstreetmap.org/?mlat=50.728438=-1.143728=17 Cheers, Mike -Original Message- From: Gerd Petermann [mailto:gpetermann_muenc...@hotmail.com] Sent: 16 May 2021 07:32 To: Development list for mkgmap Subject: Re: [mkgmap-dev] logging improvement Hi Mike, please help me: In what situation do we create roads from ways which were created from relations? Gerd Von: mkgmap-dev im Auftrag von Mike Baggaley Gesendet: Samstag, 15. Mai 2021 14:31 An: 'Development list for mkgmap' Betreff: [mkgmap-dev] logging improvement Hi Gerd, Please find attached a small patch that changes the logging of RoadDef when the original element is a relation. Currently it just says "generated way"; with the patch it gives the URL of the relation in the same way as it does for ways. Cheers, Mike ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Problems with sea in overview map
Hi Ticker, I've attached a test case that shows how the order is important. Depending on the order of shapes to be merged the resulting shape sometimes is self-intersecting, sometimes it isn't. I've not yet understood what the trigger is. Gerd Von: mkgmap-dev im Auftrag von Gerd Petermann Gesendet: Mittwoch, 19. Mai 2021 18:14 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Ticker, yes, I think ShapeMerger can always merge. There seems to be no need to look at the number of points. Maybe 24 is an exception. Sorting has an effect, but not always to the better. The merger is already used with rounded coord when the shapes for the overview map are merged. Attached is a patch that shows what I am playing with. Will continue tomorrow... Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Mittwoch, 19. Mai 2021 17:41 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problems with sea in overview map Hi Gerd I still haven't deciphered the complicated bits of ShapeMergeFilter. Could it do the testing of mergeable candidates with rounded-to -resolution points, then, if merging, convert all the points. Regardless, the limit checks against MAX_POINT_IN_ELEMENT could use PredictFilterPoints(). It isn't a major problem if this limit is exceeded because filtering might take it below again and, if it doesn't, PolygonSplitter will deal with it. Regarding ordering, maybe if merge succeeds for some potential shapes, they should all be done. Intersecting polygons from rounding of concave shapes with a long diagonal is slightly insoluble. Maybe something like WrongAngleFixer is going to be needed. Multipolygon cuts that are vertical or horizontal might be less of a problem. I'm considering the effect of different orders of filters, such that if a small hole becomes a point and cuts to it, after merging, become a series of spikes going into the shape, removeObsoletePoints removes it. Ticker On Wed, 2021-05-19 at 14:12 +, Gerd Petermann wrote: > Hi Ticker, > > I think ShapeMergeFilter could be part of the problem. It sometimes > finds strange ways to merge shapes, esp. when many small parts can be > merged. Maybe this can be improved by sorting so that it merges from > top to bottom. > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Mittwoch, 19. Mai 2021 14:14 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Problems with sea in overview map > > Hi Gerd > > I've just been experimenting and looking at shapes and rounding-to > -low > -resolution can easily cause self-intersections. > > It isn't just a problem with sea, or even with multipolygon hole > cutting. GPSMapEdit reports over 1000 in my test area. This number > reduces only slightly when I disable DP for polygons. Generally these > are not noticeable - a little bit of wood missing when zoomed out > isn't > obvious, but same is not true of sea. > > Ticker > > On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote: > > Hi Ticker, > > > > no need to guess, I use GpxCreator in various places to visualize > > input and output of filters. I load the gpx files into JOSM and > > maybe > > convert to Data layer to understand what happens, e.g. where the > > start /end point is. > > Reg. self-intersection: > > Think of a merged sea shape with several islands in it. The > > rounding > > may result in only three or four different nodes, visitied in a > > more > > or less random order while the correct shape would be a triangle or > > rectangle. > > Not sure if this is only a problem with sea, probably not. > > > > Feel free to try other orders of the filters, or maybe different > > algos to set the preserved flag. There is definitely room for > > improvements. I've played with this very often and sometimes one > > area > > is improved and another is worsened. > > > > Gerd > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev testMergeOrder.osm Description: testMergeOrder.osm ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
