Re: [HOT] Broken (multi)polygon cleanup
Hi! I am sure there are many other problems, but can we please keep to the issue at hand in this thread? I am not trying to solve all problems, I am trying to solve one particular set of problem related to (multi)polygons. And I didn't pick that particular problem at random. This is something that affects everybody. The short of it is: Due to upcoming software changes, some (multi)polygons that are rendered now will disappear from the map or look different in the future. Again, you can find all the details about the problems and of what we are trying to achieve at http://area.jochentopf.com/ . We are having great success working on these problems. You can see the stats at http://area.jochentopf.com/stats/ . This doesn't mean that the approach will be the best one for other problems, and it doesn't mean we can't do even better than we are doing until now with this problem. Btw: I am deliberately talking about "(multi)polygons". This is not something that only affects multipolygon relations, but all polygons, even the ones created from closed ways. They are simpler than mp relations, but that doesn't mean they can't have problems. Jochen On Thu, Mar 16, 2017 at 05:23:47PM -0400, john whelan wrote: > Date: Thu, 16 Mar 2017 17:23:47 -0400 > From: john whelan > To: Jochen Topf > Cc: hot@openstreetmap.org, Blake Girardot HOT/OSM > > Subject: Re: [HOT] Broken (multi)polygon cleanup > > This is similar to my pulling in Africa a country at a time and using JOSM > to check for errors. > > What I have noticed is one mapper will misstag then others will follow. So > sanitising the area or checking it for errors before the new HOT mappers > start would probably improve the overall data quality as they tend to > follow what is already mapped. > > Untagged ways by one mapper tend to cluster. I've seen a 100 buildings in > one small area by the same mapper left as area=yes. That's quick and easy > to fix. > > Recently I came across 42 highways left untagged by the same mapper in the > same area. The mapper had more than 2,000 edits so wasn't inexperienced. > > Crossing highways often I'll see a town where the highway residentials are > laid out in a grid but the junctions aren't nodes. > > Another nice recent one was a batch of settlements that had been tagged > highway=unclassified rather than landuse=residential. Searching for > highways above unclassified and crossing ways picked those out quite > quickly. > > These types of errors are handled more quickly directly in JOSM than > involving maproulette. > > Even the some other errors can be loaded up in the JOSM to-do plugin. > > I've been cleaning up in Nigeria for a while now. Making repeated passes > over the same areas over time. I'm seeing fewer errors so my suspicion is > the improved learnOSM, tutorials and simplifying what we ask mappers to map > is paying off. Validating new mappers work within 48 hours does pay off in > data quality terms. Giving feedback more than two weeks later is usually a > waste of time based on my own experience and Martin's report but I don't > think we have the validators to do the validation within the magic 24/48 > hours. > > Broken multipolyons do take a level of expertise to resolve. I usually > avoid them. However new mappers will fearlessly tread ahead and it is to > be expected they will break some. > > I don't think learnOSM goes into detail about them or about relationships. > Should they? That's another question. > > For some errors we need boots on the ground. Especially when it comes to > urban areas with limited access highways such as dual carriageways and a > highway approaching them can you turn in either direction? > > Often the most motivated people to fix the errors are the locals who will > be using the map. Trouble is we first have to get something in place that > is useful to get them motivated. > > So yes HOT maperthons do have a reputation of the blind leading the blind, > ie some are led by inexperienced mappers but we are putting effort into > reducing the number of errors. The training group has put together a very > good detailed set of instructions on how best to run a maperthon and if you > can identify a problem area HOT will listen and try to address the issues. > > Cheerio John > > On 16 Mar 2017 3:49 pm, "Jochen Topf" wrote: > > > Hi Blake, > > > > On Wed, Mar 15, 2017 at 05:29:04PM +0100, Blake Girardot HOT/OSM wrote: > > > Can you send me something that would help me see those 1600 examples > > > you found on OSM so I can see what might be going on and how to > > > improve things in HOT/MM workflow to reduce them? > > > > Those 1600 are some very specific errors I am currently looking at, but > > only have those on my local machine currently. But for the wider picture > > you can use the OSM Inspectors "area" view to see the scope of the problem. > > > > Just go to any place where HOT/MM was active. Here is an example: > > http://tools.geofabrik.de/osmi/?
Re: [HOT] Broken (multi)polygon cleanup
What isn't clear to me is just what you are trying to fix. The term means nothing to me. I am correcting duplicate segments and unclosed ways which appear to be part of your problem as I work. Perhaps someone can explain what you are after in simpler terms because if we don't understand the problem then trying to come up with ways to avoid it in the future isn't going to happen. Thanks John On 17 March 2017 at 08:16, Jochen Topf wrote: > Hi! > > I am sure there are many other problems, but can we please keep to the > issue at hand in this thread? I am not trying to solve all problems, I > am trying to solve one particular set of problem related to > (multi)polygons. > > And I didn't pick that particular problem at random. This is something > that affects everybody. The short of it is: Due to upcoming software > changes, some (multi)polygons that are rendered now will disappear from > the map or look different in the future. Again, you can find all the > details about the problems and of what we are trying to achieve at > http://area.jochentopf.com/ . > > We are having great success working on these problems. You can see the > stats at http://area.jochentopf.com/stats/ . This doesn't mean that the > approach will be the best one for other problems, and it doesn't mean we > can't do even better than we are doing until now with this problem. > > Btw: I am deliberately talking about "(multi)polygons". This is not > something that only affects multipolygon relations, but all polygons, > even the ones created from closed ways. They are simpler than mp > relations, but that doesn't mean they can't have problems. > > Jochen > > On Thu, Mar 16, 2017 at 05:23:47PM -0400, john whelan wrote: > > Date: Thu, 16 Mar 2017 17:23:47 -0400 > > From: john whelan > > To: Jochen Topf > > Cc: hot@openstreetmap.org, Blake Girardot HOT/OSM > > > > Subject: Re: [HOT] Broken (multi)polygon cleanup > > > > This is similar to my pulling in Africa a country at a time and using > JOSM > > to check for errors. > > > > What I have noticed is one mapper will misstag then others will follow. > So > > sanitising the area or checking it for errors before the new HOT mappers > > start would probably improve the overall data quality as they tend to > > follow what is already mapped. > > > > Untagged ways by one mapper tend to cluster. I've seen a 100 buildings > in > > one small area by the same mapper left as area=yes. That's quick and > easy > > to fix. > > > > Recently I came across 42 highways left untagged by the same mapper in > the > > same area. The mapper had more than 2,000 edits so wasn't inexperienced. > > > > Crossing highways often I'll see a town where the highway residentials > are > > laid out in a grid but the junctions aren't nodes. > > > > Another nice recent one was a batch of settlements that had been tagged > > highway=unclassified rather than landuse=residential. Searching for > > highways above unclassified and crossing ways picked those out quite > > quickly. > > > > These types of errors are handled more quickly directly in JOSM than > > involving maproulette. > > > > Even the some other errors can be loaded up in the JOSM to-do plugin. > > > > I've been cleaning up in Nigeria for a while now. Making repeated passes > > over the same areas over time. I'm seeing fewer errors so my suspicion > is > > the improved learnOSM, tutorials and simplifying what we ask mappers to > map > > is paying off. Validating new mappers work within 48 hours does pay off > in > > data quality terms. Giving feedback more than two weeks later is > usually a > > waste of time based on my own experience and Martin's report but I don't > > think we have the validators to do the validation within the magic 24/48 > > hours. > > > > Broken multipolyons do take a level of expertise to resolve. I usually > > avoid them. However new mappers will fearlessly tread ahead and it is to > > be expected they will break some. > > > > I don't think learnOSM goes into detail about them or about > relationships. > > Should they? That's another question. > > > > For some errors we need boots on the ground. Especially when it comes to > > urban areas with limited access highways such as dual carriageways and a > > highway approaching them can you turn in either direction? > > > > Often the most motivated people to fix the errors are the locals who will > > be using the map. Trouble is we first have to get something in place > that > > is useful to get them motivated. > > > > So yes HOT maperthons do have a reputation of the blind leading the > blind, > > ie some are led by inexperienced mappers but we are putting effort into > > reducing the number of errors. The training group has put together a > very > > good detailed set of instructions on how best to run a maperthon and if > you > > can identify a problem area HOT will listen and try to address the > issues. > > > > Cheerio John > > > > On 16 Mar 2017 3:49 pm, "Jochen Topf" wrote:
Re: [HOT] Broken (multi)polygon cleanup
Hi! did you read those documents linked from http://area.jochentopf.com/ ? https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/background.md https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/problems.md https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/faq.md If there is something unclear after reading them, please ask, and I am happy to explain. Jochen On Fri, Mar 17, 2017 at 08:30:27AM -0400, john whelan wrote: > Date: Fri, 17 Mar 2017 08:30:27 -0400 > From: john whelan > To: Jochen Topf > Cc: "hot@openstreetmap.org" , Blake Girardot HOT/OSM > > Subject: Re: [HOT] Broken (multi)polygon cleanup > > What isn't clear to me is just what you are trying to fix. The term means > nothing to me. I am correcting duplicate segments and unclosed ways which > appear to be part of your problem as I work. Perhaps someone can explain > what you are after in simpler terms because if we don't understand the > problem then trying to come up with ways to avoid it in the future isn't > going to happen. > > Thanks John > > On 17 March 2017 at 08:16, Jochen Topf wrote: > > > Hi! > > > > I am sure there are many other problems, but can we please keep to the > > issue at hand in this thread? I am not trying to solve all problems, I > > am trying to solve one particular set of problem related to > > (multi)polygons. > > > > And I didn't pick that particular problem at random. This is something > > that affects everybody. The short of it is: Due to upcoming software > > changes, some (multi)polygons that are rendered now will disappear from > > the map or look different in the future. Again, you can find all the > > details about the problems and of what we are trying to achieve at > > http://area.jochentopf.com/ . > > > > We are having great success working on these problems. You can see the > > stats at http://area.jochentopf.com/stats/ . This doesn't mean that the > > approach will be the best one for other problems, and it doesn't mean we > > can't do even better than we are doing until now with this problem. > > > > Btw: I am deliberately talking about "(multi)polygons". This is not > > something that only affects multipolygon relations, but all polygons, > > even the ones created from closed ways. They are simpler than mp > > relations, but that doesn't mean they can't have problems. > > > > Jochen > > > > On Thu, Mar 16, 2017 at 05:23:47PM -0400, john whelan wrote: > > > Date: Thu, 16 Mar 2017 17:23:47 -0400 > > > From: john whelan > > > To: Jochen Topf > > > Cc: hot@openstreetmap.org, Blake Girardot HOT/OSM > > > > > > Subject: Re: [HOT] Broken (multi)polygon cleanup > > > > > > This is similar to my pulling in Africa a country at a time and using > > JOSM > > > to check for errors. > > > > > > What I have noticed is one mapper will misstag then others will follow. > > So > > > sanitising the area or checking it for errors before the new HOT mappers > > > start would probably improve the overall data quality as they tend to > > > follow what is already mapped. > > > > > > Untagged ways by one mapper tend to cluster. I've seen a 100 buildings > > in > > > one small area by the same mapper left as area=yes. That's quick and > > easy > > > to fix. > > > > > > Recently I came across 42 highways left untagged by the same mapper in > > the > > > same area. The mapper had more than 2,000 edits so wasn't inexperienced. > > > > > > Crossing highways often I'll see a town where the highway residentials > > are > > > laid out in a grid but the junctions aren't nodes. > > > > > > Another nice recent one was a batch of settlements that had been tagged > > > highway=unclassified rather than landuse=residential. Searching for > > > highways above unclassified and crossing ways picked those out quite > > > quickly. > > > > > > These types of errors are handled more quickly directly in JOSM than > > > involving maproulette. > > > > > > Even the some other errors can be loaded up in the JOSM to-do plugin. > > > > > > I've been cleaning up in Nigeria for a while now. Making repeated passes > > > over the same areas over time. I'm seeing fewer errors so my suspicion > > is > > > the improved learnOSM, tutorials and simplifying what we ask mappers to > > map > > > is paying off. Validating new mappers work within 48 hours does pay off > > in > > > data quality terms. Giving feedback more than two weeks later is > > usually a > > > waste of time based on my own experience and Martin's report but I don't > > > think we have the validators to do the validation within the magic 24/48 > > > hours. > > > > > > Broken multipolyons do take a level of expertise to resolve. I usually > > > avoid them. However new mappers will fearlessly tread ahead and it is to > > > be expected they will break some. > > > > > > I don't think learnOSM goes into detail about them or about > > relationships. > > > Should they? That's another question. > > > > > > For some errors we need boots on t
Re: [HOT] Broken (multi)polygon cleanup
I've read it through but remember like many mappers I do not have a Ph.D. in GIS, just a degree in Chemistry so I'm usually not too bad on logic. The first thing that stands out is the message is too complex for the intended audience. The second "The *new style* tagging option is the recommended tagging option these days, but some mappers still disagree." So we don't have complete agreement, fine. The third is the main problem areas seem to be some rendering systems prefer one method over the other. "Are we mapping for a specific rendering system now?" I get the impression that the main problem is areas of landuse or defining countries. The defining country side I leave to specialists. Landuse=forest type things are often out of date and problematical. At a basic level I think we are talking using JOSM to do this work. I hate to say it but despite my efforts JOSM is still not main stream for HOT mapathons. So essentially we will confuse the audience and confused HOT mappers may lead to other problems. I suspect if we concentrate on unclosed ways and duplicate segments then this is something that can be done without a Ph.D. in GIS concepts and there is no disagreement. Leave the more complex problem solving to others. I have yet to see a HOT project that asks mappers to map country outlines. I have seen some that ask for landuse=residential but that is about as complicated as it normally gets. There might be one or two odd ones that ask for landuse but these days they are rare. The bulk of the mapping is buildings and highways with the occasional river thrown in and I don't think these are problem areas to you. There have been some imports in the HOT areas of questionable data quality some are fine but that is a different matter to HOT mapping. I would recommend if you wish to use the resources of HOT that you talk nicely to the HOT training group and see if they can sort out the message and what training needs to be given to support these efforts including what you would like mappers to avoid when mapping. Thanks Cheerio John On 17 March 2017 at 09:43, Jochen Topf wrote: > Hi! > > did you read those documents linked from http://area.jochentopf.com/ ? > > https://github.com/osmlab/fixing-polygons-in-osm/blob/ > master/doc/background.md > https://github.com/osmlab/fixing-polygons-in-osm/blob/ > master/doc/problems.md > https://github.com/osmlab/fixing-polygons-in-osm/blob/master/doc/faq.md > > If there is something unclear after reading them, please ask, and I am > happy to explain. > > Jochen > > On Fri, Mar 17, 2017 at 08:30:27AM -0400, john whelan wrote: > > Date: Fri, 17 Mar 2017 08:30:27 -0400 > > From: john whelan > > To: Jochen Topf > > Cc: "hot@openstreetmap.org" , Blake Girardot > HOT/OSM > > > > Subject: Re: [HOT] Broken (multi)polygon cleanup > > > > What isn't clear to me is just what you are trying to fix. The term > means > > nothing to me. I am correcting duplicate segments and unclosed ways > which > > appear to be part of your problem as I work. Perhaps someone can explain > > what you are after in simpler terms because if we don't understand the > > problem then trying to come up with ways to avoid it in the future isn't > > going to happen. > > > > Thanks John > > > > On 17 March 2017 at 08:16, Jochen Topf wrote: > > > > > Hi! > > > > > > I am sure there are many other problems, but can we please keep to the > > > issue at hand in this thread? I am not trying to solve all problems, I > > > am trying to solve one particular set of problem related to > > > (multi)polygons. > > > > > > And I didn't pick that particular problem at random. This is something > > > that affects everybody. The short of it is: Due to upcoming software > > > changes, some (multi)polygons that are rendered now will disappear from > > > the map or look different in the future. Again, you can find all the > > > details about the problems and of what we are trying to achieve at > > > http://area.jochentopf.com/ . > > > > > > We are having great success working on these problems. You can see the > > > stats at http://area.jochentopf.com/stats/ . This doesn't mean that > the > > > approach will be the best one for other problems, and it doesn't mean > we > > > can't do even better than we are doing until now with this problem. > > > > > > Btw: I am deliberately talking about "(multi)polygons". This is not > > > something that only affects multipolygon relations, but all polygons, > > > even the ones created from closed ways. They are simpler than mp > > > relations, but that doesn't mean they can't have problems. > > > > > > Jochen > > > > > > On Thu, Mar 16, 2017 at 05:23:47PM -0400, john whelan wrote: > > > > Date: Thu, 16 Mar 2017 17:23:47 -0400 > > > > From: john whelan > > > > To: Jochen Topf > > > > Cc: hot@openstreetmap.org, Blake Girardot HOT/OSM > > > > > > > > Subject: Re: [HOT] Broken (multi)polygon cleanup > > > > > > > > This is similar to my pulling in Africa a country at
Re: [HOT] Broken (multi)polygon cleanup
Hi Jochen Your focus on these problems more related to the softwares process is quite interesting and valuable. While I continue to focus on the Haiti south west peninsula after the Matthew hurricane landing in october 2016, I looked today at multipolygon relations misuse. For Buildings with only outer rings, I merged the ways if necessary, removed the relation and added the building tag to the ways. This Overpass query shows some of these multipolygons before correction. http://overpass-turbo.eu/s/nBb I also found and incorrect use is multipolygon relation as a collection of buildings, removed the relation and corrected the tagging. There were 7 buildings in this relation. This does not respect the OSM data model and we cannot easily count the number of building in GIS softwares. The mapper who added this relation has more then 600 days of contribution and hence classified by Pascal Neis as Great Mapper (Very Active). http://overpass-turbo.eu/s/nB9 The next problem I will look at is ways with only the area tag. Pierre ___ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot