Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-04-05 Thread moltonel 


On 5 April 2016 16:18:53 GMT+01:00, Greg Morgan 
>In my case, you'll have to provide more context here.  I look at OSM
>Inspector and Keep Right and see all these broken things.  That is a
>beautiful discovery.  Mappers are trying to improve the map that is
>very
>much a human endeavor and mistake prone.  I am just lost how everything
>is
>a bad import.

I don't think any of those import is bad, at least not on the criteria of 
duplicated way/relation tags. In the low count cases, the duplication is likely 
to have happened during later edits rather than the initial import. Today 
duplicating tag on way+rel is considered bad practice, but that came about 
slowly (standard practice used to be to tag the way only).

Duplication is also fairly harmless (as long as the tags don't contradict), and 
some would argue that it isn't worth the version churn.

Broken polygons on the other hand (which were probably broken manually, not by 
the imports) are worth fixing. They are easy to detect but not always easy to 
fix. Some error types might be mechanically fixable, but I guess that most need 
human intervention.
-- 
Vincent Dp

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-04-05 Thread Greg Morgan 
Context Please!

On Tue, Apr 5, 2016 at 1:49 AM, Frank Villaro-Dixon 
wrote:

> On 16-03-23 13:24:29, Andy Townsend, wrote 1.0K characters saying:
>
>> On 23/03/2016 12:22, Frank Villaro-Dixon wrote:
>>
>>> On 16-03-23 12:20:35, Andy Townsend, wrote 0.3K characters saying:
>>>
 On 23/03/2016 12:07, Frank Villaro-Dixon wrote:

> Again, that's not the goal if it. As said above, the role is NOT to
> change
> tags, but to remove redundancies.
>

 It doesn't matter.  All of the screed that I wrote yesterday applies to
 "redundant data" too - we need to understand how it got there, and what
 actually is wrong.

>>>
>>> Yep,
>>> as I said on another mail:
>>>
>>
>> Link please.  You've said lots of things, but much seemed very "handwavy"
>> and not really understanding of the issues.
>>
> TODO
>
>>
>> the majority comes from two 'bad' imports.
>>>
>>
>>

I looked back at the talk archives.  There's no context for this
discussion.  The moment I see something that says it is because of bad
imports, I start thinking that the issue and the supporting data is tainted.




> Link please - exactly which ones, run by whom and when?
>>
> Complete list (generated by me; source on git) is here:
> http://pastebin.com/Cn7gyn76
>
> Statistics are here:
> http://pastebin.com/KrZWJY6j
>
>
> Extract:
> 16 CanVec_Import_2009
> 16 PGS & Bing
> 22 PGS
> 23 Yahoo
>

Yahoo as a bad import?  How about a new mapper doing the best that they can
to improve the map added a few tags that a more experienced mapper can
build on?


> 36 CANVEC
> 38 JOSM lakewalker plugin
>


Where I see someone like Norm Abram[1] make these wonderful jigs and
patterns as labor saving devices for wood working, based on the limited
context, you are saying that the same labor saving Josm Lakewaker plugin to
help a mapper trace a lake is an import or a bad import?


> 39 bing
> 39 Lakewalker / Landsat
>

Ditto.


> 44 KSJ2
> 54 Landsat
> 56 CanVec 6.0 - NRCan
> 57 IRS
> 63 NRCan-CanVec-8.0
>118 landsat
>121 Bing
>123 NRCan-CanVec-7.0
>437 Kartverket N50
>802 NRCan-CanVec-10.0
>   1476
>   1708 NHD
>

The importers that provided me all the NHD data are my heroes.  Those
imports gave me a great deal of material to work with.

In my case, you'll have to provide more context here.  I look at OSM
Inspector and Keep Right and see all these broken things.  That is a
beautiful discovery.  Mappers are trying to improve the map that is very
much a human endeavor and mistake prone.  I am just lost how everything is
a bad import.

Please Advise,
Greg


[1] https://en.wikipedia.org/wiki/Norm_Abram
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-04-05 Thread Frank Villaro-Dixon 

On 16-03-23 13:24:29, Andy Townsend, wrote 1.0K characters saying:

On 23/03/2016 12:22, Frank Villaro-Dixon wrote:

On 16-03-23 12:20:35, Andy Townsend, wrote 0.3K characters saying:

On 23/03/2016 12:07, Frank Villaro-Dixon wrote:
Again, that's not the goal if it. As said above, the role is NOT 
to change

tags, but to remove redundancies.


It doesn't matter.  All of the screed that I wrote yesterday 
applies to "redundant data" too - we need to understand how it got 
there, and what actually is wrong.


Yep,
as I said on another mail:


Link please.  You've said lots of things, but much seemed very "handwavy" 
and not really understanding of the issues.

TODO



the majority comes from two 'bad' imports.


Link please - exactly which ones, run by whom and when?

Complete list (generated by me; source on git) is here:
http://pastebin.com/Cn7gyn76

Statistics are here:
http://pastebin.com/KrZWJY6j


Extract:
16 CanVec_Import_2009
16 PGS & Bing
22 PGS
23 Yahoo
36 CANVEC
38 JOSM lakewalker plugin
39 bing
39 Lakewalker / Landsat
44 KSJ2
54 Landsat
56 CanVec 6.0 - NRCan
57 IRS
63 NRCan-CanVec-8.0
   118 landsat
   121 Bing
   123 NRCan-CanVec-7.0
   437 Kartverket N50
   802 NRCan-CanVec-10.0
  1476
  1708 NHD


We can see that of the 5477 ways that need modification, half of them 
(1708 + 802 + 437 + 123 + 63 = 57% of 5477) were created by automated 
imports.



Cheers,

Frank

--
frank.villaro-dixon.eu   - PGP: 6F36914A
Envie d'électricité 100% verte ? Enercoop.fr
What is a Velomobile ?   www.sans-essence.eu

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread Martijn van Exel 
Possibly! I am not sure if I understand the full scope of the issues, but if 
someone could catch some of the issues presented in one or more Overpass 
queries, I (or anyone, really) can use geojson2maproulette to make a 
MapRoulette challenge out of it.

Martijn

> On Mar 22, 2016, at 10:37 AM, David Fawcett  wrote:
> 
> I think that this would be a great Maproulette subject!
> 
> 
> 
>> On Mar 22, 2016, at 9:23 AM, Maarten Deen  wrote:
>> 
>>> On 2016-03-22 14:10, Frank Villaro-Dixon wrote:
>>> 
>>> # First goal:
>>> First goal is quite simple. The idea is to work only on relations
>>> which have a natural=water .  Then, it will:
>>>   * Delete natural=water from all the ways if they are NOT closed or
>>> ring 0.
>> 
>> I have been fixing waterways and water areas in the New Orleans area. I 
>> think most problems are due to a botched import. There are a number of 
>> unclosed areas, some water, some wetland. They just need to be closed which 
>> sometimes is just connecting the last two dots, sometimes it is merging two 
>> ways.
>> 
>> It would be very unhelpful if the natural=water tags of these ways would be 
>> deleted. Then you have to go search in the history to see what it was. Is it 
>> natural=water or natural=wetland? It would increase the workload in fixing 
>> this.
>> 
>> Please, do not do this. You can flag them as defective so that someone has a 
>> look, but don't delete the tags.
>> 
>> Since deleting tags from unclosed ways does not solve anything (and IMHO 
>> makes things worse), this should be done manually. It would be a perfect 
>> case to enter in maproulette 
>> 
>> Regards,
>> Maarten
>> 
>> 
>> ___
>> talk mailing list
>> talk@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk
> 
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread malenki 
Frank Villaro-Dixon wrote:

>On 16-03-22 17:25:37, malenki, wrote 2.1K characters saying:
  this is nonsense
>>Example:
>>Think of an MP where the way has – additionally to the tags you want
>>to fix – intermittent=yes and the relation has not.
>>
>>Would your script really enhance the situation?  
>In that case, it wouldn't touch this tag.

You did not answer the question.

>>Where is the problem /not/ to fix this by script, then?  
>Huh ?

Exactly what fellow mappers think of your bot. :)

>>Recently I was told by somebody: Deleting data is not a real deletion
>>in OSM. It is just hiding that data.  
>I don't know how is data stored back office, but if it doesn't show
>any more on the API query, that's good, no ?

Obviously you don't know much about OSM.

>>Why not stop replying mails here reflex-like and start to think about
>>the problem? Again? With the new information you were given? And
>>without trying to squeeze all the information in your scheme of
>>thinking (or "problem solving").  
>Well, this fix was a simple fix […]

Several people already told you this wasn't really a fix.

eod for me
Thomas



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread Mike Thompson 
On Tue, Mar 22, 2016 at 8:12 AM, Frank Villaro-Dixon  wrote:

>
>> Can you give me an example ?
>
> Two lakes, collectively known as "Sheep Lakes" but individually known as
"North Sheep Lake" and "South Sheep Lake"
R [natural=water name=Sheep Lakes]
W [natural=water name=North Sheep Lake]
W [natural=water name=South Sheep Lake]

If you remove "natural=water" from the ways, then someone looking for a
water body named "North Sheep Lake" or "South Sheep Lake" wouldn't easily
be able to find it.

Mike
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread moltonel 3x Combo 
On 23/03/2016, Frank Villaro-Dixon  wrote:
> Maybe it's an higher priority, but that doesn't interest me. If someone
> wants to, good for them.

Fair enough, we all have different priorities.

> Now, the lower priority 'redundant tag deleting'
> is still needed, and if it can be done automatically (nobody still hasn't
> given me a fuckup example), then why not ? It's still usefull if you want
> to extract all the lakes of the world, for example.

How do redundant tags prevent you from extracting all the lakes of the
world ? It should be harmless, you'll just have some objects that will
be selected by two different criterias. For better or worse, any OSM
consumer has to deal with some level of redundant data.

As for problems with your algorythm, people in this list _have_
pointed out potential problems, such as missing non-duplicated tags
that are on the way rather than on the relation (To properly fix that
you need a human eye. Fix the automatically-detectable part of the
problem only and you've just succeeded in making the other problems
harder to detect). There's no need to point at actual osm objects with
those issues that your script changed when the failure case are so
easy to come up with.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread moltonel 3x Combo 
On 22/03/2016, Pierre Béland  wrote:
> This is a good proposition to look at unclosed polygons and see if a
> potential incorrect keys to fix.
>
> I agree with others that using a Bot is not a safe way to handle these
> problems. Could a script to extract such unclosed polygons be proposed?
> This way, each local community could take care to fix data for their area,
> importing and examining the data in the JOSM Editor. The Todo plugin could
> be used to go through the items to correct.

I was going to point you towards
http://wiki.openstreetmap.org/wiki/Quality_Assurance_Tools_script but
for some reason it doesn't support OSMI's multipolygon checks. Might
be worth pinging the qat devs.

Possible alternatives:
* Load OSMI's multipolygon checks as an imagery layer. If only it
didn't flag type=boundary as an error. As it is it's a bit noisy :/
* Use the OSMI website directly
(http://tools.geofabrik.de/osmi/?view=multipolygon). You can tweak the
error types to display, and load object via remote command.
* Use the qat plugin but with other sources than OSMI. For
multipolygon errors, it's IMHO not as good.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread moltonel 3x Combo 
On 22/03/2016, Frank Villaro-Dixon  wrote:
> On 16-03-22 10:23:51, Nicolás Alvarez, wrote 1.2K characters saying:
>>
>>> El 22 mar 2016, a las 10:10, Frank Villaro-Dixon 
>>> escribió:
>>>
>>> # First goal:
>>> First goal is quite simple. The idea is to work only on relations which
>>> have a natural=water .  Then, it will:
>>>* Delete natural=water from all the ways if they are NOT closed or
>>> ring 0.
>>
>>What if there is a way legitimately with natural=water that isn't closed
>> because of an error? The correct fix is to close it, not to remove the
>> tag. This cannot be done automatically.
>>
> It doesn't matter because this way will also be referenced by a relation
> which has the natural=water tag. Thus, it can be removed from the way.

If the way being closed or not doesn't matter, then it shouldn't be
part of your criterias ? I see this as a red flag indicating you
didn't think long enough about the problem, and that doesn't bode well
for an automated world-wide edit.

You're conflating two different problems:

1) Some multipolygon relations have improper geometry, such as non-closed rings.
2) Some tags are "needlessly" repeated between the relation and its members.

Neither of those problems are specific to water features.

The first problem has many QA tools to point it out, for example
OSMI[1]. Maybe some of them can be fixed automatically, but I'd much
rather go through them manually[2] because some cases are strange, and
often when one MP is broken there's lots of other bugs in the same
area by the same user that deserve a fix.

The second problem actually has the community a bit divided (last I
heard of it), as some people *want* to have the actual ways tagged
(usually for ease of consumption, but also for historical reasons). I
usually delete the superfluous tag when I happen to be editing the
object, but I wouldn't dare do this automatically on a global scale.
And as long as those duplicate tags aren't contradicting each other,
they are harmless.

[1]:http://tools.geofabrik.de/osmi/?view=multipolygon
[2]:http://wiki.openstreetmap.org/wiki/Quality_Assurance_Tools_script

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread Andy Townsend 

On 23/03/2016 12:22, Frank Villaro-Dixon wrote:

On 16-03-23 12:20:35, Andy Townsend, wrote 0.3K characters saying:

On 23/03/2016 12:07, Frank Villaro-Dixon wrote:
Again, that's not the goal if it. As said above, the role is NOT to 
change

tags, but to remove redundancies.


It doesn't matter.  All of the screed that I wrote yesterday applies 
to "redundant data" too - we need to understand how it got there, and 
what actually is wrong.


Yep,
as I said on another mail: 


Link please.  You've said lots of things, but much seemed very 
"handwavy" and not really understanding of the issues.


the majority comes from two 'bad' imports. 


Link please - exactly which ones, run by whom and when?

The rest is so scarce that it could be considered as noise (edits made 
before multipolygons existed, etc…).




... and again, where users have edited any of the data post import, we 
still need to understand how the current situation (imported data + 
modifications) got to be as it is.


Cheers,

Andy



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread Frank Villaro-Dixon 

On 16-03-23 12:20:35, Andy Townsend, wrote 0.3K characters saying:

On 23/03/2016 12:07, Frank Villaro-Dixon wrote:
Again, that's not the goal if it. As said above, the role is NOT to 
change

tags, but to remove redundancies.


It doesn't matter.  All of the screed that I wrote yesterday applies 
to "redundant data" too - we need to understand how it got there, and 
what actually is wrong.


Yep,
as I said on another mail: the majority comes from two 'bad' imports. The 
rest is so scarce that it could be considered as noise (edits made before 
multipolygons existed, etc…).


--
frank.villaro-dixon.eu   - PGP: 6F36914A
Envie d'électricité 100% verte ? Enercoop.fr
What is a Velomobile ?   www.sans-essence.eu

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread Frank Villaro-Dixon 

On 16-03-22 16:46:03, Christoph Hormann, wrote 1.5K characters saying:

On Tuesday 22 March 2016, Frank Villaro-Dixon wrote:


Can you give me an example ?


I probably could (after all i am on record for saying waterbody mapping
in OSM is a practical case of the infinite monkey theorem) but right
now i don't have the time to look for a good real world example.

In principle many cases where the way you'd remove a tag from has
additional tags the meaning of those tags will change when you remove a
tag. 

For example ? I'm only focusing on natural= and water=
Also cases where the way in question forms the division between two water 
areas you will usually run into trouble.

No, because they are part of a relation multipolygon.


On a principal level removing an ambiguity will always mean you remove
possible interpretations of the data and your bot simply cannot know if
the interpretations it removes are actually incorrect and the remaining
interpretation is the correct one.

No,
If you have a lake with multiple ways grouped with a multipoly as shown:
R [natural=water water=lake]
W [water=lake intermittent=yes]
W [intermittent=yes]
W [natural=water]

It will be transformed as:
R [natural=water water=lake]
W [intermittent=yes]
W [intermittent=yes]
W []

Where's the other possible interpretation here ?




This algorithm won't loose any information. It's merely deleting
duplicates, so where is the problem ?


You are only looking at it from a formal side and based on a specific,
subjective interpretation of tagging rules which is not appropriate.
See also

http://wiki.openstreetmap.org/wiki/Automated_Edits_code_of_conduct

As others have said the best way to improve waterbody data in OSM is by
actually doing mapping work and correcting errors based on local
knowledge or imagery.  You can believe me when i say that factual
errors and inaccuracies are much more common and harmful problems
w.r.t. water area mapping than basic tagging inconsistencies.
Try exporting all waterbodies them; you'll see that in the actual state, 
it's a royal pain in the ass. It's a shame because the quality of OSM is 
good, but it's really hard to work with this data…




--
frank.villaro-dixon.eu   - PGP: 6F36914A
Envie d'électricité 100% verte ? Enercoop.fr
What is a Velomobile ?   www.sans-essence.eu

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread Andy Townsend 

On 23/03/2016 12:07, Frank Villaro-Dixon wrote:
Again, that's not the goal if it. As said above, the role is NOT to 
change

tags, but to remove redundancies.


It doesn't matter.  All of the screed that I wrote yesterday applies to 
"redundant data" too - we need to understand how it got there, and what 
actually is wrong.


Cheers,

Andy


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread Frank Villaro-Dixon 

On 16-03-23 09:54:34, Warin, wrote 6.0K characters saying:

On 23/03/2016 2:47 AM, Pierre Béland wrote:

On 2016-03-22 14:10, Frank Villaro-Dixon wrote:

# First goal:
First goal is quite simple. The idea is to work only on relations
which have a natural=water .  Then, it will:
   * Delete natural=water from all the ways if they are NOT closed or
   ring 0.



This is a good proposition to look at unclosed polygons and see if a 
potential incorrect keys to fix.


I agree with others that using a Bot is not a safe way to handle 
these problems. Could a script to extract such unclosed polygons be 
proposed?  This way, each local community could take care to fix 
data for their area, importing and examining the data in the JOSM 
Editor. The Todo plugin could be used to go through the items to 
correct.


Pierre



An unclosed polygon is an error ... and should be fixed.
These can be detected using OSM Inspector. There are lots of them!

Surely that is a higher priority than deleting duplicate tags from 
ways that exist in relations (closed of not)?
So go fix the unclosed polygons, that will have much more benefit to 
the map than the deletion of duplicates.
Maybe it's an higher priority, but that doesn't interest me. If someone 
wants to, good for them. Now, the lower priority 'redundant tag deleting' 
is still needed, and if it can be done automatically (nobody still hasn't 
given me a fuckup example), then why not ? It's still usefull if you want 
to extract all the lakes of the world, for example.



--
frank.villaro-dixon.eu   - PGP: 6F36914A
Envie d'électricité 100% verte ? Enercoop.fr
What is a Velomobile ?   www.sans-essence.eu

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-23 Thread Frank Villaro-Dixon 

On 16-03-22 14:52:55, Andy Townsend, wrote 7.8K characters saying:

On 22/03/2016 13:10, Frank Villaro-Dixon wrote:
Technically, it was already run on the whole planet, and so far no bugs 
were found.

That's not true.  Many people complained and all your work was reverted.*

The complaining was that the thing was not accepted by the community, not
because some bugs were produced. Actually, there was 2 complainings for bugs
but they revealed false afterwards after not having taken into account the
relation.




Now, I need your comments and/or your approval, critiques, etc.
Tell me what you think ;-)



Here's what I think you should do, when you detect a potential problem:

1) Firstly, before fixing anything, try and understand what the cause was.
Perhaps an inexperienced mapper has edited some existing data that broke
something that they didn't understand?  You'll need to look at the mappers
who have contributed to the problem, their relative experience, and what
editors they are using (for example, an iD user may been not have seen the
complicated reationship between multipoloygons, and a JOSM user may have
stopped thinking about real-world data and thought _only_ in terms of
multipolygons - both can cause errors).

True. A lot of errors though were made after automated imports; mainly in the
Canada/USA regions.



If these three all agree, and it was just a tagging error (for example
I've seen people add "natural=foo" instead of "name=foo" recently) then it
makes sense to "just correct the data".  However, it's quite likely that
these three might disagree, and perhaps you need to explain to an earlier
mapper how multipolygons work, or to someone who has come along and
"corrected" data in the interim that what they've changed something to is
a valid OSM tag, but doesn't actually match what's on the ground in this
case.

Yep, true to. But the idea here is not to correct the data, but to remove
duplicates. In the script, if the relation and the way have not the same tags,
then it doesn't do anything. ONLY in a case of 'perfect copy' then the
redundant tag is removed.



There may be many scenarios that you haven't considered when designed what
automatic changes you are trying to make.  Other mappers will be able to
help you understand those when you discuss your plans with them.

True to. But we should separate the trivial bot with other changes made.
concerning the bot, nobody hasn't still showed me a concrete error…


Also, please don't think that "changing a tag to one that is valid within
OSM" means "making the data correct" - it doesn't.

Again, that's not the goal if it. As said above, the role is NOT to change
tags, but to remove redundancies.


Cheers,

Frank

--
frank.villaro-dixon.eu   - PGP: 6F36914A
Envie d'électricité 100% verte ? Enercoop.fr
What is a Velomobile ?   www.sans-essence.eu


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread Warin 

On 23/03/2016 2:47 AM, Pierre Béland wrote:

On 2016-03-22 14:10, Frank Villaro-Dixon wrote:


># First goal:
>First goal is quite simple. The idea is to work only on relations
>which have a natural=water .  Then, it will:
>* Delete natural=water from all the ways if they are NOT closed or
>ring 0.
>

This is a good proposition to look at unclosed polygons and see if a 
potential incorrect keys to fix.


I agree with others that using a Bot is not a safe way to handle these 
problems. Could a script to extract such unclosed polygons be 
proposed?  This way, each local community could take care to fix data 
for their area, importing and examining the data in the JOSM Editor. 
The Todo plugin could be used to go through the items to correct.


Pierre



An unclosed polygon is an error ... and should be fixed.
These can be detected using OSM Inspector. There are lots of them!

Surely that is a higher priority than deleting duplicate tags from ways 
that exist in relations (closed of not)?
So go fix the unclosed polygons, that will have much more benefit to the 
map than the deletion of duplicates.



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread David Fawcett 
I think that this would be a great Maproulette subject!



> On Mar 22, 2016, at 9:23 AM, Maarten Deen  wrote:
> 
>> On 2016-03-22 14:10, Frank Villaro-Dixon wrote:
>> 
>> # First goal:
>> First goal is quite simple. The idea is to work only on relations
>> which have a natural=water .  Then, it will:
>>* Delete natural=water from all the ways if they are NOT closed or
>> ring 0.
> 
> I have been fixing waterways and water areas in the New Orleans area. I think 
> most problems are due to a botched import. There are a number of unclosed 
> areas, some water, some wetland. They just need to be closed which sometimes 
> is just connecting the last two dots, sometimes it is merging two ways.
> 
> It would be very unhelpful if the natural=water tags of these ways would be 
> deleted. Then you have to go search in the history to see what it was. Is it 
> natural=water or natural=wetland? It would increase the workload in fixing 
> this.
> 
> Please, do not do this. You can flag them as defective so that someone has a 
> look, but don't delete the tags.
> 
> Since deleting tags from unclosed ways does not solve anything (and IMHO 
> makes things worse), this should be done manually. It would be a perfect case 
> to enter in maproulette 
> 
> Regards,
> Maarten
> 
> 
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread malenki 
On Tue, 22 Mar 2016 15:12:42 +0100,
Frank Villaro-Dixon wrote:
> >Mappers frequently map things in ambiguous ways or change existing
> >mapping in ways that make it ambiguous and it is hard to decide from
> >the data alone how to interpret such mapping.  A bot will not be any
> >better in doing that than a human mapper and you would destroy any
> >chance of actually determining the original intent of the mapper with
> >such edits.  
> Can you give me an example ?

Example:
Think of an MP where the way has – additionally to the tags you want to
fix – intermittent=yes and the relation has not.

Would your script really enhance the situation?
Does your script even take in consideration if the ways are inner or
outer?

> This algorithm won't loose any information. It's merely deleting 
> duplicates, so where is the problem ?

Where is the problem /not/ to fix this by script, then?

Recently I was told by somebody: Deleting data is not a real deletion
in OSM. It is just hiding that data. 

Subjective that may be true like your statement above – but it makes
editing harder without adding any value.

You already caused a lot of additional work (review of your edits and
reverts) and time used for discussions.
Why not stop replying mails here reflex-like and start to think about
the problem? Again? With the new information you were given? And
without trying to squeeze all the information in your scheme of thinking
(or "problem solving").

Thomas



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread Pierre Béland 
On 2016-03-22 14:10, Frank Villaro-Dixon wrote:


># First goal:
>First goal is quite simple. The idea is to work only on relations
>which have a natural=water .  Then, it will:
>    * Delete natural=water from all the ways if they are NOT closed or 
>    ring 0.
>

This is a good proposition to look at unclosed polygons and see if a potential 
incorrect keys to fix. 

I agree with others that using a Bot is not a safe way to handle these 
problems. Could a script to extract such unclosed polygons be proposed?  This 
way, each local community could take care to fix data for their area, importing 
and examining the data in the JOSM Editor. The Todo plugin could be used to go 
through the items to correct. 
Pierre 

  ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread Christoph Hormann 
On Tuesday 22 March 2016, Frank Villaro-Dixon wrote:
>
> Can you give me an example ?

I probably could (after all i am on record for saying waterbody mapping 
in OSM is a practical case of the infinite monkey theorem) but right 
now i don't have the time to look for a good real world example.

In principle many cases where the way you'd remove a tag from has 
additional tags the meaning of those tags will change when you remove a 
tag.  Also cases where the way in question forms the division between 
two water areas you will usually run into trouble.

On a principal level removing an ambiguity will always mean you remove 
possible interpretations of the data and your bot simply cannot know if 
the interpretations it removes are actually incorrect and the remaining 
interpretation is the correct one.

> This algorithm won't loose any information. It's merely deleting
> duplicates, so where is the problem ?

You are only looking at it from a formal side and based on a specific, 
subjective interpretation of tagging rules which is not appropriate.  
See also 

http://wiki.openstreetmap.org/wiki/Automated_Edits_code_of_conduct

As others have said the best way to improve waterbody data in OSM is by 
actually doing mapping work and correcting errors based on local 
knowledge or imagery.  You can believe me when i say that factual 
errors and inaccuracies are much more common and harmful problems 
w.r.t. water area mapping than basic tagging inconsistencies.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread Andy Townsend 

On 22/03/2016 13:10, Frank Villaro-Dixon wrote:

Hi everybody,


So, what do you think ?


I think it's a silly idea.

Identifying complex potentially problem areas is one thing - as you've 
found, attempting to fix them automatically is quite another.  In among 
the "obvious" fixes will be many harder to find new problems that you 
have introduced.





Technically, it was already run on the whole planet, and so far no 
bugs were found. 


That's not true.  Many people complained and all your work was reverted.*



Now, I need your comments and/or your approval, critiques, etc.
Tell me what you think ;-)



Here's what I think you should do, when you detect a potential problem:

1) Firstly, before fixing anything, try and understand what the cause 
was.  Perhaps an inexperienced mapper has edited some existing data that 
broke something that they didn't understand?  You'll need to look at the 
mappers who have contributed to the problem, their relative experience, 
and what editors they are using (for example, an iD user may been not 
have seen the complicated reationship between multipoloygons, and a JOSM 
user may have stopped thinking about real-world data and thought _only_ 
in terms of multipolygons - both can cause errors).


2) If you can, go and actually survey the area.  No, really, do actually 
go there.  That way you'll get a full 3d picture in your head of what's 
there and how it relates to the aerial imagery.  It also enables you to 
recognise features from imagery better, so you can see what sort of 
surface a path is, and (with water features) tell man-made ones from 
natural rivers and streams (difficult from imagery, especially when made 
by man 200 years ago).  Maybe the area is inaccessible to everyone, in 
which case anyone would have to work from imagery and other out of 
copyright sources, but if it is accessible to local mappers then they 
are the best people to fix any problem because they will be able to do a 
proper survey.


3) You'll now have a picture of (a) what the original mapper had in mind 
when they mapped it, (b) what subsequent mappers were trying to do and 
(c) what you'd have mapped it as, if you had mapped it from scratch.


If these three all agree, and it was just a tagging error (for example 
I've seen people add "natural=foo" instead of "name=foo" recently) then 
it makes sense to "just correct the data".  However, it's quite likely 
that these three might disagree, and perhaps you need to explain to an 
earlier mapper how multipolygons work, or to someone who has come along 
and "corrected" data in the interim that what they've changed something 
to is a valid OSM tag, but doesn't actually match what's on the ground 
in this case.


The best way to try and communicate with a specific previous mapper is 
via a changeset discussion comment.  The advantages of doing it this way 
are that the discussion is public, and the context is obvious, as it's 
visible with the changeset.  Other local mappers can also add comments 
there too - perhaps someone else locally has more knowledge about a 
particular water body.  If that doesn't work, or you need to contact all 
local mappers you can try adding an OSM note explaining the issue.  This 
might not get picked up immediately but notes sometimes do get fixed 
many months after they were added. Another option is to try and contact 
local users via a country's mailing list, forum or IRC channel.  They 
may know someone who is local, or know someone who knows someone (not 
necessarily an OSMer) who may be able to answer questions.



Based on the above, I don't believe that it is possible to do the sort 
of "water fixup" that you were trying to do entirely automatically. 
http://wiki.openstreetmap.org/wiki/Automated_Edits_code_of_conduct has 
sections that require you to "document and discuss your plans" for a 
reason.  To take a specific example, I noticed local edits made by you 
in http://www.openstreetmap.org/changeset/37086092 that simply failed to 
understand what the original mapper was trying to represent.  I pointed 
this out on the changeset discussion, and was frankly amazed when you 
created a bot account to make more of exactly the same sort of error, 
again.  Maybe I'm the frog in 
https://en.wikipedia.org/wiki/The_Scorpion_and_the_Frog :)


There may be many scenarios that you haven't considered when designed 
what automatic changes you are trying to make.  Other mappers will be 
able to help you understand those when you discuss your plans with them.


Also, please don't think that "changing a tag to one that is valid 
within OSM" means "making the data correct" - it doesn't.  All it means 
is that it is no longer possible to automatically find potentially 
problematical areas needing survey, or find mappers who may need help to 
map better.  In an analogy, if someone has described a "horse" as a 
"kow" correcting the spelling to "cow" does not make the description 
correct.


Finally, please remember -

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread Frank Villaro-Dixon 

On 16-03-22 15:23:44, Maarten Deen, wrote 1.7K characters saying:

On 2016-03-22 14:10, Frank Villaro-Dixon wrote:


# First goal:
First goal is quite simple. The idea is to work only on relations
which have a natural=water .  Then, it will:
	* Delete natural=water from all the ways if they are NOT closed or 
	ring 0.


I have been fixing waterways and water areas in the New Orleans area. 
I think most problems are due to a botched import. There are a number 
of unclosed areas, some water, some wetland. They just need to be 
closed which sometimes is just connecting the last two dots, sometimes 
it is merging two ways.


Again: this bot will be applied ONLY to ways belonging to a relation ! So 
the water areas are already valid. What is the problem then ?


It would be very unhelpful if the natural=water tags of these ways 
would be deleted. Then you have to go search in the history to see 
what it was. Is it natural=water or natural=wetland? It would increase 
the workload in fixing this.


Please, do not do this. You can flag them as defective so that someone 
has a look, but don't delete the tags.

The idea is not to touch defective water surfaces.



Since deleting tags from unclosed ways does not solve anything (and 
IMHO makes things worse), this should be done manually. It would be a 
perfect case to enter in maproulette 


Can you give me an example too ? It's WRONG to have a tag in a way AND in 
a relation.


For example, take this relation:
https://www.openstreetmap.org/relation/5474652
It has:
natural=water
water=lake

and a way:
https://www.openstreetmap.org/way/368404998
which also has:
natural=water
water=lake

and it is WRONG. Only the relation should have these attributes. Where is 
the problem then ?


Thanks,

Frank

--
frank.villaro-dixon.eu   - PGP: 6F36914A
Envie d'électricité 100% verte ? Enercoop.fr
What is a Velomobile ?   www.sans-essence.eu

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread Maarten Deen 

On 2016-03-22 14:10, Frank Villaro-Dixon wrote:


# First goal:
First goal is quite simple. The idea is to work only on relations
which have a natural=water .  Then, it will:
	* Delete natural=water from all the ways if they are NOT closed or 
	ring 0.


I have been fixing waterways and water areas in the New Orleans area. I 
think most problems are due to a botched import. There are a number of 
unclosed areas, some water, some wetland. They just need to be closed 
which sometimes is just connecting the last two dots, sometimes it is 
merging two ways.


It would be very unhelpful if the natural=water tags of these ways would 
be deleted. Then you have to go search in the history to see what it 
was. Is it natural=water or natural=wetland? It would increase the 
workload in fixing this.


Please, do not do this. You can flag them as defective so that someone 
has a look, but don't delete the tags.


Since deleting tags from unclosed ways does not solve anything (and IMHO 
makes things worse), this should be done manually. It would be a perfect 
case to enter in maproulette 


Regards,
Maarten


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread Frank Villaro-Dixon 

On 16-03-22 14:53:42, Christoph Hormann, wrote 1.9K characters saying:

On Tuesday 22 March 2016, Frank Villaro-Dixon wrote:


# Why ?
Well, OSM has a quite exhaustive lakes/water surfaces database, but
it's a complete pain to work on because:
* Some non closed ways have a natural=water or a water=* tag,
which makes no sense and is forbidden.
* Attributes (natural, water, name, intermittent) are in the
relation and in the way itself, which is anti normal form (and not
logical).

# First goal:
First goal is quite simple. The idea is to work only on relations
which have a natural=water .  Then, it will:
* Delete natural=water from all the ways if they are NOT closed or
ring 0.
* delete the corresponding water=x IF the relation has the same
tag



Mappers frequently map things in ambiguous ways or change existing
mapping in ways that make it ambiguous and it is hard to decide from
the data alone how to interpret such mapping.  A bot will not be any
better in doing that than a human mapper and you would destroy any
chance of actually determining the original intent of the mapper with
such edits.

Can you give me an example ?

This algorithm won't loose any information. It's merely deleting 
duplicates, so where is the problem ?





--
frank.villaro-dixon.eu   - PGP: 6F36914A
Envie d'électricité 100% verte ? Enercoop.fr
What is a Velomobile ?   www.sans-essence.eu

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread Frank Villaro-Dixon 

On 16-03-22 10:23:51, Nicolás Alvarez, wrote 1.2K characters saying:



El 22 mar 2016, a las 10:10, Frank Villaro-Dixon  
escribió:

# First goal:
First goal is quite simple. The idea is to work only on relations which have a 
natural=water .  Then, it will:
   * Delete natural=water from all the ways if they are NOT closed orring 0.


What if there is a way legitimately with natural=water that isn't closed 
because of an error? The correct fix is to close it, not to remove the tag. 
This cannot be done automatically.

It doesn't matter because this way will also be referenced by a relation 
which has the natural=water tag. Thus, it can be removed from the way.


(didn't reply to the list)

--
frank.villaro-dixon.eu   - PGP: 6F36914A
Envie d'électricité 100% verte ? Enercoop.fr
What is a Velomobile ?   www.sans-essence.eu

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread Christoph Hormann 
On Tuesday 22 March 2016, Frank Villaro-Dixon wrote:
>
> # Why ?
> Well, OSM has a quite exhaustive lakes/water surfaces database, but
> it's a complete pain to work on because:
>   * Some non closed ways have a natural=water or a water=* tag,
>   which makes no sense and is forbidden.
>   * Attributes (natural, water, name, intermittent) are in the
>   relation and in the way itself, which is anti normal form (and not
> logical).
>
> # First goal:
> First goal is quite simple. The idea is to work only on relations
> which have a natural=water .  Then, it will:
>   * Delete natural=water from all the ways if they are NOT closed or
>   ring 0.
>   * delete the corresponding water=x IF the relation has the same
>   tag

This is not going to work ('work' in the sense of turning incorrect 
information into correct information with some reasonable degree of 
reliability).

Mappers frequently map things in ambiguous ways or change existing 
mapping in ways that make it ambiguous and it is hard to decide from 
the data alone how to interpret such mapping.  A bot will not be any 
better in doing that than a human mapper and you would destroy any 
chance of actually determining the original intent of the mapper with 
such edits.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] [BOT] [RFC]: water surfaces

2016-03-22 Thread Nicolás Alvarez 

> El 22 mar 2016, a las 10:10, Frank Villaro-Dixon  
> escribió:
> 
> Hi everybody,
> 
> I launched a bot (FrankVD_bot) this week which didn't make everyone happy, as 
> it wasn't discussed with the community, which is quite normal. Here's then 
> the RFC for (let's call it) scorpion.
> 
> 
> # Targets
> The targeted zones are actually the multipolygons with natural=water on them. 
> Working box is all the planet.
> 
> # Why ?
> Well, OSM has a quite exhaustive lakes/water surfaces database, but it's a 
> complete pain to work on because:
>* Some non closed ways have a natural=water or a water=* tag,which 
> makes no sense and is forbidden.
>* Attributes (natural, water, name, intermittent) are in therelation 
> and in the way itself, which is anti normal form (and not logical).
> 
> # First goal:
> First goal is quite simple. The idea is to work only on relations which have 
> a natural=water .  Then, it will:
>* Delete natural=water from all the ways if they are NOT closed orring 
> 0.

What if there is a way legitimately with natural=water that isn't closed 
because of an error? The correct fix is to close it, not to remove the tag. 
This cannot be done automatically.


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk