Re: [OSM-talk] A thought on bot edits

[Safwat Halaby]

Thank you. 

On Sat, 2017-10-07 at 10:55 +0200, Christoph Hormann wrote:
> I would be careful interpreting the lack of objections to your
> automated 
> edits in the local community as universal approval though.  There
> are 
> likely also locals who do not think this is a good idea but due to
> the 
> low intensity and low volume of edits they don't see it necessary to 
> say anything.

That's a good point. I'll keep it in mind. 

It's worth noting I always look at a small sample of changes before
hitting the final "upload" button. The bot does not upload fully
autonomously. I'm still uncomfortable with a fully automatic bot. This
human supervision already proved helpful on several occasions. In one
recent case, the bot would have conflicted with Yuri's recent Wiki
edits, and the manual supervision caught this and we then coordinated
properly: 
https://www.openstreetmap.org/changeset/51350214

If my bot were fully automatic, it would have annoyed Yuri and
interfered with his project without discussion. Worst still, If the two
edits were being made by fully automatic bots, a perpetual bot edit war
would have possibly ensued. (Depending on the way the bots were coded)

As for local particularities, I completely agree. For instance, Israel
has a unique case where the "name" tag is always duplicated at
"name:he" or "name:ar". A theoretical global bot which removes
name/name:lang whenever the values are duplicated would cause local
damage and wouldn't be aware of the local conventions, even though such
a change may be welcome elsewhere.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Christoph Hormann]

Thanks,

i think this is a constructive approach to automated edits and if 
everyone worked this way i don't think we would have a problem.

In particular:

* your bot is well documented
* you discussed it with the local community
* it has a good supervision to editing volume ratio
* it runs only on a small area you are familiar with

The forum discussion by the way also in this case shows how important it 
is to look in depth at the area you work on and its particularities.

I would be careful interpreting the lack of objections to your automated 
edits in the local community as universal approval though.  There are 
likely also locals who do not think this is a good idea but due to the 
low intensity and low volume of edits they don't see it necessary to 
say anything.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Safwat Halaby]

Drudgery is evil, well written bots save us from drudgery, and allow us
to use human time more productively, therefore well written bots are
good.

Why should a human clean up whitespace, or add the "cuisine" tag to a
hundred "Burger King" branches? Shouldn't our creative brains invest
their time elsewhere?

I don't think a generic sweeping rule is a good idea. Each bot should
be analyzed individually. Badly written bots, the ones that add work
rather than save work, should be stopped. Namespace or database
separation mean the bots cannot as effectively save us from drudgery.
The point of good bots is to save the work that needs to be done on the
actual OSM data.

However, I do think the "burden of proof" lies on the bot owner; It is
the owner's job to explain why the bot is needed, why it is "good", and
to document it extensively and make its operations very transparent. I
try *very hard* to do that with my scripts. Every single changeset and
algorithm is logged at my talk page, all changesets have the bot=yes
tag, a dedicated bot user is used, etc.

Again, I think every case is to be handled individually, with BOP on
the bot owner. bad/undocumented/undiscussed/non-transparent bots are
the problem.

https://wiki.openstreetmap.org/wiki/User:SafwatHalaby#SafwatHalaby_bot

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Yuri Astrakhan]

Speaking from my Wikipedia bot experience (I wrote bots and created
Wikipedia API over 10 years ago to help bots):

Bots were successful in Wikipedia because all users felt empowered. Users
could very easily see what the bot edited, fix or undo bot edits, and
easily communicate with the bot authors.  OSM does not have as good of
tools to compare and undo. Hence, some users in OSM may feel powerless -
they feel like they cannot influence this process, e.g. easily undo a
mistake, or know how bad the mistake really is - does it affect just a few
or thousands of places? As OSM gets more contributors, and moves more
towards maintenance, we should address these two:

* There is no easy way to view changes side-by-side at osm.org. We need to
be able to view both the object history and the entire changeset history,
and compare any two revisions. The diff view should show geometry changes
together with tag changes. JOSM has a good diff viewer, but it is per
object, and requires the use of the app.
* There is no easy way to undo a specific edit. In Wikipedia, undoing is a
simple two click process - "undo this change" in the history view, "save".
In OSM, one has to use a JOSM plugin!

Note that some of these capabilities may exist as separate tools, but most
users may not be even aware of them. They need to be part of the OSM.org.

A few more comments:

* Don't confuse maintenance bots with batch imports. Maintenance bots
cleans up obvious mistakes and simplify things that are too tedious for
humans.  Batch import add large amount of sometimes unverified data. M-bot
cleans up wikipedia page redirects. Import bots create "botopedias" like
ceb-wiki.

* Assume the good faith - bot authors care about the project as much as
everyone else, and want to make the project better as much as everyone
else. Lets find solutions that benefit everyone.

* Bots are tools, just like JOSM. They can be used for good and cause
problems. Banning JOSM just because someone could use it badly doesn't make
sense. Instead we should encourage bot operators to contribute, but make
sure they are benefit rather than nuisance.

On Fri, Oct 6, 2017 at 4:40 AM Jo  wrote:

> True indeed. What this means, is that there can be a 'mismatch' between
> the Wikipedia tag and the Wikidata tag, if the Wikidata tag is more
> specific than what Wikipedia wants to create pages for.
>
> It's normal that this happens, as both projects have a different notion of
> notability. Aldi Nord and Aldi Süd will definitely not be the only cases of
> this. In fact I would expect this to happen very often.
>
> At least to me it happens quite a lot that I want to create an article on
> Wikpedia, but the powers that be don't consider the subject notable.
>
> Often this is a person with a street named after him or her. Or a bus
> line. But it could be a single statue in a park, or a part of a collection
> in a museum. So there will be many things we map that will have Wikidata
> items, but not Wikipedia articles. And some where our information is more
> specific that what WP has. Wikidata is actually an opendata project that
> stands closer to OSM than WP, or it certainly can be.
>
> Polyglot
>
> 2017-10-06 10:18 GMT+02:00 Martin Koppenhoefer :
>
>> 2017-10-06 10:10 GMT+02:00 Jo :
>>
>>> What I don't understand is the problems people seem to have with
>>> wikidata. If an existing wikidata entry doesn't align with what we mapped,
>>> then create a new wikidata entry that does and link it to the existing
>>> entries.
>>>
>>
>>
>> it's actually not that easy. I tried to do this and gave up (in the
>> infamous ALDI case). Andy Mabbett had created 1 new "sub-entity" for each
>> of the 2 enterprises which together are described in the wikipedia article,
>> but you cannot add the wikipedia article to the new wikidata object without
>> removing it from the other wikidata object (for both). As the wikidata
>> object that covers both enterprises is the best fit for the WP article, I
>> decided to keep the Wikipedia article linked to this, but then it didn't
>> make sense to use the more precise wikidata object as reference in OSM as
>> it hadn't any wikipedia article linked to it.
>>
>> Cheers,
>> Martin
>>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Jo]

True indeed. What this means, is that there can be a 'mismatch' between the
Wikipedia tag and the Wikidata tag, if the Wikidata tag is more specific
than what Wikipedia wants to create pages for.

It's normal that this happens, as both projects have a different notion of
notability. Aldi Nord and Aldi Süd will definitely not be the only cases of
this. In fact I would expect this to happen very often.

At least to me it happens quite a lot that I want to create an article on
Wikpedia, but the powers that be don't consider the subject notable.

Often this is a person with a street named after him or her. Or a bus line.
But it could be a single statue in a park, or a part of a collection in a
museum. So there will be many things we map that will have Wikidata items,
but not Wikipedia articles. And some where our information is more specific
that what WP has. Wikidata is actually an opendata project that stands
closer to OSM than WP, or it certainly can be.

Polyglot

2017-10-06 10:18 GMT+02:00 Martin Koppenhoefer :

> 2017-10-06 10:10 GMT+02:00 Jo :
>
>> What I don't understand is the problems people seem to have with
>> wikidata. If an existing wikidata entry doesn't align with what we mapped,
>> then create a new wikidata entry that does and link it to the existing
>> entries.
>>
>
>
> it's actually not that easy. I tried to do this and gave up (in the
> infamous ALDI case). Andy Mabbett had created 1 new "sub-entity" for each
> of the 2 enterprises which together are described in the wikipedia article,
> but you cannot add the wikipedia article to the new wikidata object without
> removing it from the other wikidata object (for both). As the wikidata
> object that covers both enterprises is the best fit for the WP article, I
> decided to keep the Wikipedia article linked to this, but then it didn't
> make sense to use the more precise wikidata object as reference in OSM as
> it hadn't any wikipedia article linked to it.
>
> Cheers,
> Martin
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Martin Koppenhoefer]

2017-10-06 10:10 GMT+02:00 Jo :

> What I don't understand is the problems people seem to have with wikidata.
> If an existing wikidata entry doesn't align with what we mapped, then
> create a new wikidata entry that does and link it to the existing entries.
>


it's actually not that easy. I tried to do this and gave up (in the
infamous ALDI case). Andy Mabbett had created 1 new "sub-entity" for each
of the 2 enterprises which together are described in the wikipedia article,
but you cannot add the wikipedia article to the new wikidata object without
removing it from the other wikidata object (for both). As the wikidata
object that covers both enterprises is the best fit for the WP article, I
decided to keep the Wikipedia article linked to this, but then it didn't
make sense to use the more precise wikidata object as reference in OSM as
it hadn't any wikipedia article linked to it.

Cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Jo]

Or a bot=https://fancyurl.iou/lawyeredcontract.json

to clearly define what the bot can and cannot do?

Personally I think we need all the help we can get from automation, but it
needs to remain 'overseen' by an actual mapper.

That's why I like the todo list plugin in JOSM a lot. And why I try to help
with developing tools to find errors and do trivial fixes. Especially
relations are relatively brittle in the OSM world.

So I understand the resistance against Yuri's automated handling of
wikidata tags. What he should do, is make his suggestions for improvement
available through our validation tools and then have mappers process them.

What I don't understand is the problems people seem to have with wikidata.
If an existing wikidata entry doesn't align with what we mapped, then
create a new wikidata entry that does and link it to the existing entries.

You could argue that's not strictly mapping anymore, but it does enhance
open data as a whole. So I think it is worthwhile to do it.

If it were possible to link from Wikidata to OSM, I'm sure it would be done
that way, but since there are no stable ids on our side, tags are the only
way to do it.

In JOSM it's possible to see which labels are behind the numbers. It should
be trivial to do so in Id as well. And why not on the standard rendering
too?

Polyglot

2017-10-06 9:45 GMT+02:00 Martin Koppenhoefer :

>
>
> sent from a phone
>
> > On 6. Oct 2017, at 06:02, Yves  wrote:
> >
> > @JB, I understood the bot=no tag like the add=no sticker on your
> physical mailbox
>
>
> yes, just like every active mapper having  tens of thousands of mailboxes
> to add stickers to. What about an opt in? Add a bot=yes if you want your
> edits modified by bots...
>
> cheers,
> Martin
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Martin Koppenhoefer]

sent from a phone

> On 6. Oct 2017, at 06:02, Yves  wrote:
> 
> @JB, I understood the bot=no tag like the add=no sticker on your physical 
> mailbox


yes, just like every active mapper having  tens of thousands of mailboxes to 
add stickers to. What about an opt in? Add a bot=yes if you want your edits 
modified by bots...

cheers,
Martin 
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Warin]

On 06-Oct-17 02:37 PM, JB wrote:


Le 05/10/2017 à 22:50, Yuri Astrakhan a écrit :
I like the "bot=no" flag, or a more specific one for a given field -  
"name:en:bot=no" - as long as those flags are not added by a bot :)

Ho…
We are now manually contributing one more tag to say it was 
contributed manually…
So many people seem to think one geodatabase can be created only 
through bots, imports, etc… why not go create it?

JB.


First import? OSM :-D



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Yves]

@JB, I understood the bot=no tag like the add=no sticker on your physical 
mailbox. 

Yves 

Le 6 octobre 2017 05:37:37 GMT+02:00, JB  a écrit :
>
>Le 05/10/2017 à 22:50, Yuri Astrakhan a écrit :
>> I like the "bot=no" flag, or a more specific one for a given field - 
>
>> "name:en:bot=no" - as long as those flags are not added by a bot :)
>Ho…
>We are now manually contributing one more tag to say it was contributed
>
>manually…
>So many people seem to think one geodatabase can be created only
>through 
>bots, imports, etc… why not go create it?
>JB.
>
>___
>talk mailing list
>talk@openstreetmap.org
>https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[JB]

Le 05/10/2017 à 22:50, Yuri Astrakhan a écrit :
I like the "bot=no" flag, or a more specific one for a given field -  
"name:en:bot=no" - as long as those flags are not added by a bot :)

Ho…
We are now manually contributing one more tag to say it was contributed 
manually…
So many people seem to think one geodatabase can be created only through 
bots, imports, etc… why not go create it?

JB.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Yuri Astrakhan]

I like the "bot=no" flag, or a more specific one for a given field -
"name:en:bot=no" - as long as those flags are not added by a bot :)

Would it make sense, judging how wikidata* tags have been mostly auto-added
by iD, as well as user's bot efforts, including my own, to treat wikidata
explicitly as a bot tag?  In a way, it is already being treated as such by
many - why not make it official?

On Tue, Oct 3, 2017 at 4:55 AM Christoph Hormann  wrote:

> On Tuesday 03 October 2017, Frederik Ramm wrote:
> > Did your proposal also extend to geoemtries? You said something about
> > bot:* tags, but if a bot were to orthogonalize an existing building,
> > would it then have to create a copy of that tagged
> > "bot:building=yes"? And how could that be differentiated from a
> > building that originally had building=YES and the bot only lowercased
> > the tag value?
>
> My original idea was only about tags but it could be extended to
> geometries of course - as i sketched in my reply to Martin, which would
> essentially mean creating a copy for the building a bot orthogonalizes
> if the building already has a manual building=yes tag.  If the bot only
> changes the tag the building would remain a normal hand mapped geometry
> but would get a bot:building=yes in addition to the building=YES.
>
> Of course duplicating geometry data would make it much more difficult
> for data users to make decisions about selectively using data and it
> would make it much more difficult for editors to allow mappers to edit
> the data correctly.  This is why i originally suggested this only for
> tags - after all the vast majority of bot edits are tag modifications
> only, geometry edits by bots are technically much more complicated to
> do right so they happen less frequently.
>
> As already said - if this approach is not considered favorably it is
> always possible to use the other method and forbid bots to touch
> anything with a bot=no tag and thereby allow mappers to opt out of bot
> edits on a case-by-case basis.
>
> --
> Christoph Hormann
> http://www.imagico.de/
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Christoph Hormann]

On Tuesday 03 October 2017, Frederik Ramm wrote:
> Did your proposal also extend to geoemtries? You said something about
> bot:* tags, but if a bot were to orthogonalize an existing building,
> would it then have to create a copy of that tagged
> "bot:building=yes"? And how could that be differentiated from a
> building that originally had building=YES and the bot only lowercased
> the tag value?

My original idea was only about tags but it could be extended to 
geometries of course - as i sketched in my reply to Martin, which would 
essentially mean creating a copy for the building a bot orthogonalizes 
if the building already has a manual building=yes tag.  If the bot only 
changes the tag the building would remain a normal hand mapped geometry 
but would get a bot:building=yes in addition to the building=YES.

Of course duplicating geometry data would make it much more difficult 
for data users to make decisions about selectively using data and it 
would make it much more difficult for editors to allow mappers to edit 
the data correctly.  This is why i originally suggested this only for 
tags - after all the vast majority of bot edits are tag modifications 
only, geometry edits by bots are technically much more complicated to 
do right so they happen less frequently.

As already said - if this approach is not considered favorably it is 
always possible to use the other method and forbid bots to touch 
anything with a bot=no tag and thereby allow mappers to opt out of bot 
edits on a case-by-case basis.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Martin Koppenhoefer]

2017-10-03 2:25 GMT+02:00 Frederik Ramm :

>
> Did your proposal also extend to geoemtries? You said something about
> bot:* tags, but if a bot were to orthogonalize an existing building,
> would it then have to create a copy of that tagged "bot:building=yes"?
>


is automatically orthogonalizing geometry something that people actually do
right now, maybe even on a global level? I'd consider it harmful, and would
propose to revert these edits (as there hasn't been any
announcement/discussion about it like requested by the guidelines). Or was
this just a hypothetical case? Similarly, applying Douglas-Peucker will
always reduce detail (that's why it is used in the end), if there are
curves the detail, even if subtile, is generally wanted (the straighter the
curves are, the bigger the loss).

Cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Frederik Ramm]

Hi,

On 02.10.2017 18:50, Christoph Hormann wrote:
> Of course i am certainly not representative for the typical mappers.  I 
> would suspect there are probably mappers that would be attracted and 
> motivated by an OSM project where bots routinely 'fix' data 
> inconsistencies like typos in tags, different spellings of common names 
> or automatically orthogonalize building geometries.  But there are 
> others who don't like this.  One motivation behind my suggestion was 
> that this would allow mappers to embrace bot edits but also allows them 
> to reject this and decide they only want to interact with other craft 
> mappers and not with bots.

Did your proposal also extend to geoemtries? You said something about
bot:* tags, but if a bot were to orthogonalize an existing building,
would it then have to create a copy of that tagged "bot:building=yes"?
And how could that be differentiated from a building that originally had
building=YES and the bot only lowercased the tag value?

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Simon Poole]

I would suggest simply adapting my old suggestion (for imports) that as long as 
you fix the same number of elements from a broken import you can bot 
edit/import to your hearts desire.

Totally serious :-)

Simon

On 2. Oktober 2017 16:58:02 MESZ, Christoph Hormann  wrote:
>On Monday 02 October 2017, Martin Koppenhoefer wrote:
>>
>> yes, keeping a lot of additional tags for a huge amount of objects in
>> the main db would still be a burden on everyone working with the
>> planet file or geographic extracts, so it seems logical to
>> externalize the bot-tags. But how would you link one db to the other?
>> If people don't see those tags (or only by request), their edits will
>> erode the information in this external db (e.g. by splitting ways,
>> deleting and redrawing parts, combining ways, etc.). What about
>> versions, will there be different versions of the same object in the
>> main db and this bot db? Is this a serious suggestion or just another
>> way of saying there are too many automated activities going on?
>
>It is a serious idea although i don't seriously expect this to be 
>implemented any time soon.  Less for technical reasons as you mentioned
>
>but for social reasons.  A huge part of the interest in making bot 
>edits stems from the idea to have the OSM community as cheap labour to 
>clean up after the bots and if you remove that incentive a lot of 
>motivation for making bot edits vanishes.
>
>Linking a separate bot editing database to the main OSM database is not
>
>that difficult in principle as long as we are only talking about tag 
>modifications on the bot side.  You would simply have a separate and 
>separately versioned 'bot tags' object for every object that has bot 
>tags.  Of course if bots should also be able to make geometry edits you
>
>would need rules for that - like bots may only edit geometries that 
>have no tag starting with something other than 'bot:' and that are not 
>member of a way or relation with tags other than 'bot:*'.  This would 
>then essentially mean any geometry edits by bots stay within the bot 
>database which would make things easier (you would have a 'bot tags' 
>table plus supplemental bot only geometries tables).
>
>That is of course all theoretical.  The more likely scenarios what will
>
>happen if bot editing activities spread even further are probably
>
>a) That more and more craft mappers get fed up with bots messing with 
>their work and manual editing activity declines overall -> OSM transits
>
>into a primarily bot maintained database.
>b) The craft mappers get fed up with the bots and decide to separate
>out 
>their work instead of that of the bots in form of some protection 
>(could be as simple as adding a 'bot=no' tag to features allowing 
>mappers to indicate 'bots may not touch this object i have just 
>mapped').
>
>-- 
>Christoph Hormann
>http://www.imagico.de/
>
>___
>talk mailing list
>talk@openstreetmap.org
>https://lists.openstreetmap.org/listinfo/talk

-- 
Diese Nachricht wurde von meinem Android-Mobiltelefon mit Kaiten Mail gesendet.___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Yves]

Actually, if you find the way to keep a db handling a property (or tag) of OSM 
element in sync with OSM, you have solved the need for UID. And if you happen 
to do so without UID or API change , it's very nice ! 


Le 2 octobre 2017 15:59:48 GMT+02:00, Christoph Hormann  a 
écrit :
>
>With all the recent endeavors to push more automated edits in OSM and 
>with the related rules and policies clearly failing (just look at 
>https://wiki.openstreetmap.org/wiki/Category:Automated_edits_log and 
>compare that to what is actually taking place in terms of automated 
>edits these days) i just had the idea that it might be a lot easier and
>
>better if we replace all current regulations of automated edits with a 
>simple rule:
>
>Automated edits of any kind may freely add or edit tags with keys 
>starting with 'bot:' but may not under any circumstances touch any 
>other tags.
>
>This way people could go crazy bot editing whatever they want in that 
>namespace but would not interfere with manual mapping activity and data
>
>consumers could choose freely if the want bot edited information and if
>
>they do if they want to give it priority over manually verified data.  
>And mappers could configure their editors to hide the bot tags if they 
>are not interested in them.
>
>Of course considering the big volume of editing activity that would 
>likely take place in the 'bot:' namespace in that scenario it might be 
>a good idea to put those tags into a separate database for efficiency 
>reasons.
>
>-- 
>Christoph Hormann
>http://www.imagico.de/
>
>___
>talk mailing list
>talk@openstreetmap.org
>https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Pierre Béland]

Christoph Hormann wrote:
> a) I am unmotivated to map in areas where imports are in progress or 
> regularly taking place (yes, i am talking about Canada).

We often see such reactions (what is good, what is bad) without any analysis of 
the situation.
Do you know Canada, have you tried to measure the effort to map the millions of 
lakes, the efforts to spot nordic villages, roads, industrie, tourism 
activities spread over a huge territory and where this is not the priority to 
provide new high resolution maps?  

This is quite demotivating for us working hard to map north of Canada to 
continously see such negative messages about our work ;)
 regard

Pierre 


  De : Christoph Hormann <o...@imagico.de>
 À : talk@openstreetmap.org 
 Envoyé le : lundi 2 octobre 2017 12h53
 Objet : Re: [OSM-talk] A thought on bot edits
   
On Monday 02 October 2017, Martijn van Exel wrote:
> I find this discussion and your proposal interesting to explore, at
> least as a hypothetical. Do we know 1) what the volume of bot edits
> is and how it has grown 

No, but i thought as well this would be an interesting thing to study.  
Of course you would need to make some definition of what a bot edit is 
that can be automatically analyzed - which is difficult.  But even a 
hairy definition might allow to identify rough trends.

There is little doubt that the volume of bot edits has grown recently 
but if it has actually grown much faster than the manual editing volume 
overall is not easy to determine.  I mostly look at remote areas and 
there the raise in dominance of automated editing activities is massive 
but the manual editing activity in these areas has always been small 
and sporadic so this is certainly not an observation you can 
extrapolate to the whole.

> 2) how many mappers have actually given up 
> based upon this? 

Again i can only answer this based on my own experience and

a) I am unmotivated to map in areas where imports are in progress or 
regularly taking place (yes, i am talking about Canada).
b) My primary motivation for mapping in OSM is that what i map gets 
improved by other craft mappers so what we produce together is better 
than what each of us can produce on our own.  If the only changes that 
are going to be made to my mapping work after i upload it to OSM are 
made by bots there would be no results from that that would be any 
better than what i could produce on my own because i could simply run 
the bots on my own privately mapped data.

Of course i am certainly not representative for the typical mappers.  I 
would suspect there are probably mappers that would be attracted and 
motivated by an OSM project where bots routinely 'fix' data 
inconsistencies like typos in tags, different spellings of common names 
or automatically orthogonalize building geometries.  But there are 
others who don't like this.  One motivation behind my suggestion was 
that this would allow mappers to embrace bot edits but also allows them 
to reject this and decide they only want to interact with other craft 
mappers and not with bots.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk


   ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Christoph Hormann]

On Monday 02 October 2017, Martijn van Exel wrote:
> I find this discussion and your proposal interesting to explore, at
> least as a hypothetical. Do we know 1) what the volume of bot edits
> is and how it has grown 

No, but i thought as well this would be an interesting thing to study.  
Of course you would need to make some definition of what a bot edit is 
that can be automatically analyzed - which is difficult.  But even a 
hairy definition might allow to identify rough trends.

There is little doubt that the volume of bot edits has grown recently 
but if it has actually grown much faster than the manual editing volume 
overall is not easy to determine.  I mostly look at remote areas and 
there the raise in dominance of automated editing activities is massive 
but the manual editing activity in these areas has always been small 
and sporadic so this is certainly not an observation you can 
extrapolate to the whole.

> 2) how many mappers have actually given up 
> based upon this? 

Again i can only answer this based on my own experience and

a) I am unmotivated to map in areas where imports are in progress or 
regularly taking place (yes, i am talking about Canada).
b) My primary motivation for mapping in OSM is that what i map gets 
improved by other craft mappers so what we produce together is better 
than what each of us can produce on our own.  If the only changes that 
are going to be made to my mapping work after i upload it to OSM are 
made by bots there would be no results from that that would be any 
better than what i could produce on my own because i could simply run 
the bots on my own privately mapped data.

Of course i am certainly not representative for the typical mappers.  I 
would suspect there are probably mappers that would be attracted and 
motivated by an OSM project where bots routinely 'fix' data 
inconsistencies like typos in tags, different spellings of common names 
or automatically orthogonalize building geometries.  But there are 
others who don't like this.  One motivation behind my suggestion was 
that this would allow mappers to embrace bot edits but also allows them 
to reject this and decide they only want to interact with other craft 
mappers and not with bots.

-- 
Christoph Hormann
http://www.imagico.de/

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Pierre Béland]

We often see critics about Bots and import accounts. Should we oppose crafters 
vs Bots? Who are crafters, who are Bots?  I suspect that they often can be the 
same ;) from the few thousand intensely active OSM contributors. And not all 
imports or Bots harm our database content. For an informed decision we simply 
need to know better about these Bots and Imports.  

To compile statististices about the OSM Contributors profiles, I am actually 
going through the http://planet.osm.org/replication/changesets/. Not easy to 
identify Bots and Imports from the Changesets metadata. Before 2012, there was 
no specific account for imports. And since 2012, you often have to read the 
contributors user profile from the OSM API to verify if this is an import 
account since not all use a prefix or suffix with import. 
For Bots, you can try to identify the user name that contains words such as 
Bot, mechanical, repair, fix, etc. But this is relatively imprecise.  You can 
also searh the Changesets metadata to see reference to Bot Edit sessions.
If somebody knows a better way to identify Import accounts and Bots, I am 
interested about that.
 
Pierre 


  De : Martijn van Exel <m...@rtijn.org>
 À : Christoph Hormann <o...@imagico.de> 
Cc : "talk@openstreetmap.org" <talk@openstreetmap.org>
 Envoyé le : lundi 2 octobre 2017 11h17
 Objet : Re: [OSM-talk] A thought on bot edits
   
I find this discussion and your proposal interesting to explore, at least as a 
hypothetical. Do we know 1) what the volume of bot edits is and how it has 
grown 2) how many mappers have actually given up based upon this? My guess is 
that instead of coming up with a global solution, this could be left to the 
local communities to decide. For example, where I live (USA) there does not 
seem to be as much resistance to automated edits to make such a change 
desirable / necessary. The effect of introducing a new tagging requirement for, 
or even entirely separating out automated edits into a different database, may 
have a different (or even an opposite) effect in communities that look more 
favorably upon these types of edits.
Martijn

   ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Martijn van Exel]

I find this discussion and your proposal interesting to explore, at least
as a hypothetical. Do we know 1) what the volume of bot edits is and how it
has grown 2) how many mappers have actually given up based upon this? My
guess is that instead of coming up with a global solution, this could be
left to the local communities to decide. For example, where I live (USA)
there does not seem to be as much resistance to automated edits to make
such a change desirable / necessary. The effect of introducing a new
tagging requirement for, or even entirely separating out automated edits
into a different database, may have a different (or even an opposite)
effect in communities that look more favorably upon these types of edits.
Martijn

On Mon, Oct 2, 2017 at 8:58 AM, Christoph Hormann  wrote:

> On Monday 02 October 2017, Martin Koppenhoefer wrote:
> >
> > yes, keeping a lot of additional tags for a huge amount of objects in
> > the main db would still be a burden on everyone working with the
> > planet file or geographic extracts, so it seems logical to
> > externalize the bot-tags. But how would you link one db to the other?
> > If people don't see those tags (or only by request), their edits will
> > erode the information in this external db (e.g. by splitting ways,
> > deleting and redrawing parts, combining ways, etc.). What about
> > versions, will there be different versions of the same object in the
> > main db and this bot db? Is this a serious suggestion or just another
> > way of saying there are too many automated activities going on?
>
> It is a serious idea although i don't seriously expect this to be
> implemented any time soon.  Less for technical reasons as you mentioned
> but for social reasons.  A huge part of the interest in making bot
> edits stems from the idea to have the OSM community as cheap labour to
> clean up after the bots and if you remove that incentive a lot of
> motivation for making bot edits vanishes.
>
> Linking a separate bot editing database to the main OSM database is not
> that difficult in principle as long as we are only talking about tag
> modifications on the bot side.  You would simply have a separate and
> separately versioned 'bot tags' object for every object that has bot
> tags.  Of course if bots should also be able to make geometry edits you
> would need rules for that - like bots may only edit geometries that
> have no tag starting with something other than 'bot:' and that are not
> member of a way or relation with tags other than 'bot:*'.  This would
> then essentially mean any geometry edits by bots stay within the bot
> database which would make things easier (you would have a 'bot tags'
> table plus supplemental bot only geometries tables).
>
> That is of course all theoretical.  The more likely scenarios what will
> happen if bot editing activities spread even further are probably
>
> a) That more and more craft mappers get fed up with bots messing with
> their work and manual editing activity declines overall -> OSM transits
> into a primarily bot maintained database.
> b) The craft mappers get fed up with the bots and decide to separate out
> their work instead of that of the bots in form of some protection
> (could be as simple as adding a 'bot=no' tag to features allowing
> mappers to indicate 'bots may not touch this object i have just
> mapped').
>
> --
> Christoph Hormann
> http://www.imagico.de/
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Lester Caine]

On 02/10/17 15:13, Martin Koppenhoefer wrote:
> Of course considering the big volume of editing activity that would
> likely take place in the 'bot:' namespace in that scenario it might be
> a good idea to put those tags into a separate database for efficiency
> reasons.
> 
> yes, keeping a lot of additional tags for a huge amount of objects in
> the main db would still be a burden on everyone working with the planet
> file or geographic extracts, so it seems logical to externalize the
> bot-tags. But how would you link one db to the other? If people don't
> see those tags (or only by request), their edits will erode the
> information in this external db (e.g. by splitting ways, deleting and
> redrawing parts, combining ways, etc.). What about versions, will there
> be different versions of the same object in the main db and this bot db?
> Is this a serious suggestion or just another way of saying there are too
> many automated activities going on?

There are many reasons for wanting unique id's IN OSM that can be used
to cross reference external databases. Add to your list archiving
historic versions of the objects, something that should be automated
into OHM. So there should be serious consideration of the idea but what
section of tags should be moved to a separate database?

There is a good case for using wikidata to provide a higher level of
hierarchy such as street names, and all of the place data that overlays
that, so OSM only needs to use the wikidata namespace for all of that
material. I don't think that the idea of 'bot' space actually fits into
that model as it is the unique ID that is fixed and 'bot' tags either
need to be accessible in 'mapping' space, or remain in the secondary
data space.

-- 
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] A thought on bot edits

[Martin Koppenhoefer]

2017-10-02 15:59 GMT+02:00 Christoph Hormann :

>
>
>
> Of course considering the big volume of editing activity that would
> likely take place in the 'bot:' namespace in that scenario it might be
> a good idea to put those tags into a separate database for efficiency
> reasons.
>


yes, keeping a lot of additional tags for a huge amount of objects in the
main db would still be a burden on everyone working with the planet file or
geographic extracts, so it seems logical to externalize the bot-tags. But
how would you link one db to the other? If people don't see those tags (or
only by request), their edits will erode the information in this external
db (e.g. by splitting ways, deleting and redrawing parts, combining ways,
etc.). What about versions, will there be different versions of the same
object in the main db and this bot db? Is this a serious suggestion or just
another way of saying there are too many automated activities going on?

Cheers,
Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk