Re: [OSM-talk] Code of conduct for automated (mass-) edits
"maning sambale" <[EMAIL PROTECTED]> writes: >> >> This is all based on positive thinking, assuming that people are >> basically well-meaning. > Agree, most mistakes in my area are not vandalism, but just mistakes. > I know I made many. Simply informing the mapper the problem is good > enough. And sometimes that mapper maybe the most prolific. > > This may not be a good analogy, but I often consult wikipedia > eventhough a lot there are lot of vandalism/crappy articles there. > > I believe a code of conduct is sufficient in the spirit and principles of OSM. ... and if it really is not there is still time to think about other means of "enforcing" compliance with the rules. Matthias ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
"Nic Roets" <[EMAIL PROTECTED]> writes: > To get back to the issue of a bot running in a specific country : Let's say > the vast majority of occurrences of Strasse are spelling mistakes and let's > say there are a few occurrences of Strasse where it is not a spelling > mistake, like a surname. Should we run the bot ? No, I don't think so. Everybody can make mistakes and just not know about ligitimate use of a 'misspelling'. But I don't think one should just accept a couple of false positives and say "It was only 0.1%". If you can not train your bot to omit all known false positives have it just fix the errors you are absolutely certain about and put the rest on a list on the web somwhere and ask the community to have a look at it. Matthias ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
> > This is all based on positive thinking, assuming that people are > basically well-meaning. Agree, most mistakes in my area are not vandalism, but just mistakes. I know I made many. Simply informing the mapper the problem is good enough. And sometimes that mapper maybe the most prolific. This may not be a good analogy, but I often consult wikipedia eventhough a lot there are lot of vandalism/crappy articles there. I believe a code of conduct is sufficient in the spirit and principles of OSM. > I'll leave it to others to deal with those who > are not ;-) > > Bye > Frederik > > -- > Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" > > ___ > talk mailing list > talk@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk > -- |-|--| | __.-._ |"Ohhh. Great warrior. Wars not make one great." -Yoda | | '-._"7' |"Freedom is still the most radical idea of all" -N.Branden| | /'.-c |Linux registered user #402901, http://counter.li.org/ | | | /T |http://esambale.wikispaces.com| | _)_/LI |-|--| ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hi, Sebastian Spaeth wrote: > Those who will read and follow the code of conduct are not those who > will blindly break stuff. Those who don't think through their actions > and do a 30 line script in python/PHP/... that breaks stuff will break > things, and they won't read through the code of conduct anyway. Well. Most people running a bot have a certain interest of helping or improving OSM. The rules they build into their bots are often based on discussions on the mailing list or suggestions found on Wiki pages. So they *do* read stuff before they act, and they *want* to do it "right". The aim of that "code of conduct" is to give people a better idea of what we think is "right". Many come from a strong IT background and tend to spend little or no time on thinking about the social component their bots might have - the "respect the work of others" part. I want the "code of conduct" to have a status like Map Features has - it is not something you *have* to follow, not a strict rule and has not been voted upon by anybody, but if you *do* adhere to it then you're less likely to encounter problems. This is all based on positive thinking, assuming that people are basically well-meaning. I'll leave it to others to deal with those who are not ;-) Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
To get back to the issue of a bot running in a specific country : Let's say the vast majority of occurrences of Strasse are spelling mistakes and let's say there are a few occurrences of Strasse where it is not a spelling mistake, like a surname. Should we run the bot ? -- Spaeth wrote "Those who don't think through their actions and do a 30 line script in python/PHP/... that breaks stuff will break things, and they won't read through the code of conduct anyway." I agree with you and let me explain why I do not think it's a big issue : I often install free software on my computer (proprietary and FOSS, poorly documented and well documented, reviewed and unreviewed) and I have no idea if that software will waste my time, open a massive security hole or even worse, do something malicious. And I expect the same of others, e.g. gosmore users. At least the OSM data structure is so simple, that we have so many ways to fight the bot problem : * Statistical detection of bots. From the output of my awk script I can deduce that 'DaBear' has edited an unhumanly number of objects during October and is most likely the cause of this thread. * Tools for targeted reverts. * Responding with simple server changes *when necessary*, like banning users, limits to upload sizes, Captcha on signup etc. On Fri, Oct 3, 2008 at 11:05 AM, Sebastian Spaeth <[EMAIL PROTECTED]>wrote: > Nick Barnes wrote: > > Frederik Ramm wrote: > >> One example to which I took exception is ... changing "Strasse" in the > name to > >> "Straße", which is the correct spelling (but nonetheless "Strasse" is > >> often found on signs). > > > > "Straße" may well be the correct spelling in German speaking countries, > > but it certainly isn't in the UK. (e.g. > > > http://www.openstreetmap.org/?lat=53.7998&lon=-1.75262&zoom=17&layers=B000FTF > ). > > Slightly OT, but it's not even valid for German speaking countries. > Switzerland doesn't even have the "ß" letter, so it uses "Strasse" as > correct and proper spelling. > > spaetz > > ___ > talk mailing list > talk@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Frederik Ramm wrote: > Hi, > > Frederik Ramm wrote: >> I am in favour of setting up a code of conduct for automated edits. Hi Frederik, while i have full understanding and sympathize with your approach, I am doubtful about its outcome. Those who will read and follow the code of conduct are not those who will blindly break stuff. Those who don't think through their actions and do a 30 line script in python/PHP/... that breaks stuff will break things, and they won't read through the code of conduct anyway. Same with a type of robots.txt tag on nodes. Those adhering to those conventions are not the ones who would break things. Still, can't hurt so why not explicitly document stuff... ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Nick Barnes wrote: > Frederik Ramm wrote: >> One example to which I took exception is ... changing "Strasse" in the name >> to >> "Straße", which is the correct spelling (but nonetheless "Strasse" is >> often found on signs). > > "Straße" may well be the correct spelling in German speaking countries, > but it certainly isn't in the UK. (e.g. > http://www.openstreetmap.org/?lat=53.7998&lon=-1.75262&zoom=17&layers=B000FTF). Slightly OT, but it's not even valid for German speaking countries. Switzerland doesn't even have the "ß" letter, so it uses "Strasse" as correct and proper spelling. spaetz ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Frederik Ramm schrieb: > Hi, > > Frederik Ramm wrote: >> I am in favour of setting up a code of conduct for automated edits. > > I have now created > > http://wiki.openstreetmap.org/index.php/Automated_Edits > > with a sub-page for a "code of conduct". Feel free to modify the text > until it reflects a stable consensus ;-) > Hmmm, please let's start a formal voting process on any of those automated changes ... Regards, ULFL ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Nice work Frederick, Thanks for all your efforts on this :-) Cheers Andy >-Original Message- >From: [EMAIL PROTECTED] [mailto:talk- >[EMAIL PROTECTED] On Behalf Of Frederik Ramm >Sent: 02 October 2008 1:37 AM >To: Talk Openstreetmap >Subject: Re: [OSM-talk] Code of conduct for automated (mass-) edits > >Hi, > >Frederik Ramm wrote: >> I am in favour of setting up a code of conduct for automated edits. > >I have now created > >http://wiki.openstreetmap.org/index.php/Automated_Edits > >with a sub-page for a "code of conduct". Feel free to modify the text >until it reflects a stable consensus ;-) > >Bye >Frederik > >-- >Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" > >___ >talk mailing list >talk@openstreetmap.org >http://lists.openstreetmap.org/listinfo/talk > >No virus found in this incoming message. >Checked by AVG - http://www.avg.com >Version: 8.0.173 / Virus Database: 270.7.5/1702 - Release Date: 01/10/2008 >9:05 AM ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hi, Frederik Ramm wrote: > I am in favour of setting up a code of conduct for automated edits. I have now created http://wiki.openstreetmap.org/index.php/Automated_Edits with a sub-page for a "code of conduct". Feel free to modify the text until it reflects a stable consensus ;-) Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Ulf Lamping skrev: > P.S: If you would only know how many of such obvious mistakes like > "aerialway=cinema" I've seen while I was having a detailed look at the > tagwatch output - and not even mentioning the common typos ... /reminder to self : Stop tagging inflight movies, while passing over inhabited areas... Dutch ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
DavidD schrieb: > 2008/9/29 Ulf Lamping <[EMAIL PROTECTED]>: > >> P.S: While I was doing a lot of changes in the past weeks, you were the >> only one complaining - any other reaction I got was simply positive ... > > Here is another complaint from last month. > http://openstreetmap.org/api/0.5/way/25207084/history > > I don't think deleting a horse=bridleway tag from a bridleway is > really helping that much. Probably the tag was entered by mistake but > why delete it? Is it really causing any harm? > First of all, you saying yourself that the change did not do real harm ;-) If you think about how to detect/fix any misstagged bridleways that are only tagged as horse=bridleway and *not* tagged as highway=bridleway (well, which is not the case here), you may encounter that horse=bridleway is a very good candidate to look at. So in JOSM I've just manually added highway=bridleway and removed horse=bridleway (so next time I have a look only at new potential problems). Doing this in JOSM for each case individually is a PITA and to be honest I don't see a loss of information here. Getting 30 "new" bridleways on the map without really loosing any information is worth this change IMHO - but your milage may vary ... Regards, ULFL P.S: If you would only know how many of such obvious mistakes like "aerialway=cinema" I've seen while I was having a detailed look at the tagwatch output - and not even mentioning the common typos ... ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Le Tuesday 30 September 2008 19:13:13, vous avez écrit : > Hi, > > M. S. wrote: > > yes it is more then urgent, I noticed today that 2 OSMers have screwed up > > 100's of my work hours and changed place names and coastlines shapes > > bringing them back to innacurate/false state > > What exactly happened, and are you sure it was the result of an > automated process? > > Bye > Frederik Well there are two users in the history (up to now ) Dmgroom and Randbewohner I contacted them thru their Wikipages, I am waiting for their reply will tell you more ASAP Thanks ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hi, Hendrik T. Voelker wrote: > OSM is providing the framework and execution means for mass changes. [...] > To get something done, one has to create a Java class that described the bot. Security implications galore. Plus, of course, we don't want to enforce Java. Plus the totally un-OSM concept of submitting something for inspection and approval before you use it - that would really turn the spirit of the project on its head. Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hi, DavidD wrote: > I don't think deleting a horse=bridleway tag from a bridleway is > really helping that much. Probably the tag was entered by mistake but > why delete it? Is it really causing any harm? I don't want to comment on the actual case as I have no knowledge about it, but I think the idea that one should generally only remove something that causes harm is a good idea. (As opposed to removing things that *you* think are unnecessary.) Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hi, M. S. wrote: > yes it is more then urgent, I noticed today that 2 OSMers have screwed up > 100's of my work hours and changed place names and coastlines shapes bringing > them back to innacurate/false state What exactly happened, and are you sure it was the result of an automated process? Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
2008/9/29 Ulf Lamping <[EMAIL PROTECTED]>: > P.S: While I was doing a lot of changes in the past weeks, you were the > only one complaining - any other reaction I got was simply positive ... Here is another complaint from last month. http://openstreetmap.org/api/0.5/way/25207084/history I don't think deleting a horse=bridleway tag from a bridleway is really helping that much. Probably the tag was entered by mistake but why delete it? Is it really causing any harm? -- DavidD ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
On Tue, Sep 30, 2008 at 02:58:48PM +, Hendrik T. Voelker wrote: To get something done, one has to create a Java class that described the bot. No, please, no! I have been programming in many different languages (at least one per family, i.e. iterative, functional, i+OOP, f+OOP) for about 20 years now and am currently writing a Java program for money (so I do know what I'm talking about, even if that sounds a bit arrogant). There's no way I would program an OSM tool in Java. This is my spare time, I don't want to spoil it with writing programs in such a horrible language. If you want to program in Java, that's fine by me. But please don't force everyone else to use the language of _your_ choice. CU Sascha -- http://sascha.silbe.org/ http://www.infra-silbe.de/ signature.asc Description: Digital signature ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
El Martes, 30 de Septiembre de 2008, Richard Fairhurst escribió: > Hendrik T. Voelker wrote: > > Granted, that might limit the development to Java programmers but hey, if > > you know one iterative language, you can easily learn another. > > I can't be the only person on this list spluttering in disbelief at > this. Absolutely no way. At all. > > Hey, I don't even know what an "iterative language" _is_. "procedural". -- -- Iván Sánchez Ortega <[EMAIL PROTECTED]> MSN:[EMAIL PROTECTED] Jabber:[EMAIL PROTECTED] ; [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Le Tuesday 30 September 2008 16:58:48 Hendrik T. Voelker, vous avez écrit : > Guys, > > after we now had a longer discussion I guess several things are clear: > >We agree that as a first step a code of conduct concerning mass changes > is a very good idea and maybe Frederik can draft one and add it to the wiki > as a proposal. yes it is more then urgent, I noticed today that 2 OSMers have screwed up 100's of my work hours and changed place names and coastlines shapes bringing them back to innacurate/false state cheers > >Following that most agree that a code of conduct alone might not be > enough and we would need to define some more potent means to prevent > disasters. Several were named, like a sandbox and test suit for verifying > the tool works, like some kind of certification and authorisation for mass > changes, and including the self-evident things like logs and undo files. > >It has become evident, that to most feared problems are stupid edits and > the loss of data because older sources (planet files) are used as the > change base. > > In today's IT business it is normal in these kinds of situations, to not > only work on most current data sets - like Paolo's approach does - but also > to lock the record being worked on to prevent race conditions. > > If we all the names requirements we maybe could think about the following > solution: > > OSM is providing the framework and execution means for mass changes. > > To get something done, one has to create a Java class that described the > bot. That then is submitted to the OSM bot execution facility - or what > ever you want to name it. The framework that executes that class provides > for record selection and locking, and also provides the means to roll back > instead of commit the changes in case of errors. And also the means for > logging and presenting the results officially on the wiki or where ever. As > this can run directly on the DB this is faster and more reliable than an > HTTP(S) connection. > > The developers for the bot class can be provided with test means like sand > box, test data and JUnit frameworks. > > Granted, that might limit the development to Java programmers but hey, if > you know one iterative language, you can easily learn another. And yes, it > might prevent a fast hack - but then, that's what we want, isn't it? > >And yes, I know that it takes time to agree upon this, time to implement > it, time and money to set it up, time to certify all the classes, ... > > In the end, this is just another idea :) > > Cheers > > Hendrik > > > ___ > talk mailing list > talk@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hendrik, I feel that you take the right approach in creating concensus. Do not forget to tackle somewhere the unlimited power of JOSM in uncaring hands. Which in itself is an amazing piece of software for something written in an "iterative language" and it's getting 'better and better. Gert Gremmen - Openstreetmap.nl (alias: cetest) -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Hendrik T. Voelker Verzonden: Tuesday, September 30, 2008 4:59 PM Aan: Talk Openstreetmap Onderwerp: Re: [OSM-talk] Code of conduct for automated (mass-) edits Guys, after we now had a longer discussion I guess several things are clear: We agree that as a first step a code of conduct concerning mass changes is a very good idea and maybe Frederik can draft one and add it to the wiki as a proposal. Following that most agree that a code of conduct alone might not be enough and we would need to define some more potent means to prevent disasters. Several were named, like a sandbox and test suit for verifying the tool works, like some kind of certification and authorisation for mass changes, and including the self-evident things like logs and undo files. It has become evident, that to most feared problems are stupid edits and the loss of data because older sources (planet files) are used as the change base. In today's IT business it is normal in these kinds of situations, to not only work on most current data sets - like Paolo's approach does - but also to lock the record being worked on to prevent race conditions. If we all the names requirements we maybe could think about the following solution: OSM is providing the framework and execution means for mass changes. To get something done, one has to create a Java class that described the bot. That then is submitted to the OSM bot execution facility - or what ever you want to name it. The framework that executes that class provides for record selection and locking, and also provides the means to roll back instead of commit the changes in case of errors. And also the means for logging and presenting the results officially on the wiki or where ever. As this can run directly on the DB this is faster and more reliable than an HTTP(S) connection. The developers for the bot class can be provided with test means like sand box, test data and JUnit frameworks. Granted, that might limit the development to Java programmers but hey, if you know one iterative language, you can easily learn another. And yes, it might prevent a fast hack - but then, that's what we want, isn't it? And yes, I know that it takes time to agree upon this, time to implement it, time and money to set it up, time to certify all the classes, ... In the end, this is just another idea :) Cheers Hendrik ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
On Tue, Sep 30, 2008 at 3:58 PM, Hendrik T. Voelker <[EMAIL PROTECTED]> wrote: > Guys, > > after we now had a longer discussion I guess several things are clear: > > We agree that as a first step a code of conduct concerning mass changes is > a very good idea and maybe Frederik can draft one and add it to the wiki as a > proposal. > > Following that most agree that a code of conduct alone might not be enough > and we would need to define some more potent means to prevent disasters. > Several were named, like a sandbox and test suit for verifying the tool works, > like some kind of certification and authorisation for mass changes, and > including the self-evident things like logs and undo files. > > It has become evident, that to most feared problems are stupid edits and > the loss of data because older sources (planet files) are used as the change > base. > > In today's IT business it is normal in these kinds of situations, to not only > work on most current data sets - like Paolo's approach does - but also to lock > the record being worked on to prevent race conditions. > > If we all the names requirements we maybe could think about the following > solution: > > OSM is providing the framework and execution means for mass changes. > > To get something done, one has to create a Java class that described the bot. > That then is submitted to the OSM bot execution facility - or what ever you > want to name it. The framework that executes that class provides for record > selection and locking, and also provides the means to roll back instead of > commit the changes in case of errors. And also the means for logging and > presenting the results officially on the wiki or where ever. As this can run > directly on the DB this is faster and more reliable than an HTTP(S) > connection. Please take a look at the API 0.6 changes being worked on. Two features included in that will help here: 1) Optimistic locking of OSM objects -- it will no longer be possible to accidentally upload an earlier version from a planet etc as the version number will have changed 2) Diff upload -- you can supply an entire file of changes to be made within one atomic transaction These aren't quite the same as what you suggest, and is somewhat less powerful, but a heck of a lot easier to manage than arbitrary code. Dave ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hendrik T. Voelker wrote: > Granted, that might limit the development to Java programmers but hey, if you > know one iterative language, you can easily learn another. I can't be the only person on this list spluttering in disbelief at this. Absolutely no way. At all. Hey, I don't even know what an "iterative language" _is_. cheers Richard ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Guys, after we now had a longer discussion I guess several things are clear: We agree that as a first step a code of conduct concerning mass changes is a very good idea and maybe Frederik can draft one and add it to the wiki as a proposal. Following that most agree that a code of conduct alone might not be enough and we would need to define some more potent means to prevent disasters. Several were named, like a sandbox and test suit for verifying the tool works, like some kind of certification and authorisation for mass changes, and including the self-evident things like logs and undo files. It has become evident, that to most feared problems are stupid edits and the loss of data because older sources (planet files) are used as the change base. In today's IT business it is normal in these kinds of situations, to not only work on most current data sets - like Paolo's approach does - but also to lock the record being worked on to prevent race conditions. If we all the names requirements we maybe could think about the following solution: OSM is providing the framework and execution means for mass changes. To get something done, one has to create a Java class that described the bot. That then is submitted to the OSM bot execution facility - or what ever you want to name it. The framework that executes that class provides for record selection and locking, and also provides the means to roll back instead of commit the changes in case of errors. And also the means for logging and presenting the results officially on the wiki or where ever. As this can run directly on the DB this is faster and more reliable than an HTTP(S) connection. The developers for the bot class can be provided with test means like sand box, test data and JUnit frameworks. Granted, that might limit the development to Java programmers but hey, if you know one iterative language, you can easily learn another. And yes, it might prevent a fast hack - but then, that's what we want, isn't it? And yes, I know that it takes time to agree upon this, time to implement it, time and money to set it up, time to certify all the classes, ... In the end, this is just another idea :) Cheers Hendrik ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
On Tue, Sep 30, 2008 at 1:58 PM, Paolo Molaro <[EMAIL PROTECTED]> wrote: > The first requirement, IMHO, is a better way to communicate with other > users. I would be in favor of sharing the email address of users > to make this easier: currently my tool has to parse the web pages > and submit web forms. There is no reason to make communication harder, > so let's allow sharing the email address to other (authenticated) users. Let's not. Aside from the many privacy issues, this would become a spammer's dream -- I get enough spam to this address because it's on a public mailing list, I don't want my nice clean OSM registered address making a break for the wild too. Dave ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Great Contribution. I will definitely look into your tools. Gert Gremmen - Openstreetmap.nl (alias: cetest) -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Paolo Molaro Verzonden: Tuesday, September 30, 2008 2:59 PM Aan: talk@openstreetmap.org Onderwerp: Re: [OSM-talk] Code of conduct for automated (mass-) edits Sorry for breaking the email history, I just subscribed to this list. Frederik Ramm wrote: > Another issue is, *if* something is changed, *how* this is done. Lacking > 0.6's versioning, if anyone analyzes yesterday's planet file to find > ways he'd like to fix and uploads changed versions of each, chances are > he'll overwrite all those that have been changed between the generation > of the planet file and his script run. Whoever wants to run an automated > update should know exactly what he's doing, and be in a position to > exactly revert his changes should it turn out they were faulty. When on the italian mailing list we recently agreed on the capitalization convention for the street type (using Via, Viale etc instead of via and viale, basically the same convention as explained on the wiki about using the Street capitalization), the risks of doing the mass edits prompted me to write a tool that reduced the risk as far as possible with the current db API. The tool basically downloads the latest version of each node/way from the db at the time of the upload and it checks that the fields/tags that need changing have still the old value before the change is applied. All the other tags are left unchanged. This way the data overwrite race lasts just a fraction of a second. I suggest people use the same approach in their scripts or simply use my code/tool. It can be download from: http://www.oddwiz.org/~lupus/osm/osm-helpers-0.3.tar.gz I didn't announce it here yet because I don't have much time to deal with a large user base, but this thread seemed like a good enough reason. People may also find the included osm-history tool useful as it displays the changes in a db object in a diff-like way (I always found it very hard to spot just the differences between versions by looking at all the data shown on the web page). Note the the tools should work on windows/OSX, but I only tested it with Mono on Linux. Feel free to mail me any bug report (and always remember to check the changes file before uploading it). > And still another thing is documentation; I somewhat expect that any > automated, large-scale change should be documented. When was it done, > what exactly was done, how many objects were affected, what were the > "source" and/or username settings for the job so that it can be > identified later. The tools that I wrote work this way: 1) a checker tool generates a changes file, this includes the object id and the new/old values for the changed tags. 2) the changes file can be inspected for error/mistakes etc 3) another tool takes the changes file and updates the objects with the protocol explained above. 4) the same upload tool can optionally notify the last contributor of the changed objects: the message will include the object id and the old/new values. So both the changes file and the email notification serve as the 'documentation' of the changes. More details are in the README file in the tarball. > 1. Make a plan of what you want to change, and discuss in relevant forum > (usu. mailing list). If there are many objections; drop the plan. If > there are few objections, maybe exempt certain areas or objects created > by certain people in order to respect their objections. Remember that > they can easily change things back again if you act against their will, > so don't even try to play the superiority card. > 2. Make sure your tools and knowledge are good: You have to be able to > revert your changes if something goes wrong, and you need to keep any > collateral damage to an absolute minimum. If you cannot guarantee that, > ask someone for help who can. I think my osm-upload-changes tool involves the minimum amount of overwriting risk as allowed by the db API. Also, separating the checker and the upload tool allows (or favours) inspecting the changes file for mistakes or unintended changes. The changes file shows just the changes, so it an be easily read (vs looking at a huge osm file with all the data, even the data that is unchanged). > 4. Provide documentation that tells people what exactly you have done. The wiki message notification in my tool properly documents the changes. Keeping the changes file around also helps with that if there is any issue in the future. That said, I agree with the other points in the email. I hope that my code will help people writing more responsible bots, but I think some changes at the project level
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Sorry for breaking the email history, I just subscribed to this list. Frederik Ramm wrote: > Another issue is, *if* something is changed, *how* this is done. Lacking > 0.6's versioning, if anyone analyzes yesterday's planet file to find > ways he'd like to fix and uploads changed versions of each, chances are > he'll overwrite all those that have been changed between the generation > of the planet file and his script run. Whoever wants to run an automated > update should know exactly what he's doing, and be in a position to > exactly revert his changes should it turn out they were faulty. When on the italian mailing list we recently agreed on the capitalization convention for the street type (using Via, Viale etc instead of via and viale, basically the same convention as explained on the wiki about using the Street capitalization), the risks of doing the mass edits prompted me to write a tool that reduced the risk as far as possible with the current db API. The tool basically downloads the latest version of each node/way from the db at the time of the upload and it checks that the fields/tags that need changing have still the old value before the change is applied. All the other tags are left unchanged. This way the data overwrite race lasts just a fraction of a second. I suggest people use the same approach in their scripts or simply use my code/tool. It can be download from: http://www.oddwiz.org/~lupus/osm/osm-helpers-0.3.tar.gz I didn't announce it here yet because I don't have much time to deal with a large user base, but this thread seemed like a good enough reason. People may also find the included osm-history tool useful as it displays the changes in a db object in a diff-like way (I always found it very hard to spot just the differences between versions by looking at all the data shown on the web page). Note the the tools should work on windows/OSX, but I only tested it with Mono on Linux. Feel free to mail me any bug report (and always remember to check the changes file before uploading it). > And still another thing is documentation; I somewhat expect that any > automated, large-scale change should be documented. When was it done, > what exactly was done, how many objects were affected, what were the > "source" and/or username settings for the job so that it can be > identified later. The tools that I wrote work this way: 1) a checker tool generates a changes file, this includes the object id and the new/old values for the changed tags. 2) the changes file can be inspected for error/mistakes etc 3) another tool takes the changes file and updates the objects with the protocol explained above. 4) the same upload tool can optionally notify the last contributor of the changed objects: the message will include the object id and the old/new values. So both the changes file and the email notification serve as the 'documentation' of the changes. More details are in the README file in the tarball. > 1. Make a plan of what you want to change, and discuss in relevant forum > (usu. mailing list). If there are many objections; drop the plan. If > there are few objections, maybe exempt certain areas or objects created > by certain people in order to respect their objections. Remember that > they can easily change things back again if you act against their will, > so don't even try to play the superiority card. > 2. Make sure your tools and knowledge are good: You have to be able to > revert your changes if something goes wrong, and you need to keep any > collateral damage to an absolute minimum. If you cannot guarantee that, > ask someone for help who can. I think my osm-upload-changes tool involves the minimum amount of overwriting risk as allowed by the db API. Also, separating the checker and the upload tool allows (or favours) inspecting the changes file for mistakes or unintended changes. The changes file shows just the changes, so it an be easily read (vs looking at a huge osm file with all the data, even the data that is unchanged). > 4. Provide documentation that tells people what exactly you have done. The wiki message notification in my tool properly documents the changes. Keeping the changes file around also helps with that if there is any issue in the future. That said, I agree with the other points in the email. I hope that my code will help people writing more responsible bots, but I think some changes at the project level will be needed to better deal with script usage (or simply with mistakes happening in GUI editors). The first requirement, IMHO, is a better way to communicate with other users. I would be in favor of sharing the email address of users to make this easier: currently my tool has to parse the web pages and submit web forms. There is no reason to make communication harder, so let's allow sharing the email address to other (authenticated) users. With proper communication all the changes to the objects could be notified (the current web interf
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Unfortunately, I am a mapper, not a programmer. Unfortunately I have opinions and ideas but not the skills to bring them into practice, I am the first to regret that. I am from a generation been raised with PDP-11 basic IBM-p qbasic and managed to understand C, a bit of perl and some python to the level that I can install and modify simple scripts. If you have some assembler projects in 8080 or 6801/3/9 I can be of help. XML is to me, structured ASCII only. To all on the list: Don't tell me to shut-up because just I cannot (yet) cope with the last 10 year of programming, and thus cannot be of help in coding. I know what it is about, just too old. Gert Gremmen -Oorspronkelijk bericht- Van: Shaun McDonald [mailto:[EMAIL PROTECTED] Verzonden: dinsdag 30 september 2008 0:22 Aan: Gert Gremmen CC: Frederik Ramm; Talk Openstreetmap Onderwerp: Re: [OSM-talk] Code of conduct for automated (mass-) edits Hi Gert, On 29 Sep 2008, at 23:10, Gert Gremmen wrote: > > I Just want to implement obliged automated > Administration/recording of activity that we are in desperate need > of now, or will be very soon. Why don't you spend some time to work on the 0.6 API, which will help towards some of what you want? Shaun ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hi Gert, On 29 Sep 2008, at 23:10, Gert Gremmen wrote: I Just want to implement obliged automated Administration/recording of activity that we are in desperate need of now, or will be very soon. Why don't you spend some time to work on the 0.6 API, which will help towards some of what you want? Shaun smime.p7s Description: S/MIME cryptographic signature ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Don't exaggerate, this is a serious discussion, not in need of overdriven arguments. Keep your arguments to what I propose, not to what you think I might propose !!! >>But I'm utterly, totally, completely against any kind of half-hearted >>measures that work only against those >>who cannot compile their own version >>of an editor I did not propose , just a temporarily measure, quick to release. See remarks later. >>Next thing you'll say is that the JOSM .jar file has to be signed and only >>the "official" signed version may >>connect, and before you know it you have >>a situation where Central Command dictates what software people may >>use. No >>thanks! This is what you make up, not what I propose. Read well Frederik !!! You are right, it goes beyond good old josm and beyond -we can trust anyone- sentiments. OSM is growing. Growing beyond -we trust all members-. Much growing beyond -we know all users- Growing beyond -we can reach all users-- Growing beyond -we can expect all users to read any "code of conduct"- let alone follow that code Growing beyond -if something goes wrong Andy or Steve or Frederik will fix it- My ideas are not that invasive at all; it won't Stop anyone from doing what he wants or needs. The upload limit will not impact any ordinary user, He just needs to push the upload button more frequently. (I understand that it won't protect against anyone able to modify The source, something probably very easy to do. Just for the time, until the API will take care of that.) Alternately, JOSM could provide a STRONG warning if a user wants to upload more data then normally expected, before to proceed. More error prone, but also more "open" friendly. I Just want to implement obliged automated Administration/recording of activity that we are in desperate need of now, or will be very soon. Gert Gremmen -Oorspronkelijk bericht- Van: Frederik Ramm [mailto:[EMAIL PROTECTED] Verzonden: maandag 29 september 2008 22:59 Aan: Gert Gremmen CC: Talk Openstreetmap Onderwerp: Re: [OSM-talk] Code of conduct for automated (mass-) edits Hi, Gert Gremmen wrote: > - Let's start with limiting JOSM and Merkaartor to a reasonable number > of uploads per manual upload action. I am against implementing any controlling measures beyond "is the username/password valid" at this point. But I'm utterly, totally, completely against any kind of half-hearted measures that work only against those who cannot compile their own version of an editor! Next thing you'll say is that the JOSM .jar file has to be signed and only the "official" signed version may connect, and before you know it you have a situation where Central Command dictates what software people may use. No thanks! Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Andy Robinson (blackadder-lists) schrieb: > > Agreed. I've been the victim of bot changes over the last few months which > have been wholly inappropriate and they are a pain in the ass to deal with, Hmmm, maybe you simply not got aware of all the changes to the better and only got aware of the changes that messed things up? Is it good or bad if someone improves 1000 tags and by mistake messing up 1 (because of errors, missunderstandings, ...)? BTW: I'm not related in any way to the Straße vs. Strasse changes/discussions! Changing a lot of stuff because the german Duden tells you so is (at least to me) obviously a bad idea. > mainly because they have to be sorted out manually, which takes up valuable > mapping/editing time. I'd be a little happier if there was plenty of > information written in the tags about the bot and what the bot was looking > for in the tags when it makes a change. If the user understands what the bot > is doing and its basis plus who to contact re bugs then its less of a > problem to revert or amend. I'd also like to see tags that have been changed > remaining in some format in the tags, perhaps with a bot_change: namespace > or something. That's easy to do if you use a real script bot, but a lot of additional manual work if you use tagwatch/osmxapi/JOSM to edit stuff (semi-)manually as I do - and I could imagine a lot of others as well. I manually load stuff into JOSM and have a look (and not programmed a bot), exactly to see and prevent mistakes that would be really ugly on a larger scale. > Yes, I know the data is in the history but quite frankly I > don't have time to look at the history for each individual item to fix > problems. I want to map. > I understand you, but that's in no way related to bots, but the "OSM wiki way" - and simply shows our current lack of good tools for change management ... Regards, ULFL P.S: While I was doing a lot of changes in the past weeks, you were the only one complaining - any other reaction I got was simply positive ... ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hi, Gert Gremmen wrote: > - Let's start with limiting JOSM and Merkaartor to a reasonable number > of uploads per manual upload action. I am against implementing any controlling measures beyond "is the username/password valid" at this point. But I'm utterly, totally, completely against any kind of half-hearted measures that work only against those who cannot compile their own version of an editor! Next thing you'll say is that the JOSM .jar file has to be signed and only the "official" signed version may connect, and before you know it you have a situation where Central Command dictates what software people may use. No thanks! Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
It's a way we need to go. We may not like it, and it may take months to go, but it is the only way to save the database against a number of mistakes, hacks, osm-terrorism or whatever bot, script or other mass modification might happen. We had this affair with the Russian guy with JOSM using simplifier, that by accident deleted many valuable details in the database in a huge area, just using JOSM. A strong argument to limit JOSM uploads to a limited number of modifications at a time. Large changes go best by script, and NEED to go documented. OSM community is getting larger and larger, and we cannot expect that every bot-writer reads (or want to read) the "code of conduct" you suggested, AND follow it. "Edits of mass destruction": I have been experimenting with a text editor and regular expressions on raw downloaded osm-data (JOSM, save as .osm) Great tool for changing all street names into "guess what" !! Set the changed flag, reload in JOSM and upload it will do the job. The system I proposed will delay the execution of script by a moderate amount of effort, namely that to document it on a central place, available to everyone. It also allows the system to create an unique identifier per mass modification allowing them to easily be reverted. (If done fast enough). Not all I suggested needs to be done at once, however: - Let's start with limiting JOSM and Merkaartor to a reasonable number of uploads per manual upload action. This would prevent the errors of the type "select all" plus "shift 1 km" or the "edits of mass destruction" type to get uploaded. It also would get rid of those just fake editing to get their name on top of the lists. - Then allow the API to refuse uploads bigger then say 500 edits at a time, unless a assigned username is delivered by the webform script accepting the documentation. These two measures would assure that no faulty mass edit will go hidden behind a common username such as AND (read: yours), who contributes valuable data normally. Gert -Oorspronkelijk bericht- Van: Frederik Ramm [mailto:[EMAIL PROTECTED] Verzonden: maandag 29 september 2008 20:24 Aan: Gert Gremmen CC: Hendrik T. Voelker; Talk Openstreetmap Onderwerp: Re: [OSM-talk] Code of conduct for automated (mass-) edits Hi, Gert Gremmen wrote: > I want to insist however, on the > website form approach, to oblige > fast-thinking fast-acting "bot writers" to > express at least what they intend to do somewhere. > The automated undo system allows for a quick undo, > before thousands of volunteers added data that will got > lost when undone. Nothing of what you say is going to be either fast or quick. It's going to take months to implement. Not what I had in mind. Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Frederik Ramm wrote: >Sent: 29 September 2008 7:22 PM >To: Hendrik T. Voelker >Cc: Talk Openstreetmap >Subject: Re: [OSM-talk] Code of conduct for automated (mass-) edits > >Hi, > >Hendrik T. Voelker wrote: >> Guys, think about one of the most fundamental design principles for >robust >> code: Keep it simple. > >[...] > >> 1. Require scripts to use an SSL certificate that is signed by OSM for >> authentication > >[...] > >> 2. As the first step request a Universally Unique Identifier (UUID) from >the >> OSM server which acts as transaction number for the batch process and tag >each >> Object with it > >Maybe everyone has misunderstood me. > >I was indeed trying to keep things simple, and the ONLY thing I wanted >was to reach some agreement between HUMANS that can be communicated to >potential bot users. > >Not a huge battery of technical measures that attempt to regulate bot >usage, and are prone to fail. > >As has been pointed out, 0.6 will bring some advances in the grouping >and tracking department so PLEASE let's not go overboard with some >semi-0.6 ideas. > Agreed. I've been the victim of bot changes over the last few months which have been wholly inappropriate and they are a pain in the ass to deal with, mainly because they have to be sorted out manually, which takes up valuable mapping/editing time. I'd be a little happier if there was plenty of information written in the tags about the bot and what the bot was looking for in the tags when it makes a change. If the user understands what the bot is doing and its basis plus who to contact re bugs then its less of a problem to revert or amend. I'd also like to see tags that have been changed remaining in some format in the tags, perhaps with a bot_change: namespace or something. Yes, I know the data is in the history but quite frankly I don't have time to look at the history for each individual item to fix problems. I want to map. Cheers Andy ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
On Mon, Sep 29, 2008 at 1:48 PM, Hugh Barnes <[EMAIL PROTECTED]> wrote: >> No, because the user agent is not recorded as part of the edits. Nor is >> it planned to record it as a property of the changeset in API 0.6. > > Right, that's surprising, it's a pretty handy thing. I was about to ask how > you know the editor in the history, but the penny's dropped that you actually > use a tag to record it. I guess that's a legacy of editors existing outside > of the API. Well, when I was at the hackathon it was certainly intended to include the user agent in the changeset. As a replacement for the created_by tag. JOSM in API 0.6 will include it's version number in the changeset for example. Have a nice day, -- Martijn van Oosterhout <[EMAIL PROTECTED]> http://svana.org/kleptog/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hi, Gert Gremmen wrote: > I want to insist however, on the > website form approach, to oblige > fast-thinking fast-acting "bot writers" to > express at least what they intend to do somewhere. > The automated undo system allows for a quick undo, > before thousands of volunteers added data that will got > lost when undone. Nothing of what you say is going to be either fast or quick. It's going to take months to implement. Not what I had in mind. Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hi, Hendrik T. Voelker wrote: > Guys, think about one of the most fundamental design principles for robust > code: Keep it simple. [...] > 1. Require scripts to use an SSL certificate that is signed by OSM for > authentication [...] > 2. As the first step request a Universally Unique Identifier (UUID) from the > OSM server which acts as transaction number for the batch process and tag > each > Object with it Maybe everyone has misunderstood me. I was indeed trying to keep things simple, and the ONLY thing I wanted was to reach some agreement between HUMANS that can be communicated to potential bot users. Not a huge battery of technical measures that attempt to regulate bot usage, and are prone to fail. As has been pointed out, 0.6 will bring some advances in the grouping and tracking department so PLEASE let's not go overboard with some semi-0.6 ideas. Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hi, Gert Gremmen wrote: > The usertype should have the username of the uploader [...] > The username should be enabled shortly before [...] > The form would return by email the username > to be used [...] > The notification script could also create a backup of > all changes [...] > Normal users should have their number of changes per upload > limited to say a few hundreds a time. [...] I was not suggesting any changes in the API. What you write above would require a lot of changes (and a lot of manpower). I just wanted to implement a code of conduct which bot users would be *expected* to follow - I didn't want to implement technical measures that *forces* people to do so. Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Nick wrote: > Seems to me that everybody replying thinks that there ought to > be some > sort of framework for mass changes and I'll go along with that > too. > I was at the Bradford Mapping Party this weekend I've just realised the irony of what I've just done. I had a quick look to spot how the mapping party went, zooming out from Nick's link and comparing Mapnik to Osmarender versions. I then noticed about 15 mini_roundabouts that hadn't been tagged direction=clockwise (which I assume being on UK roads is the case), so tagged them. I then thought that perhaps doing such a "mini" mass change was wrong. Please, Nick, reassure me that Bradford's mini roundabouts aren't anticlockwise ones just to confuse visitors? Ed ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Nick Barnes wrote: > 'Hamm Strasse' - see link above). One of the guys there commented that > after he spent some time mapping an area called 'Batley Carr', some > numbskull thought it would be a good idea to correct all the > misspellings of 'car' and changed every single reference. First of all this is not a problem limited to bots. Second there is no known way against stupidity and ignorance. That said we should try nonetheless to get things as fool prove as possible. Discussions resulting in rules and guidelines are a good step in that direction. Cheers Hendrik ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Seems to me that everybody replying thinks that there ought to be some sort of framework for mass changes and I'll go along with that too. However, the OP raised an interesting point... Frederik Ramm wrote: > One example to which I took exception is ... changing "Strasse" in the name > to > "Straße", which is the correct spelling (but nonetheless "Strasse" is > often found on signs). "Straße" may well be the correct spelling in German speaking countries, but it certainly isn't in the UK. (e.g. http://www.openstreetmap.org/?lat=53.7998&lon=-1.75262&zoom=17&layers=B000FTF). I was at the Bradford Mapping Party this weekend (which is how I spotted 'Hamm Strasse' - see link above). One of the guys there commented that after he spent some time mapping an area called 'Batley Carr', some numbskull thought it would be a good idea to correct all the misspellings of 'car' and changed every single reference. I have no evidence that either of these changes were applied globally, but all it takes is for some blinkered xenophobe to decide that everybody ought to use their character set, language rules and dictionary and we're in a right mess. Anyway, rant over. Nick. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Gert Gremmen wrote: > I want to insist however, on the > website form approach, to oblige > fast-thinking fast-acting "bot writers" to > express at least what they intend to do somewhere. Yes, sure. But not for every run of the bot. And you can add a description to the One could go as far as some kind of certification for bots. Only question is who does that, and what requirements do the bots have to full fill to pass certification? Some kind of sand box for testing - like Johannes suggested - and some kind of test suits would be a beginning. On the other hand we don't want to discourage people by complicating the process too much. Cheers Hendrik ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
>>Guys, think about one of the most fundamental design principles for robust >>code: Keep it simple. Agree, but it should meet the requirements !! Certificates might simplify the identification. The UUID might replace my username, but is not fundamentally different. I want to insist however, on the website form approach, to oblige fast-thinking fast-acting "bot writers" to express at least what they intend to do somewhere. The automated undo system allows for a quick undo, before thousands of volunteers added data that will got lost when undone. The webpage allows others to understand what's going on without having to ask, or worse : guess !! Gert Gremmen - Openstreetmap.nl (alias: cetest) -Oorspronkelijk bericht- Van: Hendrik T. Voelker [mailto:[EMAIL PROTECTED] Verzonden: Monday, September 29, 2008 1:49 PM Aan: Gert Gremmen CC: Talk Openstreetmap Onderwerp: Re: [OSM-talk] Code of conduct for automated (mass-) edits Gert Gremmen wrote: > The serverscript (see below) could create a web page with the > temporary accountname as a URL, summarizing all the submitted data, > for consulting by anyone finding a questionable dataelement. Guys, think about one of the most fundamental design principles for robust code: Keep it simple. I would use two mechanisms for a script mass edit 1. Require scripts to use an SSL certificate that is signed by OSM for authentication 2. As the first step request a Universally Unique Identifier (UUID) from the OSM server which acts as transaction number for the batch process and tag each Object with it when it is submitted. Either by the script itself or, even more secure, by the OSM server API. And when you want to document who did what when, additionally create Cheers Hendrik ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Philip Homburg wrote: > There is not much you can do if someone just rents a botnet and then pollutes > then database. Do backups. Regularly. ;))) Cheers Hendrik ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
In your letter dated Mon, 29 Sep 2008 11:55:16 +0100 you wrote: >On Mon, Sep 29, 2008 at 8:53 AM, Philip Homburg >> As far as politics go, I think that it would a good idea to just re-use the >> current structure for introducing new map features. Before you run a script >> you first propose it and only run it when enough people cast a vote in favor >> of running the script. > >Step 1) Write OSM bot >Step 2) Write OSM Wiki vote rigging bot >Step 3) Propose bot >Step 4) Rig vote >Step 5) Run OSM bot all the while pointing at the wiki shouting "look! >13 people approved it!" > >I'm not so convinced :-) Ignoring the smiley for the moment, I think that the biggest risk are the people who are just trying to help, and then get the details wrong. There is not much you can do if someone just rents a botnet and then pollutes then database. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
On Monday 29 September 2008 21:24:29 Shaun McDonald wrote: > Hugh Barnes wrote: > > On Monday 29 September 2008 18:37:53 Gert Gremmen wrote: > > > > [..]] > > All worthy ideas. > > > > I would probably look at using a custom, temporarily-useful User Agent > > string in the HTTP request rather than specific "user" hoodoo, to > > summarise it lazily. Is this a possibility? > > No, because the user agent is not recorded as part of the edits. Nor is > it planned to record it as a property of the changeset in API 0.6. > Right, that's surprising, it's a pretty handy thing. I was about to ask how you know the editor in the history, but the penny's dropped that you actually use a tag to record it. I guess that's a legacy of editors existing outside of the API. I think for a RESTful API you should look to use HTTP features as much as possible. Sorry, going off-topic. Cheers. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Gert Gremmen wrote: > The serverscript (see below) could create a web page with the > temporary accountname as a URL, summarizing all the submitted data, > for consulting by anyone finding a questionable dataelement. Guys, think about one of the most fundamental design principles for robust code: Keep it simple. I would use two mechanisms for a script mass edit 1. Require scripts to use an SSL certificate that is signed by OSM for authentication 2. As the first step request a Universally Unique Identifier (UUID) from the OSM server which acts as transaction number for the batch process and tag each Object with it when it is submitted. Either by the script itself or, even more secure, by the OSM server API. And when you want to document who did what when, additionally create Cheers Hendrik ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Fredrik wrote : "2. Make sure your tools and knowledge are good: You have to be able to revert your changes if something goes wrong, and you need to keep any collateral damage to an absolute minimum. If you cannot guarantee that, ask someone for help who can." If a bot writer sees this, he's either going to abandon his plans, or continue without having these tools. A better solution is for someone to write proper, general purpose revert tools for use by the sysadmin / community. > > This is mostly what the 0.6 API is about. Changesets will report to > you what happened, and have space for meta data (which will allow the > identification of particular bots/editors). And that'll give a basis > for reversion tools to operate. > Currently most bot writers already add identification tags to their data. For example "tiger:", "AND_" etc. The positive side is that we can already use those tags to track bots. The negative side is pollution : We all know what tiger and AND refers to, but who know what sagns stands for ? Where can we look it up ? An import in Japan has the the "note" and "note:ja" tags for millions of objects, so if someone tries to render or index it, it may not work for Japan. So it's not clear that changesets will improve the bot situation. The improvement will come from a tool that looks for a specific user or tag (or changeset) and then change to the last revision where that user or tag was absent. It looks like Frederik already has an unpolished version of such a tool for 0.5. The advantage of 0.6 will be that a client doing an upload or delete will be required to provide the version number of the object it's modifying. This is race condition is currently quite rare, and the easiest way to reduce the probability of it happening is to explain your bot on the talk list of the relevant country. > > > > > > As far as politics go, I think that it would a good idea to just re-use > the > > current structure for introducing new map features. Before you run a > script > > you first propose it and only run it when enough people cast a vote in > favor > > of running the script. > > Step 1) Write OSM bot > Step 2) Write OSM Wiki vote rigging bot > Step 3) Propose bot > Step 4) Rig vote > Step 5) Run OSM bot all the while pointing at the wiki shouting "look! > 13 people approved it!" > > Agreed. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Hugh Barnes wrote: > On Monday 29 September 2008 18:37:53 Gert Gremmen wrote: > > [..]] > All worthy ideas. > > I would probably look at using a custom, temporarily-useful User Agent string > in the HTTP request rather than specific "user" hoodoo, to summarise it > lazily. Is this a possibility? > No, because the user agent is not recorded as part of the edits. Nor is it planned to record it as a property of the changeset in API 0.6. Shaun ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dave Stubbs wrote: > On Mon, Sep 29, 2008 at 8:53 AM, Philip Homburg > <[EMAIL PROTECTED]> wrote: [] > Step 1) Write OSM bot > Step 2) Write OSM Wiki vote rigging bot > Step 3) Propose bot > Step 4) Rig vote > Step 5) Run OSM bot all the while pointing at the wiki shouting "look! > 13 people approved it!" > > I'm not so convinced :-) > > Dave > Are you in fact a cynicism bot ? Mark -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI4LpiJfMmcSPNh94RAp0SAJ40k9R/CC2Sw/ux5+/m6aymqeu4EACeIOWQ 4rDEahzFSt9LMUipt4Ig0rQ= =bH/x -END PGP SIGNATURE- ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
On Monday 29 September 2008 18:37:53 Gert Gremmen wrote: > I'd suggest a special usertype for batch operations, > and combine that with a notification system > so as to enable that account for one batch. > > The usertype should have the username of the uploader > with a random number attached (fe cetest to cetest) > to distinguish between sessions. > > The username should be enabled shortly before > carrying out the scripts/upload by the server. > The uploader could enable his temp account by filling in a web form > with sufficient "required fields" > > I suggest: > username > bounding box for all changes > description of change > modified data (list of tags) > intention of batch operation > > The form would return by email the username > to be used for this upload. The email ensures > that the uploader can be contacted. > All worthy ideas. I would probably look at using a custom, temporarily-useful User Agent string in the HTTP request rather than specific "user" hoodoo, to summarise it lazily. Is this a possibility? Cheers ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
Some enhancements: The serverscript (see below) could create a web page with the temporary accountname as a URL, summarizing all the submitted data, for consulting by anyone finding a questionable dataelement. This webpage may contain a reverse request button. A system should be invented for approval of such a request. In it's best form the script would reverse automatically. A report will be created with reversals that create problems, such as modified elements after the batch, for manual review who will do that ??? Gert Gremmen - Openstreetmap.nl (alias: cetest) -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Gert Gremmen Verzonden: Monday, September 29, 2008 10:38 AM Aan: Philip Homburg; Talk Openstreetmap Onderwerp: Re: [OSM-talk] Code of conduct for automated (mass-) edits I'd suggest a special usertype for batch operations, and combine that with a notification system so as to enable that account for one batch. The usertype should have the username of the uploader with a random number attached (fe cetest to cetest) to distinguish between sessions. The username should be enabled shortly before carrying out the scripts/upload by the server. The uploader could enable his temp account by filling in a web form with sufficient "required fields" I suggest: username bounding box for all changes description of change modified data (list of tags) intention of batch operation The form would return by email the username to be used for this upload. The email ensures that the uploader can be contacted. The notification script could also create a backup of all changes done immediately after, so as to facilitate removal if required. May this is redundant, as date and time are available to select. Normal users should have their number of changes per upload limited to say a few hundreds a time. This could be supported in JOSM and merkaartor by a warning system when 90% of the maximum upload has been reached and the user should upload. Some finetuning of the above may be necessary. Gert Gremmen - Openstreetmap.nl (alias: cetest) Gert Gremmen - Openstreetmap.nl (alias: cetest) -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Philip Homburg Verzonden: Monday, September 29, 2008 9:53 AM Aan: Talk Openstreetmap Onderwerp: Re: [OSM-talk] Code of conduct for automated (mass-) edits In your letter dated Mon, 29 Sep 2008 01:08:50 +0200 you wrote: > as OpenStreetMap draws more and more sophisticated users, we're > also seeing more scripts or, as they would be called in Wikipedia, > "bots", modifying data. > > 1. Make a plan of what you want to change, and discuss in relevant > forum (usu. mailing list). If there are many objections; drop the > plan. If there are few objections, maybe exempt certain areas or > objects created by certain people in order to respect their > objections. Remember that they can easily change things back again > if you act against their will, so don't even try to play the > superiority card. > > I would also accompany this by the notion that if you see an > automated edit that you believe has problems, and it has not been > discussed or documented, it's ok to revert it. I think there should be two technical things in place: One thing is a structured way of rolling back edits. There should be a way of reporting large scale edits, and getting them removed from the database. The second thing is a reporting script that reports on large scale edits in a timely fashing. As far as politics go, I think that it would a good idea to just re-use the current structure for introducing new map features. Before you run a script you first propose it and only run it when enough people cast a vote in favor of running the script. For example, a good way of completely destroying the JOSM/Validator's duplicate node detection feature is to fill the database with a huge number of aumatically generated duplicate nodes. :-( ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
On Mon, Sep 29, 2008 at 8:53 AM, Philip Homburg <[EMAIL PROTECTED]> wrote: > In your letter dated Mon, 29 Sep 2008 01:08:50 +0200 you wrote: >> as OpenStreetMap draws more and more sophisticated users, we're >> also seeing more scripts or, as they would be called in Wikipedia, >> "bots", modifying data. >> >> 1. Make a plan of what you want to change, and discuss in relevant >> forum (usu. mailing list). If there are many objections; drop the >> plan. If there are few objections, maybe exempt certain areas or >> objects created by certain people in order to respect their >> objections. Remember that they can easily change things back again >> if you act against their will, so don't even try to play the >> superiority card. >> >> I would also accompany this by the notion that if you see an >> automated edit that you believe has problems, and it has not been >> discussed or documented, it's ok to revert it. > > I think there should be two technical things in place: > > One thing is a structured way of rolling back edits. There should be a way > of reporting large scale edits, and getting them removed from the database. > > The second thing is a reporting script that reports on large scale edits in > a timely fashing. This is mostly what the 0.6 API is about. Changesets will report to you what happened, and have space for meta data (which will allow the identification of particular bots/editors). And that'll give a basis for reversion tools to operate. > > As far as politics go, I think that it would a good idea to just re-use the > current structure for introducing new map features. Before you run a script > you first propose it and only run it when enough people cast a vote in favor > of running the script. Step 1) Write OSM bot Step 2) Write OSM Wiki vote rigging bot Step 3) Propose bot Step 4) Rig vote Step 5) Run OSM bot all the while pointing at the wiki shouting "look! 13 people approved it!" I'm not so convinced :-) Dave ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
[..Proposal for bots..] I completly agree. > 2. Make sure your tools and knowledge are good: You have to be able to > revert your changes if something goes wrong, and you need to keep any > collateral damage to an absolute minimum. If you cannot guarantee that, > ask someone for help who can. Is there any "sandbox" available for - testing the reliability - demonstration of before/after to visualize in discussion -jha- ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
I'd suggest a special usertype for batch operations, and combine that with a notification system so as to enable that account for one batch. The usertype should have the username of the uploader with a random number attached (fe cetest to cetest) to distinguish between sessions. The username should be enabled shortly before carrying out the scripts/upload by the server. The uploader could enable his temp account by filling in a web form with sufficient "required fields" I suggest: username bounding box for all changes description of change modified data (list of tags) intention of batch operation The form would return by email the username to be used for this upload. The email ensures that the uploader can be contacted. The notification script could also create a backup of all changes done immediately after, so as to facilitate removal if required. May this is redundant, as date and time are available to select. Normal users should have their number of changes per upload limited to say a few hundreds a time. This could be supported in JOSM and merkaartor by a warning system when 90% of the maximum upload has been reached and the user should upload. Some finetuning of the above may be necessary. Gert Gremmen - Openstreetmap.nl (alias: cetest) Gert Gremmen - Openstreetmap.nl (alias: cetest) -Oorspronkelijk bericht- Van: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Namens Philip Homburg Verzonden: Monday, September 29, 2008 9:53 AM Aan: Talk Openstreetmap Onderwerp: Re: [OSM-talk] Code of conduct for automated (mass-) edits In your letter dated Mon, 29 Sep 2008 01:08:50 +0200 you wrote: > as OpenStreetMap draws more and more sophisticated users, we're > also seeing more scripts or, as they would be called in Wikipedia, > "bots", modifying data. > > 1. Make a plan of what you want to change, and discuss in relevant > forum (usu. mailing list). If there are many objections; drop the > plan. If there are few objections, maybe exempt certain areas or > objects created by certain people in order to respect their > objections. Remember that they can easily change things back again > if you act against their will, so don't even try to play the > superiority card. > > I would also accompany this by the notion that if you see an > automated edit that you believe has problems, and it has not been > discussed or documented, it's ok to revert it. I think there should be two technical things in place: One thing is a structured way of rolling back edits. There should be a way of reporting large scale edits, and getting them removed from the database. The second thing is a reporting script that reports on large scale edits in a timely fashing. As far as politics go, I think that it would a good idea to just re-use the current structure for introducing new map features. Before you run a script you first propose it and only run it when enough people cast a vote in favor of running the script. For example, a good way of completely destroying the JOSM/Validator's duplicate node detection feature is to fill the database with a huge number of aumatically generated duplicate nodes. :-( ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Code of conduct for automated (mass-) edits
In your letter dated Mon, 29 Sep 2008 01:08:50 +0200 you wrote: > as OpenStreetMap draws more and more sophisticated users, we're > also seeing more scripts or, as they would be called in Wikipedia, > "bots", modifying data. > > 1. Make a plan of what you want to change, and discuss in relevant > forum (usu. mailing list). If there are many objections; drop the > plan. If there are few objections, maybe exempt certain areas or > objects created by certain people in order to respect their > objections. Remember that they can easily change things back again > if you act against their will, so don't even try to play the > superiority card. > > I would also accompany this by the notion that if you see an > automated edit that you believe has problems, and it has not been > discussed or documented, it's ok to revert it. I think there should be two technical things in place: One thing is a structured way of rolling back edits. There should be a way of reporting large scale edits, and getting them removed from the database. The second thing is a reporting script that reports on large scale edits in a timely fashing. As far as politics go, I think that it would a good idea to just re-use the current structure for introducing new map features. Before you run a script you first propose it and only run it when enough people cast a vote in favor of running the script. For example, a good way of completely destroying the JOSM/Validator's duplicate node detection feature is to fill the database with a huge number of aumatically generated duplicate nodes. :-( ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
