Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Lester Caine

ThomasB wrote:

Hang on, you've got this completely wrong.
.

Seems what you mean and what you wrote differ somehow

Richard Fairhurst wrote

And no - this isn't intended to hit restoring a single way via P1 (while
it still exists) or whatever.

But I read it so. Also selecting 10 buildings in JOSM and pressing Q would
fall below your proposal (automated geometry fixup) and require me to add
these extra tags.


Oh come on ...
This is the sort of nit picking that is the whole problem. If we have to write 
down rules that define which key press you are allowed to use then I doubt 
anybody would contribute.


Having said that though, that simple 10 building 'tidy up' would have to be 
justified if it is being applied to 10 buildings that were imported from a 
reliable data source? A lot of buildings are not 'square' certainly around here, 
and 'styling' them is as bad as deleting ones that have been tidied by other 
means and replacing them with a 'stylised' import?


I do get the impression that the Cadastre import has it's own rules on how to 
use it, and those are only available in French, which irritates. But it does 
need to follow the generic rules, which simply allow some tracking of where 
detail is sourced, and also the accuracy of that data. We have been getting some 
information on their process, but I still don't understand some of the problems 
they are flagging as reasons for the practices?


I also think that there needs to be a major distinction between BOTS which 
modify the core data, and import processes which are simply mapping a data 
source into a format that can be merged. I think that THIS is another 
misconception that is causing confusion. I would certainly not view manually 
marking up data from a data source layer, adding additional tags, and then 
merging into the core data. As I've been proposing in other threads, carrying a 
uneditable tags with data seems essential these days? In this case the core 
elements that are copied would have a tag identifying the source and having now 
spent some time looking into this I'd even go as far as flagging when someone 
tries to edit detail that IS identified as coming from a trusted source? Or more 
important that it has been processed DURING the base data 'import'. Which is the 
main reason I'm seeing 'layers' as the main path forward here. Cadastre is at 
least two layers, the original raw data, and a processed view of that data from 
which objects are 'imported' into 'osm'. Both of those layers should be 
accessible for all of us to at least view and it's managing that which is the 
first step here?


So in light of the above, I don't recognise the use of 'bot' tags in some of 
these cases. That really is a different matter relating to changing the data? A 
bot that 'squares up all buildings' would certainly not be appropriate, but it 
might well be as part of creating a 'stylised layer' view of the data?


The project IS the data, not the maps, and ALL of the data is important, not 
just a 'current view' of what people think matter for them?


--
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Richard Fairhurst
ThomasB wrote:
 Seems what you mean and what you wrote differ somehow

I'm not sure where you read the extra requirement for discussion or
bureaucracy in what I wrote. Could you clarify?

 But I read it so. Also selecting 10 buildings in JOSM and 
 pressing Q would fall below your proposal (automated 
 geometry fixup) and require me to add these extra tags.

That qualifies as manual drawing actions rather than automated. I was
seeking to address things like xybot's bulk geometry corrections. But if you
have a suggestion for better wording, I'm all ears. :)

cheers
Richard





--
View this message in context: 
http://gis.19327.n5.nabble.com/Proposal-for-import-guidelines-tp5727448p5727567.html
Sent from the General Discussion mailing list archive at Nabble.com.

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Jochen Topf
I think there is a misunderstandig here. You seem to suggest that according to
those new guidelines you are supposed to import the data with one account and
then in a second step fix things with the normal account. This is not the case.
It has been a long-standing policy that (whether you do normal edits or any
kind of import) you are supposed to always leave the map in a good state.

In fact it is the biggest problem with imports that people do the import but
don't integrate the data with whats already there. The French community has
been doing that right all along.

Jochen

On Tue, Sep 25, 2012 at 11:36:51PM +0200, Eric Marsden wrote:
 Date: Tue, 25 Sep 2012 23:36:51 +0200
 From: Eric Marsden eric.mars...@free.fr
 To: talk@openstreetmap.org
 Subject: Re: [OSM-talk] Proposal for import guidelines
 
 Thank you for making this constructive proposal. My feeling is that it
 would constitute a positive change to the current DWG import guidelines,
 which are greatly lacking in subtlety. 
 
 Allow me to point out, and illustrate with the French cadastre case, a
 problem posed by the wish strictly to separate the import component of
 a bulk edit (corrected/checked building geometries) from the
 integration component (resolving conflicts with existing building
 geometries and their tags, improving highway geometries using the high
 resolution cadastre information, etc.). Under the current (French)
 community guidelines for integrating this data, these two steps are
 combined in a single changeset. Separating them would lead to a
 situation where, during the period between these two changesets, the
 database is in an inconsistent state (overlapping buildings, highways
 passing through buildings, etc.).
 
 Whilst this temporary inconsistency in the data is not as severe as it
 would be in a software development project, for instance (the dreaded
 FTBFS), it is rather dirty and could lead to false alerts in error
 checking software.
 
 Whether this data consistency problem is more or less significant than
 the improved tracability of data source copyright that is afforded by
 the proposed import/integration separation is debatable. In my view, the
 costs of the proposed change outweigh its benefits (at least as far as
 the French cadastre situation is concerned -- other bulk edits/imports
 will likely have different tradeoffs). 
 
 -- 
 Eric Marsden
 
 
 ___
 talk mailing list
 talk@openstreetmap.org
 http://lists.openstreetmap.org/listinfo/talk

-- 
Jochen Topf  joc...@remote.org  http://www.remote.org/jochen/  +49-721-388298

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread THEVENON Julien
 De : Lester Caine les...@lsces.co.uk

Hi Lester

 I do get the impression that the Cadastre import has it's own rules on 
how to use it, and those are only available in French, which irritates.
You mean you would appreciate a translation of French Cadastre wiki page ?


 But it does need to follow the generic rules, which simply allow some 
tracking of where detail is sourced, and also the accuracy of that data.
Every objects coming from cadastre have a tag source mentionning cadastre. 
This is mandatory and required by French Cadastre to be integrated in 
OSM


 We have been getting some information on their process, but I still 
don't understand some of the problems they are flagging as reasons for 
the practices?
Could you precise which problems you do not understand so we can clarify ?

 import processes which are simply mapping a data source into a format that 
 can be merged.
Concerning French cadastre the manual work to merged the data is quite huge 
whereas for CLC import it was automatically done. Do you consider that this 
manual work is part of the import process or not ?

 I thinks that THIS is another misconception that is causing confusion.
To avoid confusion and misconception is there somewhere an exhaustive clear and 
preceise list of objectives of Guidelines imports rules ?
It would be good to have such a list to be sure that we discuss about the same 
things and the same goals.
It would also allow to discuss on facts and technical points and perform an 
objective evaluation of strengh and weaknesses of various proposal.
For the moment I`m not sure that everyone has the same vision of what we are 
doing and which goals we want to reach.


 Cadastre is at least two layers, the original raw data, and a processed 
view of that data from which objects are 'imported' into 'osm'. Both of 
those layers should be accessible for all of us to at least view
The original data are available on cadastre website cadastre.gouv.fr
The processed data are available town by town on cadastre.openstreetmap.fr
The data after user manual processing in order to solve issues ( coherency with 
existing OSM data, artefacts not detectable by processing scripts etc ) are the 
one sent to OSM ( Data coming from cadastre avec source flag)


Cheers 

Julien
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Eric Marsden
 jt == Jochen Topf joc...@remote.org writes:

  jt I think there is a misunderstandig here. You seem to suggest that 
according to
  jt those new guidelines you are supposed to import the data with one account 
and
  jt then in a second step fix things with the normal account. This is not the 
case.
  jt It has been a long-standing policy that (whether you do normal edits or 
any
  jt kind of import) you are supposed to always leave the map in a good state.

  This means that the import account will agglomerate contributions
  sourced from the import (cleaned up building outlines, for instance)
  and a large number of ordinary, manual edits/improvements (improving
  the road network's geometric precision, adding tags) which are sourced
  from Bing, GPS traces, personal knowledge, etc. Individual changesets
  will be a mix of data from different sources, with different
  copyrights.

  Since there is no change to the clarity of copyright status, I really
  don't understand the point of a separate account for this type of
  import (obviously country-wide, mechanical CLC-type edits are different). 

-- 
Eric Marsden


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Richard Mann
I think the distinction between mechanical and manual needs to be fleshed
out a bit. To me manual implies a degree of care to other data (relative
location of existing objects, links to other objects, existing versions of
the same or related objects, other tags, consideration of the quality of
the source, cross-referencing to other sources, that sort of thing). The
amount of care depends on what you're doing, but obviously has a tendency
to decline if you're dealing with more data. Telling people that they're
not taking enough care is likely to annoy them. Asking them to
self-describe themselves as a bot when they're taking a lot of care is
also insulting.

So I'd choose some tag names that are a bit less loaded.

But the principle that changesets should have a licence tag where that's
clear/available is a sensible one. As is the message keep your changesets
at human-scale or set up a separate account.

Richard
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Peter Wendorff

Am 26.09.2012 10:30, schrieb THEVENON Julien:

* De :* Lester Caine les...@lsces.co.uk

Hi Lester

* *I do get the impression that the Cadastre import has it's own 
rules on how to use it, and those are only available in French, which 
irritates.

You mean you would appreciate a translation of French Cadastre wiki page ?

* *But it does need to follow the generic rules, which simply 
allow some tracking of where detail is sourced, and also the accuracy 
of that data.
Every objects coming from cadastre have a tag source mentionning 
cadastre. This is mandatory and required by French Cadastre to be 
integrated in OSM


* *We have been getting some information on their process, but I 
still don't understand some of the problems they are flagging as 
reasons for the practices?

Could you precise which problems you do not understand so we can clarify ?

* *import processes which are simply mapping a data source into a 
format that can be merged.
Concerning French cadastre the manual work to merged the data is quite 
huge whereas for CLC import it was automatically done. Do you consider 
that this manual work is part of the import process or not ?
Yes, it is part of the import process, as it's the main preparation of 
the import.
When we import a list of facilities we get from a third party, e.g. 
the fuel station import last year, most of the time the raw data is not 
fitting to osm needs very well.
Most of the time there's some kind of preprocessing, be it manually or 
automatically - and if it's done by a bot that bot usually has to be 
developed, too.

All this is part of the import process IMHO, yes.
* *I thinks that THIS is another misconception that is causing 
confusion.
To avoid confusion and misconception is there somewhere an exhaustive 
clear and preceise list of objectives of Guidelines imports rules ?
It would be good to have such a list to be sure that we discuss about 
the same things and the same goals.
It would also allow to discuss on facts and technical points and 
perform an objective evaluation of strengh and weaknesses of various 
proposal.
For the moment I`m not sure that everyone has the same vision of what 
we are doing and which goals we want to reach.


* *Cadastre is at least two layers, the original raw data, and a 
processed view of that data from which objects are 'imported' into 
'osm'. Both of those layers should be accessible for all of us to at 
least view

The original data are available on cadastre website cadastre.gouv.fr

All versions?
It was said that there are new versions every few years. Do the old, 
then outdated versions stay there?
Apart from that while OSM basically has English as the lingua franca, I 
think it's a bad idea to rely on French only for documentation - inside 
osm as well as for outside documentation like cadastre.gouv.fr.


regards
Peter
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Lester Caine

THEVENON Julien wrote:

* *I do get the impression that the Cadastre import has it's own rules on
how to use it, and those are only available in French, which irritates.
You mean you would appreciate a translation of French Cadastre wiki page ?

That has been a previous request :) The google translation is a little strange.


* *But it does need to follow the generic rules, which simply allow some
tracking of where detail is sourced, and also the accuracy of that data.
Every objects coming from cadastre have a tag source mentionning cadastre. This
is mandatory and required by French Cadastre to be integrated in OSM

And if we can automate that process it would help you?


* *We have been getting some information on their process, but I still don't
understand some of the problems they are flagging as reasons for the practices?
Could you precise which problems you do not understand so we can clarify ?
I'm just repeating things that were being quoted earlier, but I would like to 
have a look at the data myself to see if we are talking about the same things. 
Accessing the French data is difficult for a non French speaker :(



* *import processes which are simply mapping a data source into a format
that can be merged.
Concerning French cadastre the manual work to merged the data is quite huge
whereas for CLC import it was automatically done. Do you consider that this
manual work is part of the import process or not ?

That part of the process simply needs to be identified.
I'm still pushing here for a more robust process that will simplify merging this 
data in future, and the bulk delete that prompted the debate is an example of 
where data HAS already been imported, so the PROCESS to import it needs to be 
addressed ... which is partly being addressed by the layers discussion.
Importing material to work with is the 'automatic' bit, and processing a new 
version of the data to merge with existing data is 'automatic'. See below ...



* *I thinks that THIS is another misconception that is causing confusion.
To avoid confusion and misconception is there somewhere an exhaustive clear and
preceise list of objectives of Guidelines imports rules ?
It would be good to have such a list to be sure that we discuss about the same
things and the same goals.
It would also allow to discuss on facts and technical points and perform an
objective evaluation of strengh and weaknesses of various proposal.
For the moment I`m not sure that everyone has the same vision of what we are
doing and which goals we want to reach.

Totally agree ...
Many of my own requirements have already been addressed via other 'improvements' 
and being able to access a range of historic maps geo-referenced to the base 
'layer' is allowing me to create my own data. But *I* would like to be able to 
combine the simple 'start_date' information into a single layer whch is fine 
where an object still exists, but often nowadays newer 'estates' replace the old 
terraces, or areas bombed out in the past :(



* *Cadastre is at least two layers, the original raw data, and a processed
view of that data from which objects are 'imported' into 'osm'. Both of those
layers should be accessible for all of us to at least view
The original data are available on cadastre website cadastre.gouv.fr
The processed data are available town by town on cadastre.openstreetmap.fr
The data after user manual processing in order to solve issues ( coherency with
existing OSM data, artefacts not detectable by processing scripts etc ) are the
one sent to OSM ( Data coming from cadastre avec source flag.
I can't access the government data as an overlay? The English translation is a 
little light, but I suspect this only give hard copy output? IS the raw data 
available in a manor that we can access directly in a josm or something similar?


The cadastre.openstreetmap.fr seems to a similar problem? I can select a 
Department and then in some cases Ville, but nothing seems to happen?


There is obviously good data available, and all I'm asking is some help in 
viewing it. In the same manor as data is being made usable elsewhere ;) The UK's 
OS data is available as 'layers' we can work with, and it would be nice to see 
the French data similarly available. We have to process the raw OS data into a 
raw format that is correctly geo-referenced, and that raw data is available, so 
I would anticipate that the French data has had similar processing? Is that raw 
data available anywhere? Having got the raw layer(s) we NOW need to process them 
into a 'staging' layer which can be used to integrate later updates - all of 
which should be made available as raw layers - but the staging layer would be 
the one that flags what has or has not been merged. So I'm not sure if 
cadastre.openstreetmap.fr is your 'raw data' layer, in which case there should 
be a 'version select' or your staging layer?


We are missing tools to make all this work, but they are the same tools world 
wide, and we just need 

Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Lester Caine

Richard Mann wrote:

But the principle that changesets should have a licence tag where that's
clear/available is a sensible one. As is the message keep your changesets at
human-scale or set up a separate account.


Tagging the change set is against the data source is a must.
But I think that a simple 'read-only' tag that automatically identifies the base 
data set is something that is making sense to me. On each object, since looking 
down change set history to find it does not make sense?


This would at least allow editors to issue warnings when a change is made to 
detail provided from a 'trusted' source. The 'squaring up' buildings hit a raw 
nerve with me simply because I DO have to take care of the non-squareness of 
many structures around here and it's those sorts of tidies that could cause more 
problems! - even in the French data?


Being able to pull up an overlay of the original source data when a warning is 
created would also be helpful.


--
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Tobias Knerr
On 26.09.2012 10:13, Richard Fairhurst wrote:
 I'm not sure where you read the extra requirement for discussion or
 bureaucracy in what I wrote. Could you clarify?

Discussion and bureaucracy requirements exist for automated/mechanical
edits according to the policy pages you would like to see combined into
one with with this guideline. Thus your attempted definition of
automated edits seems quite relevant.

Also, your proposed text includes the requirement to create and link to
a documentation page, which imo qualifies as bureaucracy:

 bot_url=link to a page describing the automated edit

 But I read it so. Also selecting 10 buildings in JOSM and 
 pressing Q would fall below your proposal (automated 
 geometry fixup) and require me to add these extra tags.
 
 That qualifies as manual drawing actions rather than automated. I was
 seeking to address things like xybot's bulk geometry corrections. But if you
 have a suggestion for better wording, I'm all ears. :)

If you want to address changes performed by scripts/bots, then why don't
you just say so explicitly and avoid any potential misunderstandings?

==

An 'automated edit' is one where the editing is not carried out by
manual drawing actions. This includes (but is not limited to):

- imports of external data without inspection of individual objects
- any changes performed by a script/bot

==

Of course there are special cases where e.g. a powerful editor is used
to blindly do the exact same thing a script would do, but things like
these are what the not limited to is for.

As for reverts, I wouldn't mention them here at all. There should be
guidelines for them, yes, but they should be looked at separately. Some
of the concepts related to reverts (such as contacting the original
changeset's author personally first, or even dealing with explicit
revert requests by said author) don't really exist elsewhere.

Tobias

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Richard Fairhurst
Tordanik wrote:
 If you want to address changes performed by scripts/bots, then  
 why don't you just say so explicitly and avoid any potential 
 misunderstandings?

Because it's not just about scripts and bots. The Cadastre situation, which
started all of this off, is often people loading .osm files into JOSM,
running a quick validator check over it, and uploading. In terms of impact
on the map and on the community, there is no significant difference between
this and the same operation using upload.py.

(On a matter of language: if you want to... then why don't you just say
so? comes across as really quite hostile in English. I won't assume that
it's meant as such, as I recognise that English isn't everyone's first
language on this list. However, this is intended as a constructive
suggestion to solve an impasse which we've reached and a rather less hostile
tone would be nice. It doesn't actually make any difference to me personally
- I only _use_ OSM data for the UK, where we don't have imports, and I'm not
on DWG so I don't have to deal with the angry mails. I'm simply trying to
help and getting hostile doesn't really encourage that.)

Richard





--
View this message in context: 
http://gis.19327.n5.nabble.com/Proposal-for-import-guidelines-tp5727448p5727607.html
Sent from the General Discussion mailing list archive at Nabble.com.

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread THEVENON Julien

 De : Peter Wendorff wendo...@uni-paderborn.de

 Yes, it is part of the import process, as it's the main preparation of the 
 import.
  When we import a list of facilities we get from a third party, e.g. the 
 fuel station import last year, most of the time the raw data is not 
 fitting to osm needs very well.
  Most of the time there's some kind of preprocessing, be it manually or 
 automatically - and if it's done by a bot that bot usually has to be 
 developed, too.
  All this is part of the import process IMHO, yes

Ok, in our case this is done manually during the merge.

 All versions?

No. On cadastre.gouv.fr you have the current version

 It was said that there are new versions every few years. Do the old, then 
 outdated versions stay there?
On cadastre.openstreetmap.fr data are regenerated periodically or on demand

  Apart from that while OSM basically has English as the lingua franca, I 
 think it's a bad idea to rely on French only for documentation - inside 
 osm as well as for outside documentation like cadastre.gouv.fr.
I guess this at been written in french only because it was targeting french 
communauty for OSM documentation

Cheers 

Julien___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Lester Caine

Richard Fairhurst wrote:

It doesn't actually make any difference to me personally
- I only_use_  OSM data for the UK, where we don't have imports

Yet ;)
I want to get the border layer stuff working directly from the import rather 
than 'importing' it piecemeal into the base data ...


--
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread THEVENON Julien
 De : Richard Fairhurst rich...@systemed.net


Because it's not just about scripts and bots. The Cadastre situation, which
started all of this off, is often people loading .osm files into JOSM,
running a quick validator check over it, and uploading. In terms of impact
on the map and on the community, there is no significant difference between
this and the same operation using upload.py.

What you mention is normally and exception.
The french wiki page about cadastre building integration explicitely mention 
that data should be checked carrefully and merged with the existing to keep 
database coherency and quality.
The french community has also some tools to detect people doing what you 
mention and often perform a revert of the changeset, contact the contributor to 
explain that there are some quality requirements.
Clean cadastre integration is a process that take quite a long time when done 
correctly and that could not be automated, that's why it has been decided to 
not perform a national automated import like CLC but rather to rely on 
contributors which do that city by city

Cheers
Julien
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Jason Cunningham
On 26 September 2012 11:02, Richard Fairhurst rich...@systemed.net wrote:

 - I only _use_ OSM data for the UK, where we don't have imports, and I'm
 not
 on DWG so I don't have to deal with the angry mails. I'm simply trying to
 help and getting hostile doesn't really encourage that.)

 Richard


I was typing up a response when your email came through. I was based around
the numerous imports in the UK of Ordnance Survey VectorMapDistrict Data. I
guess I'm using a broad definition of import

In the UK we have available data from Ordnance Survey in Rasta and Vector
format. There is a wiki page on how we should use the data with a
requirement we should add a source tag.
http://wiki.openstreetmap.org/wiki/Ordnance_Survey_Opendata
For me the main use of the vector data is adding rivers, water bodies,
coastlines, and boundaries. For these types the Ordnance Survey data is
commonly the best source, but I'd argue strongly it still needs checking. I
read the initial proposal as meaning that if I added a 3 ponds by tracing
over rasta image availabe in Potlatch or JOSM it would not be considered a
manual drawn action, but if copied the vector data then it would fall
under automated edit because it would be an imports of external data.
It would not be a bulk edit but would still require several actions I'd
consider to be over the top (eg adding bot= and bot_url=)

Subsequent discussion suggests that the addition of this vector data would
not be considered an 'automated edit', but I think it would help to make
this clearer. I prefer the wording used by Tobias Knerr

On 26 September 2012 10:51, Tobias Knerr o...@tobias-knerr.de wrote:


 An 'automated edit' is one where the editing is not carried out by
 manual drawing actions. This includes (but is not limited to):

 - imports of external data without inspection of individual objects
 - any changes performed by a script/bot

 Of course there are special cases where e.g. a powerful editor is used
 to blindly do the exact same thing a script would do, but things like
 these are what the not limited to is for.



I'd suggest changing 'automated edit' with something along the lines of
'blind import', and defining it based around lack of inspection of impact
of individual objects. I'd slightly change one of the lines to
-  imports of external data without inspection of individual objects, or
consideration of impact on existing surrounding objects.

Jason
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Paul Norman
 From: Richard Fairhurst [mailto:rich...@systemed.net]
 Sent: Wednesday, September 26, 2012 3:02 AM
 To: talk@openstreetmap.org
 Subject: Re: [OSM-talk] Proposal for import guidelines
 
 Tordanik wrote:
  If you want to address changes performed by scripts/bots, then why
  don't you just say so explicitly and avoid any potential
  misunderstandings?
 
 Because it's not just about scripts and bots. 

My brief thoughts on what would fall on the mechanical side vs not

Mechanical:
Using (j)xapi to download all post boxes in a city without operator=* and
adding operator=* to them
Changing all McDonalds to McDonald's
Mass-fixing up import tagging
Anything done with absolutely no user intervention (e.g. xybot)

Not:
Using (j)xapi to download all post boxes in a city that you added without
operator=* because you realized they were all one operator and wanted to go
back to fix your earlier surveyed data.

In practice most mechanical edits fall extremely far on the side of being a
mechanical edit so even if it's a somewhat fuzzy line most will clearly be
on one side or the other.


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread THEVENON Julien
 De : Lester Caine les...@lsces.co.uk

 That has been a previous request :) The google translation is a little 
 strange.
OK I was not aware about that



 And if we can automate that process it would help you?
The add of source tag is already automatic


 I'm just repeating things that were being quoted earlier, but I would 
like to have a look at the data myself to see if we are talking about 
the same things. Accessing the French data is difficult for a non French 
speaker :(
Ok. So by exemple to see official data goto on www.cadastre.gouv.fr
In Commune field type Saint Galmier
In Departement liste choose 42 - LOIRE
Then click on rechercher button

In Ville commune list choose Saint Galmier 42330  ( I know this is 
painfull )
Click on Rechercher ( bottom right )
Click on Vue d ensemble de la commune.
Normally you should have a pop up window that display cadastre data. You can 
zoom/unzoom on it using left panel to make details like river, building, 
streets appear.
This is for the official data.

The corresponding processed data that you can find on 
http://cadastre.openstreetmap.fr/data/ are those data downloaded in PDF format 
processed by a C++ script that analyse geometrical forms and colours to extract 
buildings, railways, rivers and produced separated OSM files.
You have one directory per french department ( in the case of Saint Galmier 
this is number 42 named LOIRE ) containing all the OSM files of cities 
belonging to this department. In my example case the data corresponding to what 
you have seen on cadastre.gouv.fr are located in these files:
http://cadastre.openstreetmap.fr/data/042/I1222-SAINT-GALMIER-city-limit.osm
http://cadastre.openstreetmap.fr/data/042/I1222-SAINT-GALMIER-rails.osm
http://cadastre.openstreetmap.fr/data/042/I1222-SAINT-GALMIER-water.osm
http://cadastre.openstreetmap.fr/data/042/I1222-SAINT-GALMIER.tar.bz2

The french wiki cadastre page provide a big warning aout poor quality of water 
data and the fact that we need to perform some verifications and cleanup of all 
these data ( building, railways, river etc ) has the analyse is not perfect.


 I can't access the government data as an overlay? The English 
translation is a little light, but I suspect this only give hard copy 
output? IS the raw data available in a manor that we can access directly in a 
josm or something similar?

You can have cadastre has overlay in JOSM using french cadastre plugin.
If you want to perform the test on Saint Galmier you will have to install the 
plugin in JOSM, restart JOSM. Change the projection to Lambert 9 zones and 
choose Lambert CC zone 5, restart JOSM.

In cadastre preferences ( just behind plugin part of preferences dialog window 
) choose vector Grab Multplier 100m thanks to the radio button.
Then in JOSM Cadastre Menu choose Cadastre grab
In the dialog box type SAINT-GLAMIER in Commune field. In department list 
choose 42 - loire. Click on OK and normally you should slowly have a new 
overlay representing french cadastre data. Notice that their are coming in 
picture format.

I'm not sure about if this is possible to do the same with Merkaartor


 The cadastre.openstreetmap.fr seems to a similar problem? I can select a 
 Department and then in some cases Ville, but nothing seems to happen?
Please refer to explainations at the beginning of my email and let me know if 
you are still facing issues


 There is obviously good data available, and all I'm asking is some help 
in viewing it. In the same manor as data is being made usable elsewhere 
;)
 The UK's OS data is available as 'layers' we can work with, and it 
would be nice to see the French data similarly available.
 We have to 
process the raw OS data into a raw format that is correctly 
geo-referenced, and that raw data is available, so I would anticipate 
that the French data has had similar processing?
 Is that raw data 
available anywhere? Having got the raw layer(s) we NOW need to process 
them into a 'staging' layer which can be used to integrate later updates - all 
of which should be made available as raw layers - but the staging layer would 
be the one that flags what has or has not been merged. 

 So 
I'm not sure if cadastre.openstreetmap.fr is your 'raw data' layer, in 
which case there should be a 'version select' or your staging layer?

The georeferencing is already there for vectorised data of cadastre ( for 
raster cadastre data this is much more complex than the example I provide to 
you and we hae no automatic tools at all to 'integrate' them).
We have currently no historic versionning.
For the other points I`m not sure to understand what you mean so I hope that 
the explaination on how to access to official cadastre data and processed data 
will make things more clear

Cheers
Julien
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Lester Caine

THEVENON Julien wrote:

Clean cadastre integration is a process that take quite a long time when done
correctly and that could not be automated, that's why it has been decided to not
perform a national automated import like CLC but rather to rely on contributors
which do that city by city


But I'm still not clear if that is done of a properly geo-referenced 
overlay/layer? The initial automatic process would be creating that layer 
although I would accept that keeping historic versions is something that could 
be a cost that nobody will cover?


How is the cadastre.openstreetmap.fr data structured? It sounds as if this IS 
the base import of the cadastre data? So what is missing is a 'staging' layer, 
which identifies what has been imported. I presume that the current view is that 
it's this 'automation' that is not practical yet? But the 'extra' tools being 
provided simply allow large blocks of raw data to be copied over once they have 
been identified as building or what ever?


--
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread THEVENON Julien
 De : THEVENON Julien julien_theve...@yahoo.fr

 You can have cadastre has overlay in JOSM using french cadastre plugin.
 If you want to perform the test on Saint Galmier you will have to install 
the plugin in JOSM, restart JOSM. Change the projection to Lambert 9 
zones and choose Lambert CC zone 5, restart JOSM.

 In cadastre preferences ( just behind plugin part of preferences dialog 
window ) choose vector Grab Multplier 100m thanks to the radio button.

Sorry I forgot one step. It is mandatory to first get some OSM data because the 
cadastre plugin rely on current JOSM bbox to perform request on Official french 
cadastre servers so
Download some OSM data in Saint Galmier region ( to have an idea where it is 
located 
http://www.openstreetmap.org/?lat=45.588947lon=4.315972zoom=18layers=M  ) 

 Then in JOSM Cadastre Menu choose Cadastre grab
 In the dialog box type SAINT-GLAMIER in Commune field. In department 
list choose 42 - loire. Click on OK and normally you should slowly 
have a new overlay 
 representing french cadastre data. Notice that their are 
coming in picture format.
 I'm not sure about if this is possible to do the same with Merkaartor

Cheers
Julien
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread THEVENON Julien

 De : Lester Caine les...@lsces.co.uk

But I'm still not clear if that is done of a properly geo-referenced 
overlay/layer? The initial automatic process would be creating that 
layer although I would accept that keeping historic versions is 
something that could be a cost that nobody will cover?

yes the automatic process create a properly geo-referenced OSM file available 
on cadastre.openstreetmap.fr



How is the cadastre.openstreetmap.fr data structured?
Please refer to my previous email ( 13:07 ) if you have not read it at the time 
you wrote this email. I you need more details about it please quote the part of 
text which is not clear, it will be easier for me to answer ;-)


It sounds as if this IS the 
base import of the cadastre data? So what is missing is a 'staging' 
layer, which identifies what has been imported. I presume that the 
current view is that it's this 'automation' that is not practical yet?
There are such tools for administrative boundaries of cities but not for 
buildings. If I remember well nobody mentionned this kind of need up to now has 
the process is manual normally user should perform the import if data are 
already there.

But the 'extra' tools being provided simply allow large blocks of raw 
data to be copied over once they have been identified as building or 
what ever?No the problem with automated tool is that generated building data 
are sometimes artificially splitted due to the fact that cadastre landuse is 
splitted according to landuse ownership whereas in reality building is not 
splitted. You also have some case where OSM data has been draw on top of Bing 
that was not precisely georeference compared to cadastre so you need to adjust 
data. Sometimes water coming from cadastre doesn't exist in real life so that's 
why we need to perform manual check


Cheers
Julien___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread Lester Caine

THEVENON Julien wrote:

The corresponding processed data that you can find on
http://cadastre.openstreetmap.fr/data/ are those data downloaded in PDF format
processed by a C++ script that analyse geometrical forms and colours to extract
buildings, railways, rivers and produced separated OSM files.


Sees the light :)

SO while we have this type of raster data from as a background in potlatch and 
josm and some elements of it in vector files from OS and other sources. You are 
having to stitch together 'pictures' and then your 'automatic processing' is 
recreating vector data from the 'pictures'?


I understand the problems now ... and it only really works because the pictures 
can be simply vectorised.


SO I would be asking if the 'pictures' can be merged to create a single raster 
overlay for France to use as a background 'source' which could potentially be 
used to trace from, but can be used in conjunction with BING imagery to corss 
check? I would classify that as my base import since it's not externally 
available? We have several versions of the OS data along with the historic maps 
for the UK, and I feel sure that should be achievable in France as well?


The vectorised files are the 'staging layer' and I'm sure that since you are 
providing each building as a shape then in the future it should be possible to 
maintain a 'building_id' that can be used with the sort of merge tools I am 
looking for to handle this type of data?


--
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-26 Thread THEVENON Julien

 De : Lester Caine les...@lsces.co.uk

 Sees the light :)
Great !


 SO while we have this type of raster data from as a background in 
potlatch and josm and some elements of it in vector files from OS and 
other sources. You are having to stitch together 'pictures' and then 
your 'automatic processing' is recreating vector data from the 
'pictures'?

I don`t know all details but yes this is something like that


 I understand the problems now ... and it only really works because the 
 pictures can be simply vectorised.

yes has pdf is vectorised but if i rember well what you see as a simple yellow 
rectangle is an overlay of different rectangle inside pdf code explaining 
partly why we have some geomtry problems with contigous buildings.


 SO I would be asking if the 'pictures' can be merged to create a single 
raster overlay for France to use as a background 'source' which could 
potentially be used to trace from, but can be used in conjunction with 
BING imagery to corss check?
The French cadastre licence doesn't allow us to redistribute cadastre data as 
they are so I think it is not legally possible to do that.
The other issue is related to projection : Mercator for Bing and Lambert 9 
zones for French cadastre. In addition to that not all the french cities have 
vectorised cadastre data. There are still a lot of cities which have raster 
data that are just scan of cadaster paper plan without georeferencing ( Feurs 
in 42-Loire by example )


 I would classify that as my base import 
since it's not externally available? We have several versions of the OS 
data along with the historic maps for the UK, and I feel sure that 
should be achievable in France as well?
I don't know if some people are interested in historic maps or such kind of map 
aspects in french community

 The vectorised files are 
the 'staging layer' and I'm sure that since you are providing each 
building as a shape then in the future it should be possible to maintain a 
'building_id' that can be used with the sort of merge tools I am 
looking for to handle this type of data?
We have a tool called bati fusion that try to perfom a diff between OSM files 
to make merging easier by using building shape comparison I think but I don`t 
know if it is massively used. I personnally know the guy who developped it if 
you are interested

Cheers
Julien___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-25 Thread Petr Morávek [Xificurk]
Richard Fairhurst wrote:
 bot_source_licence=machine-readable licence name
...
 - encouraging a machine-readable licence tag helps to avoid the issues
 identifying changesets that were encountered in the redaction.

I don't like the name of this tag, it seems ambiguous.

From it's name I would guess that it's the license for the source code
of the bot itself, but your description suggest it's meant for the data.
In that case I would suggest using simple source_license (or
source:license ?) or something similar, the bottom line is that the word
bot should not be used.

Best regards,
Petr Morávek aka Xificurk

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-25 Thread Christian Quest
I think the additional tags on the changeset are a good approach...
and when used properly they make the dedicated account useless
(whatever the size of the changeset) as they provide much more
details.

The API could even reject changesets that are above a given size if
these tags are not present.

The tag names should not be too much bot related.

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-25 Thread Jochen Topf
On Tue, Sep 25, 2012 at 06:11:35PM +0100, Richard Fairhurst wrote:
 [...]
 An 'automated edit' is one where the editing is not carried out by
 manual drawing actions. This includes (but is not limited to):
 
 - imports of external data
 - search-and-replace tag changes
 - automated geometry fixup
 - reverting edits
 
 and applies equally to scripted edits and to those carried out
 within an editor program.
 [...]

What exactly is meant with reverting edits? Bringing back deleted objects in
Potlatch is something I have done during normal OSM editing. Thats certainly
not an automated edit.

I think it is rather difficult do exactly define what an automated edit is
and what not. And trying to define this better and better is just an invitation
to language lawyers to argue about minutiae.

I suggest we have an example section in this policy document that describes
several common cases that are and several that aren't automated edits. Thats
easier for most people to understand than complex rules.

And the policy should have a clear guideline on what to do if you think your
case falls into the grey area between those cases. Something like this: If you
are unsure whether something you want to do falls under this 'automated edit'
guideline we encourage you to discuss your case on the mailing list at ... or
... . If you can not get the information you need from there you can also
contact the Data Working Group at ... 

Jochen
-- 
Jochen Topf  joc...@remote.org  http://www.remote.org/jochen/  +49-721-388298

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-25 Thread Jean-Marc Liotier
I like this proposal - from my very personal point of view it safeguards
all the conflicting interests and reaffirms essential inflexible
principles while cutting some slack to users who perform small local
imports :

The bot=yes tag identifies the import as such, to help moderators
focus on that class of potentially widely damaging changes.

bot_url=link to a page describing the automated edit provides all the
necessary context about the import, including quality and methodology
issues specific to the source of data.

The bot_source_licence tag clarifies the license status of the source
at that point in time.

The specific conditions (imports of more than a given number of nodes,
continuously running scripts, edits affecting more than one country) for
changesets for which a separate account is necessary are clear and
non-equivocal, reaffirming the current requirement for a separate
account while letting the users of occasional small-scale imports at a
local level perform them with their personal account.

The need to keep these conditions open-ended is a weakness that lets
detractors claim that they are arbitrary, but I'm guessing that this is
necessary to prevent users gaming the rules with stupid technical
loopholes... Not quite transparent but practical.

This proposal hits all the goals I have seen stated so far... Or are
there others that are not satisfied by this proposal ?

On the French list, some contributors are complaining that the
changeset-level tagging makes the separate account requirement entirely
obsolete. Technically, I believe they are right... But I hope they'll
see that this proposal could be a fair meeting ground for an opportunity
to improve the import process with better metadata and make it more
flexible where necessary while not messing too much with the current
international consensus.






On 09/25/2012 07:11 PM, Richard Fairhurst wrote:
 A propos of the recent contretemps about Cadastre imports and separate
 accounts (excessive use of French in this sentence is unintentional),
 I'd like to propose the following modification to the import/bulk edit
 guidelines:

 ==

 An 'automated edit' is one where the editing is not carried out by
 manual drawing actions. This includes (but is not limited to):

 - imports of external data
 - search-and-replace tag changes
 - automated geometry fixup
 - reverting edits

 and applies equally to scripted edits and to those carried out within
 an editor program.

 All changesets including automated edits MUST have the following
 additional tags:

 bot=yes
 bot_url=link to a page describing the automated edit

 Users are also encouraged to add these tags:

 bot_type=machine-readable description of the edit type
 bot_source_licence=machine-readable licence name

 For example, bot_type=import, bot_source_licence=public_domain; or
 bot_type=revert.

 The tags should be added to the changeset, not the individual objects.
 Authors of software facilitating such edits (e.g. editor plugins)
 should provide relevant tags as a default.


 In addition, all automated edits of a high-volume, sustained or
 continuous nature MUST also be carried out from a separate OSM
 account. This includes (but is not limited to):

 - large-scale imports (for example, 20,000 nodes or greater)
 - continuously running scripts
 - edits affecting more than one country


 Like all other mappers, authors of automated edits must monitor the
 OSM inbox for any accounts they use, and be prepared to respond to
 messages and queries about their edits.

 We recognise that complying with this rule may seem onerous, but we
 would remind authors of automated edits that with great power comes
 great responsibility. OpenStreetMap's value, and differentiation from
 other data providers, comes from the local knowledge, skill and
 enthusiasm of its community, rather than from simply agglomerating
 data available elsewhere. These guidelines are designed to retain
 visibility of automated edits and thereby safeguard our most precious
 resource.

 ==

 (end of proposed text)

 I hope you can see the intentions behind this proposal, but in essence:

 - requiring particular tags makes visibility easier, so that DWG et al
 have a better view of automated edits;
 - it also helps to spread awareness of automated edits through the
 community, since these edits can be easily visualised by client
 software - thereby bringing many eyeballs to the edits;
 - encouraging a machine-readable licence tag helps to avoid the issues
 identifying changesets that were encountered in the redaction.

 A brief clarification on this message: This is a personal posting. I
 have already proposed to the OSMF board that the three similar sets of
 guidelines on the wiki (imports, automated edits, mechanical edits) be
 combined into one, and that the result is endorsed as an OSMF policy.
 If this suggestion is received reasonably positively, then I'll bring
 it forward for incorporation into such a policy.

 

Re: [OSM-talk] Proposal for import guidelines

2012-09-25 Thread Lester Caine

Jochen Topf wrote:

I think it is rather difficult do exactly define what an automated edit is
and what not. And trying to define this better and better is just an invitation
to language lawyers to argue about minutiae.


I've deliberately take this out of context because I'm beginning to see plan 
coming together - at least in my shrinking brain.


The starting point is the discussion on 'Multiple Layers' I'd like to propose 
that every import is made available as a complete geo-referenced that we can all 
select and view. Layers will all be date stamped, so that we can select 
particular point in time. Registering the layer will also record all of the 
licensing details and where the material has come form. Along with documenting 
the steps taken to process the source into the correct format and alignment.


This will give us a 'layer number' which will be used as part of any unique 
object ID's and when merging that data into other layers, the 'source' tag 
simply contains the layer number - automatically.


I am seeing tools in qgis to run diffs between layers? But basically when a new 
version of an import arrives we can copy the conversion details from the 
existing version, and generate a new layer. Then diff tools allow easy 
identification of new elements that can simply be imported into a 'staging 
layer' ... HOW that is imported is something that needs to be fine tuned, but 
potentially could simply be an automatic import? But all of this is 'automated 
edit' process.


Since the original source element can be identified, MANUAL edits can be 
referenced back. And deletes simply tag 'end_date=xxx' so information is still 
accessible. However I would anticipate that the manual processing is simply 
grouping and identifying ways from the source layer and tagging each created 
object with a source of the layer number and either an inherited id or one 
generated within the staging layer. At this stage I'm sure that 'source' layer 
should be read only, but there should be an additional object table which 
provides a cross reference to any merged data?


I'm sure that we do not need to break things down as much as 
http://wiki.openstreetmap.org/wiki/OSM_Layers proposes, since selecting a view 
of the data that just has a particular sub set of objects is not difficult, but 
I can see the advantage of secondary caches of data in a layer framework.


--
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-25 Thread Paul Norman
My main machine is down at the moment so this isn't as detailed as I'd like, but I have a few thoughts.On Sep 25, 2012, at 10:11 AM, Richard Fairhurst rich...@systemed.net wrote:A propos of the recent contretemps about Cadastre imports and separate  accounts (excessive use of French in this sentence is unintentional),  I'd like to propose the following modification to the import/bulk edit  guidelines:  ==  An 'automated edit' is one where the editing is not carried out by  manual drawing actions. This includes (but is not limited to):  - imports of external data - search-and-replace tag changes - automated geometry fixup - reverting edits  and applies equally to scripted edits and to those carried out within an  editor program.  All changesets including automated edits MUST have the following  additional tags:  bot=yes bot_url=link to a page describing the automated edit  Users are also encouraged to add these tags:  bot_type=machine-readable description of the edit type bot_source_licence=machine-readable licence name  For example, bot_type=import, bot_source_licence=public_domain; or  bot_type=revert.  The tags should be added to the changeset, not the individual objects.  Authors of software facilitating such edits (e.g. editor plugins) should  provide relevant tags as a default.I'd like to see a standardized tag to indicate reverted changesets. The redaction bot usedredacted_changesets=*, perhaps reverted_changesets=cs1;cs2;cs3 (etc) could be used as a standard way to indicate changesetsIn addition, all automated edits of a high-volume, sustained or  continuous nature MUST also be carried out from a separate OSM account.  This includes (but is not limited to):  - large-scale imports (for example, 20,000 nodes or greater) - continuously running scripts - edits affecting more than one countryAlthough I agree that these should or do require a separate account, I wouldn't classify large-scale imports as automated edits, I'd classify them both as types of bulk edits.I see bulk edits as falling into two groups- Mechanical edits, some of which would be automated edits- ImportsIn theory you can have edits which blur the boundaries (e.g. recent Czech edits) but in practice these are infrequent. Most bulk edits clearly fall into one group or another.Like all other mappers, authors of automated edits must monitor the OSM  inbox for any accounts they use, and be prepared to respond to messages  and queries about their edits.Speaking as someone who both maintains multiple (four) accounts and frequently has to contact separate accounts, I prefer a link to the person's main account, either to /user/name or /message/user/new. In theory all messages sent to any of my accounts result in an email to my main account but in practice I've found these sometimes get routed to spam. I regularly check my main account but I normally only check the others on demand. Even if I was checking these daily my main account has more detailed contact information and information on how to get in touch with me quickly.We recognise that complying with this rule may seem onerous, but we  would remind authors of automated edits that "with great power comes  great responsibility". OpenStreetMap's value, and differentiation from  other data providers, comes from the local knowledge, skill and  enthusiasm of its community, rather than from simply agglomerating data  available elsewhere. These guidelines are designed to retain visibility  of automated edits and thereby safeguard our most precious resource.  ==  (end of proposed text)  I hope you can see the intentions behind this proposal, but in essence:  - requiring particular tags makes visibility easier, so that DWG et al  have a better view of automated edits; - it also helps to spread awareness of automated edits through the  community, since these edits can be easily visualised by client software  - thereby bringing "many eyeballs" to the edits; - encouraging a machine-readable licence tag helps to avoid the issues  identifying changesets that were encountered in the redaction.  A brief clarification on this message: This is a personal posting. I  have already proposed to the OSMF board that the three similar sets of  guidelines on the wiki (imports, automated edits, mechanical edits) be  combined into one, and that the result is endorsed as an OSMF policy. If  this suggestion is received reasonably positively, then I'll bring it  forward for incorporation into such a policy.  I would welcome your comments. :)___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-25 Thread Tobias Knerr
I'm worried about the ongoing push to extend the reach of rules
originally designed (and supported by the community) for imports and
scripts to actions initiated by human mappers using editor software.

Even though your mail's subject mentions import guidelines, your
proposed text switches to the much wider term automated edit and
includes such things as ...

On 25.09.2012 19:11, Richard Fairhurst wrote:
 - search-and-replace tag changes
 - automated geometry fixup
 - reverting edits

In my opinion, none of that (if performed though editing software on a
moderate amount of data) is something that should require the same
amount of discussion and bureaucracy as a country-wide import.

These are simply different things than imports and scripts, should be
considered separately, and there should be much lower barriers for
performing these actions.

Tobias

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-25 Thread Eric Marsden
Thank you for making this constructive proposal. My feeling is that it
would constitute a positive change to the current DWG import guidelines,
which are greatly lacking in subtlety. 

Allow me to point out, and illustrate with the French cadastre case, a
problem posed by the wish strictly to separate the import component of
a bulk edit (corrected/checked building geometries) from the
integration component (resolving conflicts with existing building
geometries and their tags, improving highway geometries using the high
resolution cadastre information, etc.). Under the current (French)
community guidelines for integrating this data, these two steps are
combined in a single changeset. Separating them would lead to a
situation where, during the period between these two changesets, the
database is in an inconsistent state (overlapping buildings, highways
passing through buildings, etc.).

Whilst this temporary inconsistency in the data is not as severe as it
would be in a software development project, for instance (the dreaded
FTBFS), it is rather dirty and could lead to false alerts in error
checking software.

Whether this data consistency problem is more or less significant than
the improved tracability of data source copyright that is afforded by
the proposed import/integration separation is debatable. In my view, the
costs of the proposed change outweigh its benefits (at least as far as
the French cadastre situation is concerned -- other bulk edits/imports
will likely have different tradeoffs). 

-- 
Eric Marsden


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-25 Thread Richard Fairhurst
I'm off to bed but would just like to respond to this one before I do.

Tordanik wrote:
 On 25.09.2012 19:11, Richard Fairhurst wrote:
  - search-and-replace tag changes
  - automated geometry fixup
  - reverting edits

 In my opinion, none of that (if performed though editing software 
 on a moderate amount of data) is something that should require 
 the same amount of discussion and bureaucracy as a country-
 wide import.

Hang on, you've got this completely wrong.

There is no extra discussion involved in this proposal. No extra
bureaucracy. None. This proposal is _purely_ about how edits (that are
already happening) are flagged up.

The proposal is just to add two extra tags, on the changeset, that permit
extra visibility. It's not much. I run a revert bot
(http://www.openstreetmap.org/user/General%20Dreedle) and would be very
happy to add one line of Perl to add these tags and thereby flag up this is
an automated edit. It doesn't seem onerous to me.

And no - this isn't intended to hit restoring a single way via P1 (while it
still exists) or whatever. Though I have to admit I'm rather flattered that
Jochen has admitted to using Potlatch. ;)

cheers
Richard





--
View this message in context: 
http://gis.19327.n5.nabble.com/Proposal-for-import-guidelines-tp5727448p5727548.html
Sent from the General Discussion mailing list archive at Nabble.com.

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


Re: [OSM-talk] Proposal for import guidelines

2012-09-25 Thread ThomasB
Richard Fairhurst wrote
 Hang on, you've got this completely wrong.
 .

Seems what you mean and what you wrote differ somehow


Richard Fairhurst wrote
 And no - this isn't intended to hit restoring a single way via P1 (while
 it still exists) or whatever. 

But I read it so. Also selecting 10 buildings in JOSM and pressing Q would
fall below your proposal (automated geometry fixup) and require me to add
these extra tags. 




--
View this message in context: 
http://gis.19327.n5.nabble.com/Proposal-for-import-guidelines-tp5727448p5727552.html
Sent from the General Discussion mailing list archive at Nabble.com.

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk