John,
I'm sorry to keep saying this, but I really do not think this is an
acceptable import approval process.
You're saying there was no wiki describing the plan when this went to
the imports mailing list - only a link to a similar plan with related
data. You did not follow the import guidelines and you need to go back
and read that page line by line and follow the procedures that we have
in place.
https://wiki.openstreetmap.org/wiki/Import/Guidelines
I'll go ahead and add a mention of this plan to the imports catalogue to
get us started. I'll also add some sections to the wiki and try to leave
some indication of where things can be better documented.
You may think I'm quibbling over procedural details, but I think this
process is really important. If we were talking about importing
buildings in one neighborhood, I would look the other way, but this is
all of Canada. This is a huge, huge import and we need to take the time
to do things right, and especially to document the process so people can
get involved that aren't already.
Best,
Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban Planning
NateWessel.com <http://natewessel.com>
On 1/18/19 3:48 PM, John Whelan wrote:
The import mailing list was pointed to the correct page of the wiki.
The initial post was to say this is what we were thinking of and there
was a comment saying we needed to change the comment line.
>There is no mention of this proposed import on the import catalogue
The import process was reviewed by the person who set up the Ottawa
import did we miss that step on the Ottawa import as well? Neither
was it raised as a concern on the import mailing list. I think this is
very minor and can be corrected.
We learnt a fair bit on the Ottawa import and my expectation is since
we are using experienced mappers to do the import conflation would be
either handled by them or the building not imported. We aren't using
new mappers in a mapathon here and with experienced mappers then I
think you have to trust them. The world isn't perfect. Think in terms
of service level.
>There are 2X more nodes than needed to represent the building accurately.
The problem with correcting this is you are introducing
approximations. This will vary according to the source and this can
be simplified or corrected once its in OSM. I think this is a
different issue of a mechanical edit that needs to be considered
separately.
If we are concerned with database size then I suggest we change the
instructions to say put the source comment on the change set rather
than on the building outline.
Cheerio John
Nate Wessel wrote on 2019-01-18 3:06 PM:
John,
You seem to be playing the long game with this data - it sounds like
you've been working with this a lot longer than I have, and you've
put in the time and effort to help make this
actually-quite-incredible dataset available to us. I don't want to
stop the import from happening - quite the opposite. I just want to
make sure that the time is taken to do this right. OSM deserves that.
Your (our) long awaited victory will be the sweeter for our patience
now.
There are several specific issues I see where the I's are not
crossed, nor the t's dotted. I've mentioned several already, so I'll
try to be brief (I really need to get back to working on my
dissertation).
1) There was extremely limited discussion on the imports mailing
list. The initial email did not make clear the scope of the project.
I read the email and did not think twice at it, thinking it was
entirely about Ottawa. The link in that email was actually to the
Ottawa import, and not this one, which seems to have been only in
draft at the time.
https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
As such, this project has NOT been reviewed by the imports list,
which is a requirement for proceeding with the import.
2) There is no mention of this proposed import on the import
catalogue (https://wiki.openstreetmap.org/wiki/Import/Catalogue)
which is required in the imports guidelines. I suspect many other
guidelines have not been followed.
3) The wiki page describing the import is not adequate to assess the
quality of the data or of the proposed import. See for example:
https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks
The import guidelines call for a description of how conflation will
be handled. The fact that two of the major importers seem to have a
substantial disagreement about how to handle existing data indicates
this was not well discussed and I can see that it isn't well documented.
4) The buildings need to be simplified, quite a bit actually. Most
buildings have multiple nodes representing straight lines. This
bloats the database and makes things harder to edit by hand later.
There are probably 2x more nodes than are needed to represent the
data accurately, making it harder for editors and data consumers to
work with down the road.This is a simple fix that will save countless
hours later.
... I could go on, but I think this is plenty sufficient to justify
pressing pause on all this.
Again, I don't in any way want to disrespect the work that has gone
into this effort already. We're all volunteers here and I know how
much time this all takes. However. importing all/most of the
buildings in Canada is a monstrously large task, which will have to
dance around a lot of people's toes. We should expect this to take a
really damn long time if we're going to do it right. We need to have
the patience to learn from experience, from critique, and from the
wisdom of the people who've learned from flawed imports in the past
and have devised guidelines and processes so that we can have better
experiences with this in the future.
Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban Planning
NateWessel.com <http://natewessel.com>
On 1/18/19 2:24 PM, john whelan wrote:
My background is I'm a retired civil servant who has written and
overseen procurement documents and fairly large procurements.
Dotting the is and crossing the Ts are my speciality.
There are two parts to an import. The first part is the part played
by the import mailing group. They confine themselves to is the
license correct and do you have a reasonable plan. In this case the
license is one of the few that has been confirmed by the Legal
Working Group of OpenStreetMap and as such no questions were raised
about it on the import mailing list. We have methodology that has
been used before successfully with the Ottawa building outline
import. There were major discussions both on talk-ca and the import
mailing group before that import took place and we took note of the
issues raised and addressed them. The licensing issue goes back
about eight years to when I was talking to Federal Government
Treasury Board and explaining their Open Data license did not align
with OSM. That is why their license is now known as 2.0.
The second part is the local group makes the decision to import they
are the authority no one else.
Apparently you were not part of the talk-ca when the discussions
took place which would have been the time and place to raise concerns.
When the Ottawa import was done there were one or two places where
the existing buildings and the import overlapped. In the
instructions on the import there are instructions to cover this.
Specifically there is a validation step. I seem to recall the error
rate was of the order of 1% and I expect this latest batch to be
roughly the same.
If you can identify which municipalities data is of poor quality
then I'm sure we can remove these. For the most part these are from
the foundation plans recorded by the municipality using professional
surveying techniques.
Would you like to clarify exactly where I failed to dot the Is and
cross the Ts please.
Many Thanks
John
On Fri, 18 Jan 2019 at 13:37, Nate Wessel <bike...@gmail.com
<mailto:bike...@gmail.com>> wrote:
Hi John,
As Steve has said, you seem to be the only one suggesting that
thousands of import committees might need to be formed.
Certainly I'm not suggesting that.
My understanding of OSM import procedure (and wiki-style
projects more generally) is that imports should operate in an
essentially consensual way where possible. The goal is to build
consent and bring people on board with a project or a change by
addressing their concerns in a meaningful and respectful way.
I think that I have made some substantive and troubling claims
about the quality of the data being imported. I've pointed out
that this project has not followed the import procedures that
were produced by a community of mappers larger than just those
in Canada.
So to respond to your implication, I am in some sense the one
reviewing the project, just as I would welcome you to find ways
that my own contributions could be better. If you want my
credentials for reviewing your work, here they are:
1) I am an active contributor to OSM in Toronto, where I live
(and elsewhere)
2) I am currently helping to lead a building import in Hamilton
County Ohio that has better addressed some of the issues I see
this import struggling with. I can help you do the same.
3) I've been doing research in GIS for a long time now, though I
don't need that to tell you that the issues I've described are
hardly insurmountable technically or even all that difficult to
fix. It would take maybe one day's hard work to get the
technical side of this right.
I think Canadian OSMers will agree that we can take a pause to
get things right on such a massive import. If they don't - if
I'm shouted down or better, if my critiques are adequately
addressed, then I will leave you to finish the project in peace.
I might even lend a hand if all goes well, as I sincerely hope
it does :-)
Best,
Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban
Planning
NateWessel.com <http://natewessel.com>
On 1/18/19 1:11 PM, john whelan wrote:
I know of no other way to contact him but he made an
interesting comment that the project is on hold in the wiki
pending review.
Would he care to comment on who is supposed to be reviewing the
project?
My understanding is that the import was raised in talk-ca
before it commenced for comment and these were generally
favourable. I took that as the local mappers to Canada had
been consulted and they are the "local mappers" authority in
this case.
I understand he has concerns about local mappers making
decisions but in Canada we have been importing similar data
through CANVEC for some time. CANVEC data comes from a number
of sources including municipal data.
Is he suggesting that each of the 3,700 municipalities in
Canada should form a group of local mappers who can make
individual decisions on whether their municipal data should be
imported and we should end up with 3,700 import plans?
Thanks John
_______________________________________________
Talk-ca mailing list
Talk-ca@openstreetmap.org <mailto:Talk-ca@openstreetmap.org>
https://lists.openstreetmap.org/listinfo/talk-ca
_______________________________________________
Talk-ca mailing list
Talk-ca@openstreetmap.org <mailto:Talk-ca@openstreetmap.org>
https://lists.openstreetmap.org/listinfo/talk-ca
--
Sent from Postbox
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
_______________________________________________
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca