I'm not familiar with the tool, but that is essentially what I'm asking
for - nothing all that complicated. We would need to make sure we're
not losing any valuable detail though, and ensure that topology is
preserved where buildings share nodes.
Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban Planning
NateWessel.com <http://natewessel.com>
On 1/18/19 4:24 PM, James wrote:
I can run all the shapefiles through qgis simplify tool if this
resolves the issue...
On Fri., Jan. 18, 2019, 4:08 p.m. Nate Wessel <bike...@gmail.com
<mailto:bike...@gmail.com> wrote:
With default settings in JOSM, sure. In the import I was working
on, we used a Douglas-Peucker algorithm with a 20cm threshold
(before the import started) and it worked beautifully. We had many
points that seemed to have been introduced in the shapefiles as
some kind of data artifact - they didn't add any detail to the
shape at all. This procedure removed almost all of them with no
discernible reduction in quality.
Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban
Planning
NateWessel.com <http://natewessel.com>
On 1/18/19 4:03 PM, James wrote:
dare you to run simplify tool on anything remotely round, it will
make it look like garbage
On Fri., Jan. 18, 2019, 3:49 p.m. John Whelan
<jwhelan0...@gmail.com <mailto:jwhelan0...@gmail.com> wrote:
The import mailing list was pointed to the correct page of
the wiki. The initial post was to say this is what we were
thinking of and there was a comment saying we needed to
change the comment line.
>There is no mention of this proposed import on the import
catalogue
The import process was reviewed by the person who set up the
Ottawa import did we miss that step on the Ottawa import as
well? Neither was it raised as a concern on the import
mailing list. I think this is very minor and can be corrected.
We learnt a fair bit on the Ottawa import and my expectation
is since we are using experienced mappers to do the import
conflation would be either handled by them or the building
not imported. We aren't using new mappers in a mapathon here
and with experienced mappers then I think you have to trust
them. The world isn't perfect. Think in terms of service level.
>There are 2X more nodes than needed to represent the
building accurately.
The problem with correcting this is you are introducing
approximations. This will vary according to the source and
this can be simplified or corrected once its in OSM. I think
this is a different issue of a mechanical edit that needs to
be considered separately.
If we are concerned with database size then I suggest we
change the instructions to say put the source comment on the
change set rather than on the building outline.
Cheerio John
Nate Wessel wrote on 2019-01-18 3:06 PM:
John,
You seem to be playing the long game with this data - it
sounds like you've been working with this a lot longer than
I have, and you've put in the time and effort to help make
this actually-quite-incredible dataset available to us. I
don't want to stop the import from happening - quite the
opposite. I just want to make sure that the time is taken to
do this right. OSM deserves that. Your (our) long awaited
victory will be the sweeter for our patience now.
There are several specific issues I see where the I's are
not crossed, nor the t's dotted. I've mentioned several
already, so I'll try to be brief (I really need to get back
to working on my dissertation).
1) There was extremely limited discussion on the imports
mailing list. The initial email did not make clear the scope
of the project. I read the email and did not think twice at
it, thinking it was entirely about Ottawa. The link in that
email was actually to the Ottawa import, and not this one,
which seems to have been only in draft at the time.
https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
As such, this project has NOT been reviewed by the imports
list, which is a requirement for proceeding with the import.
2) There is no mention of this proposed import on the import
catalogue (https://wiki.openstreetmap.org/wiki/Import/Catalogue)
which is required in the imports guidelines. I suspect many
other guidelines have not been followed.
3) The wiki page describing the import is not adequate to
assess the quality of the data or of the proposed import.
See for example:
https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks
The import guidelines call for a description of how
conflation will be handled. The fact that two of the major
importers seem to have a substantial disagreement about how
to handle existing data indicates this was not well
discussed and I can see that it isn't well documented.
4) The buildings need to be simplified, quite a bit
actually. Most buildings have multiple nodes representing
straight lines. This bloats the database and makes things
harder to edit by hand later. There are probably 2x more
nodes than are needed to represent the data accurately,
making it harder for editors and data consumers to work with
down the road.This is a simple fix that will save countless
hours later.
... I could go on, but I think this is plenty sufficient to
justify pressing pause on all this.
Again, I don't in any way want to disrespect the work that
has gone into this effort already. We're all volunteers here
and I know how much time this all takes. However. importing
all/most of the buildings in Canada is a monstrously large
task, which will have to dance around a lot of people's
toes. We should expect this to take a really damn long time
if we're going to do it right. We need to have the patience
to learn from experience, from critique, and from the wisdom
of the people who've learned from flawed imports in the past
and have devised guidelines and processes so that we can
have better experiences with this in the future.
Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in
Urban Planning
NateWessel.com <http://natewessel.com>
On 1/18/19 2:24 PM, john whelan wrote:
My background is I'm a retired civil servant who has
written and overseen procurement documents and fairly large
procurements. Dotting the is and crossing the Ts are my
speciality.
There are two parts to an import. The first part is the
part played by the import mailing group. They confine
themselves to is the license correct and do you have a
reasonable plan. In this case the license is one of the
few that has been confirmed by the Legal Working Group of
OpenStreetMap and as such no questions were raised about it
on the import mailing list. We have methodology that has
been used before successfully with the Ottawa building
outline import. There were major discussions both on
talk-ca and the import mailing group before that import
took place and we took note of the issues raised and
addressed them. The licensing issue goes back about eight
years to when I was talking to Federal Government Treasury
Board and explaining their Open Data license did not align
with OSM. That is why their license is now known as 2.0.
The second part is the local group makes the decision to
import they are the authority no one else.
Apparently you were not part of the talk-ca when the
discussions took place which would have been the time and
place to raise concerns.
When the Ottawa import was done there were one or two
places where the existing buildings and the import
overlapped. In the instructions on the import there are
instructions to cover this. Specifically there is a
validation step. I seem to recall the error rate was of
the order of 1% and I expect this latest batch to be
roughly the same.
If you can identify which municipalities data is of poor
quality then I'm sure we can remove these. For the most
part these are from the foundation plans recorded by the
municipality using professional surveying techniques.
Would you like to clarify exactly where I failed to dot the
Is and cross the Ts please.
Many Thanks
John
On Fri, 18 Jan 2019 at 13:37, Nate Wessel
<bike...@gmail.com <mailto:bike...@gmail.com>> wrote:
Hi John,
As Steve has said, you seem to be the only one
suggesting that thousands of import committees might
need to be formed. Certainly I'm not suggesting that.
My understanding of OSM import procedure (and
wiki-style projects more generally) is that imports
should operate in an essentially consensual way where
possible. The goal is to build consent and bring people
on board with a project or a change by addressing their
concerns in a meaningful and respectful way.
I think that I have made some substantive and troubling
claims about the quality of the data being imported.
I've pointed out that this project has not followed the
import procedures that were produced by a community of
mappers larger than just those in Canada.
So to respond to your implication, I am in some sense
the one reviewing the project, just as I would welcome
you to find ways that my own contributions could be
better. If you want my credentials for reviewing your
work, here they are:
1) I am an active contributor to OSM in Toronto, where
I live (and elsewhere)
2) I am currently helping to lead a building import in
Hamilton County Ohio that has better addressed some of
the issues I see this import struggling with. I can
help you do the same.
3) I've been doing research in GIS for a long time now,
though I don't need that to tell you that the issues
I've described are hardly insurmountable technically or
even all that difficult to fix. It would take maybe one
day's hard work to get the technical side of this right.
I think Canadian OSMers will agree that we can take a
pause to get things right on such a massive import. If
they don't - if I'm shouted down or better, if my
critiques are adequately addressed, then I will leave
you to finish the project in peace. I might even lend a
hand if all goes well, as I sincerely hope it does :-)
Best,
Nate Wessel
Jack of all trades, Master of Geography, PhD candidate
in Urban Planning
NateWessel.com <http://natewessel.com>
On 1/18/19 1:11 PM, john whelan wrote:
I know of no other way to contact him but he made an
interesting comment that the project is on hold in the
wiki pending review.
Would he care to comment on who is supposed to be
reviewing the project?
My understanding is that the import was raised in
talk-ca before it commenced for comment and these were
generally favourable. I took that as the local
mappers to Canada had been consulted and they are the
"local mappers" authority in this case.
I understand he has concerns about local mappers
making decisions but in Canada we have been importing
similar data through CANVEC for some time. CANVEC
data comes from a number of sources including
municipal data.
Is he suggesting that each of the 3,700 municipalities
in Canada should form a group of local mappers who can
make individual decisions on whether their municipal
data should be imported and we should end up with
3,700 import plans?
Thanks John
_______________________________________________
Talk-ca mailing list
Talk-ca@openstreetmap.org <mailto:Talk-ca@openstreetmap.org>
https://lists.openstreetmap.org/listinfo/talk-ca
_______________________________________________
Talk-ca mailing list
Talk-ca@openstreetmap.org
<mailto:Talk-ca@openstreetmap.org>
https://lists.openstreetmap.org/listinfo/talk-ca
--
Sent from Postbox
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
_______________________________________________
Talk-ca mailing list
Talk-ca@openstreetmap.org <mailto:Talk-ca@openstreetmap.org>
https://lists.openstreetmap.org/listinfo/talk-ca
_______________________________________________
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca
