Re: [Talk-ca] Ongoing Canadian building import needs to be stopped, possibly reverted

[James]

As Frederik Ramm once said(sorry i'm paraphrasing from memory please don't
shoot me) There has never been a GO-Nogo for imports, you bring it up on
the mailing lists with reasonable delay, is there no objections(in this
case no one was saying anything about it for 2-3 weeks) then email the list
that the import would start.

On Fri., Jan. 18, 2019, 12:59 a.m. Alan Richards  Along the lines of what Jarek said, sometimes silence just means tacit
> acceptance, or that it's not that controversial. There's quite a bit of
> government data here that is supposedly "open" but unavailable for OSM, so
> I'm very glad Stats Can was able to find a way to collect municipal data
> and publish it under one national license. I was surprised myself it hadn't
> got more attention, but I'm firmly onboard with more imports if done with
> care.
> Manually adding buildings - especially residential neighborhoods, is about
> the most boring task I can think of, yet it does add a lot to the map.
>
> I'll admit I hadn't looked at the data quality myself, but I just did
> review several task squares around BC and they look pretty good. Houses
> were all in the right place, accurate, and generally as much or even more
> detailed than I typically see. Issues seemed to be mostly the larger
> commercial buildings being overly large or missing detail, but in general
> these are the buildings most likely to be already mapped. To a large
> degree, it's up the individual importer to do some quality control, review
> against existing object, satellite, etc. If we have specific issues we can
> and should address them, but if the data is largely good then I see no need
> to abort or revert.
>
> alarobric
>
> On Thu, Jan 17, 2019 at 7:41 PM Jarek Piórkowski 
> wrote:
>
>> On Thu, 17 Jan 2019 at 21:46, OSM Volunteer stevea
>>  wrote:
>> > Thanks, Jarek.  Considering I am a proponent of "perfection must not be
>> the enemy of good" (regarding OSM data entry), I think data which are "darn
>> good, though not perfect" DO deserve to enter into OSM.  Sometimes "darn
>> good" might be 85%, 95% "good," as then we'll get it to 99% and then 100%
>> over time.  But if the focus on "how" isn't sharp enough to get it to 85%
>> (or so) during initial entry, go back and start over to get that number
>> up.  85% sounds arbitrary, I know, but think of it as "a solid B" which
>> might be "passes the class for now" without failing.  And it's good we
>> develop a "meanwhile strategy" to take it to 99% and then 100% in the
>> (near- or at most mid-term) future.  This isn't outrageously difficult,
>> though it does take patience and coordination.  Open communication is a
>> prerequisite.
>>
>> Thank you for this commitment. I wish others shared it. Unfortunately
>> the reality I've been seeing in OSM is that edits which are 90+% good
>> (like this import) are challenged, while edits which are 50+% bad
>> (maps.me submissions, wheelmap/rosemary v0.4.4 going to completely
>> wrong locations for _years_) go unchallenged or are laboriously
>> manually fixed afterward.
>>
>> --Jarek
>>
>> ___
>> Talk-ca mailing list
>> Talk-ca@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-ca
>>
> ___
> Talk-ca mailing list
> Talk-ca@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-ca
>
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] Ongoing Canadian building import needs to be stopped, possibly reverted

[Kyle Nuttall]

The pilot project that took place in Ottawa for all these building imports is 
what got me hooked into OSM in the first place. I would make only very minor 
changes here and there. I even attempted to draw building footprints but got 
burnt out after only doing a single street, which was very discouraging for me 
to continue.

When I saw the entire neighbourhood get flooded with new buildings that weren't 
there before, I was entirely intrigued and actually got on board with the 
locals to help with the process. I've been hooked since and have been to many 
meetups afterwards. Helping out with projects completely unrelated to the 
initial building import.

I'm entirely of the belief that it is much more encouraging for a new user to 
make a minor change (eg. changing `building=yes` to `building=detached`) than 
it is to add every single minor detail to each object from scratch (visiting 
the location, drawing the building footprints manually, adding address data, 
etc.). It's just overwhelming for a new user.

It is very much a cat-and-mouse type scenario with community driven projects 
like OSM. Apparently the issue with this import is the lack of community 
involvement but I can for sure tell you that this import will help flourish the 
community in the local areas. Especially if they only need to add or change 
minor tags than if they would have had to create all of this data by hand. With 
an import this size there is bound to be some errors that slip through. That's 
where the community comes through to correct these minor things.

This is the whole point of OSM. A user creates an object with as much 
information as they know and the next user comes and adds onto that, and the 
next user adds and/or updates even more. Neither of those users on their own 
could have added as much detail as all of their knowledge combined.

Are we supposed to just wait for a user who can add every single building with 
centimetre precision and every bit of detail simply because we can't? No, of 
course not. We do the best we can and have other users who know more than we do 
build on that.

I fully endorse this import because I would love to see what it does for the 
local communities that apparently need to figure this import out for themselves.

Cheers,
Kyle

On Jan. 18, 2019 05:40, James  wrote:
As Frederik Ramm once said(sorry i'm paraphrasing from memory please don't 
shoot me) There has never been a GO-Nogo for imports, you bring it up on the 
mailing lists with reasonable delay, is there no objections(in this case no one 
was saying anything about it for 2-3 weeks) then email the list that the import 
would start.

On Fri., Jan. 18, 2019, 12:59 a.m. Alan Richards 
mailto:alarob...@gmail.com> wrote:
Along the lines of what Jarek said, sometimes silence just means tacit 
acceptance, or that it's not that controversial. There's quite a bit of 
government data here that is supposedly "open" but unavailable for OSM, so I'm 
very glad Stats Can was able to find a way to collect municipal data and 
publish it under one national license. I was surprised myself it hadn't got 
more attention, but I'm firmly onboard with more imports if done with care.
Manually adding buildings - especially residential neighborhoods, is about the 
most boring task I can think of, yet it does add a lot to the map.

I'll admit I hadn't looked at the data quality myself, but I just did review 
several task squares around BC and they look pretty good. Houses were all in 
the right place, accurate, and generally as much or even more detailed than I 
typically see. Issues seemed to be mostly the larger commercial buildings being 
overly large or missing detail, but in general these are the buildings most 
likely to be already mapped. To a large degree, it's up the individual importer 
to do some quality control, review against existing object, satellite, etc. If 
we have specific issues we can and should address them, but if the data is 
largely good then I see no need to abort or revert.

alarobric

On Thu, Jan 17, 2019 at 7:41 PM Jarek Piórkowski 
mailto:ja...@piorkowski.ca>> wrote:
On Thu, 17 Jan 2019 at 21:46, OSM Volunteer stevea
mailto:stevea...@softworkers.com>> wrote:
> Thanks, Jarek.  Considering I am a proponent of "perfection must not be the 
> enemy of good" (regarding OSM data entry), I think data which are "darn good, 
> though not perfect" DO deserve to enter into OSM.  Sometimes "darn good" 
> might be 85%, 95% "good," as then we'll get it to 99% and then 100% over 
> time.  But if the focus on "how" isn't sharp enough to get it to 85% (or so) 
> during initial entry, go back and start over to get that number up.  85% 
> sounds arbitrary, I know, but think of it as "a solid B" which might be 
> "passes the class for now" without failing.  And it's good we develop a 
> "meanwhile strategy" to take it to 99% and then 100% in the (near- or at most 
> mid-term) future.  This isn't outrageously difficult, though it does take 
> patienc

[Talk-ca] Waylens camera available in Montreal

[Fabián Rodríguez]

Hi,

Last year OSM US lent me a Waylens camera for this project:

  * https://www.openstreetmap.us/2018/05/camera-lending-program/
  * 
https://wiki.openstreetmap.org/wiki/Foundation/Local_Chapters/United_States/Waylens_Camera_Lending_Program

I can't continue contributing, so if someone wants to join this effort,
please let me know off-list. We'd need to arrange for the kit I received
to be picked up. I am in Montreal.

Please read both links, they include important details and context. If
you aren't already an active OSM contributor, please do not contact me.

Cheers,

- Fabian

-- 
Fabián Rodríguez
http://openstreetmap.magicfab.ca/

___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] Ongoing Canadian building import needs to be stopped, possibly reverted

[Nate Wessel]

Hi all,

I've just joined the talk-ca list, so please accept my apologies for not 
addressing this list earlier. I'm happy to take this thread off the 
imports list for now and onto talk-ca until things are ready to begin 
again. The next person to reply can please feel free to remove that 
email if they agree.


I've just made a note on the draft import plan wiki page noting that the 
import has been stopped:


https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan

I would really appreciate it if the person with admin access to the 
tasking manager projects could please take those offline for the moment, 
or perhaps place them in a validation-only mode if that's possible.


Like I said in my last email, which perhaps didn't make it to the 
talk-ca list 
(https://lists.openstreetmap.org/pipermail/imports/2019-January/005886.html) 
I'm now proposing that we leave the data that has already been imported 
and enter a phase of thorough validation on that data.


My plan, over the next several days, is to do a general survey of the 
quality of the data that has been imported so far and make a list of 
systematic issues I see that should be addressed before we can consider 
moving forward again. I'll add those comments to the conversation in 
talk-ca and on the wiki page (link above), as I feel is appropriate. As 
I said before, I'm of the mind that this import did not get adequate 
review or approval and did not follow all the import guidelines. I think 
therefore we need to take stock, cross the t's, dot the i's, and move 
this thing back toward where it needs to be. Step one is a thoroughly 
documented wiki page outlining the proposal and responding to everything 
required in the import guidelines.

https://wiki.openstreetmap.org/wiki/Import/Guidelines

I know there are people excited about this import, and people who are 
eager to get back to work bringing buildings in, but I think everyone 
will be happier in the end if we take the time to do this right. We 
don't need to stop forever - we just need to stop until we get things 
right. I sincerely respect the good intentions of everyone involved in 
this and I hope we can all work together to make OSM a map known for 
it's coverage AND it's quality.


Best,

Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban Planning
NateWessel.com 

On 1/17/19 9:05 PM, OSM Volunteer stevea wrote:

The thread link is:  
https://lists.openstreetmap.org/pipermail/imports/2019-January/005878.html

SteveA
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

[Talk-ca] 2020 building import wiki comment by Nate Wessel

[john whelan]

I know of no other way to contact him but he made an interesting comment
that the project is on hold in the wiki pending review.

Would he care to comment on who is supposed to be reviewing the project?

My understanding is that the import was raised in talk-ca before it
commenced for comment and these were generally favourable.  I took that as
the local mappers to Canada had been consulted and they are the "local
mappers" authority in this case.

I understand he has concerns about local mappers making decisions but in
Canada we have been importing similar data through CANVEC for some time.
CANVEC data comes from a number of sources including municipal data.

Is he suggesting that each of the 3,700 municipalities in Canada should
form a group of local mappers who can make individual decisions on whether
their municipal data should be imported and we should end up with 3,700
import plans?

Thanks John
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[Nate Wessel]

Hi John,

As Steve has said, you seem to be the only one suggesting that thousands 
of import committees might need to be formed. Certainly I'm not 
suggesting that.


My understanding of OSM import procedure (and wiki-style projects more 
generally) is that imports should operate in an essentially consensual 
way where possible. The goal is to build consent and bring people on 
board with a project or a change by addressing their concerns in a 
meaningful and respectful way.


I think that I have made some substantive and troubling claims about the 
quality of the data being imported. I've pointed out that this project 
has not followed the import procedures that were produced by a community 
of mappers larger than just those in Canada.


So to respond to your implication, I am in some sense the one reviewing 
the project, just as I would welcome you to find ways that my own 
contributions could be better. If you want my credentials for reviewing 
your work, here they are:


1) I am an active contributor to OSM in Toronto, where I live (and 
elsewhere)


2) I am currently helping to lead a building import in Hamilton County 
Ohio that has better addressed some of the issues I see this import 
struggling with. I can help you do the same.


3) I've been doing research in GIS for a long time now, though I don't 
need that to tell you that the issues I've described are hardly 
insurmountable technically or even all that difficult to fix. It would 
take maybe one day's hard work to get the technical side of this right.


I think Canadian OSMers will agree that we can take a pause to get 
things right on such a massive import. If they don't - if I'm shouted 
down or better, if my critiques are adequately addressed, then I will 
leave you to finish the project in peace. I might even lend a hand if 
all goes well, as I sincerely hope it does :-)


Best,

Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban Planning
NateWessel.com 

On 1/18/19 1:11 PM, john whelan wrote:
I know of no other way to contact him but he made an interesting 
comment that the project is on hold in the wiki pending review.


Would he care to comment on who is supposed to be reviewing the project?

My understanding is that the import was raised in talk-ca before it 
commenced for comment and these were generally favourable. I took that 
as the local mappers to Canada had been consulted and they are the 
"local mappers" authority in this case.


I understand he has concerns about local mappers making decisions but 
in Canada we have been importing similar data through CANVEC for some 
time.  CANVEC data comes from a number of sources including municipal 
data.


Is he suggesting that each of the 3,700 municipalities in Canada 
should form a group of local mappers who can make individual decisions 
on whether their municipal data should be imported and we should end 
up with 3,700 import plans?


Thanks John



___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] Ongoing Canadian building import needs to be stopped, possibly reverted

[Jarek Piórkowski]

Some more thoughts from me.

Building outlines, particularly for single-family subdivisions as seen
in Canadian suburbs, are extremely labour-intensive to map manually.

My parents' house is now on OSM - accurately. They live in a city with
about 10,000 buildings, and about 0.5 active mappers. This wouldn't
been completed manually in the next 5 years.

An option to do this automatically with a computer algorithm detecting
objects from imagery could be suggested, but this has not been very
accurate in OSM in the past, even when there is decent imagery. The
only other feasible data source is government, where they have such
data more or less.

The alternative is of course the opinion that we should not have
building outlines until someone goes through and adds the buildings
manually. In practice what I've seen done in Toronto is that bigger
buildings are mapped on best-effort basis from survey and imagery,
while areas of single-family houses are left blank. This isn't
_wrong_, and maybe some prefer this.

I would also like to note that building outlines will _never_ be
completely verifiably up to date. I can't go into most people's
backyards and verify that there isn't a new addition on their house. A
building might be legally split into two different properties without
it being evident from the street. Imagery is out of date the day after
it's taken, and proper offset can be difficult to establish in big
cities where GPS signal is erratic. Pragmatically, I can tell you from
personal experience that building data in lovingly-mapped Berlin is
also worse than 1 meter accuracy. So again: best effort.

What do we get from having buildings? A sense of land use (arguably
replaceable with larger landuse areas). A way to roughly estimate
population density. A way to gauge built-up density. A data source for
locating buildings in possible flood zones, or fire risk. Statistics:
as open data, queryable by APIs that are already used, in format
more-or-less common worldwide.

Examples were given of rowhouse- or de-facto rowhouse-buildings where
a part is attached to the wrong building. This does not alter any of
the above examples. It's wrong, but is it substantially more wrong
than a blank subdivision, or one with only a few buildings mapped? Is
it better to have a null, or be off by 5%? The legal truth is in
property records, and we can't measure houses with a ruler, so OSM can
only be a statistical source. And then there's the question of
verifiability - some of these buildings are connected to their
neighbour building inside. I've really struggled at distinguishing
what exactly is a "building" on Old Toronto avenues even with
street-side survey.

Bluntly, OSM is not perfect in Canada. I have pet peeves I can quote,
and I'm sure many of you do as well. If we import, the question is:
are we making it better?

1. Do we want this data?
2. Is it generally of acceptable quality?
3. Is there a mechanism to spot and reject where data is particularly bad?

Cheers,
Jarek, who should really get back to updating built-last-year stuff at Fort York

On Fri, 18 Jan 2019 at 09:31, Kyle Nuttall  wrote:
>
> The pilot project that took place in Ottawa for all these building imports is 
> what got me hooked into OSM in the first place. I would make only very minor 
> changes here and there. I even attempted to draw building footprints but got 
> burnt out after only doing a single street, which was very discouraging for 
> me to continue.
>
> When I saw the entire neighbourhood get flooded with new buildings that 
> weren't there before, I was entirely intrigued and actually got on board with 
> the locals to help with the process. I've been hooked since and have been to 
> many meetups afterwards. Helping out with projects completely unrelated to 
> the initial building import.
>
> I'm entirely of the belief that it is much more encouraging for a new user to 
> make a minor change (eg. changing `building=yes` to `building=detached`) than 
> it is to add every single minor detail to each object from scratch (visiting 
> the location, drawing the building footprints manually, adding address data, 
> etc.). It's just overwhelming for a new user.
>
> It is very much a cat-and-mouse type scenario with community driven projects 
> like OSM. Apparently the issue with this import is the lack of community 
> involvement but I can for sure tell you that this import will help flourish 
> the community in the local areas. Especially if they only need to add or 
> change minor tags than if they would have had to create all of this data by 
> hand. With an import this size there is bound to be some errors that slip 
> through. That's where the community comes through to correct these minor 
> things.
>
> This is the whole point of OSM. A user creates an object with as much 
> information as they know and the next user comes and adds onto that, and the 
> next user adds and/or updates even more. Neither of those users on their own 
> could have added

Re: [Talk-ca] Ongoing Canadian building import needs to be stopped, possibly reverted

[Yaro Shkvorets]

Nate,
I'll change the project name to reflect that the import is on hold. As a
local mapper, if you want to take a lead on the Toronto import that'd be
great.
I did review some of DannyMcD's edits last night
(Mississauga-Brampton-Vaughan) and to be honest was rather disappointed
with the quality. It appears Danny chose to import only new buildings (i.e.
residential homes mostly), leaving most of the existing hand-traced
non-residential building outlines in OSM untouched. That's unfortunate, the
dataset offers some really good data and leaving half of it behind makes it
more difficult to revisit in the future.
In my edits (Markham-Scarborough-East York) I was aiming to replace as many
existing geometries with outlines from the import as possible. I think
that's what we should be trying to do going forward.
Looking forward to your comments and discussion.



On Fri, Jan 18, 2019 at 1:07 PM Nate Wessel  wrote:

> Hi all,
>
> I've just joined the talk-ca list, so please accept my apologies for not
> addressing this list earlier. I'm happy to take this thread off the imports
> list for now and onto talk-ca until things are ready to begin again. The
> next person to reply can please feel free to remove that email if they
> agree.
>
> I've just made a note on the draft import plan wiki page noting that the
> import has been stopped:
>
>
> https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan
>
> I would really appreciate it if the person with admin access to the
> tasking manager projects could please take those offline for the moment, or
> perhaps place them in a validation-only mode if that's possible.
>
> Like I said in my last email, which perhaps didn't make it to the talk-ca
> list (
> https://lists.openstreetmap.org/pipermail/imports/2019-January/005886.html)
> I'm now proposing that we leave the data that has already been imported and
> enter a phase of thorough validation on that data.
>
> My plan, over the next several days, is to do a general survey of the
> quality of the data that has been imported so far and make a list of
> systematic issues I see that should be addressed before we can consider
> moving forward again. I'll add those comments to the conversation in
> talk-ca and on the wiki page (link above), as I feel is appropriate. As I
> said before, I'm of the mind that this import did not get adequate review
> or approval and did not follow all the import guidelines. I think therefore
> we need to take stock, cross the t's, dot the i's, and move this thing back
> toward where it needs to be. Step one is a thoroughly documented wiki page
> outlining the proposal and responding to everything required in the import
> guidelines.
> https://wiki.openstreetmap.org/wiki/Import/Guidelines
>
> I know there are people excited about this import, and people who are
> eager to get back to work bringing buildings in, but I think everyone will
> be happier in the end if we take the time to do this right. We don't need
> to stop forever - we just need to stop until we get things right. I
> sincerely respect the good intentions of everyone involved in this and I
> hope we can all work together to make OSM a map known for it's coverage AND
> it's quality.
>
> Best,
> Nate Wessel
> Jack of all trades, Master of Geography, PhD candidate in Urban Planning
> NateWessel.com 
>
> On 1/17/19 9:05 PM, OSM Volunteer stevea wrote:
>
> The thread link is:  
> https://lists.openstreetmap.org/pipermail/imports/2019-January/005878.html
>
> SteveA
>
>

-- 
Best Regards,
  Yaro Shkvorets
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[john whelan]

My background is I'm a retired civil servant who has written and overseen
procurement documents and fairly large procurements.  Dotting the is and
crossing the Ts are my speciality.

There are two parts to an import.  The first part is the part played by the
import mailing group.  They confine themselves to is the license correct
and do you have a reasonable plan.  In this case the license is one of the
few that has been confirmed by the Legal Working Group of OpenStreetMap and
as such no questions were raised about it on the import mailing list.  We
have methodology that has been used before successfully with the Ottawa
building outline import. There were major discussions both on talk-ca and
the import mailing group before that import took place and we took note of
the issues raised and addressed them.  The licensing issue goes back about
eight years to when I was talking to Federal Government Treasury Board and
explaining their Open Data license did not align with OSM.  That is why
their license is now known as 2.0.

The second part is the local group makes the decision to import they are
the authority no one else.

Apparently you were not part of the talk-ca when the discussions took place
which would have been the time and place to raise concerns.

When the Ottawa import was done there were one or two places where the
existing buildings and the import overlapped.  In the instructions on the
import there are instructions to cover this. Specifically there is a
validation step.  I seem to recall the error rate was of the order of 1%
and I expect this latest batch to be roughly the same.

If you can identify which municipalities data is of poor quality then I'm
sure we can remove these.  For the most part these are from the foundation
plans recorded by the municipality using professional surveying techniques.

Would you like to clarify exactly where I failed to dot the Is and cross
the Ts please.

Many Thanks

John



On Fri, 18 Jan 2019 at 13:37, Nate Wessel  wrote:

> Hi John,
>
> As Steve has said, you seem to be the only one suggesting that thousands
> of import committees might need to be formed. Certainly I'm not suggesting
> that.
>
> My understanding of OSM import procedure (and wiki-style projects more
> generally) is that imports should operate in an essentially consensual way
> where possible. The goal is to build consent and bring people on board with
> a project or a change by addressing their concerns in a meaningful and
> respectful way.
>
> I think that I have made some substantive and troubling claims about the
> quality of the data being imported. I've pointed out that this project has
> not followed the import procedures that were produced by a community of
> mappers larger than just those in Canada.
>
> So to respond to your implication, I am in some sense the one reviewing
> the project, just as I would welcome you to find ways that my own
> contributions could be better. If you want my credentials for reviewing
> your work, here they are:
>
> 1) I am an active contributor to OSM in Toronto, where I live (and
> elsewhere)
>
> 2) I am currently helping to lead a building import in Hamilton County
> Ohio that has better addressed some of the issues I see this import
> struggling with. I can help you do the same.
>
> 3) I've been doing research in GIS for a long time now, though I don't
> need that to tell you that the issues I've described are hardly
> insurmountable technically or even all that difficult to fix. It would take
> maybe one day's hard work to get the technical side of this right.
>
> I think Canadian OSMers will agree that we can take a pause to get things
> right on such a massive import. If they don't - if I'm shouted down or
> better, if my critiques are adequately addressed, then I will leave you to
> finish the project in peace. I might even lend a hand if all goes well, as
> I sincerely hope it does :-)
>
> Best,
> Nate Wessel
> Jack of all trades, Master of Geography, PhD candidate in Urban Planning
> NateWessel.com 
>
> On 1/18/19 1:11 PM, john whelan wrote:
>
> I know of no other way to contact him but he made an interesting comment
> that the project is on hold in the wiki pending review.
>
> Would he care to comment on who is supposed to be reviewing the project?
>
> My understanding is that the import was raised in talk-ca before it
> commenced for comment and these were generally favourable.  I took that as
> the local mappers to Canada had been consulted and they are the "local
> mappers" authority in this case.
>
> I understand he has concerns about local mappers making decisions but in
> Canada we have been importing similar data through CANVEC for some time.
> CANVEC data comes from a number of sources including municipal data.
>
> Is he suggesting that each of the 3,700 municipalities in Canada should
> form a group of local mappers who can make individual decisions on whether
> their municipal data should be imported and we should

Re: [Talk-ca] Ongoing Canadian building import needs to be stopped, possibly reverted

[john whelan]

And that is a problem with imports.  Traditionally or the party line is if
its been mapped already then to preserve the history you either leave it
alone or manually correct it.  Manually correcting it is very time
consuming.  Often the decision is made to leave the existing way in the map.

I'm not going to say one method is correct over the other but the least
contentious is to add only things are are not there already.

Cheerio John

On Fri, 18 Jan 2019 at 14:17, Yaro Shkvorets  wrote:

> Nate,
> I'll change the project name to reflect that the import is on hold. As a
> local mapper, if you want to take a lead on the Toronto import that'd be
> great.
> I did review some of DannyMcD's edits last night
> (Mississauga-Brampton-Vaughan) and to be honest was rather disappointed
> with the quality. It appears Danny chose to import only new buildings (i.e.
> residential homes mostly), leaving most of the existing hand-traced
> non-residential building outlines in OSM untouched. That's unfortunate, the
> dataset offers some really good data and leaving half of it behind makes it
> more difficult to revisit in the future.
> In my edits (Markham-Scarborough-East York) I was aiming to replace as
> many existing geometries with outlines from the import as possible. I think
> that's what we should be trying to do going forward.
> Looking forward to your comments and discussion.
>
>
>
> On Fri, Jan 18, 2019 at 1:07 PM Nate Wessel  wrote:
>
>> Hi all,
>>
>> I've just joined the talk-ca list, so please accept my apologies for not
>> addressing this list earlier. I'm happy to take this thread off the imports
>> list for now and onto talk-ca until things are ready to begin again. The
>> next person to reply can please feel free to remove that email if they
>> agree.
>>
>> I've just made a note on the draft import plan wiki page noting that the
>> import has been stopped:
>>
>>
>> https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan
>>
>> I would really appreciate it if the person with admin access to the
>> tasking manager projects could please take those offline for the moment, or
>> perhaps place them in a validation-only mode if that's possible.
>>
>> Like I said in my last email, which perhaps didn't make it to the talk-ca
>> list (
>> https://lists.openstreetmap.org/pipermail/imports/2019-January/005886.html)
>> I'm now proposing that we leave the data that has already been imported and
>> enter a phase of thorough validation on that data.
>>
>> My plan, over the next several days, is to do a general survey of the
>> quality of the data that has been imported so far and make a list of
>> systematic issues I see that should be addressed before we can consider
>> moving forward again. I'll add those comments to the conversation in
>> talk-ca and on the wiki page (link above), as I feel is appropriate. As I
>> said before, I'm of the mind that this import did not get adequate review
>> or approval and did not follow all the import guidelines. I think therefore
>> we need to take stock, cross the t's, dot the i's, and move this thing back
>> toward where it needs to be. Step one is a thoroughly documented wiki page
>> outlining the proposal and responding to everything required in the import
>> guidelines.
>> https://wiki.openstreetmap.org/wiki/Import/Guidelines
>>
>> I know there are people excited about this import, and people who are
>> eager to get back to work bringing buildings in, but I think everyone will
>> be happier in the end if we take the time to do this right. We don't need
>> to stop forever - we just need to stop until we get things right. I
>> sincerely respect the good intentions of everyone involved in this and I
>> hope we can all work together to make OSM a map known for it's coverage AND
>> it's quality.
>>
>> Best,
>> Nate Wessel
>> Jack of all trades, Master of Geography, PhD candidate in Urban Planning
>> NateWessel.com 
>>
>> On 1/17/19 9:05 PM, OSM Volunteer stevea wrote:
>>
>> The thread link is:  
>> https://lists.openstreetmap.org/pipermail/imports/2019-January/005878.html
>>
>> SteveA
>>
>>
>
> --
> Best Regards,
>   Yaro Shkvorets
> ___
> Talk-ca mailing list
> Talk-ca@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-ca
>
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] Ongoing Canadian building import needs to be stopped, possibly reverted

[Yaro Shkvorets]

Jarek,
There is no question we want this data. I went through much of it in
Toronto and Kingston and I found it to be very good, consistent and
precise. Time-wise it's somewhat current with 2016 ESRI imagery (sometimes
ahead, sometimes slightly behind) and is well-aligned with it. It offers 3D
features (when several buildings appear overlapped in the dataset) but you
just need to be familiar with `building:part` tag to sort through it. I
haven't looked at other provinces but in Ontario I really have no
complaints about dataset quality whatsoever. Also I don't get Nate's
"wildly unsimplified geometries" comment. IMO geometries are just perfectly
detailed.


On Fri, Jan 18, 2019 at 2:00 PM Jarek Piórkowski 
wrote:

> Some more thoughts from me.
>
> Building outlines, particularly for single-family subdivisions as seen
> in Canadian suburbs, are extremely labour-intensive to map manually.
>
> My parents' house is now on OSM - accurately. They live in a city with
> about 10,000 buildings, and about 0.5 active mappers. This wouldn't
> been completed manually in the next 5 years.
>
> An option to do this automatically with a computer algorithm detecting
> objects from imagery could be suggested, but this has not been very
> accurate in OSM in the past, even when there is decent imagery. The
> only other feasible data source is government, where they have such
> data more or less.
>
> The alternative is of course the opinion that we should not have
> building outlines until someone goes through and adds the buildings
> manually. In practice what I've seen done in Toronto is that bigger
> buildings are mapped on best-effort basis from survey and imagery,
> while areas of single-family houses are left blank. This isn't
> _wrong_, and maybe some prefer this.
>
> I would also like to note that building outlines will _never_ be
> completely verifiably up to date. I can't go into most people's
> backyards and verify that there isn't a new addition on their house. A
> building might be legally split into two different properties without
> it being evident from the street. Imagery is out of date the day after
> it's taken, and proper offset can be difficult to establish in big
> cities where GPS signal is erratic. Pragmatically, I can tell you from
> personal experience that building data in lovingly-mapped Berlin is
> also worse than 1 meter accuracy. So again: best effort.
>
> What do we get from having buildings? A sense of land use (arguably
> replaceable with larger landuse areas). A way to roughly estimate
> population density. A way to gauge built-up density. A data source for
> locating buildings in possible flood zones, or fire risk. Statistics:
> as open data, queryable by APIs that are already used, in format
> more-or-less common worldwide.
>
> Examples were given of rowhouse- or de-facto rowhouse-buildings where
> a part is attached to the wrong building. This does not alter any of
> the above examples. It's wrong, but is it substantially more wrong
> than a blank subdivision, or one with only a few buildings mapped? Is
> it better to have a null, or be off by 5%? The legal truth is in
> property records, and we can't measure houses with a ruler, so OSM can
> only be a statistical source. And then there's the question of
> verifiability - some of these buildings are connected to their
> neighbour building inside. I've really struggled at distinguishing
> what exactly is a "building" on Old Toronto avenues even with
> street-side survey.
>
> Bluntly, OSM is not perfect in Canada. I have pet peeves I can quote,
> and I'm sure many of you do as well. If we import, the question is:
> are we making it better?
>
> 1. Do we want this data?
> 2. Is it generally of acceptable quality?
> 3. Is there a mechanism to spot and reject where data is particularly bad?
>
> Cheers,
> Jarek, who should really get back to updating built-last-year stuff at
> Fort York
>
> On Fri, 18 Jan 2019 at 09:31, Kyle Nuttall 
> wrote:
> >
> > The pilot project that took place in Ottawa for all these building
> imports is what got me hooked into OSM in the first place. I would make
> only very minor changes here and there. I even attempted to draw building
> footprints but got burnt out after only doing a single street, which was
> very discouraging for me to continue.
> >
> > When I saw the entire neighbourhood get flooded with new buildings that
> weren't there before, I was entirely intrigued and actually got on board
> with the locals to help with the process. I've been hooked since and have
> been to many meetups afterwards. Helping out with projects completely
> unrelated to the initial building import.
> >
> > I'm entirely of the belief that it is much more encouraging for a new
> user to make a minor change (eg. changing `building=yes` to
> `building=detached`) than it is to add every single minor detail to each
> object from scratch (visiting the location, drawing the building footprints
> manually, adding address data, etc.). It's 

Re: [Talk-ca] Ongoing Canadian building import needs to be stopped, possibly reverted

[Yaro Shkvorets]

John,
>> Traditionally or the party line is if its been mapped already then to
preserve the history you either leave it alone or manually correct it.
Manually correcting it is very time consuming.  Often the decision is made
to leave the existing way in the map.

JOSM offers very convenient way to do it called "Replace geometry". Select
both ways, old and new, press Ctrl-Shift-G, merge any conflicting tags and
you preserve the history, tags and have new improved outline in a couple of
clicks.

On Fri, Jan 18, 2019 at 2:30 PM john whelan  wrote:

> And that is a problem with imports.  Traditionally or the party line is if
> its been mapped already then to preserve the history you either leave it
> alone or manually correct it.  Manually correcting it is very time
> consuming.  Often the decision is made to leave the existing way in the map.
>
> I'm not going to say one method is correct over the other but the least
> contentious is to add only things are are not there already.
>
> Cheerio John
>
> On Fri, 18 Jan 2019 at 14:17, Yaro Shkvorets  wrote:
>
>> Nate,
>> I'll change the project name to reflect that the import is on hold. As a
>> local mapper, if you want to take a lead on the Toronto import that'd be
>> great.
>> I did review some of DannyMcD's edits last night
>> (Mississauga-Brampton-Vaughan) and to be honest was rather disappointed
>> with the quality. It appears Danny chose to import only new buildings (i.e.
>> residential homes mostly), leaving most of the existing hand-traced
>> non-residential building outlines in OSM untouched. That's unfortunate, the
>> dataset offers some really good data and leaving half of it behind makes it
>> more difficult to revisit in the future.
>> In my edits (Markham-Scarborough-East York) I was aiming to replace as
>> many existing geometries with outlines from the import as possible. I think
>> that's what we should be trying to do going forward.
>> Looking forward to your comments and discussion.
>>
>>
>>
>> On Fri, Jan 18, 2019 at 1:07 PM Nate Wessel  wrote:
>>
>>> Hi all,
>>>
>>> I've just joined the talk-ca list, so please accept my apologies for not
>>> addressing this list earlier. I'm happy to take this thread off the imports
>>> list for now and onto talk-ca until things are ready to begin again. The
>>> next person to reply can please feel free to remove that email if they
>>> agree.
>>>
>>> I've just made a note on the draft import plan wiki page noting that the
>>> import has been stopped:
>>>
>>>
>>> https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan
>>>
>>> I would really appreciate it if the person with admin access to the
>>> tasking manager projects could please take those offline for the moment, or
>>> perhaps place them in a validation-only mode if that's possible.
>>>
>>> Like I said in my last email, which perhaps didn't make it to the
>>> talk-ca list (
>>> https://lists.openstreetmap.org/pipermail/imports/2019-January/005886.html)
>>> I'm now proposing that we leave the data that has already been imported and
>>> enter a phase of thorough validation on that data.
>>>
>>> My plan, over the next several days, is to do a general survey of the
>>> quality of the data that has been imported so far and make a list of
>>> systematic issues I see that should be addressed before we can consider
>>> moving forward again. I'll add those comments to the conversation in
>>> talk-ca and on the wiki page (link above), as I feel is appropriate. As I
>>> said before, I'm of the mind that this import did not get adequate review
>>> or approval and did not follow all the import guidelines. I think therefore
>>> we need to take stock, cross the t's, dot the i's, and move this thing back
>>> toward where it needs to be. Step one is a thoroughly documented wiki page
>>> outlining the proposal and responding to everything required in the import
>>> guidelines.
>>> https://wiki.openstreetmap.org/wiki/Import/Guidelines
>>>
>>> I know there are people excited about this import, and people who are
>>> eager to get back to work bringing buildings in, but I think everyone will
>>> be happier in the end if we take the time to do this right. We don't need
>>> to stop forever - we just need to stop until we get things right. I
>>> sincerely respect the good intentions of everyone involved in this and I
>>> hope we can all work together to make OSM a map known for it's coverage AND
>>> it's quality.
>>>
>>> Best,
>>> Nate Wessel
>>> Jack of all trades, Master of Geography, PhD candidate in Urban Planning
>>> NateWessel.com 
>>>
>>> On 1/17/19 9:05 PM, OSM Volunteer stevea wrote:
>>>
>>> The thread link is:  
>>> https://lists.openstreetmap.org/pipermail/imports/2019-January/005878.html
>>>
>>> SteveA
>>>
>>>
>>
>> --
>> Best Regards,
>>   Yaro Shkvorets
>> ___
>> Talk-ca mailing list
>> Talk-ca@openstreetmap.org
>> https://lists.openstreetmap.o

Re: [Talk-ca] Ongoing Canadian building import needs to be stopped, possibly reverted

[john whelan]

Could you update the wiki to include these instructions please.

Thanks John

On Fri, 18 Jan 2019 at 14:53, Yaro Shkvorets  wrote:

> John,
> >> Traditionally or the party line is if its been mapped already then to
> preserve the history you either leave it alone or manually correct it.
> Manually correcting it is very time consuming.  Often the decision is made
> to leave the existing way in the map.
>
> JOSM offers very convenient way to do it called "Replace geometry". Select
> both ways, old and new, press Ctrl-Shift-G, merge any conflicting tags and
> you preserve the history, tags and have new improved outline in a couple of
> clicks.
>
> On Fri, Jan 18, 2019 at 2:30 PM john whelan  wrote:
>
>> And that is a problem with imports.  Traditionally or the party line is
>> if its been mapped already then to preserve the history you either leave it
>> alone or manually correct it.  Manually correcting it is very time
>> consuming.  Often the decision is made to leave the existing way in the map.
>>
>> I'm not going to say one method is correct over the other but the least
>> contentious is to add only things are are not there already.
>>
>> Cheerio John
>>
>> On Fri, 18 Jan 2019 at 14:17, Yaro Shkvorets  wrote:
>>
>>> Nate,
>>> I'll change the project name to reflect that the import is on hold. As a
>>> local mapper, if you want to take a lead on the Toronto import that'd be
>>> great.
>>> I did review some of DannyMcD's edits last night
>>> (Mississauga-Brampton-Vaughan) and to be honest was rather disappointed
>>> with the quality. It appears Danny chose to import only new buildings (i.e.
>>> residential homes mostly), leaving most of the existing hand-traced
>>> non-residential building outlines in OSM untouched. That's unfortunate, the
>>> dataset offers some really good data and leaving half of it behind makes it
>>> more difficult to revisit in the future.
>>> In my edits (Markham-Scarborough-East York) I was aiming to replace as
>>> many existing geometries with outlines from the import as possible. I think
>>> that's what we should be trying to do going forward.
>>> Looking forward to your comments and discussion.
>>>
>>>
>>>
>>> On Fri, Jan 18, 2019 at 1:07 PM Nate Wessel  wrote:
>>>
 Hi all,

 I've just joined the talk-ca list, so please accept my apologies for
 not addressing this list earlier. I'm happy to take this thread off the
 imports list for now and onto talk-ca until things are ready to begin
 again. The next person to reply can please feel free to remove that email
 if they agree.

 I've just made a note on the draft import plan wiki page noting that
 the import has been stopped:


 https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan

 I would really appreciate it if the person with admin access to the
 tasking manager projects could please take those offline for the moment, or
 perhaps place them in a validation-only mode if that's possible.

 Like I said in my last email, which perhaps didn't make it to the
 talk-ca list (
 https://lists.openstreetmap.org/pipermail/imports/2019-January/005886.html)
 I'm now proposing that we leave the data that has already been imported and
 enter a phase of thorough validation on that data.

 My plan, over the next several days, is to do a general survey of the
 quality of the data that has been imported so far and make a list of
 systematic issues I see that should be addressed before we can consider
 moving forward again. I'll add those comments to the conversation in
 talk-ca and on the wiki page (link above), as I feel is appropriate. As I
 said before, I'm of the mind that this import did not get adequate review
 or approval and did not follow all the import guidelines. I think therefore
 we need to take stock, cross the t's, dot the i's, and move this thing back
 toward where it needs to be. Step one is a thoroughly documented wiki page
 outlining the proposal and responding to everything required in the import
 guidelines.
 https://wiki.openstreetmap.org/wiki/Import/Guidelines

 I know there are people excited about this import, and people who are
 eager to get back to work bringing buildings in, but I think everyone will
 be happier in the end if we take the time to do this right. We don't need
 to stop forever - we just need to stop until we get things right. I
 sincerely respect the good intentions of everyone involved in this and I
 hope we can all work together to make OSM a map known for it's coverage AND
 it's quality.

 Best,
 Nate Wessel
 Jack of all trades, Master of Geography, PhD candidate in Urban Planning
 NateWessel.com 

 On 1/17/19 9:05 PM, OSM Volunteer stevea wrote:

 The thread link is:  
 https://lists.openstreetmap.org/pipermail/impor

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[Nate Wessel]

John,

You seem to be playing the long game with this data - it sounds like 
you've been working with this a lot longer than I have, and you've put 
in the time and effort to help make this actually-quite-incredible 
dataset available to us. I don't want to stop the import from happening 
- quite the opposite. I just want to make sure that the time is taken to 
do this right. OSM deserves that. Your (our) long awaited victory will 
be the sweeter for our patience now.


There are several specific issues I see where the I's are not crossed, 
nor the t's dotted. I've mentioned several already, so I'll try to be 
brief (I really need to get back to working on my dissertation).


1) There was extremely limited discussion on the imports mailing list. 
The initial email did not make clear the scope of the project. I read 
the email and did not think twice at it, thinking it was entirely about 
Ottawa. The link in that email was actually to the Ottawa import, and 
not this one, which seems to have been only in draft at the time. 
https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
As such, this project has NOT been reviewed by the imports list, which 
is a requirement for proceeding with the import.


2) There is no mention of this proposed import on the import catalogue 
(https://wiki.openstreetmap.org/wiki/Import/Catalogue)
which is required in the imports guidelines. I suspect many other 
guidelines have not been followed.


3) The wiki page describing the import is not adequate to assess the 
quality of the data or of the proposed import. See for example: 
https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks
The import guidelines call for a description of how conflation will be 
handled. The fact that two of the major importers seem to have a 
substantial disagreement about how to handle existing data indicates 
this was not well discussed and I can see that it isn't well documented.


4) The buildings need to be simplified, quite a bit actually. Most 
buildings have multiple nodes representing straight lines. This bloats 
the database and makes things harder to edit by hand later. There are 
probably 2x more nodes than are needed to represent the data accurately, 
making it harder for editors and data consumers to work with down the 
road.This is a simple fix that will save countless hours later.


... I could go on, but I think this is plenty sufficient to justify 
pressing pause on all this.


Again, I don't in any way want to disrespect the work that has gone into 
this effort already. We're all volunteers here and I know how much time 
this all takes. However. importing all/most of the buildings in Canada 
is a monstrously large task, which will have to dance around a lot of 
people's toes. We should expect this to take a really damn long time if 
we're going to do it right. We need to have the patience to learn from 
experience, from critique, and from the wisdom of the people who've 
learned from flawed imports in the past and have devised guidelines and 
processes so that we can have better experiences with this in the future.


Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban Planning
NateWessel.com 

On 1/18/19 2:24 PM, john whelan wrote:
My background is I'm a retired civil servant who has written and 
overseen procurement documents and fairly large procurements. Dotting 
the is and crossing the Ts are my speciality.


There are two parts to an import.  The first part is the part played 
by the import mailing group.  They confine themselves to is the 
license correct and do you have a reasonable plan.  In this case the 
license is one of the few that has been confirmed by the Legal Working 
Group of OpenStreetMap and as such no questions were raised about it 
on the import mailing list.  We have methodology that has been used 
before successfully with the Ottawa building outline import. There 
were major discussions both on talk-ca and the import mailing group 
before that import took place and we took note of the issues raised 
and addressed them.  The licensing issue goes back about eight years 
to when I was talking to Federal Government Treasury Board and 
explaining their Open Data license did not align with OSM.  That is 
why their license is now known as 2.0.


The second part is the local group makes the decision to import they 
are the authority no one else.


Apparently you were not part of the talk-ca when the discussions took 
place which would have been the time and place to raise concerns.


When the Ottawa import was done there were one or two places where the 
existing buildings and the import overlapped.  In the instructions on 
the import there are instructions to cover this. Specifically there is 
a validation step.  I seem to recall the error rate was of the order 
of 1% and I expect this latest batch to be roughly the same.


If you can identify w

Re: [Talk-ca] [Imports] Ongoing Canadian building import needs to be stopped, possibly reverted

[James]

https://wiki.openstreetmap.org/wiki/JOSM/Plugins/utilsplugin2

On Fri., Jan. 18, 2019, 3:02 p.m. Kevin Kenny  On Fri, Jan 18, 2019 at 2:54 PM Yaro Shkvorets 
> wrote:
> > JOSM offers very convenient way to do it called "Replace geometry".
> Select both ways, old and new, press Ctrl-Shift-G, merge any conflicting
> tags and you preserve the history, tags and have new improved outline in a
> couple of clicks.
>
> Good point. I use that a *lot* when updating the New York public land
> boundaries. Is it in a stock JOSM now? You used to have to install a
> plugin (with some uninformative name like 'Utilities') to get it. It's
> an absolute necessity for importers.
>
> ___
> Imports mailing list
> impo...@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
>
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[John Whelan]

The import mailing list was pointed to the correct page of the wiki.  
The initial post was to say this is what we were thinking of and there 
was a comment saying we needed to change the comment line.


>There is no mention of this proposed import on the import catalogue


The import process was reviewed by the person who set up the Ottawa 
import did we miss that step on the Ottawa import as well?  Neither was 
it raised as a concern on the import mailing list. I think this is very 
minor and can be corrected.


We learnt a fair bit on the Ottawa import and my expectation is since we 
are using experienced mappers to do the import conflation would be 
either handled by them or the building not imported. We aren't using new 
mappers in a mapathon here and with experienced mappers then I think you 
have to trust them.  The world isn't perfect. Think in terms of service 
level.


>There are 2X more nodes than needed to represent the building accurately.

The problem with correcting this is you are introducing approximations. 
This will vary according to the source and this can be simplified or 
corrected once its in OSM. I think this is a different issue of a 
mechanical edit that needs to be considered separately.


If we are concerned with database size then I suggest we change the 
instructions to say put the source comment on the change set rather than 
on the building outline.


Cheerio John


Nate Wessel wrote on 2019-01-18 3:06 PM:


John,

You seem to be playing the long game with this data - it sounds like 
you've been working with this a lot longer than I have, and you've put 
in the time and effort to help make this actually-quite-incredible 
dataset available to us. I don't want to stop the import from 
happening - quite the opposite. I just want to make sure that the time 
is taken to do this right. OSM deserves that. Your (our) long awaited 
victory will be the sweeter for our patience now.


There are several specific issues I see where the I's are not crossed, 
nor the t's dotted. I've mentioned several already, so I'll try to be 
brief (I really need to get back to working on my dissertation).


1) There was extremely limited discussion on the imports mailing list. 
The initial email did not make clear the scope of the project. I read 
the email and did not think twice at it, thinking it was entirely 
about Ottawa. The link in that email was actually to the Ottawa 
import, and not this one, which seems to have been only in draft at 
the time. 
https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
As such, this project has NOT been reviewed by the imports list, which 
is a requirement for proceeding with the import.


2) There is no mention of this proposed import on the import catalogue 
(https://wiki.openstreetmap.org/wiki/Import/Catalogue)
which is required in the imports guidelines. I suspect many other 
guidelines have not been followed.


3) The wiki page describing the import is not adequate to assess the 
quality of the data or of the proposed import. See for example: 
https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks
The import guidelines call for a description of how conflation will be 
handled. The fact that two of the major importers seem to have a 
substantial disagreement about how to handle existing data indicates 
this was not well discussed and I can see that it isn't well documented.


4) The buildings need to be simplified, quite a bit actually. Most 
buildings have multiple nodes representing straight lines. This bloats 
the database and makes things harder to edit by hand later. There are 
probably 2x more nodes than are needed to represent the data 
accurately, making it harder for editors and data consumers to work 
with down the road.This is a simple fix that will save countless hours 
later.


... I could go on, but I think this is plenty sufficient to justify 
pressing pause on all this.


Again, I don't in any way want to disrespect the work that has gone 
into this effort already. We're all volunteers here and I know how 
much time this all takes. However. importing all/most of the buildings 
in Canada is a monstrously large task, which will have to dance around 
a lot of people's toes. We should expect this to take a really damn 
long time if we're going to do it right. We need to have the patience 
to learn from experience, from critique, and from the wisdom of the 
people who've learned from flawed imports in the past and have devised 
guidelines and processes so that we can have better experiences with 
this in the future.


Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban Planning
NateWessel.com 

On 1/18/19 2:24 PM, john whelan wrote:
My background is I'm a retired civil servant who has written and 
overseen procurement documents and fairly large procurements. Dotting 
the is and crossing the Ts are my speciality.


There are t

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[James]

dare you to run simplify tool on anything remotely round, it will make it
look like garbage

On Fri., Jan. 18, 2019, 3:49 p.m. John Whelan  The import mailing list was pointed to the correct page of the wiki.  The
> initial post was to say this is what we were thinking of and there was a
> comment saying we needed to change the comment line.
>
> >There is no mention of this proposed import on the import catalogue
>
>
> The import process was reviewed by the person who set up the Ottawa import
> did we miss that step on the Ottawa import as well?  Neither was it raised
> as a concern on the import mailing list. I think this is very minor and can
> be corrected.
>
> We learnt a fair bit on the Ottawa import and my expectation is since we
> are using experienced mappers to do the import conflation would be either
> handled by them or the building not imported. We aren't using new mappers
> in a mapathon here and with experienced mappers then I think you have to
> trust them.  The world isn't perfect. Think in terms of service level.
>
> >There are 2X more nodes than needed to represent the building accurately.
>
> The problem with correcting this is you are introducing approximations.
> This will vary according to the source and this can be simplified or
> corrected once its in OSM. I think this is a different issue of a
> mechanical edit that needs to be considered separately.
>
> If we are concerned with database size then I suggest we change the
> instructions to say put the source comment on the change set rather than on
> the building outline.
>
> Cheerio John
>
>
> Nate Wessel wrote on 2019-01-18 3:06 PM:
>
> John,
>
> You seem to be playing the long game with this data - it sounds like
> you've been working with this a lot longer than I have, and you've put in
> the time and effort to help make this actually-quite-incredible dataset
> available to us. I don't want to stop the import from happening - quite the
> opposite. I just want to make sure that the time is taken to do this right.
> OSM deserves that. Your (our) long awaited victory will be the sweeter for
> our patience now.
>
> There are several specific issues I see where the I's are not crossed, nor
> the t's dotted. I've mentioned several already, so I'll try to be brief (I
> really need to get back to working on my dissertation).
>
> 1) There was extremely limited discussion on the imports mailing list. The
> initial email did not make clear the scope of the project. I read the email
> and did not think twice at it, thinking it was entirely about Ottawa. The
> link in that email was actually to the Ottawa import, and not this one,
> which seems to have been only in draft at the time.
> https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
> As such, this project has NOT been reviewed by the imports list, which is
> a requirement for proceeding with the import.
>
> 2) There is no mention of this proposed import on the import catalogue (
> https://wiki.openstreetmap.org/wiki/Import/Catalogue)
> which is required in the imports guidelines. I suspect many other
> guidelines have not been followed.
>
> 3) The wiki page describing the import is not adequate to assess the
> quality of the data or of the proposed import. See for example:
> https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks
> The import guidelines call for a description of how conflation will be
> handled. The fact that two of the major importers seem to have a
> substantial disagreement about how to handle existing data indicates this
> was not well discussed and I can see that it isn't well documented.
>
> 4) The buildings need to be simplified, quite a bit actually. Most
> buildings have multiple nodes representing straight lines. This bloats the
> database and makes things harder to edit by hand later. There are probably
> 2x more nodes than are needed to represent the data accurately, making it
> harder for editors and data consumers to work with down the road.This is a
> simple fix that will save countless hours later.
>
> ... I could go on, but I think this is plenty sufficient to justify
> pressing pause on all this.
>
> Again, I don't in any way want to disrespect the work that has gone into
> this effort already. We're all volunteers here and I know how much time
> this all takes. However. importing all/most of the buildings in Canada is a
> monstrously large task, which will have to dance around a lot of people's
> toes. We should expect this to take a really damn long time if we're going
> to do it right. We need to have the patience to learn from experience, from
> critique, and from the wisdom of the people who've learned from flawed
> imports in the past and have devised guidelines and processes so that we
> can have better experiences with this in the future.
> Nate Wessel
> Jack of all trades, Master of Geography, PhD candidate in Urban Planning
> NateWessel.com 

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[John Whelan]

James you know I could never resist a dare!

Cheerio John

James wrote on 2019-01-18 4:03 PM:
dare you to run simplify tool on anything remotely round, it will make 
it look like garbage


On Fri., Jan. 18, 2019, 3:49 p.m. John Whelan  wrote:


The import mailing list was pointed to the correct page of the
wiki.  The initial post was to say this is what we were thinking
of and there was a comment saying we needed to change the comment
line.

>There is no mention of this proposed import on the import catalogue


The import process was reviewed by the person who set up the
Ottawa import did we miss that step on the Ottawa import as well? 
Neither was it raised as a concern on the import mailing list. I
think this is very minor and can be corrected.

We learnt a fair bit on the Ottawa import and my expectation is
since we are using experienced mappers to do the import conflation
would be either handled by them or the building not imported. We
aren't using new mappers in a mapathon here and with experienced
mappers then I think you have to trust them.  The world isn't
perfect. Think in terms of service level.

>There are 2X more nodes than needed to represent the building
accurately.

The problem with correcting this is you are introducing
approximations. This will vary according to the source and this
can be simplified or corrected once its in OSM. I think this is a
different issue of a mechanical edit that needs to be considered
separately.

If we are concerned with database size then I suggest we change
the instructions to say put the source comment on the change set
rather than on the building outline.

Cheerio John


Nate Wessel wrote on 2019-01-18 3:06 PM:


John,

You seem to be playing the long game with this data - it sounds
like you've been working with this a lot longer than I have, and
you've put in the time and effort to help make this
actually-quite-incredible dataset available to us. I don't want
to stop the import from happening - quite the opposite. I just
want to make sure that the time is taken to do this right. OSM
deserves that. Your (our) long awaited victory will be the
sweeter for our patience now.

There are several specific issues I see where the I's are not
crossed, nor the t's dotted. I've mentioned several already, so
I'll try to be brief (I really need to get back to working on my
dissertation).

1) There was extremely limited discussion on the imports mailing
list. The initial email did not make clear the scope of the
project. I read the email and did not think twice at it, thinking
it was entirely about Ottawa. The link in that email was actually
to the Ottawa import, and not this one, which seems to have been
only in draft at the time.
https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
As such, this project has NOT been reviewed by the imports list,
which is a requirement for proceeding with the import.

2) There is no mention of this proposed import on the import
catalogue (https://wiki.openstreetmap.org/wiki/Import/Catalogue)
which is required in the imports guidelines. I suspect many other
guidelines have not been followed.

3) The wiki page describing the import is not adequate to assess
the quality of the data or of the proposed import. See for
example:

https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks
The import guidelines call for a description of how conflation
will be handled. The fact that two of the major importers seem to
have a substantial disagreement about how to handle existing data
indicates this was not well discussed and I can see that it isn't
well documented.

4) The buildings need to be simplified, quite a bit actually.
Most buildings have multiple nodes representing straight lines.
This bloats the database and makes things harder to edit by hand
later. There are probably 2x more nodes than are needed to
represent the data accurately, making it harder for editors and
data consumers to work with down the road.This is a simple fix
that will save countless hours later.

... I could go on, but I think this is plenty sufficient to
justify pressing pause on all this.

Again, I don't in any way want to disrespect the work that has
gone into this effort already. We're all volunteers here and I
know how much time this all takes. However. importing all/most of
the buildings in Canada is a monstrously large task, which will
have to dance around a lot of people's toes. We should expect
this to take a really damn long time if we're going to do it
right. We need to have the patience to learn from experience,
from critique, and from the wisdom o

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[Nate Wessel]

With default settings in JOSM, sure. In the import I was working on, we 
used a Douglas-Peucker algorithm with a 20cm threshold (before the 
import started) and it worked beautifully. We had many points that 
seemed to have been introduced in the shapefiles as some kind of data 
artifact - they didn't add any detail to the shape at all. This 
procedure removed almost all of them with no discernible reduction in 
quality.


Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban Planning
NateWessel.com 

On 1/18/19 4:03 PM, James wrote:
dare you to run simplify tool on anything remotely round, it will make 
it look like garbage


On Fri., Jan. 18, 2019, 3:49 p.m. John Whelan  wrote:


The import mailing list was pointed to the correct page of the
wiki. The initial post was to say this is what we were thinking of
and there was a comment saying we needed to change the comment line.

>There is no mention of this proposed import on the import catalogue


The import process was reviewed by the person who set up the
Ottawa import did we miss that step on the Ottawa import as well? 
Neither was it raised as a concern on the import mailing list. I
think this is very minor and can be corrected.

We learnt a fair bit on the Ottawa import and my expectation is
since we are using experienced mappers to do the import conflation
would be either handled by them or the building not imported. We
aren't using new mappers in a mapathon here and with experienced
mappers then I think you have to trust them.  The world isn't
perfect. Think in terms of service level.

>There are 2X more nodes than needed to represent the building
accurately.

The problem with correcting this is you are introducing
approximations.  This will vary according to the source and this
can be simplified or corrected once its in OSM. I think this is a
different issue of a mechanical edit that needs to be considered
separately.

If we are concerned with database size then I suggest we change
the instructions to say put the source comment on the change set
rather than on the building outline.

Cheerio John


Nate Wessel wrote on 2019-01-18 3:06 PM:


John,

You seem to be playing the long game with this data - it sounds
like you've been working with this a lot longer than I have, and
you've put in the time and effort to help make this
actually-quite-incredible dataset available to us. I don't want
to stop the import from happening - quite the opposite. I just
want to make sure that the time is taken to do this right. OSM
deserves that. Your (our) long awaited victory will be the
sweeter for our patience now.

There are several specific issues I see where the I's are not
crossed, nor the t's dotted. I've mentioned several already, so
I'll try to be brief (I really need to get back to working on my
dissertation).

1) There was extremely limited discussion on the imports mailing
list. The initial email did not make clear the scope of the
project. I read the email and did not think twice at it, thinking
it was entirely about Ottawa. The link in that email was actually
to the Ottawa import, and not this one, which seems to have been
only in draft at the time.
https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
As such, this project has NOT been reviewed by the imports list,
which is a requirement for proceeding with the import.

2) There is no mention of this proposed import on the import
catalogue (https://wiki.openstreetmap.org/wiki/Import/Catalogue)
which is required in the imports guidelines. I suspect many other
guidelines have not been followed.

3) The wiki page describing the import is not adequate to assess
the quality of the data or of the proposed import. See for
example:

https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks
The import guidelines call for a description of how conflation
will be handled. The fact that two of the major importers seem to
have a substantial disagreement about how to handle existing data
indicates this was not well discussed and I can see that it isn't
well documented.

4) The buildings need to be simplified, quite a bit actually.
Most buildings have multiple nodes representing straight lines.
This bloats the database and makes things harder to edit by hand
later. There are probably 2x more nodes than are needed to
represent the data accurately, making it harder for editors and
data consumers to work with down the road.This is a simple fix
that will save countless hours later.

... I could go on, but I think this is plenty sufficient to
justify pressing pause on all this.

Again, I don't in a

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[James]

I can run all the shapefiles through qgis simplify tool if this resolves
the issue...

On Fri., Jan. 18, 2019, 4:08 p.m. Nate Wessel  With default settings in JOSM, sure. In the import I was working on, we
> used a Douglas-Peucker algorithm with a 20cm threshold (before the import
> started) and it worked beautifully. We had many points that seemed to have
> been introduced in the shapefiles as some kind of data artifact - they
> didn't add any detail to the shape at all. This procedure removed almost
> all of them with no discernible reduction in quality.
> Nate Wessel
> Jack of all trades, Master of Geography, PhD candidate in Urban Planning
> NateWessel.com 
>
> On 1/18/19 4:03 PM, James wrote:
>
> dare you to run simplify tool on anything remotely round, it will make it
> look like garbage
>
> On Fri., Jan. 18, 2019, 3:49 p.m. John Whelan  wrote:
>
>> The import mailing list was pointed to the correct page of the wiki.  The
>> initial post was to say this is what we were thinking of and there was a
>> comment saying we needed to change the comment line.
>>
>> >There is no mention of this proposed import on the import catalogue
>>
>>
>> The import process was reviewed by the person who set up the Ottawa
>> import did we miss that step on the Ottawa import as well?  Neither was it
>> raised as a concern on the import mailing list. I think this is very minor
>> and can be corrected.
>>
>> We learnt a fair bit on the Ottawa import and my expectation is since we
>> are using experienced mappers to do the import conflation would be either
>> handled by them or the building not imported. We aren't using new mappers
>> in a mapathon here and with experienced mappers then I think you have to
>> trust them.  The world isn't perfect. Think in terms of service level.
>>
>> >There are 2X more nodes than needed to represent the building accurately.
>>
>> The problem with correcting this is you are introducing approximations.
>> This will vary according to the source and this can be simplified or
>> corrected once its in OSM. I think this is a different issue of a
>> mechanical edit that needs to be considered separately.
>>
>> If we are concerned with database size then I suggest we change the
>> instructions to say put the source comment on the change set rather than on
>> the building outline.
>>
>> Cheerio John
>>
>>
>> Nate Wessel wrote on 2019-01-18 3:06 PM:
>>
>> John,
>>
>> You seem to be playing the long game with this data - it sounds like
>> you've been working with this a lot longer than I have, and you've put in
>> the time and effort to help make this actually-quite-incredible dataset
>> available to us. I don't want to stop the import from happening - quite the
>> opposite. I just want to make sure that the time is taken to do this right.
>> OSM deserves that. Your (our) long awaited victory will be the sweeter for
>> our patience now.
>>
>> There are several specific issues I see where the I's are not crossed,
>> nor the t's dotted. I've mentioned several already, so I'll try to be brief
>> (I really need to get back to working on my dissertation).
>>
>> 1) There was extremely limited discussion on the imports mailing list.
>> The initial email did not make clear the scope of the project. I read the
>> email and did not think twice at it, thinking it was entirely about Ottawa.
>> The link in that email was actually to the Ottawa import, and not this one,
>> which seems to have been only in draft at the time.
>> https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
>> As such, this project has NOT been reviewed by the imports list, which is
>> a requirement for proceeding with the import.
>>
>> 2) There is no mention of this proposed import on the import catalogue (
>> https://wiki.openstreetmap.org/wiki/Import/Catalogue)
>> which is required in the imports guidelines. I suspect many other
>> guidelines have not been followed.
>>
>> 3) The wiki page describing the import is not adequate to assess the
>> quality of the data or of the proposed import. See for example:
>> https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks
>> The import guidelines call for a description of how conflation will be
>> handled. The fact that two of the major importers seem to have a
>> substantial disagreement about how to handle existing data indicates this
>> was not well discussed and I can see that it isn't well documented.
>>
>> 4) The buildings need to be simplified, quite a bit actually. Most
>> buildings have multiple nodes representing straight lines. This bloats the
>> database and makes things harder to edit by hand later. There are probably
>> 2x more nodes than are needed to represent the data accurately, making it
>> harder for editors and data consumers to work with down the road.This is a
>> simple fix that will save countless hours later.
>>
>> ... I could go on, but I think this is plenty sufficient to jus

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[Yaro Shkvorets]

I'm on board with removing redundant nodes, as long as it doesn't affect
actual geometries much.
Also there are quite a few duplicated nodes in buildings. They can be
removed in JOSM in one click before upload but better to do it at the
source.

On Fri, Jan 18, 2019 at 4:25 PM James  wrote:

> I can run all the shapefiles through qgis simplify tool if this resolves
> the issue...
>
> On Fri., Jan. 18, 2019, 4:08 p.m. Nate Wessel 
>> With default settings in JOSM, sure. In the import I was working on, we
>> used a Douglas-Peucker algorithm with a 20cm threshold (before the import
>> started) and it worked beautifully. We had many points that seemed to have
>> been introduced in the shapefiles as some kind of data artifact - they
>> didn't add any detail to the shape at all. This procedure removed almost
>> all of them with no discernible reduction in quality.
>> Nate Wessel
>> Jack of all trades, Master of Geography, PhD candidate in Urban Planning
>> NateWessel.com 
>>
>> On 1/18/19 4:03 PM, James wrote:
>>
>> dare you to run simplify tool on anything remotely round, it will make it
>> look like garbage
>>
>> On Fri., Jan. 18, 2019, 3:49 p.m. John Whelan > wrote:
>>
>>> The import mailing list was pointed to the correct page of the wiki.
>>> The initial post was to say this is what we were thinking of and there was
>>> a comment saying we needed to change the comment line.
>>>
>>> >There is no mention of this proposed import on the import catalogue
>>>
>>>
>>> The import process was reviewed by the person who set up the Ottawa
>>> import did we miss that step on the Ottawa import as well?  Neither was it
>>> raised as a concern on the import mailing list. I think this is very minor
>>> and can be corrected.
>>>
>>> We learnt a fair bit on the Ottawa import and my expectation is since we
>>> are using experienced mappers to do the import conflation would be either
>>> handled by them or the building not imported. We aren't using new mappers
>>> in a mapathon here and with experienced mappers then I think you have to
>>> trust them.  The world isn't perfect. Think in terms of service level.
>>>
>>> >There are 2X more nodes than needed to represent the building
>>> accurately.
>>>
>>> The problem with correcting this is you are introducing approximations.
>>> This will vary according to the source and this can be simplified or
>>> corrected once its in OSM. I think this is a different issue of a
>>> mechanical edit that needs to be considered separately.
>>>
>>> If we are concerned with database size then I suggest we change the
>>> instructions to say put the source comment on the change set rather than on
>>> the building outline.
>>>
>>> Cheerio John
>>>
>>>
>>> Nate Wessel wrote on 2019-01-18 3:06 PM:
>>>
>>> John,
>>>
>>> You seem to be playing the long game with this data - it sounds like
>>> you've been working with this a lot longer than I have, and you've put in
>>> the time and effort to help make this actually-quite-incredible dataset
>>> available to us. I don't want to stop the import from happening - quite the
>>> opposite. I just want to make sure that the time is taken to do this right.
>>> OSM deserves that. Your (our) long awaited victory will be the sweeter for
>>> our patience now.
>>>
>>> There are several specific issues I see where the I's are not crossed,
>>> nor the t's dotted. I've mentioned several already, so I'll try to be brief
>>> (I really need to get back to working on my dissertation).
>>>
>>> 1) There was extremely limited discussion on the imports mailing list.
>>> The initial email did not make clear the scope of the project. I read the
>>> email and did not think twice at it, thinking it was entirely about Ottawa.
>>> The link in that email was actually to the Ottawa import, and not this one,
>>> which seems to have been only in draft at the time.
>>> https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
>>> As such, this project has NOT been reviewed by the imports list, which
>>> is a requirement for proceeding with the import.
>>>
>>> 2) There is no mention of this proposed import on the import catalogue (
>>> https://wiki.openstreetmap.org/wiki/Import/Catalogue)
>>> which is required in the imports guidelines. I suspect many other
>>> guidelines have not been followed.
>>>
>>> 3) The wiki page describing the import is not adequate to assess the
>>> quality of the data or of the proposed import. See for example:
>>> https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks
>>> The import guidelines call for a description of how conflation will be
>>> handled. The fact that two of the major importers seem to have a
>>> substantial disagreement about how to handle existing data indicates this
>>> was not well discussed and I can see that it isn't well documented.
>>>
>>> 4) The buildings need to be simplified, quite a bit actually. Most
>>> buildings have multiple nodes repr

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[James]

You guys can analyze the simplified version of ontario:
https://drive.google.com/file/d/1OK83yrPwMW4nefyu-6JsIInu0meK2rW6/view?usp=sharing
If you think it's good, I can simplify the other files and process them
into mbtiles.
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[John Marshall]

I found the building footprints to be very good. JOSM cleaned up most of
the errors..I'm not sure it would be worth the risk to do more processing.
Sometimes there were crossing ways, usually with  terrace or buildings in
the downtown core that need to be fixed manually. Which took a ton of time.
I always removed all the errors from import buildings before I added the
OSM data.

 I also keeped the city name on the imported data so I could tell which was
the imported data/ OSM data. I removed the city name later before
uploading. If there was already a building in OSM, I used the "Replace
Geometry" unless the building in OSM was better. Sometimes , the local
mapper had done a better job than the city, so I just left it.The building
in the Niagara Region had tons of very good buildings that I just left.

I also tired to fixed other JOSM errors, Like adding road names, missing
tags, spelling error, natural=land ect.  Using Geo Base I added about 300
names to roads mostly around  Muskoka and Goderich. This usually took more
time than adding the buildings.

I would say adding buildings in rural, residential, & Industrial areas
there is low risk for problems. The downtown areas even in small
cities Port Colborne were very time consuming and require an experienced
mapper.. Personally I wouldn't even want to try downtown Toronto.

FYI,  The top 3 CDN OSM mappers are importing the building data.
http://osmstats.neis-one.org/?item=countries&country=Canada


Cheers

John


On Fri, Jan 18, 2019 at 5:30 PM James  wrote:

> You guys can analyze the simplified version of ontario:
>
> https://drive.google.com/file/d/1OK83yrPwMW4nefyu-6JsIInu0meK2rW6/view?usp=sharing
> If you think it's good, I can simplify the other files and process them
> into mbtiles.
> ___
> Talk-ca mailing list
> Talk-ca@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-ca
>
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[Pierre Béland]

John,
Il y a local et local. Compte-tenu des différences culturelles Québec vs Canada 
en général et que les contributeurs du Québec ne fréquentent pratiquement pas 
cette liste, vous ne devriez pas prendre pour acquis que vous représentez cette 
communauté et pouvez démarrer des projets en son nom.
 
Pierre 
 

Le vendredi 18 janvier 2019 13 h 11 min 37 s HNE, john whelan 
 a écrit :  
 
 I know of no other way to contact him but he made an interesting comment that 
the project is on hold in the wiki pending review.
Would he care to comment on who is supposed to be reviewing the project?
My understanding is that the import was raised in talk-ca before it commenced 
for comment and these were generally favourable.  I took that as the local 
mappers to Canada had been consulted and they are the "local mappers" authority 
in this case.
I understand he has concerns about local mappers making decisions but in Canada 
we have been importing similar data through CANVEC for some time.  CANVEC data 
comes from a number of sources including municipal data.
Is he suggesting that each of the 3,700 municipalities in Canada should form a 
group of local mappers who can make individual decisions on whether their 
municipal data should be imported and we should end up with 3,700 import plans?
Thanks John

___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca
  ___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[John Whelan]

I agree and we are sensitive to Quebec's position.

I think the hope was we would make the data available and that local 
mappers would be involved in the import over time as happens with 
CANVEC.  One comment I heard early on was this isn't so much an import 
as a marathon.


What seems to have happened is a lot of buildings have been imported 
very quickly.  In rural areas or places where there are few buildings 
this isn't so much of a problem. Certain locations have run in very 
smoothly.


I think the data quality is considerably better than iD and Mapathons 
with new mappers.


The original data files are available on an Open Data portal with a 
license that is compatible with OSM and to be honest we have very little 
control over who can download them or what they do with them.


What we do have is a process that was used in Ottawa and is fairly 
robust. Data quality is very dependent on the individual mappers doing 
the import though.


Looking at the stats I don't think much has been done in Quebec and I 
feel James would be happy to restrict access Quebec in someway if that 
would make you happier for the moment.  It has been set up as a separate 
set of tiles so can be isolated fairly easily.


Could you be nice and chat to the Quebec mappers and sound them out on 
what they would like to do?  The data for Quebec is from Quebec 
municipalities by the way.  Please bear in mind that Microsoft are 
rumoured to be about to release building data for Canada in the same way 
as they have for the US.  This is scanned from images data and I suspect 
the data quality will not be as high as the Municipal data.  I 
understand there are multiple imports going on with the US Microsoft 
building outline data currently.


I seem to recall that Daniel Begin, who I believe is a Quebec mapper, 
made comments on the project in talk-ca some time ago.  I also seem to 
recall it was his suggestion that we made it a single import plan.


Thoughts?

Thanks John

Pierre Béland wrote on 2019-01-18 6:54 PM:

John,

Il y a local et local. Compte-tenu des différences culturelles Québec 
vs Canada en général et que les contributeurs du Québec ne fréquentent 
pratiquement pas cette liste, vous ne devriez pas prendre pour acquis 
que vous représentez cette communauté et pouvez démarrer des projets 
en son nom.



Pierre


Le vendredi 18 janvier 2019 13 h 11 min 37 s HNE, john whelan 
 a écrit :



I know of no other way to contact him but he made an interesting 
comment that the project is on hold in the wiki pending review.


Would he care to comment on who is supposed to be reviewing the project?

My understanding is that the import was raised in talk-ca before it 
commenced for comment and these were generally favourable.  I took 
that as the local mappers to Canada had been consulted and they are 
the "local mappers" authority in this case.


I understand he has concerns about local mappers making decisions but 
in Canada we have been importing similar data through CANVEC for some 
time.  CANVEC data comes from a number of sources including municipal 
data.


Is he suggesting that each of the 3,700 municipalities in Canada 
should form a group of local mappers who can make individual decisions 
on whether their municipal data should be imported and we should end 
up with 3,700 import plans?


Thanks John


___
Talk-ca mailing list
Talk-ca@openstreetmap.org 
https://lists.openstreetmap.org/listinfo/talk-ca


--
Sent from Postbox 

___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] [Imports] Ongoing Canadian building import needs to be stopped, possibly reverted

[Kevin Kenny]

On Fri, Jan 18, 2019 at 2:54 PM Yaro Shkvorets  wrote:
> JOSM offers very convenient way to do it called "Replace geometry". Select 
> both ways, old and new, press Ctrl-Shift-G, merge any conflicting tags and 
> you preserve the history, tags and have new improved outline in a couple of 
> clicks.

Good point. I use that a *lot* when updating the New York public land
boundaries. Is it in a stock JOSM now? You used to have to install a
plugin (with some uninformative name like 'Utilities') to get it. It's
an absolute necessity for importers.

___
Talk-ca mailing list
Talk-ca@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-ca

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[Nate Wessel]

I'm not familiar with the tool, but that is essentially what I'm asking 
for -  nothing all that complicated. We would need to make sure we're 
not losing any valuable detail though, and ensure that topology is 
preserved where buildings share nodes.


Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban Planning
NateWessel.com 

On 1/18/19 4:24 PM, James wrote:
I can run all the shapefiles through qgis simplify tool if this 
resolves the issue...


On Fri., Jan. 18, 2019, 4:08 p.m. Nate Wessel  wrote:


With default settings in JOSM, sure. In the import I was working
on, we used a Douglas-Peucker algorithm with a 20cm threshold
(before the import started) and it worked beautifully. We had many
points that seemed to have been introduced in the shapefiles as
some kind of data artifact - they didn't add any detail to the
shape at all. This procedure removed almost all of them with no
discernible reduction in quality.

Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban
Planning
NateWessel.com 

On 1/18/19 4:03 PM, James wrote:

dare you to run simplify tool on anything remotely round, it will
make it look like garbage

On Fri., Jan. 18, 2019, 3:49 p.m. John Whelan
mailto:jwhelan0...@gmail.com> wrote:

The import mailing list was pointed to the correct page of
the wiki.  The initial post was to say this is what we were
thinking of and there was a comment saying we needed to
change the comment line.

>There is no mention of this proposed import on the import
catalogue


The import process was reviewed by the person who set up the
Ottawa import did we miss that step on the Ottawa import as
well?  Neither was it raised as a concern on the import
mailing list. I think this is very minor and can be corrected.

We learnt a fair bit on the Ottawa import and my expectation
is since we are using experienced mappers to do the import
conflation would be either handled by them or the building
not imported. We aren't using new mappers in a mapathon here
and with experienced mappers then I think you have to trust
them.  The world isn't perfect. Think in terms of service level.

>There are 2X more nodes than needed to represent the
building accurately.

The problem with correcting this is you are introducing
approximations.  This will vary according to the source and
this can be simplified or corrected once its in OSM. I think
this is a different issue of a mechanical edit that needs to
be considered separately.

If we are concerned with database size then I suggest we
change the instructions to say put the source comment on the
change set rather than on the building outline.

Cheerio John


Nate Wessel wrote on 2019-01-18 3:06 PM:


John,

You seem to be playing the long game with this data - it
sounds like you've been working with this a lot longer than
I have, and you've put in the time and effort to help make
this actually-quite-incredible dataset available to us. I
don't want to stop the import from happening - quite the
opposite. I just want to make sure that the time is taken to
do this right. OSM deserves that. Your (our) long awaited
victory will be the sweeter for our patience now.

There are several specific issues I see where the I's are
not crossed, nor the t's dotted. I've mentioned several
already, so I'll try to be brief (I really need to get back
to working on my dissertation).

1) There was extremely limited discussion on the imports
mailing list. The initial email did not make clear the scope
of the project. I read the email and did not think twice at
it, thinking it was entirely about Ottawa. The link in that
email was actually to the Ottawa import, and not this one,
which seems to have been only in draft at the time.

https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
As such, this project has NOT been reviewed by the imports
list, which is a requirement for proceeding with the import.

2) There is no mention of this proposed import on the import
catalogue (https://wiki.openstreetmap.org/wiki/Import/Catalogue)
which is required in the imports guidelines. I suspect many
other guidelines have not been followed.

3) The wiki page describing the import is not adequate to
assess the quality of the data or of the proposed import.
See for example:

https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan

Re: [Talk-ca] Ongoing Canadian building import needs to be stopped, possibly reverted

[Nate Wessel]

Hi Yaro,

Thanks for marking this as on-hold in the tasking manager. I know I came 
in like a wrecking ball and I really appreciate y'all holding things up 
while we discuss.


I'd be happy to validate data and help import the rest of central 
Toronto once we're up and running again! I use the data in this area a 
lot in my work... so I have a vested interest in keeping it at it's best :-)


As to the conflation issue, one of the things we're doing in the other 
import I'm working on is that we've essentially split it into two parts. 
First we're importing buildings that don't conflict with OSM at all - 
this is the easy part - and only later will we go in a bit more 
surgically and try to add tags to existing ways and replace geometries 
with better data. We haven't started that part yet, though I imagine it 
will be a real slog. IMO, it seems like a lot to ask that editors do 
both things at once as these are really very different tasks, especially 
given the size of the tasks here.


I wonder if you'd have any interest in a similar separation of tasks for 
this import? I think one of the benefits is that less experienced 
mappers can get their hands dirty on the easier new-data-import part, 
without having to be expert on which geometry is better, how to preserve 
way histories and tags, etc. Like I said, we haven't started this part 
yet in the other import, but even I find the prospect a little daunting!


Best,

Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban Planning
NateWessel.com 

On 1/18/19 2:15 PM, Yaro Shkvorets wrote:

Nate,
I'll change the project name to reflect that the import is on hold. As 
a local mapper, if you want to take a lead on the Toronto import 
that'd be great.
I did review some of DannyMcD's edits last night 
(Mississauga-Brampton-Vaughan) and to be honest was rather 
disappointed with the quality. It appears Danny chose to import only 
new buildings (i.e. residential homes mostly), leaving most of the 
existing hand-traced non-residential building outlines in OSM 
untouched. That's unfortunate, the dataset offers some really good 
data and leaving half of it behind makes it more difficult to revisit 
in the future.
In my edits (Markham-Scarborough-East York) I was aiming to replace as 
many existing geometries with outlines from the import as possible. I 
think that's what we should be trying to do going forward.

Looking forward to your comments and discussion.



On Fri, Jan 18, 2019 at 1:07 PM Nate Wessel > wrote:


Hi all,

I've just joined the talk-ca list, so please accept my apologies
for not addressing this list earlier. I'm happy to take this
thread off the imports list for now and onto talk-ca until things
are ready to begin again. The next person to reply can please feel
free to remove that email if they agree.

I've just made a note on the draft import plan wiki page noting
that the import has been stopped:


https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan

I would really appreciate it if the person with admin access to
the tasking manager projects could please take those offline for
the moment, or perhaps place them in a validation-only mode if
that's possible.

Like I said in my last email, which perhaps didn't make it to the
talk-ca list
(https://lists.openstreetmap.org/pipermail/imports/2019-January/005886.html)
I'm now proposing that we leave the data that has already been
imported and enter a phase of thorough validation on that data.

My plan, over the next several days, is to do a general survey of
the quality of the data that has been imported so far and make a
list of systematic issues I see that should be addressed before we
can consider moving forward again. I'll add those comments to the
conversation in talk-ca and on the wiki page (link above), as I
feel is appropriate. As I said before, I'm of the mind that this
import did not get adequate review or approval and did not follow
all the import guidelines. I think therefore we need to take
stock, cross the t's, dot the i's, and move this thing back toward
where it needs to be. Step one is a thoroughly documented wiki
page outlining the proposal and responding to everything required
in the import guidelines.
https://wiki.openstreetmap.org/wiki/Import/Guidelines

I know there are people excited about this import, and people who
are eager to get back to work bringing buildings in, but I think
everyone will be happier in the end if we take the time to do this
right. We don't need to stop forever - we just need to stop until
we get things right. I sincerely respect the good intentions of
everyone involved in this and I hope we can all work together to
make OSM a map known for it's coverage AND it's quality.

Best,

N

Re: [Talk-ca] 2020 building import wiki comment by Nate Wessel

[Nate Wessel]

John,

I'm sorry to keep saying this, but I really do not think this is an 
acceptable import approval process.


You're saying there was no wiki describing the plan when this went to 
the imports mailing list - only a link to a similar plan with related 
data. You did not follow the import guidelines and you need to go back 
and read that page line by line and follow the procedures that we have 
in place.

https://wiki.openstreetmap.org/wiki/Import/Guidelines
I'll go ahead and add a mention of this plan to the imports catalogue to 
get us started. I'll also add some sections to the wiki and try to leave 
some indication of where things can be better documented.


You may think I'm quibbling over procedural details, but I think this 
process is really important. If we were talking about importing 
buildings in one neighborhood, I would look the other way, but this is 
all of Canada. This is a huge, huge import and we need to take the time 
to do things right, and especially to document the process so people can 
get involved that aren't already.


Best,

Nate Wessel
Jack of all trades, Master of Geography, PhD candidate in Urban Planning
NateWessel.com 

On 1/18/19 3:48 PM, John Whelan wrote:
The import mailing list was pointed to the correct page of the wiki.  
The initial post was to say this is what we were thinking of and there 
was a comment saying we needed to change the comment line.


>There is no mention of this proposed import on the import catalogue


The import process was reviewed by the person who set up the Ottawa 
import did we miss that step on the Ottawa import as well?  Neither 
was it raised as a concern on the import mailing list. I think this is 
very minor and can be corrected.


We learnt a fair bit on the Ottawa import and my expectation is since 
we are using experienced mappers to do the import conflation would be 
either handled by them or the building not imported. We aren't using 
new mappers in a mapathon here and with experienced mappers then I 
think you have to trust them. The world isn't perfect. Think in terms 
of service level.


>There are 2X more nodes than needed to represent the building accurately.

The problem with correcting this is you are introducing 
approximations.  This will vary according to the source and this can 
be simplified or corrected once its in OSM. I think this is a 
different issue of a mechanical edit that needs to be considered 
separately.


If we are concerned with database size then I suggest we change the 
instructions to say put the source comment on the change set rather 
than on the building outline.


Cheerio John


Nate Wessel wrote on 2019-01-18 3:06 PM:


John,

You seem to be playing the long game with this data - it sounds like 
you've been working with this a lot longer than I have, and you've 
put in the time and effort to help make this 
actually-quite-incredible dataset available to us. I don't want to 
stop the import from happening - quite the opposite. I just want to 
make sure that the time is taken to do this right. OSM deserves that. 
Your (our) long awaited victory will be the sweeter for our patience 
now.


There are several specific issues I see where the I's are not 
crossed, nor the t's dotted. I've mentioned several already, so I'll 
try to be brief (I really need to get back to working on my 
dissertation).


1) There was extremely limited discussion on the imports mailing 
list. The initial email did not make clear the scope of the project. 
I read the email and did not think twice at it, thinking it was 
entirely about Ottawa. The link in that email was actually to the 
Ottawa import, and not this one, which seems to have been only in 
draft at the time. 
https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
As such, this project has NOT been reviewed by the imports list, 
which is a requirement for proceeding with the import.


2) There is no mention of this proposed import on the import 
catalogue (https://wiki.openstreetmap.org/wiki/Import/Catalogue)
which is required in the imports guidelines. I suspect many other 
guidelines have not been followed.


3) The wiki page describing the import is not adequate to assess the 
quality of the data or of the proposed import. See for example: 
https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks
The import guidelines call for a description of how conflation will 
be handled. The fact that two of the major importers seem to have a 
substantial disagreement about how to handle existing data indicates 
this was not well discussed and I can see that it isn't well documented.


4) The buildings need to be simplified, quite a bit actually. Most 
buildings have multiple nodes representing straight lines. This 
bloats the database and makes things harder to edit by hand later. 
There are probably 2x more nodes than are needed to represent the 
data accurately, making it ha