Sounds sane to me. Are we thinking of setting up a second import to handle Microsoft building outlines on a similar basis? Or extending this one? I only ask since if we are doing it municipality by municipality it might be an idea to identify those municipalities that do not have suitable open data through stats canada. I can see us with a list of municipalities and a data source and I think it probably should be in one place.
Cheerio John On Fri, 3 Jan 2020 at 15:42, Daniel @jfd553 <jfd...@hotmail.com> wrote: > Bonjour groupe, mes excuses pour ce très long courriel !-) > > I have reviewed everything that has been written on the ODB import (aka > Canada Building Import) in Talk-ca and the wiki. I proposed changes to some > wiki pages (via talk tabs) to ease the discussions about this import and > the following. Now, in order to restart the import, here are some thoughts > and a proposal on how to proceed to complete the task. > > > > *1. Issues with the ODB Data Import* > > Many concerns were raised about the import. One major concern was to > obtain local communities’ buy-in in the Canadian context. Another concern > was to improve the quality of the data prior the import. The following > paragraphs intend to clear most of these concerns. > > *1.1. Which data import project?* > > According to the import guidelines (steps 3 & 4), a data import explicitly > refers to a single data source (ODB in our case). Discussions about the > availability and quality of Microsoft or ESRI data, while interesting, are > not relevant as they should be dealt with as other import projects. > > *1.2. What has been imported so far?* > > According to what I found [1], the ODB import is completed for 21 > municipalities. These imports seem to have kept OSM content’s history, at > least for the samples checked, but many problems were found. In some case, > the imports brought swimming pools in OSM because they were included in the > dataset (e.g. Moncton). In other cases, importing buildings with accurate > locations (XY) over content mapped from less accurate imagery resulted in > buildings that now overlap the street network (e.g. Squamish). It means > that all these 21 imports need to be carefully re-examined and corrected as > required. > > For 12 other municipalities, the import is partial, either suspended as > requested, or because previous imports had already provided most of the > buildings (often from the same municipal provider). That said the import > will definitely improve OSM accuracy and completeness if done properly. > > *2. How should ODB Data be imported?* > > I will copy the following paragraphs in the “Canada Building Import” wiki > page [3] for a detailed discussion… > > Since the data (ODB, OSM and imagery) differ from one municipality to > another, there can be no imports at the national or provincial level. We > have to work on a municipal basis and make sure to identify all the > problems and the corrective measures to apply when dealing with issues like > those I identified [1]. > > *2.1 Importing Locally* > > According to the import guidelines (step 2), we must not import the data > without local buy-in. However, and contrarily to some European country, > there is usually no such thing as a local OSM community in each > municipality. However, we may find a few local mappers from time to time. > Working on a municipal basis should allow identifying these local mappers > before doing the import. I often use this tool [2] to identify and contact > local mappers. Once identified, I suggest that… > > - We contact them to explain our intents by referring to appropriate wiki > pages. > > - We wait a week or two to let them respond nothing, that they have > concerns, or wish to help. > > - Without negative answers we could proceed to the import. > > I first suggest that when a contributor wishes to import ODB for a given > municipality, he first identifies himself as responsible for the import (we > need to create an entry for each municipality somewhere in the wiki). He > can then contact local mappers, as explain above, and go ahead with the > import once everything settled. For those who already made the import, I > suggest that they review their work since many issues were detected with > some of these imports. > > Since there are only a few local OSM communities in Canada, and because > Canada is large, I suggest not limiting the import of a given municipality > to the people of the concerned province or region. > > *2.2 Pre-processing* > > Once local mappers have agreed, some pre-processing can be done if > required. > > A few months ago, I developed a tool that could be used to process the > data [4]. Concerns were raised because the application was developed using > proprietary software. So I documented the whole process and algorithms in > order to see courageous coders converting it in open source software. In > the meantime, and as long as I have access to an FME licence, I could > process the data, when necessary, prior to make it available through the > task manager. > > Proposed pre-processing [4] includes: > > - Reading of original ODB data, > > - Removal of near collinear nodes (simplification), > > - Orthogonalization of buildings (for corners having near right angles), > > - Tagging of building footprints, > > - Providing files in OSM format. > > *Proposed tagging:* In addition to the tags produced by the > orthogonalization process [4] and the source tag (source > <https://wiki.openstreetmap.org/wiki/Key:source>=Statistics Canada - Open > Building Database), the name of the Census Subdivision provided in ODB data > [5] is used to add the addr:city tag to each building. > > The pre-processing requires parameters that are specific to the data to > process. These parameters were estimated on a municipal basis using actual > ODB data. The processing time increases exponentially according to the > number of buildings so, it may take a couple of days before the data is > available for a given municipality. Currently, the proposed pre-processing > does not convert terrace buildings into individual houses nor it tags > topological errors. > > *2.3. Import Process* > > After the local mappers, if any, agreed to the import, the pre-processing > completed when required, we can proceed to the import. > > 1- Do not bulk import the data! Always use the task manager ( > http://tasks.osmcanada.ca/). Select and open a task square in JOSM. If > it’s too big (e.g. too much work or request is too big to load in JOSM), go > back to the task manager and split the task into smaller squares. > > 2- Load imagery layer (Bing or ESRI World Imagery) and align the imagery > with ODB data (i.e. create a new image offset) if necessary because, unless > proven otherwise, ODB should be more accurate (XY) than most available > images especially in hilly areas. > > 3- Align the existing OSM content to the image (i.e. after the new offset > is applied) if required. > > 4- Currently step 2 and following as described in the wiki [2]. I suggest > merging the Conflation section [6] here and reviewing everything to take > into account the current proposal. > > *References* > > [1] https://wiki.openstreetmap.org/wiki/The_Open_Database_of_Buildings > > [2] > https://wiki.openstreetmap.org/wiki/Canada_Building_Import#Import_process > > [3] http://resultmaps.neis-one.org (“Overview of OpenStreetMap > Contributors aka who’s around me?”) > > [4] https://github.com/jfd553/OrthogonalizingBuildingFootprint > > [5] > https://www150.statcan.gc.ca/n1/pub/92-195-x/2011001/geo/csd-sdr/csd-sdr-eng.htm > > [6] https://wiki.openstreetmap.org/wiki/Canada_Building_Import#Conflation > > > > > > > > Let’s move ahead! > > Daniel > _______________________________________________ > Talk-ca mailing list > Talk-ca@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-ca >
_______________________________________________ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca