Re: [Talk-ca] Open Data Imports
Hello: I've done some partial work on collecting / writing some scripts that work with the current canvec plus shape files. If you want I can send you a copy. Currently it breaks the files down into the various layers i.e. building, waterway, hydro lines. The idea was that you could pick and choose what data layer you wanted. I haven't touched the scripts in a couple of months, so I'm not sure exactly where I'm at with them. Andrew On Thu, 2015-07-23 at 14:38 -0400, Daniel Begin wrote: Bonjour Andrew, Good initiative! And it will be perfect if you add all necessary links to good practices/warnings about imports!-) I had a look at Canvec+ details (a). - The prepackaged files (250K tiles) are going to be quite large since, from what I understand, they have merged together all (16) underneath 50K. - Custom “areas of interest” might be difficult to manage for data import. - Proposed file formats are similar to what it used to be with standard Canvec, but it does not include OSM format :-( About the script(s) used to convert Canvec to OSM, they were built using FME workbenches linked together with batch files (so, obviously not open source). Best regards, Daniel a) http://geogratis.gc.ca/site/eng/whats-new/intro-canvec -Original Message- From: Andrew MacKinnon [mailto:andrew...@gmail.com] Sent: July-23-15 14:04 To: Talk-CA OpenStreetMap Subject: [Talk-ca] Open Data Imports I am starting to work on importing Open Data datasets. I am using pnorman's ogr2osm script with modified translation files (see https://github.com/andrewpmk/ogr2osm-translations). It will be some time before I actually import anything. I would like to assemble a list of government open data portals in Canada which are compatible with the OSM license. Please add suitable open data sources to [https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Open_data]. If they have already been fully imported then you should put a note on that wiki page. Also I am trying to figure out a way to import newer CanVec data. The CanVec files in OSM format at http://ftp2.cits.rncan.gc.ca/OSM/pub/ are out of date and appear to have been created in 2010. Is the script that was used to convert CanVec to OSM open source? It looks like there is a new version of CanVec called CanVec+, has anyone here used it yet? I am hoping to do something about the large amount of broken imported data in OSM in Canada and we need a better way of fixing broken CanVec data than copying from the Geobase WMS layer or cutting and pasting from outdated .osm files from 2010. ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Highway recoding
Any objections? Of course not! Your overview of the situation worldwide is pretty exhaustive, and is in line with most comments, consideration, that were expressed so far. However, as I suggested in an earlier email, I would keep the topic alive for a couple of weeks, just to make sure everyone that may feel concerned about the subject have a chance to comment (since it is summer time). Unless there are backlashes from some contributors, I propose to keep everything as is until the end of august and then move forward to update definitions and data. Does everyone comfortable with it? Daniel From: Tristan Anderson [mailto:andersontris...@hotmail.com] Sent: July-23-15 17:18 To: Daniel Begin; 'Stewart C. Russell'; talk-ca@openstreetmap.org Subject: RE: [Talk-ca] Highway recoding So it seems like we're coming to some agreement. The current Canadian definition based on that 2005 document should be replaced with something else that is consistent with the rest of the world. Once we find this new definition, the appropriate wiki pages should be updated. I took a look around the world and finally saw some consistency in how trunk tags are used. Stewart's guidelines are basically correct, but I think I can hammer out a more specific description. There are two types of roads with are both usually tagged highway=trunk: (1) Limited access highways. This is a physical description for a road that has some of the characteristics of a motorway. They are often dual carriageways of fairly high speed. (2) Highways connecting distant population centres. This is a functional description for a road where used by cars and heavy trucks travelling long distances or between major cities. Although usually two lanes, in more remote areas these roads may have very light traffic, be unpaved, or be slow. In some parts of the world, like Germany, France and the eastern United States, all trunk roads are type (1) because long-distance travel is generally done on their dense networks of motorways. Conversely, in large swathes of Australia and Canada, as well as in much of the developing world, all trunk roads are type (2) because type (1) doesn't exist. The only country I noticed that doesn't follow the above scheme is Britain (actually just England and Wales), ironically the birthplace of the trunk. The designation there is used quite liberally, including even short roads connecting small towns and quite a few of of London's city streets. Just look at England at zoom level 5 and observe how unusually green it is. I suggest using the international model, with types (1) and (2) above being tagged as trunks in Canada. This won't change much as it largely coincides with how roads are already tagged. The wiki pages can be updated accordingly then we can look at specific roads in BC and Québec! Any objections? From: jfd...@hotmail.com To: scr...@gmail.com; talk-ca@openstreetmap.org Date: Thu, 23 Jul 2015 10:08:44 -0400 Subject: Re: [Talk-ca] Highway recoding Thank Russel, Your description is pretty close of the one I had in mind (about trunks) before I found the Canadian definition was referring to the mentioned document. Cheers, Daniel -Original Message- From: Stewart C. Russell [mailto:scr...@gmail.com] Sent: July-23-15 08:44 To: talk-ca@openstreetmap.org Subject: Re: [Talk-ca] Highway recoding The definition of trunk is a difficult one, if based on the UK understanding. Like its unwritten constitution, trunk roads in the UK are more on a know it when I see it basis. Pretty much the only definitions I can think of that would be generally applicable are: * a trunk road goes from one city/town to another. * no parking at the side of the road. * something above the urban speed limit applies (though there are often nasty brief exceptions, like a roughly 200m stretch of 30 mph that used to adorn the A80, dammit). A trunk road isn't always dual carriageway. It can have traffic lights, roundabouts or (rare, in the UK) stop signs. Depending on its age, it may bypass towns and villages. Older trunk roads may also have all the usual roads entering it, while newer ones are likely to have on-ramps. In summary, the UK definition is so riddled with unwritten exceptions that trying to apply it rigorously in even one province in Canada will be frustrating. And no matter what you do, you'll always get some rogue user that comes along and adds their own tagging. It's a sair fecht cheers, Stewart ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Highway recoding
~~ Un résumé français suit ~~ Bonjour all, The few comments we got so far show that most of us, but not all, are uncomfortable with the “strategic” approach causing inconsistent descriptions of actual road “object” within Canada and between CA/US borders. Since it is summer, I will keep the discussion alive for a while to make sure all interested people made their point. Join the conversation whenever you want :-) We are waiting for more comments… Daniel Ps: comments received off-list will stay off-list – Please join the actual conversation J En résumé, je questionne la façon d’attribuer le tag ‘trunk’ aux routes principales tel que proposé dans un document gouvernemental (a) cité dans le wiki (c) et propose de clarifier la documentation une fois un consensus obtenu. Les commentaires reçu à date vont pour la plupart (mais pas tous) dans le sens qu’une définition de type ‘’stratégique’’ (une route est importante pour l’économie d’une région) produit des résultats inconsistants par rapport à la perception qu’offre la carte par rapport aux ‘’infrastructures’’ qui la supporte (les routes ‘’trunk’’ à Toronto, sur la Côte-Nord ou au Yukon sont très différentes les unes des autres alors que les autres classes de routes sont similaires à la grandeur du pays) – bref la description ‘’physique’’ serait plus appropriée. Vos commentaires sont bienvenus a) http://www.comt.ca/english/NHS-report-english.pdf b) http://wiki.openstreetmap.org/wiki/Highway:International_equivalence c) http://wiki.openstreetmap.org/wiki/Canadian_tagging_guidelines From: Daniel Begin [mailto:jfd...@hotmail.com] Sent: July-22-15 16:44 To: 'Paul Norman'; talk-ca@openstreetmap.org Subject: Re: [Talk-ca] Highway recoding Bonjour Paul, You actually highlight what makes me uncomfortable with the “strategic” approach applied in many part of Canada. You are concerned about the road network in BC; I am concerned about the network in QC. Until few months ago, there were no trunk here; they are now everywhere. IMO, OSM classification mostly aims at describing the road infrastructures, not the strategic/economic importance a local government says about them (almost quoted you!-). I understand that Tristan has similar concerns about the consequences of such approach in road classification; even if he suggested that the current definitions (using strategic approach) are good guidelines (but need not be followed religiously). Other comments on the subject Daniel From: Paul Norman [mailto:penor...@mac.com] Sent: July-22-15 15:59 To: talk-ca@openstreetmap.org Subject: Re: [Talk-ca] Highway recoding On 7/22/2015 11:43 AM, Daniel Begin wrote: So far, I understand we have 2.5 votes for tagging trunk/motorway all roads identified as “core route” in document (a); 0.5 against (I am still torn between the two approaches!-) More comments would be appreciated Such an approach would be inconsistent with how highways are tagged in BC and expectations of locals. It would also make BC quite different than across the boarder in Washington. I can think of several motorways and trunk roads which are not on the list in the PDF, and many of the roads on the list are primary, or in at least one case, secondary. Some of the roads not on the list are more important in the transportation network than ones on it. The criteria being proposed are also inherently unverifiable. We map the world, not what a government database says. What about new roads? There's a new route that's opened up, and it's a mix of trunk and motorway, but it's not listed in the NHS report. To tag it primary when less significant roads constructed to a lower standard are tagged as trunk and motorway would be absurd. Because it has a lot of freight, it probably will become a NHS road at some point. Does its classification magically change when nothing has changed on the ground? ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Highway recoding
The definition of ‘trunk’ is a difficult one, if based on the UK understanding. Like its unwritten constitution, trunk roads in the UK are more on a know it when I see it basis. Pretty much the only definitions I can think of that would be generally applicable are: * a trunk road goes from one city/town to another. * no parking at the side of the road. * something above the urban speed limit applies (though there are often nasty brief exceptions, like a roughly 200m stretch of 30 mph that used to adorn the A80, dammit). A trunk road isn't always dual carriageway. It can have traffic lights, roundabouts or (rare, in the UK) stop signs. Depending on its age, it may bypass towns and villages. Older trunk roads may also have all the usual roads entering it, while newer ones are likely to have on-ramps. In summary, the UK definition is so riddled with unwritten exceptions that trying to apply it rigorously in even one province in Canada will be frustrating. And no matter what you do, you'll always get some rogue user that comes along and adds their own tagging. It's a sair fecht … cheers, Stewart ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Open Data Imports
On 7/23/2015 8:54 PM, Andrew MacKinnon wrote: Does anyone know which of these (and others) are compatible with the OSM license? Very likely most of them are not released under open licenses. Unfortunately, non-open data gets listed in OpenAddresses and there's no assurance that you can combine data from one source in OpenAddresses with data from another source in it. OpenAddresses is a good fallback source for a geocoder, but I don't know that there are any geocoders that do that yet. ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Open Data Imports
It might be helpful to look at http://openaddresses.io/ which is an project to aggregate address data from various open data portals. More and more cities have open data now. In Ontario openaddresses.io lists: - Burlington - Guelph - Hamilton - Kitchener - Oakville - Toronto - Waterloo - Welland - Windsor Does anyone know which of these (and others) are compatible with the OSM license? ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Highway recoding
Thank Russel, Your description is pretty close of the one I had in mind (about trunks) before I found the Canadian definition was referring to the mentioned document. Cheers, Daniel -Original Message- From: Stewart C. Russell [mailto:scr...@gmail.com] Sent: July-23-15 08:44 To: talk-ca@openstreetmap.org Subject: Re: [Talk-ca] Highway recoding The definition of ‘trunk’ is a difficult one, if based on the UK understanding. Like its unwritten constitution, trunk roads in the UK are more on a know it when I see it basis. Pretty much the only definitions I can think of that would be generally applicable are: * a trunk road goes from one city/town to another. * no parking at the side of the road. * something above the urban speed limit applies (though there are often nasty brief exceptions, like a roughly 200m stretch of 30 mph that used to adorn the A80, dammit). A trunk road isn't always dual carriageway. It can have traffic lights, roundabouts or (rare, in the UK) stop signs. Depending on its age, it may bypass towns and villages. Older trunk roads may also have all the usual roads entering it, while newer ones are likely to have on-ramps. In summary, the UK definition is so riddled with unwritten exceptions that trying to apply it rigorously in even one province in Canada will be frustrating. And no matter what you do, you'll always get some rogue user that comes along and adds their own tagging. It's a sair fecht … cheers, Stewart ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Open Data Imports
Bonjour Andrew, Good initiative! And it will be perfect if you add all necessary links to good practices/warnings about imports!-) I had a look at Canvec+ details (a). - The prepackaged files (250K tiles) are going to be quite large since, from what I understand, they have merged together all (16) underneath 50K. - Custom “areas of interest” might be difficult to manage for data import. - Proposed file formats are similar to what it used to be with standard Canvec, but it does not include OSM format :-( About the script(s) used to convert Canvec to OSM, they were built using FME workbenches linked together with batch files (so, obviously not open source). Best regards, Daniel a) http://geogratis.gc.ca/site/eng/whats-new/intro-canvec -Original Message- From: Andrew MacKinnon [mailto:andrew...@gmail.com] Sent: July-23-15 14:04 To: Talk-CA OpenStreetMap Subject: [Talk-ca] Open Data Imports I am starting to work on importing Open Data datasets. I am using pnorman's ogr2osm script with modified translation files (see https://github.com/andrewpmk/ogr2osm-translations). It will be some time before I actually import anything. I would like to assemble a list of government open data portals in Canada which are compatible with the OSM license. Please add suitable open data sources to [https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Open_data]. If they have already been fully imported then you should put a note on that wiki page. Also I am trying to figure out a way to import newer CanVec data. The CanVec files in OSM format at http://ftp2.cits.rncan.gc.ca/OSM/pub/ are out of date and appear to have been created in 2010. Is the script that was used to convert CanVec to OSM open source? It looks like there is a new version of CanVec called CanVec+, has anyone here used it yet? I am hoping to do something about the large amount of broken imported data in OSM in Canada and we need a better way of fixing broken CanVec data than copying from the Geobase WMS layer or cutting and pasting from outdated .osm files from 2010. ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
[Talk-ca] Open Data Imports
I am starting to work on importing Open Data datasets. I am using pnorman's ogr2osm script with modified translation files (see https://github.com/andrewpmk/ogr2osm-translations). It will be some time before I actually import anything. I would like to assemble a list of government open data portals in Canada which are compatible with the OSM license. Please add suitable open data sources to [https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Open_data]. If they have already been fully imported then you should put a note on that wiki page. Also I am trying to figure out a way to import newer CanVec data. The CanVec files in OSM format at http://ftp2.cits.rncan.gc.ca/OSM/pub/ are out of date and appear to have been created in 2010. Is the script that was used to convert CanVec to OSM open source? It looks like there is a new version of CanVec called CanVec+, has anyone here used it yet? I am hoping to do something about the large amount of broken imported data in OSM in Canada and we need a better way of fixing broken CanVec data than copying from the Geobase WMS layer or cutting and pasting from outdated .osm files from 2010. ___ Talk-ca mailing list Talk-ca@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-ca
