Re: [talk-au] Talk-au Digest, Vol 76, Issue 5
Hi Nick, Yes, vicmap data does contain road geometry, names plus many more attributes. Vicmap also produces are raster version from this dataset so that work is already done. I will enquiry about getting access to this. However manually tracing and editing is a LOT of work. Does anyone know of a process where a mass import can be done without manually tracing imagery? The dataset is very comprehensive. Here's a quick summary of it's features: Release under Creative commons 3.0 license, dataset can be obtained in various formats from http://www.data.vic.gov.au/ The has heaps of datasets. I've downloaded most relevant datasets in shp format and combined the datasets into logical structure, download links below. If anyone is interest in other features such as forestry reserves, national parks etc, have a look and pull the data down from data.vic. Transport shp data https://www.dropbox.com/sh/fuba69goag6c5pb/FYRs68wqDo documentationhttps://www.dropbox.com/s/p44ld1ov1i291ra/Vicmap-Transport-Prod-Desc-V3_4.pdf Hydro - shp data https://www.dropbox.com/sh/qrnmzveiu4ayxsn/bXyu5QI4Vr - documentationhttps://www.dropbox.com/s/dwsw36xs905dw1d/Vicmap-Hydro-Prod-Desc-v5.4.pdf Administrative - shp datahttps://www.dropbox.com/sh/efa8ags193e8d7b/_SxEilmoOA Li. On Fri, Oct 11, 2013 at 11:00 PM, talk-au-requ...@openstreetmap.org wrote: Send Talk-au mailing list submissions to talk-au@openstreetmap.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.openstreetmap.org/listinfo/talk-au or, via email, send a message with subject or body 'help' to talk-au-requ...@openstreetmap.org You can reach the person managing the list at talk-au-ow...@openstreetmap.org When replying, please edit your Subject line so it is more specific than Re: Contents of Talk-au digest... Today's Topics: 1. Re: vicmap data licensing (Li) 2. Re: vicmap data licensing (Nick Hocking) -- Message: 1 Date: Fri, 11 Oct 2013 07:37:31 +1100 From: Li m...@lixia.co To: Ben Kelley ben.kel...@gmail.com Cc: OSM Australian Talk List talk-au@openstreetmap.org Subject: Re: [talk-au] vicmap data licensing Message-ID: 91641499-6fb4-426c-8313-c0d82c305...@lixia.co Content-Type: text/plain; charset=us-ascii Does anyone have experience on importing data? In particular avoiding duplicates? Li. On 10 Oct 2013, at 5:16 pm, Ben Kelley ben.kel...@gmail.com wrote: Hi. I guess the thing to consider is how you would handle a second import if someone had edited the data in OSM in between. I think this kind of conflict would be very difficult to resolve. You could either plan to do a 1-off import, or maybe include a tag on the imported data matching a unique identifier for the same feature in the vicmap data. The US Tiger import did something like this. - Ben Kelley. On 10 Oct 2013 17:12, Li Xia m...@lixia.co wrote: Hi everyone, I'm meeting Vicmap and data.gov staff tomorrow to get their blessing on importing vicmap data into OSM. Once the licensing is squared away, we can move onto discussing techniques of importing the data. Snapshot data in shp format is available from data.vic.gov.au. Alternatively a vicmap provides a live feed to weekly data diffs directly. Any advice on how to import this data is much appreciated. Li ___ Talk-au mailing list Talk-au@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-au -- next part -- An HTML attachment was scrubbed... URL: http://lists.openstreetmap.org/pipermail/talk-au/attachments/20131011/98778e6c/attachment-0001.html -- Message: 2 Date: Fri, 11 Oct 2013 12:18:17 +1100 From: Nick Hocking nick.hock...@gmail.com To: talk-au@openstreetmap.org Subject: Re: [talk-au] vicmap data licensing Message-ID: caded0cxpnfdoza4jsnvn0xczcgaun0hnn5x2nk-siuxlfdv...@mail.gmail.com Content-Type: text/plain; charset=iso-8859-1 Hi Li Does the vicmap[ data include road geometry and road names. If so then a really usefull thing to do would be to create an imagery layer from this data that could be used in JOSM. This is what is done in the USA with each year's TIGER data. Then we could use the Bing imagery the vicmap layer and existing data to fill in all the unnamed streets/roads and include any new ones or ones that have not yet been surveyed or traced. It would only be a matter of months and all of Victoria's roads would be completely up to date. Curerently, I'm spending hours each day using the TIGER data and Bing imagery in helping to fix up the horrible original TIGER data but would love to be helping in fixing up Australia. Nick Hocking Canberra -- next part -- An HTML attachment was scrubbed... URL: http://lists.openstreetmap.org/pipermail/talk
Re: [talk-au] Talk-au Digest, Vol 76, Issue 5
Hi, Whatever process we undertake will have large manual elements to integrate the datasets and keep them updated. Even if that is simply merging the datasets in josm. We need to get the vicmap data in a form that will have maximum utility to OSM mappers. Ian. On 12 October 2013 09:09, Li Xia m...@lixia.co wrote: Hi Nick, Yes, vicmap data does contain road geometry, names plus many more attributes. Vicmap also produces are raster version from this dataset so that work is already done. I will enquiry about getting access to this. However manually tracing and editing is a LOT of work. Does anyone know of a process where a mass import can be done without manually tracing imagery? The dataset is very comprehensive. Here's a quick summary of it's features: Release under Creative commons 3.0 license, dataset can be obtained in various formats from http://www.data.vic.gov.au/ The has heaps of datasets. I've downloaded most relevant datasets in shp format and combined the datasets into logical structure, download links below. If anyone is interest in other features such as forestry reserves, national parks etc, have a look and pull the data down from data.vic. Transport shp data documentation Hydro - shp data - documentation Administrative - shp data Li. On Fri, Oct 11, 2013 at 11:00 PM, talk-au-requ...@openstreetmap.org wrote: Send Talk-au mailing list submissions to talk-au@openstreetmap.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.openstreetmap.org/listinfo/talk-au or, via email, send a message with subject or body 'help' to talk-au-requ...@openstreetmap.org You can reach the person managing the list at talk-au-ow...@openstreetmap.org When replying, please edit your Subject line so it is more specific than Re: Contents of Talk-au digest... Today's Topics: 1. Re: vicmap data licensing (Li) 2. Re: vicmap data licensing (Nick Hocking) -- Message: 1 Date: Fri, 11 Oct 2013 07:37:31 +1100 From: Li m...@lixia.co To: Ben Kelley ben.kel...@gmail.com Cc: OSM Australian Talk List talk-au@openstreetmap.org Subject: Re: [talk-au] vicmap data licensing Message-ID: 91641499-6fb4-426c-8313-c0d82c305...@lixia.co Content-Type: text/plain; charset=us-ascii Does anyone have experience on importing data? In particular avoiding duplicates? Li. On 10 Oct 2013, at 5:16 pm, Ben Kelley ben.kel...@gmail.com wrote: Hi. I guess the thing to consider is how you would handle a second import if someone had edited the data in OSM in between. I think this kind of conflict would be very difficult to resolve. You could either plan to do a 1-off import, or maybe include a tag on the imported data matching a unique identifier for the same feature in the vicmap data. The US Tiger import did something like this. - Ben Kelley. On 10 Oct 2013 17:12, Li Xia m...@lixia.co wrote: Hi everyone, I'm meeting Vicmap and data.gov staff tomorrow to get their blessing on importing vicmap data into OSM. Once the licensing is squared away, we can move onto discussing techniques of importing the data. Snapshot data in shp format is available from data.vic.gov.au. Alternatively a vicmap provides a live feed to weekly data diffs directly. Any advice on how to import this data is much appreciated. Li ___ Talk-au mailing list Talk-au@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-au -- next part -- An HTML attachment was scrubbed... URL: http://lists.openstreetmap.org/pipermail/talk-au/attachments/20131011/98778e6c/attachment-0001.html -- Message: 2 Date: Fri, 11 Oct 2013 12:18:17 +1100 From: Nick Hocking nick.hock...@gmail.com To: talk-au@openstreetmap.org Subject: Re: [talk-au] vicmap data licensing Message-ID: caded0cxpnfdoza4jsnvn0xczcgaun0hnn5x2nk-siuxlfdv...@mail.gmail.com Content-Type: text/plain; charset=iso-8859-1 Hi Li Does the vicmap[ data include road geometry and road names. If so then a really usefull thing to do would be to create an imagery layer from this data that could be used in JOSM. This is what is done in the USA with each year's TIGER data. Then we could use the Bing imagery the vicmap layer and existing data to fill in all the unnamed streets/roads and include any new ones or ones that have not yet been surveyed or traced. It would only be a matter of months and all of Victoria's roads would be completely up to date. Curerently, I'm spending hours each day using the TIGER data and Bing imagery in helping to fix up the horrible original TIGER data but would love to be helping in fixing up Australia. Nick Hocking Canberra -- next part -- An HTML attachment was scrubbed... URL: http
Re: [talk-au] Talk-au Digest, Vol 76, Issue 5
We need to get the vicmap data in a form that will have maximum - can you be more specific? Do you have any experience in merging datasets in josm? Li. On Sat, Oct 12, 2013 at 9:18 AM, Ian Sergeant inas66+...@gmail.com wrote: Hi, Whatever process we undertake will have large manual elements to integrate the datasets and keep them updated. Even if that is simply merging the datasets in josm. We need to get the vicmap data in a form that will have maximum utility to OSM mappers. Ian. On 12 October 2013 09:09, Li Xia m...@lixia.co wrote: Hi Nick, Yes, vicmap data does contain road geometry, names plus many more attributes. Vicmap also produces are raster version from this dataset so that work is already done. I will enquiry about getting access to this. However manually tracing and editing is a LOT of work. Does anyone know of a process where a mass import can be done without manually tracing imagery? The dataset is very comprehensive. Here's a quick summary of it's features: Release under Creative commons 3.0 license, dataset can be obtained in various formats from http://www.data.vic.gov.au/ The has heaps of datasets. I've downloaded most relevant datasets in shp format and combined the datasets into logical structure, download links below. If anyone is interest in other features such as forestry reserves, national parks etc, have a look and pull the data down from data.vic. Transport shp data documentation Hydro - shp data - documentation Administrative - shp data Li. On Fri, Oct 11, 2013 at 11:00 PM, talk-au-requ...@openstreetmap.org wrote: Send Talk-au mailing list submissions to talk-au@openstreetmap.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.openstreetmap.org/listinfo/talk-au or, via email, send a message with subject or body 'help' to talk-au-requ...@openstreetmap.org You can reach the person managing the list at talk-au-ow...@openstreetmap.org When replying, please edit your Subject line so it is more specific than Re: Contents of Talk-au digest... Today's Topics: 1. Re: vicmap data licensing (Li) 2. Re: vicmap data licensing (Nick Hocking) -- Message: 1 Date: Fri, 11 Oct 2013 07:37:31 +1100 From: Li m...@lixia.co To: Ben Kelley ben.kel...@gmail.com Cc: OSM Australian Talk List talk-au@openstreetmap.org Subject: Re: [talk-au] vicmap data licensing Message-ID: 91641499-6fb4-426c-8313-c0d82c305...@lixia.co Content-Type: text/plain; charset=us-ascii Does anyone have experience on importing data? In particular avoiding duplicates? Li. On 10 Oct 2013, at 5:16 pm, Ben Kelley ben.kel...@gmail.com wrote: Hi. I guess the thing to consider is how you would handle a second import if someone had edited the data in OSM in between. I think this kind of conflict would be very difficult to resolve. You could either plan to do a 1-off import, or maybe include a tag on the imported data matching a unique identifier for the same feature in the vicmap data. The US Tiger import did something like this. - Ben Kelley. On 10 Oct 2013 17:12, Li Xia m...@lixia.co wrote: Hi everyone, I'm meeting Vicmap and data.gov staff tomorrow to get their blessing on importing vicmap data into OSM. Once the licensing is squared away, we can move onto discussing techniques of importing the data. Snapshot data in shp format is available from data.vic.gov.au. Alternatively a vicmap provides a live feed to weekly data diffs directly. Any advice on how to import this data is much appreciated. Li ___ Talk-au mailing list Talk-au@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-au -- next part -- An HTML attachment was scrubbed... URL: http://lists.openstreetmap.org/pipermail/talk-au/attachments/20131011/98778e6c/attachment-0001.html -- Message: 2 Date: Fri, 11 Oct 2013 12:18:17 +1100 From: Nick Hocking nick.hock...@gmail.com To: talk-au@openstreetmap.org Subject: Re: [talk-au] vicmap data licensing Message-ID: caded0cxpnfdoza4jsnvn0xczcgaun0hnn5x2nk-siuxlfdv...@mail.gmail.com Content-Type: text/plain; charset=iso-8859-1 Hi Li Does the vicmap[ data include road geometry and road names. If so then a really usefull thing to do would be to create an imagery layer from this data that could be used in JOSM. This is what is done in the USA with each year's TIGER data. Then we could use the Bing imagery the vicmap layer and existing data to fill in all the unnamed streets/roads and include any new ones or ones that have not yet been surveyed or traced. It would only be a matter
[talk-au] obtaining permission to import vicmap datasets
Quick update... I met with data.vic and vicmap yesterday for permission to import vicmap data into OSM. data.vic has requested that we include attribution in the wiki pagehttp://wiki.openstreetmap.org/wiki/Contributors#Australian_government_public_information_datasets, and then they will request official approval from the policy office. I have added attribution to Vicmap transport, hydro, admin and LITE datasets on the wiki contributors pagehttp://wiki.openstreetmap.org/wiki/Contributors#Australian_government_public_information_datasets so the process can move forward. They have indicated that this process will take around 2 weeks. These attributions are required for formal approval, DO NOT REMOVE THEM! As of this moment, we have been given the green light to start importing the vicmap data. Approval is simply a formality. Li. ___ Talk-au mailing list Talk-au@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Talk-au Digest, Vol 76, Issue 5
Can I recommend for the smaller data sets, that conversion scripts are done and published to github? Ie: shp to geo json is done, then geojosn to osm, plus splitting, etc? That allows you to preview, and when the underlying data changes, version control it a little. I have the same issues to solve for SA data (roads, boundaries, etc) On Saturday, October 12, 2013, Li Xia wrote: We need to get the vicmap data in a form that will have maximum - can you be more specific? Do you have any experience in merging datasets in josm? Li. On Sat, Oct 12, 2013 at 9:18 AM, Ian Sergeant inas66+...@gmail.comwrote: Hi, Whatever process we undertake will have large manual elements to integrate the datasets and keep them updated. Even if that is simply merging the datasets in josm. We need to get the vicmap data in a form that will have maximum utility to OSM mappers. Ian. On 12 October 2013 09:09, Li Xia m...@lixia.co wrote: Hi Nick, Yes, vicmap data does contain road geometry, names plus many more attributes. Vicmap also produces are raster version from this dataset so that work is already done. I will enquiry about getting access to this. However manually tracing and editing is a LOT of work. Does anyone know of a process where a mass import can be done without manually tracing imagery? The dataset is very comprehensive. Here's a quick summary of it's features: Release under Creative commons 3.0 license, dataset can be obtained in various formats from http://www.data.vic.gov.au/ The has heaps of datasets. I've downloaded most relevant datasets in shp format and combined the datasets into logical structure, download links below. If anyone is interest in other features such as forestry reserves, national parks etc, have a look and pull the data down from data.vic. Transport shp data documentation Hydro - shp data - documentation Administrative - shp data Li. On Fri, Oct 11, 2013 at 11:00 PM, talk-au-requ...@openstreetmap.org wrote: Send Talk-au mailing list submissions to talk-au@openstreetmap.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.openstreetmap.org/listinfo/talk-au or, via email, send a message with subject or body 'help' to talk-au-requ...@openstreetmap.org You can reach the person managing the list at talk-au-ow...@openstreetmap.org When replying, please edit your Subject line so it is more specific than Re: Contents of Talk-au digest... Today's Topics: 1. Re: vicmap data licensing (Li) 2. Re: vicmap data licensing (Nick Hocking) -- Message: 1 Date: Fri, 11 Oct 2013 07:37:31 +1100 From: Li m...@lixia.co To: Ben Kelley ben.kel...@gmail.com Cc: OSM Australian Talk List talk-au@openstreetmap.org Subject: Re: [talk-au] vicmap data licensing Message-ID: 91641499-6fb4-426c-8313-c0d82c305...@lixia.co Content-Type: text/plain; charset=us-ascii Does anyone have experience on importing data? In particular avoiding duplicates? Li. On 10 Oct 2013, at 5:16 pm, Ben Kelley ben.kel...@gmail.com wrote: Hi. ___ Talk-au mailing list Talk-au@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] South Australia Suburb Boundries
Hrm, this email got filtered I think due to the attachments.
On Saturday, October 12, 2013, Daniel O'Connor wrote:
Did the bits to produce .osm files (again on github); suitable to open up
and view in JOSM.
I spot checked two areas near me that I know well, and the accuracy is
pretty high.
http://www.openstreetmap.org/browse/way/241675341
http://www.openstreetmap.org/browse/way/241667279
I guess this discussion probably does want to head over to the imports
list shortly.
Any volunteers to help write up a fairly complete plan? I really don't
fancy doing ~3000 suburb boundaries one by one in JOSM and checking them
all myself; on top of doing all of the writeups/status updates/etc.
Also, I have overseas travel in the coming weeks; so am likely to vanish
half way through the conversation unless there's at least one other mapper
with ownership.
http://wiki.openstreetmap.org/wiki/Import/Plan_Outline
On Sun, Oct 6, 2013 at 7:01 PM, Daniel O'Connor
daniel.ocon...@gmail.comjavascript:_e({}, 'cvml',
'daniel.ocon...@gmail.com');
wrote:
I did look through a few of the existing tools; but most fell into the
too hard basket. In the end, manually doing it via QGIS and exporting into
the right projection was fairly easy.
I've pushed to github what I've done; which is produce geojson kml
serializations of it - I had assumed geojson.io would let me export
easily to OSM, but unfortunately that's not the case.
Example:
https://github.com/CloCkWeRX/data.sa.gov.au-suburb-boundaries/blob/master/suburbs/suburbs_0.geojson
Repo:
https://github.com/CloCkWeRX/data.sa.gov.au-suburb-boundaries/
Anyway, I'm more or less going to stop there, unless someone wants to do
a quick bit of geojson - osm; with the appropriate tagging and shoot in a
pull request or two - I'll let others shepard it through the import process
and de-duplication.
On Sun, Oct 6, 2013 at 6:44 PM, Andrew Harvey
andrew.harv...@gmail.comjavascript:_e({}, 'cvml',
'andrew.harv...@gmail.com');
wrote:
On 6 October 2013 18:25, Paul Norman penor...@mac.comjavascript:_e({},
'cvml', 'penor...@mac.com');
wrote:
There are numerous tools for converting from shapefiles to .osm. Both
shp-to-osm and ogr2osm work and you can find more info on the wiki.
ogr2ogr v1.10 can also do it I believe.
___
Talk-au mailing list
Talk-au@openstreetmap.org javascript:_e({}, 'cvml',
'Talk-au@openstreetmap.org');
https://lists.openstreetmap.org/listinfo/talk-au
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] vicmap data licensing
Hi Li I'm convinced that you will find that it is practically impossible to mass import (or even mass merge) a new road dataset into existing data when there is a large amount of existing data. The only practical way would be to delete all of the Victorian data and start again with the mass import. This would destroy OSM Victoria. Using high quality external data to complete the existing data is a realistic and not to difficult job. Firstly any existing roads that lost their names in the licence change (or had only ever been traced rather than surveyed in the first place) will appear highlighted in OSMI and can easily be named using vicmap data by any number of armchair mappers. This is an easy and enjoyable task. Also the vicmapdata will probably have a lot of roads that are new or that have never been traced or surveyed in OSM. These can easily be identified by overlaying the vicmap data in one JOSM layer and the existing data in another. Using correct transparency and colouring the new roads stand out like.oops. Then the vicmap data (on a road by road basis) can be click selected and merged into the existing layer with all its tags. If a whole neighbourhood is new then it may be merged as a unit. You will have to manually connect the new road/s to existing ones but that is also easy. Bing imagery (which is very good in Australia) can aslo be used, at the same time, to verify or tweak the vicmap data as it is manually merged. All this results in excellent road data which will be better than any other prroviders, by a wide margin. I do this every year that new TIGER data comes out and I make sure a few cities in the USA are updated in this fashion. Every time vicmap produce new datasets this process can easily be repeated, without having to do a whole new mass import. Of course, ideally, local mappers will have surveyed any new roads way before vicmap have produced a new data set. This certainly happens in Canberra - which reminds me, there may be three of four new roads open today - I'll go have a look in a minute or two. Given the current amount of good quality existing Victorian road data in OSM, the task of getting it perfect using vicmapdata will not take too long. Of even more use would be if there is cadastral data available. This is absolutely essential if OSM is to ever succeed and this data is not easily surveyed. Walking around with a pad an pencil peering into people's letterboxes or doors is not safe or enjoyable. Therefore this data is ideal to be mass imported. (especially as it would probably be high quality data). Even then you would have to be careful not to step on peoples toes who had already done a street or two. Maybe you could import address info on a street by street basis and if your import program found any existing address data, it could ignore that street and create a list somewhere that people could use to do a partial manual import from vicmap data. How you programmatically compare streets in OSM and Vicmapdata is the difficult bit. No one in OSM has ever been brave (or maybe clever) enough to achieve this yet and I certainly couldn't. Mass imports only can work if there is no (or almost no) existing data. USA found this out when the mass import of TIGER data, effectively destroyed the USA OSM community and it has taken nearly ten years to recover. Unfortunately the original TIGER data was/is very low quality in terms of geometry and this still plagues the USA data to this day (and probably for the next 5 years or so). I'm sure that people in OSM-US can help in converting vicmap data into an imagery layer (and may even host it for you as well :-) Cheers Nick ___ Talk-au mailing list Talk-au@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-au
