Re: [talk-au] Bulk loading all the Australian Statistical Geography Standard into the OSM - a query from the Australian Bureau of Statistics [SEC=UNCLASSIFIED]
I've done a blog post about how I used the landuse data tied to the Mesh Blocks to load as an underlay in an OSM editor (JOSM). http://andrewharvey4.wordpress.com/2011/03/17/using-abs-asgs-data-in-openstreetmap/ The data is a bit coarse in places but I think when combined with NearMap tracing it can help out places in the OSM map that are currently lacking landuse cover, or don't already have detailed landuse information. Unfortunately I don't have the resources to host the WMS service (I mention in the blog post) publicly though. On Wed, Feb 23, 2011 at 12:01 PM, Marcus Blake marcus.bl...@abs.gov.auwrote: To the Australian OSM community, The Australian Bureau of Statistics has recent published the first part of a new statistical geography, the Australia Statistical Geography Standard or ASGS for short. The boundaries are based on a new basic spatial unit called a mesh block which have been aggregated to create efficient spatial units for the dissemination and analysis of statistical data. They have been released in advanced of the 2011 Australian census and are fixed for the next 5 years. The attached links and PDF file provide additional information. The ABS Geography section is presently investigating the possibility of loaded the new Australian Statistical Geography Standard into the OSM database. As a starting point, I'd like to start a discussion about how this could be achieved, if it is possible at all. From the ABS point of view the principle reason for doing this is that an the OSM database would hold a copy of the official version of the boundaries and that this point of truth would be available for all OSM users and downstream distributors. It would therefore become one of the channels by which the ABS distributes the ASGS boundaries and associated coding structures There are three main issues I can see need addressing (and probably a large number of other issues I'm not yet aware of ) * 1. Is the OSM database a suitable location for the ASGS* The ABS would like to facilitate the use of the new ASGS as much as possible and the OSM database looks to be an efficient mechanism for the distribution of the spatial boundaries and codes. But what does the community think??... * 2. Licensing* Even though ABS data (including all spatial data) is released under a CC license it does require attribution (Attribution 2.5 Australia CC BY 2.5). How is this license model handled under OSM. Is there a means to associated attribution with particular layers within the OSM database? * 3. The practicality's of loading load.* I note previous posts on loading the ABS Postal Areas and the technical problems involved. What is the most efficient and best way of load a categorising these data within the database? Our preference would be to bulk upload through an FME process. Perhaps this is a question for the imports list? * Any Questions for the ABS?* Lastly if there are any questions people have on the new ASGS (and the old ASGC) or anything on the definition or application of statistical boundaries I am happy to answer specific queries and contribute to discussions. cheers, Marcus. Marcus Blake marcus.bl...@abs.gov.au Assistant Director Geography Section Australian Bureau of Statistics - * Additional Information* *2911.0.55.003 - Census of Population and Housing: Outcomes from the 2011 Census Output Geography Discussion Paper, 2011* This publication a good diagram of the ASGS * ** http://www.abs.gov.au/AUSSTATS/abs@.nsf/ProductsbyReleaseDate/DB85CD1D52DE042DCA25783E000E0AF8?OpenDocument *http://www.abs.gov.au/AUSSTATS/abs@.nsf/ProductsbyReleaseDate/DB85CD1D52DE042DCA25783E000E0AF8?OpenDocument * ABS License details: CC Attribution 2.5 Australia* * **http://creativecommons.org/licenses/by/2.5/au/deed.en*http://creativecommons.org/licenses/by/2.5/au/deed.en * The first volume of the ASGS. * This includes all the electronic boundaries in MID/MIF and Shape formats * **Australian Statistical Geography Standard (ASGS): Volume 1 - Main Structure and Greater Capital City Statistical Areas, July 2011 (cat no. 1270.0.55.001)* http://www.abs.gov.au/ausstats/abs@.nsf/mf/1270.0.55.001 * The ABS Geography website* * **http://www.abs.gov.au/Geography* http://www.abs.gov.au/Geography * Geography Frequently Asked Questions* * ** http://www.abs.gov.au/websitedbs/D3310114.nsf/home/Frequently+Asked+Questions *http://www.abs.gov.au/websitedbs/D3310114.nsf/home/Frequently+Asked+Questions Free publications and statistics available on www.abs.gov.au ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au ___ Talk-au
Re: [talk-au] Bulk loading all the Australian Statistical Geography Standard into the OSM - a query from the Australian Bureau of Statistics [SEC=UNCLASSIFIED]
On Thu, 2011-02-24 at 15:12 +1100, {withheld} wrote:
However I still hold the community should accept the offer and be
grateful. Carping about internal politics just looks bad. And whiny. And
doesn't encourage anybody else ever offering similar largesse ever again.
Well, to be fair, the community would accept his data, but in 4 weeks it
will be removed from the OSM database and it will be unable to be
included in any future OSM.
Whether you think this is internal politics or not, it is the law and as
Marcus clearly explained what terms the data would be released under,
the fact that some within the OSM project have chosen to no longer
utilise data sources such as his is the problem, and theyre the ones you
should be talking to, not us.
David
___
Talk-au mailing list
Talk-au@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Bulk loading all the Australian Statistical Geography Standard into the OSM - a query from the Australian Bureau of Statistics [SEC=UNCLASSIFIED]
On 24 February 2011 10:52, Steve Bennett stevag...@gmail.com wrote: 1) OSM's core purpose is as a street map This hasn't been the case for quite some time. Not to mention that the previous ABS data has been very useful in regional areas for plotting physical features, like roads, that couldn't be obtained any other way at the time. I can only assume the new data would be the same, if not better, than the existing data which would continue to improve map data available in regional areas. ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Bulk loading all the Australian Statistical Geography Standard into the OSM - a query from the Australian Bureau of Statistics [SEC=UNCLASSIFIED]
On 24/02/11 12:50, John Smith wrote: On 24 February 2011 10:52, Steve Bennett stevag...@gmail.com wrote: 1) OSM's core purpose is as a street map This hasn't been the case for quite some time. Not to mention that the previous ABS data has been very useful in regional areas for plotting physical features, like roads, that couldn't be obtained any other way at the time. I can only assume the new data would be the same, if not better, than the existing data which would continue to improve map data available in regional areas. For expletive elided's sake, with due respect to Andrew Harvey and Steve Bennett, shouldn't the community be a whole lot better off expressing appropriate gratitude to Marcus Blake for offering this data and then pissing off and finding a use for it! No wonder no bloody government supplier wants to give us data. We are so damned ungrateful for what we get! I genuinely hope I have hurt a few feelings. At least I will have made the point. ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Bulk loading all the Australian Statistical Geography Standard into the OSM - a query from the Australian Bureau of Statistics [SEC=UNCLASSIFIED]
On Thu, Feb 24, 2011 at 2:33 PM, {withheld}
pheasant.cou...@gmail.com For expletive elided's sake, with due
respect to Andrew Harvey and
Steve Bennett, shouldn't the community be a whole lot better off
expressing appropriate gratitude to Marcus Blake for offering this data
and then pissing off and finding a use for it!
No wonder no bloody government supplier wants to give us data. We are so
damned ungrateful for what we get!
Yeah, I think it's great that the ABS wants to get involved with OSM.
But obviously not all data is appropriate for OSM, and the OP even
made clear that he wasn't certain that this was the case here.
Maybe someone could explain a bit more about this ASGS, and why it's a
good fit for OSM?
I genuinely hope I have hurt a few feelings. At least I will have made
the point.
Wrong on both counts, I believe.
Steve
___
Talk-au mailing list
Talk-au@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Bulk loading all the Australian Statistical Geography Standard into the OSM - a query from the Australian Bureau of Statistics [SEC=UNCLASSIFIED]
On 24/02/11 14:46, Ian Sergeant wrote:
On 24 February 2011 14:33, {withheld} pheasant.cou...@gmail.com
mailto:pheasant.cou...@gmail.com wrote:
For expletive elided's sake, with due respect to Andrew Harvey and
Steve Bennett, shouldn't the community be a whole lot better off
expressing appropriate gratitude to Marcus Blake for offering this data
and then pissing off and finding a use for it!
Marcus isn't just offering the data, he is also looking for a static
data repository.
He says that ...
From the ABS point of view the principle reason for doing this is
that an the OSM database would hold a copy of the official version
of the boundaries and that this point of truth would be available for
all OSM users and downstream distributors.
I don't believe OSM offers that. It is kindness in stating that directly
up front.
That doesn't mean that OSM can't continue to work with the ABS data in
other ways.
Not quite sure I fully accept that point, as I interpret Marcus' wording
more in the sense of its generosity than him stating restrictions. (A
government department asking for charity? Doesn't make sense.)
However I still hold the community should accept the offer and be
grateful. Carping about internal politics just looks bad. And whiny. And
doesn't encourage anybody else ever offering similar largesse ever again.
Do we really want to appear like a bunch of inconsiderate losers; or do
we have any sort of pretence to be a professional outfit? Or is this
community just comprehensively hopeless at everything we attempt?
In case I am the only person to do this: Thank you Marcus, for your fine
and generous offer. It will be looked into, and somebody will get back
to you as and if appropriate.
___
Talk-au mailing list
Talk-au@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Bulk loading all the Australian Statistical Geography Standard into the OSM - a query from the Australian Bureau of Statistics [SEC=UNCLASSIFIED]
Marcus Blake marcus.bl...@abs.gov.au wrote Wed, 23 February, 2011 11:31:50: From the ABS point of view the principle reason for doing this is that an the OSM database would hold a copy of the official version of the boundaries and that this point of truth would be available for all OSM users and downstream distributors. It would therefore become one of the channels by which the ABS distributes the ASGS boundaries and associated coding structures Hi Marcus, You may be misunderstanding how OSM works. OSM doesn't hold official, unchanging versions of things. Things get modified all the time. The ABS suburb boundaries have been imported into OSM previously. They were used as a basis for identification of the path of roads and rivers which often tend to run along the boundaries, and in many places were split and joined with those roads and rivers such that subsequent improvements to the roads and rivers have modified the position of the ABS boundaries. This makes it difficult to replace the boundaries in OSM with updated data from ABS. OSM does not have layers. All the objects are joined together according to the real world topology (e.g. roads with paths and railways at level crossings). All of OSM is released under CC-BY-SA which is an attribution license compatible with CC-BY. The attribution includes a link to a list of data providers and contributors on www.openstreetmap.org in which ABS is listed. http://wiki.openstreetmap.org/wiki/Contributors OSM has a web API and a set of community built tools which may be used to handle imports (or write your own tools). However, a lot of care needs to go into identifying and improving the existing data, without either duplicating existing boundary data or removing people's work in edits they have made to features that may be joined to the boundaries. ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Bulk loading all the Australian Statistical Geography Standard into the OSM - a query from the Australian Bureau of Statistics [SEC=UNCLASSIFIED]
Hi Marcus Unfortunately OSM has recently forced a change to it's licence agreement to a version where attribution is not required on any copies that are made of OSM data, probably to appease Microsoft and Bing maps who will then be free to charge for these maps, with no attribution at all. Anybody who has used nearmap or Government data sources for their mapping therefore cannot agree to the new terms, and all of their data is going to be removed on 1st April 2011. As you can imagine there are a lot of upset mappers, and there are alternative sites being set up where the original licence and data will be retained. There are a number of sites doing this including; http://fosm.org Creating a new Layer for your data would be a good move from the point of view of mappers, who could not change this data either deliberately or accidentally, and it would therefore be more reliable. Unfortunately these changes are recent and the alternative sites are still a work in progress, and not yet ready to adapt to new requirements. Having said that, go to http://fosm.org/p2/potlatchFosm.xml, and look at the Background drop down menu. It includes a number of options for background layers from a variety of sources. Also try http://www.openstreetmap.org, open up the edit tab, and select the checkbox option in the bottom left hand corner of the potlatch window, which then shows background layer options. I think all other editors also have these background options, and there are a number of editors out there. I would suggest to you that you make your data available in a format that is compatible with these other background sources, and host the actual data on your own servers. This would also have the advantage that your data will always be up to the minute if and when changes are made. It would then not take much for the mapping applications to import your data as a layer, and you would not need to chase up the different mapping sites and get them to include your data. It would also be a relativity small step to host your own map viewer, which could include your data as a layer as well as the option of google maps, bing maps, open street map, fosm or whatever as a reference to where the boundary's are relative to roads and creeks or coastlines. I do not know what the API's are, or even where to find them, but the nearmap http://www.nearmap.com/; people are active and if they cannot help you then I am sure they can point you in the right direction. Andrew. On 23 February 2011 09:01, Marcus Blake marcus.bl...@abs.gov.au wrote: To the Australian OSM community, The Australian Bureau of Statistics has recent published the first part of a new statistical geography, the Australia Statistical Geography Standard or ASGS for short. The boundaries are based on a new basic spatial unit called a mesh block which have been aggregated to create efficient spatial units for the dissemination and analysis of statistical data. They have been released in advanced of the 2011 Australian census and are fixed for the next 5 years. The attached links and PDF file provide additional information. The ABS Geography section is presently investigating the possibility of loaded the new Australian Statistical Geography Standard into the OSM database. As a starting point, I'd like to start a discussion about how this could be achieved, if it is possible at all. From the ABS point of view the principle reason for doing this is that an the OSM database would hold a copy of the official version of the boundaries and that this point of truth would be available for all OSM users and downstream distributors. It would therefore become one of the channels by which the ABS distributes the ASGS boundaries and associated coding structures There are three main issues I can see need addressing (and probably a large number of other issues I'm not yet aware of ) * 1. Is the OSM database a suitable location for the ASGS* The ABS would like to facilitate the use of the new ASGS as much as possible and the OSM database looks to be an efficient mechanism for the distribution of the spatial boundaries and codes. But what does the community think??... * 2. Licensing* Even though ABS data (including all spatial data) is released under a CC license it does require attribution (Attribution 2.5 Australia CC BY 2.5). How is this license model handled under OSM. Is there a means to associated attribution with particular layers within the OSM database? * 3. The practicality's of loading load.* I note previous posts on loading the ABS Postal Areas and the technical problems involved. What is the most efficient and best way of load a categorising these data within the database? Our preference would be to bulk upload through an FME process. Perhaps this is a question for the imports list? * Any Questions for the ABS?* Lastly if there are any questions people have on the new ASGS (and the old ASGC) or
Re: [talk-au] Bulk loading all the Australian Statistical Geography Standard into the OSM - a query from the Australian Bureau of Statistics [SEC=UNCLASSIFIED]
On 23 February 2011 11:35, Simon Biber simonbi...@yahoo.com.au wrote: of OSM is released under CC-BY-SA which is an attribution license compatible with CC-BY. The attribution includes a link to a list of data providers and contributors on www.openstreetmap.org in which ABS is listed. As of April 1st only accounts that agree to the new CTs (contributor terms) will be able to contribute to OSM, while the CTs suggest OSM-F will attribute data donors, there is no such guarantee that down stream users of the data will be required to do the same, so many data sources will possibly be removed from OSM-F's database in the near future, or reverted shortly after importing. CommonMap would possibly be a better fit for the ABS and other such bodies as they operate under a cc-by license and have no plans to change, ever. ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Bulk loading all the Australian Statistical Geography Standard into the OSM - a query from the Australian Bureau of Statistics [SEC=UNCLASSIFIED]
On Wed, 2011-02-23 at 10:38 +0800, Andrew Laughton wrote: Hi Marcus Unfortunately OSM has recently forced a change to it's licence agreement to a version where attribution is not required on any copies that are made of OSM data, probably to appease Microsoft and Bing maps who will then be free to charge for these maps, with no attribution at all. Unfortunately, as stated by others, this is currently the situation. The powers-that-be dont really follow this email list, so you would probably be better off contacting the OSM-talk list or the OSM legal list. While those in charge may not listen to us little folk, an organisation such ABS might have more pull. Knowing that the project might lose the tracing of some small contributors to forked sites, isnt quite as devastating as knowing government departments will prefer to use CC sites in preference to their OSM site. However another important thing to remember, is that although this major change is happening in just over 5 weeks, the Contributor Terms still havent been finalised. Really your best bet is to contact the legal list or the OSM legal working group, as the best anyone here can offer, is what we've pieced together from what little the foundation and legal team have allowed us mere users to know. David Anybody who has used nearmap or Government data sources for their mapping therefore cannot agree to the new terms, and all of their data is going to be removed on 1st April 2011. As you can imagine there are a lot of upset mappers, and there are alternative sites being set up where the original licence and data will be retained. There are a number of sites doing this including; http://fosm.org Creating a new Layer for your data would be a good move from the point of view of mappers, who could not change this data either deliberately or accidentally, and it would therefore be more reliable. Unfortunately these changes are recent and the alternative sites are still a work in progress, and not yet ready to adapt to new requirements. Having said that, go to http://fosm.org/p2/potlatchFosm.xml, and look at the Background drop down menu. It includes a number of options for background layers from a variety of sources. Also try http://www.openstreetmap.org, open up the edit tab, and select the checkbox option in the bottom left hand corner of the potlatch window, which then shows background layer options. I think all other editors also have these background options, and there are a number of editors out there. I would suggest to you that you make your data available in a format that is compatible with these other background sources, and host the actual data on your own servers. This would also have the advantage that your data will always be up to the minute if and when changes are made. It would then not take much for the mapping applications to import your data as a layer, and you would not need to chase up the different mapping sites and get them to include your data. It would also be a relativity small step to host your own map viewer, which could include your data as a layer as well as the option of google maps, bing maps, open street map, fosm or whatever as a reference to where the boundary's are relative to roads and creeks or coastlines. I do not know what the API's are, or even where to find them, but the nearmap http://www.nearmap.com/; people are active and if they cannot help you then I am sure they can point you in the right direction. Andrew. On 23 February 2011 09:01, Marcus Blake marcus.bl...@abs.gov.au wrote: To the Australian OSM community, The Australian Bureau of Statistics has recent published the first part of a new statistical geography, the Australia Statistical Geography Standard or ASGS for short. The boundaries are based on a new basic spatial unit called a mesh block which have been aggregated to create efficient spatial units for the dissemination and analysis of statistical data. They have been released in advanced of the 2011 Australian census and are fixed for the next 5 years. The attached links and PDF file provide additional information. The ABS Geography section is presently investigating the possibility of loaded the new Australian Statistical Geography Standard into the OSM database. As a starting point, I'd like to start a discussion about how this could be achieved, if it is possible at all. From the ABS point of view the principle reason for doing this is that an the OSM database would hold a copy of the official version of the boundaries and that this point of truth would be available for all OSM users and downstream distributors. It would therefore become one of the channels by
