Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

On Fri, 18 Sep 2020 at 21:25, Andrew Davidson  wrote:

> On 17/9/20 5:36 pm, Andrew Harvey wrote:
> >
> > So while I'd prefer using ref: I guess some psma specific tag
> > could be okay.
>
> Do you have a preference between ref: and note: ?
>

I can't decide, so I'm happy with whatever you or others decide. (I'd also
be fine with not including the ID at all as well)


> > So you're suggesting 1) uploading duplicate state borders, and then
> > cleanup after import or 2) doing the upload so that it immediately uses
> > the existing state borders?
>
> 1 as 2 is too complicated for my tiny brain.
>
> Except for QLD I don't expect it would take too long to replace the
> overlapping ways. QLD, on the other hand, took me a couple of evenings
> to chop off the boundaries at the state border, so it might take a while
> as there are quite a few boundaries that will have to be merged into the
> maritime boundary.
>
> Provided the outer ways have a fixme on it, it'll be easy to find the
> remaining ways that need merging.
>

I don't like the idea of uploading duplicate data to be fixed later. I took
a look in JOSM and it doesn't seem too hard to manually correct the files
to be uploaded to use the existing state border way. I'm happy to do this
manual work to get the import files ready.
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Davidson]

On 17/9/20 5:36 pm, Andrew Harvey wrote:


So while I'd prefer using ref: I guess some psma specific tag 
could be okay.


Do you have a preference between ref: and note: ?

So you're suggesting 1) uploading duplicate state borders, and then 
cleanup after import or 2) doing the upload so that it immediately uses 
the existing state borders?


1 as 2 is too complicated for my tiny brain.

Except for QLD I don't expect it would take too long to replace the 
overlapping ways. QLD, on the other hand, took me a couple of evenings 
to chop off the boundaries at the state border, so it might take a while 
as there are quite a few boundaries that will have to be merged into the 
maritime boundary.


Provided the outer ways have a fixme on it, it'll be easy to find the 
remaining ways that need merging.



___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

On Wed, 16 Sep 2020 at 20:07, Andrew Davidson  wrote:

> On 15/9/20 10:53 pm, Andrew Harvey wrote:
> >
> > 1. psma:loc_pid. Where this is a stable ID that is used as a reference,
> > the existing ref tag is better for this. If we want to be more specific
> > then ref:psma or something like that would work. No need to invent new
> > tags here when one already exists, is well documented and in widespread
> use.
>
> I've never really considered these to be more than tagging for dataset
> maintainers, I didn't really think that any data consumers would want to
> use these. I would not put these under ref=* as they aren't really
> references that end users would see.
>

I guess it could be done either way, eg. wikidata IDs are done as their own
key http://wiki.openstreetmap.org/wiki/Key:wikidata. Same for iata and icao
https://wiki.openstreetmap.org/wiki/Key:icao.

However also some ref:namespace tags like
https://wiki.openstreetmap.org/wiki/Key:ref:isil and
https://wiki.openstreetmap.org/wiki/Key:ref:wigos and
https://wiki.openstreetmap.org/wiki/Key:ref:whc

Of course the big issue with non-namespaced ref is there can only be one,
and by the looks of it these are something PSMA ad and PSMA are certainly
not the authority on defining LGAs and Suburb/Localities. So for that
reason alone, I change my mind, it shouldn't be non-namespaced ref.

So while I'd prefer using ref: I guess some psma specific tag
could be okay.

Though I'm not too fussed either way, if others have a different opinion
I'm happy to go with whatever.

> 2. Regarding source tags on objects, this might be something I added
> > originally, I can't remember, but I'm on the fence about it.
>
> The majority view at the time that we discussed this was to add a source
> tag. I understand that the source tag does have a decay function as to
> it's usefulness but it's still better than changeset tagging, which
> become useless the moment you upload the data. Maybe if Overpass could
> search for something based on the changeset tags of the last modifying
> changeset they would be useful, but till that tag I prefer to have them
> in-channel.
>
> If the general view is now that adding a source tag is not worthwhile
> then we can leave them out.
>




>
> > The "import" upload
> > should immediately be correct and not a broken state until post-import
> > changes clean things up, it should be uploaded clean in the first
> > instance.
>
> I agree with you 100%. The reason the current workflow has us deleting
> the outer ways before uploading is because this is what you wanted:
>
> https://lists.openstreetmap.org/pipermail/talk-au/2018-October/012131.html
>
> I'm happy to upload valid boundaries which is what I suggested:
>
> https://lists.openstreetmap.org/pipermail/talk-au/2018-October/012132.html


So you're suggesting 1) uploading duplicate state borders, and then cleanup
after import or 2) doing the upload so that it immediately uses the
existing state borders?
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Davidson]

On 15/9/20 10:53 pm, Andrew Harvey wrote:


1. psma:loc_pid. Where this is a stable ID that is used as a reference, 
the existing ref tag is better for this. If we want to be more specific 
then ref:psma or something like that would work. No need to invent new 
tags here when one already exists, is well documented and in widespread use.


I've never really considered these to be more than tagging for dataset 
maintainers, I didn't really think that any data consumers would want to 
use these. I would not put these under ref=* as they aren't really 
references that end users would see.


The two options would be:
1. the ref: namespace or
2. ref:AU: namespace which is similar to what the FR community seems to use:

https://wiki.openstreetmap.org/wiki/France/Liste_des_r%C3%A9f%C3%A9rences_nationales

Any opinions?

If this is how people would like to tag these types of ids I would also 
have to go back and modify all of the others I have created over the years.


2. Regarding source tags on objects, this might be something I added 
originally, I can't remember, but I'm on the fence about it.


The majority view at the time that we discussed this was to add a source 
tag. I understand that the source tag does have a decay function as to 
it's usefulness but it's still better than changeset tagging, which 
become useless the moment you upload the data. Maybe if Overpass could 
search for something based on the changeset tags of the last modifying 
changeset they would be useful, but till that tag I prefer to have them 
in-channel.


If the general view is now that adding a source tag is not worthwhile 
then we can leave them out.


The "import" upload 
should immediately be correct and not a broken state until post-import 
changes clean things up, it should be uploaded clean in the first 
instance.


I agree with you 100%. The reason the current workflow has us deleting 
the outer ways before uploading is because this is what you wanted:


https://lists.openstreetmap.org/pipermail/talk-au/2018-October/012131.html

I'm happy to upload valid boundaries which is what I suggested:

https://lists.openstreetmap.org/pipermail/talk-au/2018-October/012132.html

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

On Wed, 16 Sep 2020 at 10:44, cleary  wrote:

> Thanks to both Andrew Davidson and Andrew Harvey for their work as I see
> addition of PSMA administrative boundaries as an important improvement for
> our map.
>
> In regard to adding source tags to objects, I find them very helpful when
> editing something. If I think I have contrary information, I refer to the
> source of the existing data. If it is an authoritative source, I am
> prompted to go back and check my own information. Where information is
> unsourced or refers to a less authoritative source, then I am more likely
> to proceed with the edit or attempt to contact the mapper who entered the
> previous information.  Also I hope that by adding authoritative sources for
> my edits, other mappers will be less likely to make changes without
> contacting me or at least double-checking the accuracy of their later
> information.  If information comes from multiple sources, then it may be
> appropriate to have multiple source tags e.g. source:geometry, source:name,
> source:alt_name etc.


I listened to a talk by Maurits van der Vlugt about "authoritative data"
once (slides
https://www.slideshare.net/MauritsV/authoritative-data-must-come-from-an-official-source
).

I think we give too much credit for so-called "authoritative data". Just
because the government publishes a GIS file of LGA and suburb boundaries,
doesn't mean it's 100% accurate because it's not, and so we shouldn't treat
these objects in OSM as immutable (I know you're not claiming that).

In regard to relations, it is possible to have different sources for the
> relation and for component ways,


Yes this is exactly how the import candidates have been prepared, I'm in
support of the source:geometry=* on the way, it's just the source=* on the
relation, especially where there will be many other tags added later that
I'm not so sure about.

Ways often get split, and so checking the history to find the source can be
difficult, but this isn't much of a problem for the relations.
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[cleary]

Thanks to both Andrew Davidson and Andrew Harvey for their work as I see 
addition of PSMA administrative boundaries as an important improvement for our 
map.

In regard to adding source tags to objects, I find them very helpful when 
editing something. If I think I have contrary information, I refer to the 
source of the existing data. If it is an authoritative source, I am prompted to 
go back and check my own information. Where information is unsourced or refers 
to a less authoritative source, then I am more likely to proceed with the edit 
or attempt to contact the mapper who entered the previous information.  Also I 
hope that by adding authoritative sources for my edits, other mappers will be 
less likely to make changes without contacting me or at least double-checking 
the accuracy of their later information.  If information comes from multiple 
sources, then it may be appropriate to have multiple source tags e.g. 
source:geometry, source:name, source:alt_name etc.

In regard to relations, it is possible to have different sources for the 
relation and for component ways, e.g a relation for a waterway may show the 
source of information including the name sourced from a permitted government 
dataset but component segments of the waterway may be sourced from two 
different satellite imagery sources.  Again, I find sources helpful. Even state 
boundaries need verifying. At one time (not sure if it still the case) some SA 
Govt data showed some SA localities and protected areas as extending into NSW 
along parts of the shared border. After looking closely, I moved the OSM 
boundary slightly in some areas (generally preferring the NSW LPI data) but I 
was always careful to show the source of my data so that other mappers could 
verity or challenge me if appropriate.  Some administrative boundaries in OSM 
were sourced from ABS data which were always just approximations. ABS data was 
useful in OSM when we had nothing else, but not now that we have accurate data 
from official sources.  I have found it helpful to know the sources of the data 
I am looking at. 




On Tue, 15 Sep 2020, at 12:53 PM, Andrew Harvey wrote:
> Resurecting this old thread as Andrew Davidson has been working on this 
> import plan a bit more -> 
> https://wiki.openstreetmap.org/w/index.php?title=Import%2FCatalogue%2FPSMA_Admin_Boundaries=revision=2034132=1918537
> 
> Surfacing a few points from my review:
> 
> 1. psma:loc_pid. Where this is a stable ID that is used as a reference, 
> the existing ref tag is better for this. If we want to be more specific 
> then ref:psma or something like that would work. No need to invent new 
> tags here when one already exists, is well documented and in widespread 
> use.
> 
> 2. Regarding source tags on objects, this might be something I added 
> originally, I can't remember, but I'm on the fence about it. While on 
> one hand I can see it being helpful for mappers to understand where 
> these were sourced from, over time as people make changes in OSM the 
> source tag becomes inaccurate. I've seen this with existing source 
> tags, where people generally don't remove them if updating based on a 
> different source. Since you can always inspect the history to find the 
> source, is there really any benefit to having these on each object? 
> Given a whole bunch of other tags like contract details, website etc. 
> for LGAs can be added to, a top level source tag is not perfect. I'm 
> good with source:geometery on the ways, but not sure about source on 
> the relations.
> 
> 3. More of a question for Andrew Davidson, is the plan for the actual 
> upload to go through the state borders and other ways already existing 
> in OSM, and delete the one with FIXME from your candidates, and use the 
> existing state border as part of the new relation? The "import" upload 
> should immediately be correct and not a broken state until post-import 
> changes clean things up, it should be uploaded clean in the first 
> instance. So this means manually working on it in JOSM before uploading.
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au
>

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

Resurecting this old thread as Andrew Davidson has been working on this
import plan a bit more ->
https://wiki.openstreetmap.org/w/index.php?title=Import%2FCatalogue%2FPSMA_Admin_Boundaries=revision=2034132=1918537

Surfacing a few points from my review:

1. psma:loc_pid. Where this is a stable ID that is used as a reference, the
existing ref tag is better for this. If we want to be more specific then
ref:psma or something like that would work. No need to invent new tags here
when one already exists, is well documented and in widespread use.

2. Regarding source tags on objects, this might be something I added
originally, I can't remember, but I'm on the fence about it. While on one
hand I can see it being helpful for mappers to understand where these were
sourced from, over time as people make changes in OSM the source tag
becomes inaccurate. I've seen this with existing source tags, where people
generally don't remove them if updating based on a different source. Since
you can always inspect the history to find the source, is there really any
benefit to having these on each object? Given a whole bunch of other tags
like contract details, website etc. for LGAs can be added to, a top level
source tag is not perfect. I'm good with source:geometery on the ways, but
not sure about source on the relations.

3. More of a question for Andrew Davidson, is the plan for the actual
upload to go through the state borders and other ways already existing in
OSM, and delete the one with FIXME from your candidates, and use the
existing state border as part of the new relation? The "import" upload
should immediately be correct and not a broken state until post-import
changes clean things up, it should be uploaded clean in the first instance.
So this means manually working on it in JOSM before uploading.
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

I do remember that problem, getting it fixed should be great.

Then we should start to make plans, I assume the work-flow would span
multiple days so that we can do things removing the old boundaries, right?

On 20/12/18 3:54 pm, Andrew Harvey wrote:
> Just posting any questions here, works for me.
>
> I lost time to work on this, but I'm back again with free time heading
> into the holidays.
>
> I thought we still had some issues with the .osm files, specifically I
> needed to increase the generalisation to try to ensure ways that
> should be snapped are snapped.
>
> I'm also on the maptime slack if that's easier to work through specifics.
>
>
> On Wed, 19 Dec 2018 at 06:47, Andrew Davidson  wrote:
>> I have a few remaining tagging question left to answer and the workflow
>> still needs to be written up (and agreed on). Other than that, I think
>> we're all good to go.
>>
>> Not wanting to hijack the project I've been a bit unsure on how to push
>> this import over the last hump. I could write up a list of questions for
>> the mail list or I could try to finish the wiki page and have the
>> discussions on the talk page. Any views or objections?
>>
>> On 19/12/18 12:23 am, Joel H. wrote:
>>> Hmm, It's been almost a month since the last message in this
>>> import-related thread. I hope for a date planned so we can follow
>>> through with the plan, since it seems pretty complete.
>>>
>>> If Andrew Davidson is still interested, his script could help match and
>>> add the nodes into the relationships and possibly flag discrepancies in
>>> names.
>>>
>>> Is anyone interested in reviving this effort?
>>>
>>> On 19/11/18 8:49 pm, Andrew Davidson wrote:
 Planning for the import is not finished yet, so please wait until
 after that.

 On Mon, 19 Nov 2018 21:23 Lee Mason >>>  wrote:

 I am prepared to make a start on the Tasmania import over the next
 few days, unless there is some further discussion?



 ___
 Talk-au mailing list
 Talk-au@openstreetmap.org
 https://lists.openstreetmap.org/listinfo/talk-au
>>> ___
>>> Talk-au mailing list
>>> Talk-au@openstreetmap.org
>>> https://lists.openstreetmap.org/listinfo/talk-au
>>>
>> ___
>> Talk-au mailing list
>> Talk-au@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-au
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

Just posting any questions here, works for me.

I lost time to work on this, but I'm back again with free time heading
into the holidays.

I thought we still had some issues with the .osm files, specifically I
needed to increase the generalisation to try to ensure ways that
should be snapped are snapped.

I'm also on the maptime slack if that's easier to work through specifics.


On Wed, 19 Dec 2018 at 06:47, Andrew Davidson  wrote:
>
> I have a few remaining tagging question left to answer and the workflow
> still needs to be written up (and agreed on). Other than that, I think
> we're all good to go.
>
> Not wanting to hijack the project I've been a bit unsure on how to push
> this import over the last hump. I could write up a list of questions for
> the mail list or I could try to finish the wiki page and have the
> discussions on the talk page. Any views or objections?
>
> On 19/12/18 12:23 am, Joel H. wrote:
> > Hmm, It's been almost a month since the last message in this
> > import-related thread. I hope for a date planned so we can follow
> > through with the plan, since it seems pretty complete.
> >
> > If Andrew Davidson is still interested, his script could help match and
> > add the nodes into the relationships and possibly flag discrepancies in
> > names.
> >
> > Is anyone interested in reviving this effort?
> >
> > On 19/11/18 8:49 pm, Andrew Davidson wrote:
> >> Planning for the import is not finished yet, so please wait until
> >> after that.
> >>
> >> On Mon, 19 Nov 2018 21:23 Lee Mason  >>  wrote:
> >>
> >> I am prepared to make a start on the Tasmania import over the next
> >> few days, unless there is some further discussion?
> >>
> >>
> >>
> >> ___
> >> Talk-au mailing list
> >> Talk-au@openstreetmap.org
> >> https://lists.openstreetmap.org/listinfo/talk-au
> >
> > ___
> > Talk-au mailing list
> > Talk-au@openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/talk-au
> >
>
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Davidson]

I have a few remaining tagging question left to answer and the workflow 
still needs to be written up (and agreed on). Other than that, I think 
we're all good to go.


Not wanting to hijack the project I've been a bit unsure on how to push 
this import over the last hump. I could write up a list of questions for 
the mail list or I could try to finish the wiki page and have the 
discussions on the talk page. Any views or objections?


On 19/12/18 12:23 am, Joel H. wrote:
Hmm, It's been almost a month since the last message in this 
import-related thread. I hope for a date planned so we can follow 
through with the plan, since it seems pretty complete.


If Andrew Davidson is still interested, his script could help match and 
add the nodes into the relationships and possibly flag discrepancies in 
names.


Is anyone interested in reviving this effort?

On 19/11/18 8:49 pm, Andrew Davidson wrote:
Planning for the import is not finished yet, so please wait until 
after that.


On Mon, 19 Nov 2018 21:23 Lee Mason  wrote:


I am prepared to make a start on the Tasmania import over the next
few days, unless there is some further discussion?



___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au


___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au



___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

Hmm, It's been almost a month since the last message in this
import-related thread. I hope for a date planned so we can follow
through with the plan, since it seems pretty complete.

If Andrew Davidson is still interested, his script could help match and
add the nodes into the relationships and possibly flag discrepancies in
names.

Is anyone interested in reviving this effort?

On 19/11/18 8:49 pm, Andrew Davidson wrote:
> Planning for the import is not finished yet, so please wait until
> after that.
>
> On Mon, 19 Nov 2018 21:23 Lee Mason   wrote:
>
> I am prepared to make a start on the Tasmania import over the next
> few days, unless there is some further discussion?
>
>
>
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Davidson]

Planning for the import is not finished yet, so please wait until after
that.

On Mon, 19 Nov 2018 21:23 Lee Mason  I am prepared to make a start on the Tasmania import over the next few
> days, unless there is some further discussion?
>
>
>
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Lee Mason]

Ok!


From: Andrew Davidson 
Sent: Monday, November 19, 2018 9:45:24 PM
To: Lee Mason
Subject: Re: [talk-au] PSMA Administrative Boundaries

Planning for the import is not finished yet, so please wait until after that.

On Mon, 19 Nov 2018 21:23 Lee Mason 
mailto:lee.ma...@outlook.com.au> wrote:

I am prepared to make a start on the Tasmania import over the next few days, 
unless there is some further discussion?




From: Joel H. mailto:jo...@disroot.org>>
Sent: Wednesday, November 7, 2018 2:57:45 PM
To: talk-au@openstreetmap.org<mailto:talk-au@openstreetmap.org>; Andrew Harvey
Subject: Re: [talk-au] PSMA Administrative Boundaries

How are we with our import plan on the wiki, Do we have any blockers
stopping the PSMA import?

I think we should start setting dates state-by-state for import.


___
Talk-au mailing list
Talk-au@openstreetmap.org<mailto:Talk-au@openstreetmap.org>
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openstreetmap.org%2Flistinfo%2Ftalk-audata=02%7C01%7C%7C565081b3aeb3469940b808d6446557e2%7C84df9e7fe9f640afb435%7C1%7C0%7C636771599383310905sdata=XA%2FTFSN%2FuejCTQhpBBiR8xdhgAQTe6%2BPuYYFfrgL%2Fao%3Dreserved=0<https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openstreetmap.org%2Flistinfo%2Ftalk-au=02%7C01%7C%7Ccde3b15fd9804a92bd4208d64e0c2507%7C84df9e7fe9f640afb435%7C1%7C0%7C636782211392831029=yPo%2FqegBvwiKde7IrnQSJC%2FXp8RsVVd3me0WVsqNgCg%3D=0>
___
Talk-au mailing list
Talk-au@openstreetmap.org<mailto:Talk-au@openstreetmap.org>
https://lists.openstreetmap.org/listinfo/talk-au<https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openstreetmap.org%2Flistinfo%2Ftalk-au=02%7C01%7C%7Ccde3b15fd9804a92bd4208d64e0c2507%7C84df9e7fe9f640afb435%7C1%7C0%7C636782211392831029=yPo%2FqegBvwiKde7IrnQSJC%2FXp8RsVVd3me0WVsqNgCg%3D=0>
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Lee Mason]

I am prepared to make a start on the Tasmania import over the next few days, 
unless there is some further discussion?




From: Joel H. 
Sent: Wednesday, November 7, 2018 2:57:45 PM
To: talk-au@openstreetmap.org; Andrew Harvey
Subject: Re: [talk-au] PSMA Administrative Boundaries

How are we with our import plan on the wiki, Do we have any blockers
stopping the PSMA import?

I think we should start setting dates state-by-state for import.


___
Talk-au mailing list
Talk-au@openstreetmap.org
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openstreetmap.org%2Flistinfo%2Ftalk-audata=02%7C01%7C%7C565081b3aeb3469940b808d6446557e2%7C84df9e7fe9f640afb435%7C1%7C0%7C636771599383310905sdata=XA%2FTFSN%2FuejCTQhpBBiR8xdhgAQTe6%2BPuYYFfrgL%2Fao%3Dreserved=0
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

Just downloaded to take a look, Most seem right.

In order to QA something like this, taking random samples won't help
since any number of errors would be small.

Does the script currently check for a byte-to-byte match of the name=*
text? I think that should be the determining faction of adding a node to
the relation. The remaining un-merged entries can be determined manually.


On 10/11/18 10:44 am, Andrew Davidson wrote:
> If you want to start working on something I have had a go at matching
> place nodes:
>
> http://overpass-turbo.eu/s/Dy3
>
> with the corresponding admin_level 10 boundaries:
>
> https://github.com/FrakGart/TestAdmin10Labels
>
> These have been matched by finding the best matching place node by
> name within a bbox 150% the size of the admin boundary. In the cases
> where there are more than one perfect match the first one the program
> encountered is used.
>
> Needs reviewing to see if the most appropriate node has been selected.
> Particularly where a town/city node has been selected: do we need to
> add a suburb node? (or do we use the same node for city and suburb eg:
> Brisbane)
>
> This is v1.0, I was planning to see if I could flag if the selected
> node is inside/outside the boundaries (eg: Mount Isa) and also look
> for duplicate place nodes (ie: same name appearing on different values
> of place).
>
>
> On 7/11/18 2:57 pm, Joel H. wrote:
>> How are we with our import plan on the wiki, Do we have any blockers
>> stopping the PSMA import?
>>
>> I think we should start setting dates state-by-state for import.
>>
>>
>> ___
>> Talk-au mailing list
>> Talk-au@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-au
>>
>
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Davidson]

If you want to start working on something I have had a go at matching 
place nodes:


http://overpass-turbo.eu/s/Dy3

with the corresponding admin_level 10 boundaries:

https://github.com/FrakGart/TestAdmin10Labels

These have been matched by finding the best matching place node by name 
within a bbox 150% the size of the admin boundary. In the cases where 
there are more than one perfect match the first one the program 
encountered is used.


Needs reviewing to see if the most appropriate node has been selected. 
Particularly where a town/city node has been selected: do we need to add 
a suburb node? (or do we use the same node for city and suburb eg: 
Brisbane)


This is v1.0, I was planning to see if I could flag if the selected node 
is inside/outside the boundaries (eg: Mount Isa) and also look for 
duplicate place nodes (ie: same name appearing on different values of 
place).



On 7/11/18 2:57 pm, Joel H. wrote:

How are we with our import plan on the wiki, Do we have any blockers
stopping the PSMA import?

I think we should start setting dates state-by-state for import.


___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au



___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

How are we with our import plan on the wiki, Do we have any blockers
stopping the PSMA import?

I think we should start setting dates state-by-state for import.


___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

Thanks for expressing interest!

The integration will mostly involve... (and I'm going to specify this on
the wiki when I get a chance.)

- Changing the place=* tags on the boundaries to match the nodes that
are already in OSM (e.g. Gold Coast should be changed from
place=municipality to place=city)

- Adding the nodes into the relation with the role being set to "label"
(example:
https://www.openstreetmap.org/relation/6070036#map=14/-28.3297/153.3821)

- Reconsider the placement of the node (make sure it is over the town
centre)

- Add a place=* node if one is missing

- Reconsider the place=* value (Is it really a village or a suburb? Is
it really a city or is it a town?)


The biggest issue with iD is that it won't load big areas very well. But
if you can deal with that help would be excellent.

Again I will add this plan to the wiki when I get a chance.


On 31/10/18 6:37 am, Graeme Fitzpatrick wrote:
> Hi Joel
>
> Happy to help with Gold Coast area if there's anything I can do.
>
> I only use iD though, not JOSM, so don't know if that's a problem?
>
> Thanks
>
> Graeme
>
>
> On Tue, 30 Oct 2018 at 21:13, Joel H.  > wrote:
>
> I've put my self down for integration in QLD. 
>
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Graeme Fitzpatrick]

Hi Joel

Happy to help with Gold Coast area if there's anything I can do.

I only use iD though, not JOSM, so don't know if that's a problem?

Thanks

Graeme


On Tue, 30 Oct 2018 at 21:13, Joel H.  wrote:

> I've put my self down for integration in QLD.
>
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

I have edited the wiki with a table relating to the team approach, It's
rough but it shows who is doing what.

I have enlisted Andrew Harvey as the uploader/account holder. I've put
my self down for integration in QLD. I'm hoping that others can handle
integration in their own states and put names down on the wiki.

Thanks!

https://wiki.openstreetmap.org/wiki/Import/Catalogue/PSMA_Admin_Boundaries#Team_Approach

On 23/10/18 10:25 am, Andrew Harvey wrote:
> I see, thanks. It's in the original non-simplified files too. It can
> be manually fixed, but just adding a shared node is not the right fix,
> we need to snap the nodes and then removed the shared ways.
>
> We should be able to fix this in the processing scripts by increasing
> the tolerance that things get snapped together. In this case here, the
> two ways/nodes are 0.1m apart.
> On Sun, 21 Oct 2018 at 12:33, Joel H.  wrote:
>> On 19/10/18 5:01 pm, Andrew Harvey wrote:
>>
>>> Is there a problem with crossing ways? Why do they need a shared node
>>> when they are different admin levels?
>>
>> Yes jump to the position I told you and zoom right in to the
>> intersection, there is a crossover between two level 10 admin areas.
>>
>> I don't know if this was introduced in the simplification or it was an
>> error with PSMA. But there is only 26 of these errors in QLD so I would
>> hardly call this an issue, you can easily fix these manually.
>>
>>
>>
>> ___
>> Talk-au mailing list
>> Talk-au@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-au
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au


___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

Just for the record, be very careful removing level 10 boundaries around
QLD, PSMA appears to be missing a few around central QLD.

I'm going back now and reverting bad changes.

On 23/10/18 10:25 am, Andrew Harvey wrote:
> I see, thanks. It's in the original non-simplified files too. It can
> be manually fixed, but just adding a shared node is not the right fix,
> we need to snap the nodes and then removed the shared ways.
>
> We should be able to fix this in the processing scripts by increasing
> the tolerance that things get snapped together. In this case here, the
> two ways/nodes are 0.1m apart.
> On Sun, 21 Oct 2018 at 12:33, Joel H.  wrote:
>> On 19/10/18 5:01 pm, Andrew Harvey wrote:
>>
>>> Is there a problem with crossing ways? Why do they need a shared node
>>> when they are different admin levels?
>>
>> Yes jump to the position I told you and zoom right in to the
>> intersection, there is a crossover between two level 10 admin areas.
>>
>> I don't know if this was introduced in the simplification or it was an
>> error with PSMA. But there is only 26 of these errors in QLD so I would
>> hardly call this an issue, you can easily fix these manually.
>>
>>
>>
>> ___
>> Talk-au mailing list
>> Talk-au@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-au
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

I see, thanks. It's in the original non-simplified files too. It can
be manually fixed, but just adding a shared node is not the right fix,
we need to snap the nodes and then removed the shared ways.

We should be able to fix this in the processing scripts by increasing
the tolerance that things get snapped together. In this case here, the
two ways/nodes are 0.1m apart.
On Sun, 21 Oct 2018 at 12:33, Joel H.  wrote:
>
> On 19/10/18 5:01 pm, Andrew Harvey wrote:
>
> > Is there a problem with crossing ways? Why do they need a shared node
> > when they are different admin levels?
>
>
> Yes jump to the position I told you and zoom right in to the
> intersection, there is a crossover between two level 10 admin areas.
>
> I don't know if this was introduced in the simplification or it was an
> error with PSMA. But there is only 26 of these errors in QLD so I would
> hardly call this an issue, you can easily fix these manually.
>
>
>
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

Not sure why I can't find it on the list archives but there was some
discussion about this below. In short, I'm in favour of not uploading
duplicate state borders and during or post-import we manually fix up
the relations to use the existing state boundaries.

On Thu, 18 Oct 2018 at 12:35, Andrew Davidson  wrote:
>
> On Wed, Oct 17, 2018 at 6:29 PM Andrew Harvey  
> wrote:
>>
>> I'm in favour of first manually removing the
>> state borders so we don't dump more nodes on top and then post-upload
>> we can just join the new relations to the existing state borders where
>> they meet.
>
>
> I can see your point about not wanting to upload ways and nodes that just 
> going to be deleted immediately after. However if we don't then it's going to 
> make my idea for QA harder. I'd assumed that we'd be uploading valid 
> multi-polygons which means that we could use Overpass and the JOSM validator 
> for QA. I guess we might be able to come up with another approach but I'm not 
> sure it's worth the effort just to avoid adding and removing a few tens of 
> thousands of nodes in comparison to the size of the import.
On Mon, 22 Oct 2018 at 15:04, Joel H.  wrote:
>
> One last note (and I don't know if this has been mentioned already).
>
> But it seems that the State boundary (at least on the QLD/NSW boarder)
> doesn't match up precisely with what we already have in OSM. Should we
> remove the outer edges of the PSMA data and then add existing state ways?
>
> On 22/10/18 1:40 pm, Andrew Harvey wrote:
> >> I don't know if this was introduced in the simplification or it was an
> > error with PSMA. But there is only 26 of these errors in QLD so I would
> > hardly call this an issue, you can easily fix these manually.
> >
> > I'm not sure either, so unless there's a better/automated way, let's
> > just address this manually either during the import or immediately
> > after.
> > On Sun, 21 Oct 2018 at 12:33, Joel H.  wrote:
> >> On 19/10/18 5:01 pm, Andrew Harvey wrote:
> >>
> >>> Is there a problem with crossing ways? Why do they need a shared node
> >>> when they are different admin levels?
> >>
> >> Yes jump to the position I told you and zoom right in to the
> >> intersection, there is a crossover between two level 10 admin areas.
> >>
> >> I don't know if this was introduced in the simplification or it was an
> >> error with PSMA. But there is only 26 of these errors in QLD so I would
> >> hardly call this an issue, you can easily fix these manually.
> >>
> >>
> >>
> >> ___
> >> Talk-au mailing list
> >> Talk-au@openstreetmap.org
> >> https://lists.openstreetmap.org/listinfo/talk-au
> > ___
> > Talk-au mailing list
> > Talk-au@openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/talk-au
>
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

One last note (and I don't know if this has been mentioned already).

But it seems that the State boundary (at least on the QLD/NSW boarder)
doesn't match up precisely with what we already have in OSM. Should we
remove the outer edges of the PSMA data and then add existing state ways?

On 22/10/18 1:40 pm, Andrew Harvey wrote:
>> I don't know if this was introduced in the simplification or it was an
> error with PSMA. But there is only 26 of these errors in QLD so I would
> hardly call this an issue, you can easily fix these manually.
>
> I'm not sure either, so unless there's a better/automated way, let's
> just address this manually either during the import or immediately
> after.
> On Sun, 21 Oct 2018 at 12:33, Joel H.  wrote:
>> On 19/10/18 5:01 pm, Andrew Harvey wrote:
>>
>>> Is there a problem with crossing ways? Why do they need a shared node
>>> when they are different admin levels?
>>
>> Yes jump to the position I told you and zoom right in to the
>> intersection, there is a crossover between two level 10 admin areas.
>>
>> I don't know if this was introduced in the simplification or it was an
>> error with PSMA. But there is only 26 of these errors in QLD so I would
>> hardly call this an issue, you can easily fix these manually.
>>
>>
>>
>> ___
>> Talk-au mailing list
>> Talk-au@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-au
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

Per the import guidelines
https://wiki.openstreetmap.org/wiki/Import/Guidelines I've requested a
review from the wider imports mailing list at
https://lists.openstreetmap.org/pipermail/imports/2018-October/005760.html
On Mon, 22 Oct 2018 at 14:40, Andrew Harvey  wrote:
>
> > I don't know if this was introduced in the simplification or it was an
> error with PSMA. But there is only 26 of these errors in QLD so I would
> hardly call this an issue, you can easily fix these manually.
>
> I'm not sure either, so unless there's a better/automated way, let's
> just address this manually either during the import or immediately
> after.
> On Sun, 21 Oct 2018 at 12:33, Joel H.  wrote:
> >
> > On 19/10/18 5:01 pm, Andrew Harvey wrote:
> >
> > > Is there a problem with crossing ways? Why do they need a shared node
> > > when they are different admin levels?
> >
> >
> > Yes jump to the position I told you and zoom right in to the
> > intersection, there is a crossover between two level 10 admin areas.
> >
> > I don't know if this was introduced in the simplification or it was an
> > error with PSMA. But there is only 26 of these errors in QLD so I would
> > hardly call this an issue, you can easily fix these manually.
> >
> >
> >
> > ___
> > Talk-au mailing list
> > Talk-au@openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

> I don't know if this was introduced in the simplification or it was an
error with PSMA. But there is only 26 of these errors in QLD so I would
hardly call this an issue, you can easily fix these manually.

I'm not sure either, so unless there's a better/automated way, let's
just address this manually either during the import or immediately
after.
On Sun, 21 Oct 2018 at 12:33, Joel H.  wrote:
>
> On 19/10/18 5:01 pm, Andrew Harvey wrote:
>
> > Is there a problem with crossing ways? Why do they need a shared node
> > when they are different admin levels?
>
>
> Yes jump to the position I told you and zoom right in to the
> intersection, there is a crossover between two level 10 admin areas.
>
> I don't know if this was introduced in the simplification or it was an
> error with PSMA. But there is only 26 of these errors in QLD so I would
> hardly call this an issue, you can easily fix these manually.
>
>
>
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

Is there a problem with crossing ways? Why do they need a shared node when
they are different admin levels?

On Fri., 19 Oct. 2018, 5:56 pm Joel H.,  wrote:

> Just took a look at the crossing boarders without ways error, and it
> seems some level 10 boundaries are indeed overlaping. Check out
> -22.5639465 147.0726169 in Queensland.
>
> As for way too long, I also don't think that needs fixing, just some
> automated JOSM warning not taking context into consideration.
>
> On 18/10/18 12:30 pm, Andrew Harvey wrote:
> > I tried to find out more about two JOSM warnings but I couldn't.
> >
> > 1. Way segment too long
> >
> > What's wrong with long way segments? I'm not convinced we should add
> > nodes where they aren't necessary for detail.
> >
> > 2. Crossing borders without a shared way.
> >
> > This should only happen when you have an LGA boundary crossing a
> > Suburb/Locality boundary, do we need a shared node between these?
> >
> >> I can see your point about not wanting to upload ways and nodes that
> just going to be deleted immediately after. However if we don't then it's
> going to make my idea for QA harder.
> > Could we delay that QA step until after these have been manually fixed
> > up to use the existing state borders?
> >
> >> I'd assumed that we'd be uploading valid multi-polygons which means
> that we could use Overpass and the JOSM validator for QA. I guess we might
> be able to come up with another approach but I'm not sure it's worth the
> effort just to avoid adding and removing a few tens of thousands of nodes
> in comparison to the size of the import.
> > I was leaning this way to avoid this import dumping duplicates on top
> > of existing mappers work on the state boundaries.
> >
> >
> > On Thu, 18 Oct 2018 at 12:46, Andrew Davidson 
> wrote:
> >> On Wed, Oct 17, 2018 at 6:29 PM Andrew Harvey 
> wrote:
> >>>
> >>> So I guess at this point do people want to checkout the simplified OSM
> >>> files for any issues?
> >>>
> >> They look OK, but I would like to have an opportunity to clean up the
> JOSM warnings before we upload them (except the relations with the same
> members warning as these are real).
> > ___
> > Talk-au mailing list
> > Talk-au@openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/talk-au
>
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au
>
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

Just took a look at the crossing boarders without ways error, and it
seems some level 10 boundaries are indeed overlaping. Check out
-22.5639465 147.0726169 in Queensland.

As for way too long, I also don't think that needs fixing, just some
automated JOSM warning not taking context into consideration.

On 18/10/18 12:30 pm, Andrew Harvey wrote:
> I tried to find out more about two JOSM warnings but I couldn't.
>
> 1. Way segment too long
>
> What's wrong with long way segments? I'm not convinced we should add
> nodes where they aren't necessary for detail.
>
> 2. Crossing borders without a shared way.
>
> This should only happen when you have an LGA boundary crossing a
> Suburb/Locality boundary, do we need a shared node between these?
>
>> I can see your point about not wanting to upload ways and nodes that just 
>> going to be deleted immediately after. However if we don't then it's going 
>> to make my idea for QA harder.
> Could we delay that QA step until after these have been manually fixed
> up to use the existing state borders?
>
>> I'd assumed that we'd be uploading valid multi-polygons which means that we 
>> could use Overpass and the JOSM validator for QA. I guess we might be able 
>> to come up with another approach but I'm not sure it's worth the effort just 
>> to avoid adding and removing a few tens of thousands of nodes in comparison 
>> to the size of the import.
> I was leaning this way to avoid this import dumping duplicates on top
> of existing mappers work on the state boundaries.
>
>
> On Thu, 18 Oct 2018 at 12:46, Andrew Davidson  wrote:
>> On Wed, Oct 17, 2018 at 6:29 PM Andrew Harvey  
>> wrote:
>>>
>>> So I guess at this point do people want to checkout the simplified OSM
>>> files for any issues?
>>>
>> They look OK, but I would like to have an opportunity to clean up the JOSM 
>> warnings before we upload them (except the relations with the same members 
>> warning as these are real).
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

I tried to find out more about two JOSM warnings but I couldn't.

1. Way segment too long

What's wrong with long way segments? I'm not convinced we should add
nodes where they aren't necessary for detail.

2. Crossing borders without a shared way.

This should only happen when you have an LGA boundary crossing a
Suburb/Locality boundary, do we need a shared node between these?

> I can see your point about not wanting to upload ways and nodes that just 
> going to be deleted immediately after. However if we don't then it's going to 
> make my idea for QA harder.

Could we delay that QA step until after these have been manually fixed
up to use the existing state borders?

> I'd assumed that we'd be uploading valid multi-polygons which means that we 
> could use Overpass and the JOSM validator for QA. I guess we might be able to 
> come up with another approach but I'm not sure it's worth the effort just to 
> avoid adding and removing a few tens of thousands of nodes in comparison to 
> the size of the import.

I was leaning this way to avoid this import dumping duplicates on top
of existing mappers work on the state boundaries.


On Thu, 18 Oct 2018 at 12:46, Andrew Davidson  wrote:
>
> On Wed, Oct 17, 2018 at 6:29 PM Andrew Harvey  
> wrote:
>>
>>
>> So I guess at this point do people want to checkout the simplified OSM
>> files for any issues?
>>
> They look OK, but I would like to have an opportunity to clean up the JOSM 
> warnings before we upload them (except the relations with the same members 
> warning as these are real).

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Davidson]

On Wed, Oct 17, 2018 at 6:29 PM Andrew Harvey 
wrote:

>
> So I guess at this point do people want to checkout the simplified OSM
> files for any issues?
>
> They look OK, but I would like to have an opportunity to clean up the JOSM
warnings before we upload them (except the relations with the same members
warning as these are real).
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Davidson]

On Wed, Oct 17, 2018 at 6:29 PM Andrew Harvey 
wrote:

> I'm in favour of first manually removing the
> state borders so we don't dump more nodes on top and then post-upload
> we can just join the new relations to the existing state borders where
> they meet.
>

I can see your point about not wanting to upload ways and nodes that just
going to be deleted immediately after. However if we don't then it's going
to make my idea for QA harder. I'd assumed that we'd be uploading valid
multi-polygons which means that we could use Overpass and the JOSM
validator for QA. I guess we might be able to come up with another approach
but I'm not sure it's worth the effort just to avoid adding and removing a
few tens of thousands of nodes in comparison to the size of the import.
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

I've reprocessed these files again, I have simplified versions, which
cut the number of nodes with not much loss in detail:

Original full resolution: https://tianjara.net/data/PSMA_AdminBdy_OSM/
Simplified: https://tianjara.net/data/PSMA_AdminBdy_OSM_Simplified/

Andrew Davidson has been testing actually doing an import with the DEV
API. It looks like the upload.py scripts will do a better job of
splitting the upload. In JOSM it will upload all the empty nodes first
and then in the last changeset push the relations. It looks like
upload.py does things differently.

This does mean we have less opportunity to do this import manually, so
if we use upload.py, I'm in favour of first manually removing the
state borders so we don't dump more nodes on top and then post-upload
we can just join the new relations to the existing state borders where
they meet.

So I guess at this point do people want to checkout the simplified OSM
files for any issues?
On Tue, 16 Oct 2018 at 10:48, Joel H.  wrote:
>
> OK, just tell me when you are ready, and I'll clean up QLD ready for
> import.
>
> On 14/10/18 10:46 pm, Andrew Harvey wrote:
> > On Sun, 14 Oct 2018 at 21:15, Joel H.  wrote:
> >> Hey Andrew,
> >>
> >> Today on the QLD slack, I had a local mapper ask me about suburb areas.
> >> I personally think it would be an opportune time to start on the QLD
> >> import at least.
> >>
> >>
> >> What is the status on the PSMA data? Is it ready to import, and is
> >> anything holding us back?
> > I wouldn't say it's ready yes.
> >
> > [ ] simplification - I'm comfortable with the detail in the data, to
> > me it doesn't feel excess or over the top so I don't think we need any
> > simplification
> > [ ] website copyright notice - in progress at
> > https://github.com/openstreetmap/openstreetmap-website/pull/2026
> >
> >> Can you think of anything else? And would it be alright for me to start 
> >> cleanup now?
> > If the current boundaries aren't accurate, then yeah it's probably
> > easier to start dropping them out and they can be brought in fresh
> > through the import. Up to the local mappers to decide. Though it does
> > add pressure on us to make the import, which I'd rather not be rushed.
> >
> >>
> >> In the meantime, I have run some overpass scripts to grab all boundaries
> >> and place areas. I'm planning on starting a cleanup on the existing
> >> boundaries, todo includes:
> >>
> >> -remove admin boundaries for level 6 and 10 and integrate anything else
> >> into their own things e.g. fences or signposts
> >>
> >> -remove place=* areas
> >>
> >>
> >> Can you think of anything else? And would it be alright for me to start
> >> cleanup now?
> >>
> >>
>

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

OK, just tell me when you are ready, and I'll clean up QLD ready for
import.

On 14/10/18 10:46 pm, Andrew Harvey wrote:
> On Sun, 14 Oct 2018 at 21:15, Joel H.  wrote:
>> Hey Andrew,
>>
>> Today on the QLD slack, I had a local mapper ask me about suburb areas.
>> I personally think it would be an opportune time to start on the QLD
>> import at least.
>>
>>
>> What is the status on the PSMA data? Is it ready to import, and is
>> anything holding us back?
> I wouldn't say it's ready yes.
>
> [ ] simplification - I'm comfortable with the detail in the data, to
> me it doesn't feel excess or over the top so I don't think we need any
> simplification
> [ ] website copyright notice - in progress at
> https://github.com/openstreetmap/openstreetmap-website/pull/2026
>
>> Can you think of anything else? And would it be alright for me to start 
>> cleanup now?
> If the current boundaries aren't accurate, then yeah it's probably
> easier to start dropping them out and they can be brought in fresh
> through the import. Up to the local mappers to decide. Though it does
> add pressure on us to make the import, which I'd rather not be rushed.
>
>>
>> In the meantime, I have run some overpass scripts to grab all boundaries
>> and place areas. I'm planning on starting a cleanup on the existing
>> boundaries, todo includes:
>>
>> -remove admin boundaries for level 6 and 10 and integrate anything else
>> into their own things e.g. fences or signposts
>>
>> -remove place=* areas
>>
>>
>> Can you think of anything else? And would it be alright for me to start
>> cleanup now?
>>
>>


___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

On Sun, 14 Oct 2018 at 21:15, Joel H.  wrote:
>
> Hey Andrew,
>
> Today on the QLD slack, I had a local mapper ask me about suburb areas.
> I personally think it would be an opportune time to start on the QLD
> import at least.
>
>
> What is the status on the PSMA data? Is it ready to import, and is
> anything holding us back?

I wouldn't say it's ready yes.

[ ] simplification - I'm comfortable with the detail in the data, to
me it doesn't feel excess or over the top so I don't think we need any
simplification
[ ] website copyright notice - in progress at
https://github.com/openstreetmap/openstreetmap-website/pull/2026

> Can you think of anything else? And would it be alright for me to start 
> cleanup now?

If the current boundaries aren't accurate, then yeah it's probably
easier to start dropping them out and they can be brought in fresh
through the import. Up to the local mappers to decide. Though it does
add pressure on us to make the import, which I'd rather not be rushed.

>
>
> In the meantime, I have run some overpass scripts to grab all boundaries
> and place areas. I'm planning on starting a cleanup on the existing
> boundaries, todo includes:
>
> -remove admin boundaries for level 6 and 10 and integrate anything else
> into their own things e.g. fences or signposts
>
> -remove place=* areas
>
>
> Can you think of anything else? And would it be alright for me to start
> cleanup now?
>
>

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

Hey Andrew,

Today on the QLD slack, I had a local mapper ask me about suburb areas.
I personally think it would be an opportune time to start on the QLD
import at least.


What is the status on the PSMA data? Is it ready to import, and is
anything holding us back?


In the meantime, I have run some overpass scripts to grab all boundaries
and place areas. I'm planning on starting a cleanup on the existing
boundaries, todo includes:

-remove admin boundaries for level 6 and 10 and integrate anything else
into their own things e.g. fences or signposts

-remove place=* areas


Can you think of anything else? And would it be alright for me to start
cleanup now?



___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

Let's try to document things on the wiki
https://wiki.openstreetmap.org/wiki/Import/Catalogue/PSMA_Admin_Boundaries
to help document, it's open to all to edit.
On Wed, 10 Oct 2018 at 19:04, Andrew Harvey  wrote:
>
> On Wed, 10 Oct 2018 at 16:29, Ewen Hill  wrote:
> >
> > Good afternoon all,
> >Firstly, I assume that this has been an issue elsewhere in the world
> > where a large body of official data over a wide area needs to be integrated.
> > Is there a way of going out to other countries and seeing how they worked it
> > and what lessons they learned.
>
> There's some material at https://wiki.openstreetmap.org/wiki/Import
>
> > My plan would be to make a web based system outside OSM by having an
> > external web db having both the PSMA and the current OSM suburb level data.
> > Both would be updated daily automatically and identify
> > 1. Where the PSMA is completely missing from OSM
> > 2. Where OSM data is completely missing from PSMA or duplicated (two suburbs
> > of same name) or broken.
> > 3. Where there are significant differences in size or overlap of the
> > polygons (sorted in order of mismatch)
> > 4. Where there are small differences that can be taken as creek alignment or
> > a new roundabout and could be flagged as potentially a false positive.
> >
> > In that way, we can attack the list, suburb by suburb from 1 to 4. A map
> > could be made of the the PSMA location and have all associated nodes and
> > ways displayed and mathematically provide a best solution for the editor to
> > consider. It may use portion or entire existing lines or it may be easier
> > just to down load the new suburb and have the other suburbs meet it
> > manually.
> >
> > Once the editor has decided the approach, it creates a data set and then
> > sends them to their prefer editor to make sure all is good. The old boundary
> > is tagged as well as the new one correctly.
> >
> > The entire system doesn't need to be built immediately, just the variation
> > list perhaps. Happy to help further.
> >
> > This process removes a lot of risk out of the equation and allows for
> > constant monitoring however is clerly slower than a bulk update by state.
> >
> > Your thoughts?
>
> I think since for the most part at least, the data is either in OSM or
> not when you break it down by state, I'd be +1 for importing on a per
> state basis initially.
>
> Each quarter as the upstream data gets updated, then we can go through
> the manual process of patching the changes.
>
> If you want to build a GUI around that, I think that's fine, but I
> don't think it's required to bring this data in.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

On Wed, 10 Oct 2018 at 16:29, Ewen Hill  wrote:
>
> Good afternoon all,
>Firstly, I assume that this has been an issue elsewhere in the world
> where a large body of official data over a wide area needs to be integrated.
> Is there a way of going out to other countries and seeing how they worked it
> and what lessons they learned.

There's some material at https://wiki.openstreetmap.org/wiki/Import

> My plan would be to make a web based system outside OSM by having an
> external web db having both the PSMA and the current OSM suburb level data.
> Both would be updated daily automatically and identify
> 1. Where the PSMA is completely missing from OSM
> 2. Where OSM data is completely missing from PSMA or duplicated (two suburbs
> of same name) or broken.
> 3. Where there are significant differences in size or overlap of the
> polygons (sorted in order of mismatch)
> 4. Where there are small differences that can be taken as creek alignment or
> a new roundabout and could be flagged as potentially a false positive.
>
> In that way, we can attack the list, suburb by suburb from 1 to 4. A map
> could be made of the the PSMA location and have all associated nodes and
> ways displayed and mathematically provide a best solution for the editor to
> consider. It may use portion or entire existing lines or it may be easier
> just to down load the new suburb and have the other suburbs meet it
> manually.
>
> Once the editor has decided the approach, it creates a data set and then
> sends them to their prefer editor to make sure all is good. The old boundary
> is tagged as well as the new one correctly.
>
> The entire system doesn't need to be built immediately, just the variation
> list perhaps. Happy to help further.
>
> This process removes a lot of risk out of the equation and allows for
> constant monitoring however is clerly slower than a bulk update by state.
>
> Your thoughts?

I think since for the most part at least, the data is either in OSM or
not when you break it down by state, I'd be +1 for importing on a per
state basis initially.

Each quarter as the upstream data gets updated, then we can go through
the manual process of patching the changes.

If you want to build a GUI around that, I think that's fine, but I
don't think it's required to bring this data in.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

On Tue, 9 Oct 2018 at 20:51, Andrew Davidson  wrote:
> One question would be should we be putting a place tag on admin
> boundaries? According to taginfo only 34% of admin_level=10 has a place
> tag.

I say yes since these suburbs are an area not a single point.

> > - how we'll manage that upload in terms of breaking it up into smaller 
> > pieces
>
> I'd assumed that you'd be uploading them using an import account. How
> small to break them up into would depend on what's used to upload them.
> State-by-state does have the advantage that the existing state borders
> are going to have to be manual snapped to the imported boundaries anyway.
>
> > - if there is any re-using of existing ways or not
>
> As several others have pointed out if we import the boundaries
> completely then mappers can manually snap them to natural features,
> roads, other boundaries etc to their heart's content. This step is
> optional as the boundaries will still be valid even if no one does any
> follow up work.
>
> As a minimum the boundaries would have to be matched to the existing
> state boundaries, otherwise there will be overlaps or slivers.

+1

> > - how to ensure we're retaining the existing LGA/Suburb data in OSM.
>
> So I'm assuming here that we are not talking about NSW/SA/ACT. These
> have already been done from state data and if the PSMA data is different
> then we should be going back to the state data to check (at which point
> we're using the state data).

+1

> To keep history I thought we'd could keep record of the current relation
> numbers of the existing admin boundaries, scrub off the no longer needed
> ways, and once the new boundaries are imported swap the new for old. For
> boundaries that are currently closed ways we'd need to convert them to a
> one-member relation first.

That should be possible.

On Tue, 9 Oct 2018 at 21:14, Lee Mason  wrote:
> I didn’t know what to make of QLD, but I’ve just found this definition which 
> suggests that the OSM coastline should be sufficient:
> http://qldspatial.information.qld.gov.au/catalogue/custom/viewMetadataDetails.page?uuid=%7B3F3DBD69-647B-4833-B0A5-CC43D5E70699%7D
> “For coastal areas other than Brisbane, the LGA comprises the mainland and 
> all islands above their respective sea-shores within the encompassed area.”

Thanks Lee!

On Tue, 9 Oct 2018 at 21:53, Joel H.  wrote:
>
> Good, doing country wide I think would have been a pain. I just tried
> QLD and all is looking good except the boundaries which extend into the
> ocean.
>
> Are you going to do the upload? And when should we do it?

I'm happy to do it, but I don't think we're at the point where it's
ready just yet.

As others have pointed out, in terms of LGAs,
- WA has three already in OSM of unknown origin, some don't match the
PSMA data. I've added a changeset comment to try to determine if we
leave those in place or replace them.
- NT has one which is from 3 years ago originally from GA, which
doesn't account for a new LGA from 2014, so I'm okay to replace this.
- TAS have no LGAs in OSM currently
- Chrismas Island/Cocos Island have no LGAs in OSM currently
- SA has some LGAs, we'll just add in those missing and leave the
existing ones in place
- ACT is N/A
- NSW already in OSM, so let's skip this
- VIC already in OSM, so let's skip this
- QLD has some rough approximations for a few LGA, I propose to
replace these with the more detailed boundaries

for Suburb/Localities, VIC has a lot of big differences so some local
knowledge on that would be great, NSW is mostly complete, SA is just
missing Adelaide.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Ewen Hill]

Good afternoon all,
   Firstly, I assume that this has been an issue elsewhere in the world
where a large body of official data over a wide area needs to be integrated.
Is there a way of going out to other countries and seeing how they worked it
and what lessons they learned.

My plan would be to make a web based system outside OSM by having an
external web db having both the PSMA and the current OSM suburb level data. 
Both would be updated daily automatically and identify
1. Where the PSMA is completely missing from OSM
2. Where OSM data is completely missing from PSMA or duplicated (two suburbs
of same name) or broken.
3. Where there are significant differences in size or overlap of the
polygons (sorted in order of mismatch)
4. Where there are small differences that can be taken as creek alignment or
a new roundabout and could be flagged as potentially a false positive.

In that way, we can attack the list, suburb by suburb from 1 to 4. A map
could be made of the the PSMA location and have all associated nodes and
ways displayed and mathematically provide a best solution for the editor to
consider. It may use portion or entire existing lines or it may be easier
just to down load the new suburb and have the other suburbs meet it
manually. 

Once the editor has decided the approach, it creates a data set and then
sends them to their prefer editor to make sure all is good. The old boundary
is tagged as well as the new one correctly. 

The entire system doesn't need to be built immediately, just the variation
list perhaps. Happy to help further. 

This process removes a lot of risk out of the equation and allows for
constant monitoring however is clerly slower than a bulk update by state.

Your thoughts?



--
Sent from: http://gis.19327.n8.nabble.com/Australia-f5416966.html

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

Good, doing country wide I think would have been a pain. I just tried
QLD and all is looking good except the boundaries which extend into the
ocean.

Are you going to do the upload? And when should we do it?

On 9/10/18 6:19 pm, Andrew Harvey wrote:
> On Tue, 9 Oct 2018 at 13:10, Andrew Harvey  wrote:
>> By state https://tianjara.net/data/PSMA_AdminBdy_OSM/
> Just a note that I've just updated these files, to remove duplicate nodes
>
> Another issue is I've applied place=suburb to these
> Suburbs/Localities, but in many cases they'd probably be better as a
> town or suburb, so would likely need manually matching up with
> existing place nodes, but that can be done post-import as follow up
> tasks.
>
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Lee Mason]

I didn’t know what to make of QLD, but I’ve just found this definition which 
suggests that the OSM coastline should be sufficient:
http://qldspatial.information.qld.gov.au/catalogue/custom/viewMetadataDetails.page?uuid=%7B3F3DBD69-647B-4833-B0A5-CC43D5E70699%7D

“For coastal areas other than Brisbane, the LGA comprises the mainland and all 
islands above their respective sea-shores within the encompassed area.”


From: Andrew Davidson<mailto:thesw...@gmail.com>
Sent: Tuesday, 9 October 2018 20:56
To: talk-au@openstreetmap.org<mailto:talk-au@openstreetmap.org>
Subject: Re: [talk-au] PSMA Administrative Boundaries

On 07/10/18 23:27, Lee Mason wrote:
> Nice work on cleaning up the PSMA data, Andrew. It looks a lot easier to
> manage for importing now.
>
> A quick look at the data, and it appears there is some regional/state
> variance in the boundaries:
>
> Some of the WA LGAs extend out to the coastal waters limit.
>
> VIC LGAs mostly approximate the high tide mark (except around Port
> Welshpool) and are shared with the locality boundaries.
>
> TAS LGAs mostly extend to the low tide mark and are not shared with
> locality boundaries.
>

Weirdly in Queensland, some of the locality and LGA boundaries extend
into international waters. They probably need to be cut-off at the
coastal water boundary.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openstreetmap.org%2Flistinfo%2Ftalk-audata=02%7C01%7C%7Cfb4031c5544d440913b408d62dcd8908%7C84df9e7fe9f640afb435%7C1%7C0%7C636746758117747213sdata=YU6bIRXejeE5VCmeqRuCLA2uZp2veElbSkrviXQTYUE%3Dreserved=0

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Davidson]

On 07/10/18 23:27, Lee Mason wrote:
Nice work on cleaning up the PSMA data, Andrew. It looks a lot easier to 
manage for importing now.


A quick look at the data, and it appears there is some regional/state 
variance in the boundaries:


Some of the WA LGAs extend out to the coastal waters limit.

VIC LGAs mostly approximate the high tide mark (except around Port 
Welshpool) and are shared with the locality boundaries.


TAS LGAs mostly extend to the low tide mark and are not shared with 
locality boundaries.




Weirdly in Queensland, some of the locality and LGA boundaries extend 
into international waters. They probably need to be cut-off at the 
coastal water boundary.


___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Davidson]

On 07/10/18 18:32, Andrew Harvey wrote:


I see the next steps are to:

- discuss if this is the right format for upload or not,


One question would be should we be putting a place tag on admin 
boundaries? According to taginfo only 34% of admin_level=10 has a place 
tag.



- how we'll manage that upload in terms of breaking it up into smaller pieces


I'd assumed that you'd be uploading them using an import account. How 
small to break them up into would depend on what's used to upload them. 
State-by-state does have the advantage that the existing state borders 
are going to have to be manual snapped to the imported boundaries anyway.



- if there is any re-using of existing ways or not


As several others have pointed out if we import the boundaries 
completely then mappers can manually snap them to natural features, 
roads, other boundaries etc to their heart's content. This step is 
optional as the boundaries will still be valid even if no one does any 
follow up work.


As a minimum the boundaries would have to be matched to the existing 
state boundaries, otherwise there will be overlaps or slivers.



- how to ensure we're retaining the existing LGA/Suburb data in OSM.


So I'm assuming here that we are not talking about NSW/SA/ACT. These 
have already been done from state data and if the PSMA data is different 
then we should be going back to the state data to check (at which point 
we're using the state data).


To keep history I thought we'd could keep record of the current relation 
numbers of the existing admin boundaries, scrub off the no longer needed 
ways, and once the new boundaries are imported swap the new for old. For 
boundaries that are currently closed ways we'd need to convert them to a 
one-member relation first.



___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

On Tue, 9 Oct 2018 at 13:10, Andrew Harvey  wrote:
> By state https://tianjara.net/data/PSMA_AdminBdy_OSM/

Just a note that I've just updated these files, to remove duplicate nodes

Another issue is I've applied place=suburb to these
Suburbs/Localities, but in many cases they'd probably be better as a
town or suburb, so would likely need manually matching up with
existing place nodes, but that can be done post-import as follow up
tasks.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

By state https://tianjara.net/data/PSMA_AdminBdy_OSM/

On Tue, 9 Oct 2018 at 05:36, Andrew Harvey  wrote:
>
> I realised the JOSM PBF plugin doesn't like negative IDs. However
> there are duplicate nodes, so i'm re-processing to fix this. I'll also
> split it up by states. Originally I was hesitant at doing that since
> you want shared borders, but we're going to want to reuse the existing
> state boundaries so it will need to be done manually anyway.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

I realised the JOSM PBF plugin doesn't like negative IDs. However
there are duplicate nodes, so i'm re-processing to fix this. I'll also
split it up by states. Originally I was hesitant at doing that since
you want shared borders, but we're going to want to reuse the existing
state boundaries so it will need to be done manually anyway.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

>discuss if this is the right format for upload or not

I found that GeoJSON file you sent worked best for me. Otherwise we
stick with .osm for OSM. I'm having trouble with that PBF file you sent
(or I don't have enough memory (I have 8GB))


>how we'll manage that upload in terms of breaking it up into smaller pieces

We have to do this per-state, no question. Otherwise it will be hard to
reuse ways and make relationships.


>if there is any re-using of existing ways or not

That was what I wanted to do. If you could get that GeoJSON file you
sent before, but this time with all the relations and LGA/Suburbs done,
it would be perfect I reckon.


>how to ensure we're retaining the existing LGA/Suburb data in OSM.

The ones mapped in QLD are solid. As for the label relations, we should
leave fixme notes (like done in NSW) and encourage their removal once
place value has been checked and label relation has been added.

The boundaries mapped in QLD right now are very rough and aren't based
off any real data, these should be removed.


On 7/10/18 5:32 pm, Andrew Harvey wrote:
> I've approached this from the angle of what we'd need to get the data
> to look like if importing as is into OSM. I couldn't find any tools
> which correctly ensure that boundary ways were shared via relations
> instead of duplicated so I wrote some new tools. The process I used is
> documented at https://github.com/andrewharvey/psma-admin-bdy2osm
>
> The processed AUGUST 2018 LGA + Suburb/Locality OSM file is at
>
> https://tianjara.net/data/PSMA_Admin_Bdy.osm.pbf
>
> The file is quite big so you'll need enough memory to open it in JOSM.
>
> In JOSM you can open this directly with the PBF plugin otherwise any
> of the software at
> https://wiki.openstreetmap.org/wiki/PBF/Software_Compliance can
> deflate it.
>
> I see the next steps are to:
>
> - discuss if this is the right format for upload or not,
> - how we'll manage that upload in terms of breaking it up into smaller pieces
> - if there is any re-using of existing ways or not
> - how to ensure we're retaining the existing LGA/Suburb data in OSM.
> This includes both relations and those mapped as points (as Joel
> touched on). Those mapped as points are likely suitable for either the
> label or admin_center members of the relation.
>
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au


___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Lee Mason]

Nice work on cleaning up the PSMA data, Andrew. It looks a lot easier to manage 
for importing now.



A quick look at the data, and it appears there is some regional/state variance 
in the boundaries:

Some of the WA LGAs extend out to the coastal waters limit.

VIC LGAs mostly approximate the high tide mark (except around Port Welshpool) 
and are shared with the locality boundaries.

TAS LGAs mostly extend to the low tide mark and are not shared with locality 
boundaries.



--



In regards to discussion earlier about aligning boundaries to existing 
features; I think that if a mapper determines that a boundary and a feature 
approximate each other, they can be merged. But it should not be a requirement 
to align or not align features.



For example, when an admin boundary clearly defines the left side of a river, 
it should be separate. But as the river becomes a stream and is narrower and 
more ambiguous, it would provide a tidier map to align the two features at this 
point.



Whilst this does mean a decrease in the accuracy of the data on OSM, if a data 
user requires the precision to determine if an admin boundary is on the left or 
right side of a road or river, they should (or would) be using the 
authoritative data source anyway because OSM can never guarantee that level of 
accuracy. It also means it is less of an issue when future users inadvertently 
try to “fix” the map by aligning admin boundaries with rivers, etc.



So I would suggest that mappers use their own good judgement to align a 
boundary and a feature that very closely approximate each other if they wish to 
do so.



With the same reasoning, I think if an admin boundary closely approximates the 
hightide mark, it should be aligned with the OSM coastline.



--



Speaking from the perspective of Tasmania’s data, my suggestion is:

Tassie LGAs would not be aligned with the coastline because they follow the low 
tide mark.

Localities would be aligned to the OSM coastline because they approximate the 
high tide mark.



And I would probably import Tasmania by doing 1 LGA at a time (including the 
contained localities).



Cheers

Lee






From: Andrew Harvey 
Sent: Sunday, October 7, 2018 6:32:02 PM
To: OSM Australian Talk List
Subject: Re: [talk-au] PSMA Administrative Boundaries

I've approached this from the angle of what we'd need to get the data
to look like if importing as is into OSM. I couldn't find any tools
which correctly ensure that boundary ways were shared via relations
instead of duplicated so I wrote some new tools. The process I used is
documented at 
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fandrewharvey%2Fpsma-admin-bdy2osmdata=02%7C01%7C%7C6fa5cf91925e4f1f3a4d08d62c2734a6%7C84df9e7fe9f640afb435%7C1%7C0%7C636744944223052969sdata=wwiO1AL7k1rZAvsf5Igz6SsC758fwZ%2BxdYKdnmj4Upo%3Dreserved=0

The processed AUGUST 2018 LGA + Suburb/Locality OSM file is at

https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftianjara.net%2Fdata%2FPSMA_Admin_Bdy.osm.pbfdata=02%7C01%7C%7C6fa5cf91925e4f1f3a4d08d62c2734a6%7C84df9e7fe9f640afb435%7C1%7C0%7C636744944223052969sdata=EH3xdg4az3Ps1T8iQIjqJ6H9ui3lZZ36cjTns4xHTvk%3Dreserved=0

The file is quite big so you'll need enough memory to open it in JOSM.

In JOSM you can open this directly with the PBF plugin otherwise any
of the software at
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.openstreetmap.org%2Fwiki%2FPBF%2FSoftware_Compliancedata=02%7C01%7C%7C6fa5cf91925e4f1f3a4d08d62c2734a6%7C84df9e7fe9f640afb435%7C1%7C0%7C636744944223052969sdata=D06VwqI21v7Oqk36bFLgJUcrBWCzY7NpzDPk3zIn%2BuE%3Dreserved=0
 can
deflate it.

I see the next steps are to:

- discuss if this is the right format for upload or not,
- how we'll manage that upload in terms of breaking it up into smaller pieces
- if there is any re-using of existing ways or not
- how to ensure we're retaining the existing LGA/Suburb data in OSM.
This includes both relations and those mapped as points (as Joel
touched on). Those mapped as points are likely suitable for either the
label or admin_center members of the relation.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openstreetmap.org%2Flistinfo%2Ftalk-audata=02%7C01%7C%7C6fa5cf91925e4f1f3a4d08d62c2734a6%7C84df9e7fe9f640afb435%7C1%7C0%7C636744944223052969sdata=FEObUh5zYdC2%2FgYY0nkKfcFovtg0Fp7fqgqRbR3MbX0%3Dreserved=0
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

I've approached this from the angle of what we'd need to get the data
to look like if importing as is into OSM. I couldn't find any tools
which correctly ensure that boundary ways were shared via relations
instead of duplicated so I wrote some new tools. The process I used is
documented at https://github.com/andrewharvey/psma-admin-bdy2osm

The processed AUGUST 2018 LGA + Suburb/Locality OSM file is at

https://tianjara.net/data/PSMA_Admin_Bdy.osm.pbf

The file is quite big so you'll need enough memory to open it in JOSM.

In JOSM you can open this directly with the PBF plugin otherwise any
of the software at
https://wiki.openstreetmap.org/wiki/PBF/Software_Compliance can
deflate it.

I see the next steps are to:

- discuss if this is the right format for upload or not,
- how we'll manage that upload in terms of breaking it up into smaller pieces
- if there is any re-using of existing ways or not
- how to ensure we're retaining the existing LGA/Suburb data in OSM.
This includes both relations and those mapped as points (as Joel
touched on). Those mapped as points are likely suitable for either the
label or admin_center members of the relation.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[cleary]

I'll look there. Thanks.



On Sun, Oct 7, 2018, at 12:31 PM, Warin wrote:
> On 07/10/18 11:22, cleary wrote:
> >
> > In regard to admin boundaries sharing the coastline, I think that would 
> > also be incorrect but I am less confident of my view on this.
> >
> > I did update some administrative boundaries in South Australia using the SA 
> > Government Data and those boundaries did not coincide with the coastline 
> > (see the coastline around Streaky Bay, Ceduna, Fowlers Bay and near the 
> > Nullarbor).  The coastline in OSM needs a lot of work - I have looked at 
> > parts of WA and SA but found it very difficult to use the satellite imagery 
> > correctly to refine the map of the coastline in areas where there are 
> > extensive mudflats or large tidal flows and even rocky areas just 
> > under/above the waterline. But I suggest it is safest, and more accurate, 
> > to map the administrative boundaries and the coastline separately.
> 
> Might be good to look at Broome for hi/low tide .. there is a fair 
> distance between the two there so it would be easy to pick in that 
> location.
> 
> 
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Warin]

PS While on coast lines ... computer model of the Kimberly coastline 
over a few thousand years.. looks like it is breathing.


http://www.abc.net.au/news/2018-10-07/wa-coastline-transformed-by-sea-levels-over-thousands-of-years/10338500

On 07/10/18 11:22, cleary wrote:


In regard to admin boundaries sharing the coastline, I think that would also be 
incorrect but I am less confident of my view on this.

I did update some administrative boundaries in South Australia using the SA 
Government Data and those boundaries did not coincide with the coastline (see 
the coastline around Streaky Bay, Ceduna, Fowlers Bay and near the Nullarbor).  
The coastline in OSM needs a lot of work - I have looked at parts of WA and SA 
but found it very difficult to use the satellite imagery correctly to refine 
the map of the coastline in areas where there are extensive mudflats or large 
tidal flows and even rocky areas just under/above the waterline. But I suggest 
it is safest, and more accurate, to map the administrative boundaries and the 
coastline separately.






___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au




___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Warin]

On 07/10/18 11:22, cleary wrote:


In regard to admin boundaries sharing the coastline, I think that would also be 
incorrect but I am less confident of my view on this.

I did update some administrative boundaries in South Australia using the SA 
Government Data and those boundaries did not coincide with the coastline (see 
the coastline around Streaky Bay, Ceduna, Fowlers Bay and near the Nullarbor).  
The coastline in OSM needs a lot of work - I have looked at parts of WA and SA 
but found it very difficult to use the satellite imagery correctly to refine 
the map of the coastline in areas where there are extensive mudflats or large 
tidal flows and even rocky areas just under/above the waterline. But I suggest 
it is safest, and more accurate, to map the administrative boundaries and the 
coastline separately.


Might be good to look at Broome for hi/low tide .. there is a fair distance 
between the two there so it would be easy to pick in that location.


___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[cleary]

In regard to admin boundaries sharing the coastline, I think that would also be 
incorrect but I am less confident of my view on this.

I did update some administrative boundaries in South Australia using the SA 
Government Data and those boundaries did not coincide with the coastline (see 
the coastline around Streaky Bay, Ceduna, Fowlers Bay and near the Nullarbor).  
The coastline in OSM needs a lot of work - I have looked at parts of WA and SA 
but found it very difficult to use the satellite imagery correctly to refine 
the map of the coastline in areas where there are extensive mudflats or large 
tidal flows and even rocky areas just under/above the waterline. But I suggest 
it is safest, and more accurate, to map the administrative boundaries and the 
coastline separately.






___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Warin]

On 06/10/18 21:34, Andrew Harvey wrote:

Thanks for raising that. I'd seen some boundaries in WA defined in
legislation as, follow this road, then that road etc. but I think that
was for school zones. So the LGA and Suburb/Localities are defined by
the cadastral plans then?

I hear the points and see there is consensus to not reuse existing
roads, rivers in the admin boundaries, so I support that approach.

What about admin boundaries that border the coastline? Should they
share the existing coastline or not?


OSM has defined the 'coast line' as the high tide mark as that is easier to 
pick than the low or mid tide marks.
It is probable that the admin boundaries use the low tide mark?
Do a sample comparison?




That does simplify the import, as there is much less manual effort needed.

I guess what we need now is an OSM XML file with both the
Suburb/Localities and LGA boundaries together with shared ways (as
many ways are in common). I'll see what I can do to put this together,
is anyone else working on this too?

On Sat, 6 Oct 2018 at 20:53, cleary  wrote:

In regard to administrative boundaries being attached to other features such as 
waterways and roads, I think it is a trade-off between accuracy and convenience.

I am most familiar with NSW. Boundaries are not "defined" by words but rather by 
surveyors' charts. The surveyors may often have been directed to use waterways, roads, mountain 
ridges and similar features for their surveys. However the waterways and roads have sometimes/often 
moved but the boundaries have not.  Words are sometimes used to describe boundaries such as 
"it follows the river and then goes south along the main road ... " Such a description is 
approximate and is near enough for many purposes, especially if one's area of interest is well 
within the boundaries. However it may not be sufficiently precise if one is concerned with 
particular locations close to the boundaries.


Examples in NSW that might be considered include the boundary on the Murray 
River west of Tocumwal, the Lachlan River east of Cobb Highway, Willandra Creek 
south of Roto, Bogan River at Girilambone. If the boundaries were attached to 
the respective waterways, either the boundaries or the waterways would be 
incorrect. Where boundaries are mapped on rivers or roads, mappers may re-align 
the river or road as changes occur and the administraitve boundary becomes 
distorted, sometimes only slightly but usually increasingly significant over 
time. Alternatively we could map the waterway or road using the administrative 
boundary data (as some mappers have done in the past) and ignore the satellite 
imagery and GPS data but this affects the accuracy of the location of the 
waterway or road.

While I will accept the community's group decision, personally I think accuracy 
is to be valued over convenience.  I strongly advocate for accuracy by mapping 
administrative boundaries separate from other features on the map, even if they 
are nearby.

The decision in regard to the above issue will affect use of a source tag for 
the boundary. If the boundary is an approximation and attached to waterways or 
roads then it would be incorrect to use a boundary source tag, However if 
boundaries are mapped separately and accurately, then we should record the 
source of the boundary data. While I would suggest adding the source tag to the 
relation for the administrative boundary, it might also be added to the way if 
there is any need to specify the source for the way e.g. if using the 
administrative boundary for the geography of a river, then also give the source 
of the boundary data as the source for the waterway.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au




___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

Thanks for raising that. I'd seen some boundaries in WA defined in
legislation as, follow this road, then that road etc. but I think that
was for school zones. So the LGA and Suburb/Localities are defined by
the cadastral plans then?

I hear the points and see there is consensus to not reuse existing
roads, rivers in the admin boundaries, so I support that approach.

What about admin boundaries that border the coastline? Should they
share the existing coastline or not?

That does simplify the import, as there is much less manual effort needed.

I guess what we need now is an OSM XML file with both the
Suburb/Localities and LGA boundaries together with shared ways (as
many ways are in common). I'll see what I can do to put this together,
is anyone else working on this too?

On Sat, 6 Oct 2018 at 20:53, cleary  wrote:
> In regard to administrative boundaries being attached to other features such 
> as waterways and roads, I think it is a trade-off between accuracy and 
> convenience.
>
> I am most familiar with NSW. Boundaries are not "defined" by words but rather 
> by surveyors' charts. The surveyors may often have been directed to use 
> waterways, roads, mountain ridges and similar features for their surveys. 
> However the waterways and roads have sometimes/often moved but the boundaries 
> have not.  Words are sometimes used to describe boundaries such as "it 
> follows the river and then goes south along the main road ... " Such a 
> description is approximate and is near enough for many purposes, especially 
> if one's area of interest is well within the boundaries. However it may not 
> be sufficiently precise if one is concerned with particular locations close 
> to the boundaries.
>
>
> Examples in NSW that might be considered include the boundary on the Murray 
> River west of Tocumwal, the Lachlan River east of Cobb Highway, Willandra 
> Creek south of Roto, Bogan River at Girilambone. If the boundaries were 
> attached to the respective waterways, either the boundaries or the waterways 
> would be incorrect. Where boundaries are mapped on rivers or roads, mappers 
> may re-align the river or road as changes occur and the administraitve 
> boundary becomes distorted, sometimes only slightly but usually increasingly 
> significant over time. Alternatively we could map the waterway or road using 
> the administrative boundary data (as some mappers have done in the past) and 
> ignore the satellite imagery and GPS data but this affects the accuracy of 
> the location of the waterway or road.
>
> While I will accept the community's group decision, personally I think 
> accuracy is to be valued over convenience.  I strongly advocate for accuracy 
> by mapping administrative boundaries separate from other features on the map, 
> even if they are nearby.
>
> The decision in regard to the above issue will affect use of a source tag for 
> the boundary. If the boundary is an approximation and attached to waterways 
> or roads then it would be incorrect to use a boundary source tag, However if 
> boundaries are mapped separately and accurately, then we should record the 
> source of the boundary data. While I would suggest adding the source tag to 
> the relation for the administrative boundary, it might also be added to the 
> way if there is any need to specify the source for the way e.g. if using the 
> administrative boundary for the geography of a river, then also give the 
> source of the boundary data as the source for the waterway.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Warin]

On 06/10/18 20:52, cleary wrote:

In regard to administrative boundaries being attached to other features such as 
waterways and roads, I think it is a trade-off between accuracy and convenience.



+1.

If the admin boundaries use other features as there boundaries then it is the 
other feature that takes priority in accuracy over that of the boundary.

The tags on the way will have those of the other feature, possibly including 
the source of that other feature.

If the admin boundary is moved because the other feature is changed then so be 
it.
I have come across a few admin boundaries that are attached to things .. and 
from now on I'll move them to match the other feature, if that is a problem for 
you then make the admin boundary separate. I for one am tired of separating 
them.



___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[cleary]

In regard to administrative boundaries being attached to other features such as 
waterways and roads, I think it is a trade-off between accuracy and 
convenience. 

I am most familiar with NSW. Boundaries are not "defined" by words but rather 
by surveyors' charts. The surveyors may often have been directed to use 
waterways, roads, mountain ridges and similar features for their surveys. 
However the waterways and roads have sometimes/often moved but the boundaries 
have not.  Words are sometimes used to describe boundaries such as "it follows 
the river and then goes south along the main road ... " Such a description is 
approximate and is near enough for many purposes, especially if one's area of 
interest is well within the boundaries. However it may not be sufficiently 
precise if one is concerned with particular locations close to the boundaries.

Examples in NSW that might be considered include the boundary on the Murray 
River west of Tocumwal, the Lachlan River east of Cobb Highway, Willandra Creek 
south of Roto, Bogan River at Girilambone. If the boundaries were attached to 
the respective waterways, either the boundaries or the waterways would be 
incorrect. Where boundaries are mapped on rivers or roads, mappers may re-align 
the river or road as changes occur and the administraitve boundary becomes 
distorted, sometimes only slightly but usually increasingly significant over 
time. Alternatively we could map the waterway or road using the administrative 
boundary data (as some mappers have done in the past) and ignore the satellite 
imagery and GPS data but this affects the accuracy of the location of the 
waterway or road.

While I will accept the community's group decision, personally I think accuracy 
is to be valued over convenience.  I strongly advocate for accuracy by mapping 
administrative boundaries separate from other features on the map, even if they 
are nearby.  

The decision in regard to the above issue will affect use of a source tag for 
the boundary. If the boundary is an approximation and attached to waterways or 
roads then it would be incorrect to use a boundary source tag, However if 
boundaries are mapped separately and accurately, then we should record the 
source of the boundary data. While I would suggest adding the source tag to the 
relation for the administrative boundary, it might also be added to the way if 
there is any need to specify the source for the way e.g. if using the 
administrative boundary for the geography of a river, then also give the source 
of the boundary data as the source for the waterway.





___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

>Do you have examples of the overlapping ways? It looks pretty okay
around Brisbane to me. Here's an a mesh of the LGA's in GeoJSON which
you can import into JOSM with enough memory.
https://tianjara.net/data/LGA_mesh.geojson.xz

Just import the shapefile into JOSM and use the validation. The GeoJSON
file is looking much cleaner.


>That's fairly easy to as a preprocessing step, eg with ogr2osm or via
other scripts. I've been working on processing the PSMA data to make it
easier to import. Since I think we should reuse existing ways where
possible, if we did that it's a mostly manual process anyway. Even
without reusing existing way, to get relations you need shared ways on
the borders. One approach is to use
https://github.com/andrewharvey/geojson-mesh to get single ways for the
border which we then manually join up into the full relations in JOSM.

We might just have to do this. It would be a lot faster then trying to
find overlapping ways, at least on the LGA data.


>If there's an existing place node, then we should use that as the label
member of the relation

Yes this is what I was intending.



___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

On Sat, 6 Oct 2018 at 15:57, Joel H.  wrote:
>
> OK everyone I am currently editing the LGA shapefiles for QLD so no one 
> should attempt as to not create conflicts (although I'm not currently working 
> on the suburbs file).

Don't worry, no one should actually be doing any importing until we
get community consensus and a plan in place.

> The PSMA data isn't good IMO. And requires a lot of fixing to be imported, 
> lots of problems with overlapping ways.

Do you have examples of the overlapping ways? It looks pretty okay
around Brisbane to me. Here's an a mesh of the LGA's in GeoJSON which
you can import into JOSM with enough memory.
https://tianjara.net/data/LGA_mesh.geojson.xz

> But the biggest problem is that the names are all caps! Does anyone know a 
> way to automatically convert all of these properly so that "BRISBANE" is 
> "Brisbane"?

That's fairly easy to as a preprocessing step, eg with ogr2osm or via
other scripts.

I've been working on processing the PSMA data to make it easier to
import. Since I think we should reuse existing ways where possible, if
we did that it's a mostly manual process anyway. Even without reusing
existing way, to get relations you need shared ways on the borders.

One approach is to use https://github.com/andrewharvey/geojson-mesh to
get single ways for the border which we then manually join up into the
full relations in JOSM.

> On 6/10/18 12:07 pm, Joel H. wrote:
> My first though is to add a fixme notice (in similar fashion to the NSW 
> import), that tells us to reconsider the place label node already in OSM, and 
> to integrate it with the boundary data.

If there's an existing place node, then we should use that as the
label member of the relation.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

On Fri, 5 Oct 2018 at 11:43, cleary  wrote:
> A month ago, we celebrated the news that OSM now has approval to use the PSMA 
> Administrative Boundaries and there was some discussion, including the need 
> for a proper import process.  I am willing to start adding some boundaries in 
> areas with which I am familiar/interested but I am waiting for the proper 
> import process to be determined. I am not aware if anything has been done in 
> regard to a plan to import this data. If so, please guide me.

Glad to see people are interested in bringing this data into OSM. I'm
strongly of the view that we should discuss and document the import
process before beginning, so naturally that's the first step.

>If not, I propose the following:
> Individual mappers download most recent data from 
> https://data.gov.au/dataset/psma-administrative-boundaries and add/check data 
> in their areas of interest and/or when they have time.  (Andrew Harvey has 
> provided a script to assist working with PSMA Administrative Boundaries Data 
> and there is a link on 
> https://wiki.openstreetmap.org/wiki/Australian_Data_Catalogue)
>
> Administrative boundaries in NSW, SA, VIC and ACT be modified 
> instance-by-instance if there are discrepancies that require updating. Where 
> there are discrepancies, the most recent data is usually preferred . If there 
> are concerns about accuracy of the most recent data, further consultation 
> with other mappers or relevant government boundary authorities would be 
> appropriate.
>
> In QLD, WA, TAS and NT :  LGA and Suburb/Locality boundaries be added one at 
> a time, integrating with existing data where appropriate.  Previous data from 
> unauthorised sources be deleted or the authorised source be added if the data 
> is accurate.
>
> LGA Boundaries continue to be tagged as admin_level=6
> Suburb/Locality boundaries be tagged as admin_level=10

I agree.

> In both instances source=PSMA_Admin_Boundaries_August_2018
> (or whichever date is applicable to the most recent data being used)
> This source information may seem unwieldy but provides accuracy and 
> completeness of information.

I'm on the fence on this, while the source tag on the feature can be
very handy for mappers, I've also seen the downside where as other
tags are added later on it becomes unclear what actually came from
that source. Where do you propose this tag to go, on the relation or
on the way members of the relation?

Consistent with existing LGA boundaries in OSM I'm proposing we use
relations with the tags:

type=boundary
boundary=administrative
admin_level=6
place=municipality

with optional tags

short_name=Sydney
name=City of Sydney
wikidata=
wikipedia= (optional with wikidata tag)
website=
phone=
email=

the admin boundary ways as "outer" members.

Consistent with existing Suburb/Locality boundaries in OSM I'm
proposing we use relations with the tags:

type=boundary
boundary=administrative
admin_level=10
place=suburb
name=

with optional tags:

postal_code (although there's not 1:1 match between postal areas and
suburbs, I'm okay with adding a postal code that generally applies to
the suburb so long as it's not coming from copyrighted sources)
wikidata
wikipedia (optional with wikidata tag)

> Administrative boundaries NOT be attached to other ways such as creeks, 
> rivers, roads, etcetera so that the other features can be modified as needed 
> without affecting the administrative boundaries which are generally static.

I disagree with that. If the admin boundary seems to follow the creek,
river, road, coastline then we should use that existing way as part of
the relation.

I'd like to avoid a "mess" of multiple ways all very close to each
other and overlapping.

Plus if the boundary is defined as that river, road, coastline then by
using the existing way we are able to capture that detail in OSM in a
way that almost all other spatial data cannot.

> Multiple administrative boundaries (including national_parks and government 
> approved protected_areas or state forests) be mapped on a single way, where 
> appropriate, and with multiple relations attached to that single way i.e. a 
> single way could form the boundaries for localities, LGAs and a national 
> park, if appropriate in the particular location.

I agree.

> Electoral boundaries NOT be added at this time.

I agree.

> Other boundaries which have been discussed, such as regions and indigenous 
> areas, NOT be added as part of this particular project unless they are LGAs 
> or Suburbs/Localities (some indigenous areas may be identified as protected 
> areas or defined localities or LGAs, in which case they are added and 
> identified on that basis). Mapping of regions could be further discussed 
> separately if required. They are usually not defined by legislation and 
> therefore usually not "administrative" in the way that LGAs and 
> Suburbs/Localities are legislated.

I agree, plus these other types of regions aren't covered by the PSMA
Admin 

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

OK everyone I am currently editing the LGA shapefiles for QLD so no one
should attempt as to not create conflicts (although I'm not currently
working on the suburbs file).

The PSMA data isn't good IMO. And requires a lot of fixing to be
imported, lots of problems with overlapping ways.

But the biggest problem is that the names are all caps! Does anyone know
a way to automatically convert all of these properly so that "BRISBANE"
is "Brisbane"?

On 6/10/18 12:07 pm, Joel H. wrote:
>
> Hi cleary (and everyone),
>
> This is excellent news, I wasn't aware of this. I think this is great
> and would be interested in importing Queensland's
>
> My first though is to add a fixme notice (in similar fashion to the
> NSW import), that tells us to reconsider the place label node already
> in OSM, and to integrate it with the boundary data.
>
> I'm going to check the shapefiles, but in the meantime, what other
> considerations need to be made?
>
>
> --Joel
>
> On 5/10/18 11:42 am, cleary wrote:
>>
>> A month ago, we celebrated the news that OSM now has approval to use
>> the PSMA Administrative Boundaries and there was some discussion,
>> including the need for a proper import process.  I am willing to
>> start adding some boundaries in areas with which I am
>> familiar/interested but I am waiting for the proper import process to
>> be determined. I am not aware if anything has been done in regard to
>> a plan to import this data. If so, please guide me. If not, I propose
>> the following:
>>
>>
>> Individual mappers download most recent data from
>> https://data.gov.au/dataset/psma-administrative-boundaries and
>> add/check data in their areas of interest and/or when they have
>> time.  (Andrew Harvey has provided a script to assist working with
>> PSMA Administrative Boundaries Data and there is a link on
>> https://wiki.openstreetmap.org/wiki/Australian_Data_Catalogue)
>>
>> Administrative boundaries in NSW, SA, VIC and ACT be modified
>> instance-by-instance if there are discrepancies that require
>> updating. Where there are discrepancies, the most recent data is
>> usually preferred . If there are concerns about accuracy of the most
>> recent data, further consultation with other mappers or relevant
>> government boundary authorities would be appropriate.
>>
>> In QLD, WA, TAS and NT :  LGA and Suburb/Locality boundaries be added
>> one at a time, integrating with existing data where appropriate. 
>> Previous data from unauthorised sources be deleted or the authorised
>> source be added if the data is accurate.
>>
>> LGA Boundaries continue to be tagged as admin_level=6
>> Suburb/Locality boundaries be tagged as admin_level=10
>>
>> In both instances source=PSMA_Admin_Boundaries_August_2018 
>> (or whichever date is applicable to the most recent data being used)
>> This source information may seem unwieldy but provides accuracy and
>> completeness of information.
>>
>> Administrative boundaries NOT be attached to other ways such as
>> creeks, rivers, roads, etcetera so that the other features can be
>> modified as needed without affecting the administrative boundaries
>> which are generally static.
>>
>> Multiple administrative boundaries (including national_parks and
>> government approved protected_areas or state forests) be mapped on a
>> single way, where appropriate, and with multiple relations attached
>> to that single way i.e. a single way could form the boundaries for
>> localities, LGAs and a national park, if appropriate in the
>> particular location.
>>
>> Electoral boundaries NOT be added at this time.
>>
>> Other boundaries which have been discussed, such as regions and
>> indigenous areas, NOT be added as part of this particular project
>> unless they are LGAs or Suburbs/Localities (some indigenous areas may
>> be identified as protected areas or defined localities or LGAs, in
>> which case they are added and identified on that basis). Mapping of
>> regions could be further discussed separately if required. They are
>> usually not defined by legislation and therefore usually not
>> "administrative" in the way that LGAs and Suburbs/Localities are
>> legislated.
>>
>>
>> Comments and feedback, please.
>>
>>
>>
>>
>> On Fri, Aug 31, 2018, at 4:23 PM, Andrew Harvey wrote:
>>> The Department of Industry, Innovation and Science which manages
>>> data.gov.au  have just completed the OSMF CC BY
>>> waiver allowing the PSMA Administrative Boundaries[1] to be used
>>> within OpenStreetMap[2].
>>>
>>> Quoting their email, "The Australian Government received great
>>> encouragement from PSMA Australia Limited to make the AB
>>> [Administrative Boundaries] data more accessible. Many thanks to OSM
>>> for your ongoing global open data efforts."
>>>
>>> Others more active in working with admin boundaries in OSM might be
>>> able to comment further on my analysis, but I've taken a look at the
>>> current OSM data from a planet extract[3], using osmium tool to
>>> extract administrative 

Re: [talk-au] PSMA Administrative Boundaries

[Joel H.]

Hi cleary (and everyone),

This is excellent news, I wasn't aware of this. I think this is great
and would be interested in importing Queensland's

My first though is to add a fixme notice (in similar fashion to the NSW
import), that tells us to reconsider the place label node already in
OSM, and to integrate it with the boundary data.

I'm going to check the shapefiles, but in the meantime, what other
considerations need to be made?


--Joel

On 5/10/18 11:42 am, cleary wrote:
>
> A month ago, we celebrated the news that OSM now has approval to use
> the PSMA Administrative Boundaries and there was some discussion,
> including the need for a proper import process.  I am willing to start
> adding some boundaries in areas with which I am familiar/interested
> but I am waiting for the proper import process to be determined. I am
> not aware if anything has been done in regard to a plan to import this
> data. If so, please guide me. If not, I propose the following:
>
>
> Individual mappers download most recent data from
> https://data.gov.au/dataset/psma-administrative-boundaries and
> add/check data in their areas of interest and/or when they have time. 
> (Andrew Harvey has provided a script to assist working with PSMA
> Administrative Boundaries Data and there is a link on
> https://wiki.openstreetmap.org/wiki/Australian_Data_Catalogue)
>
> Administrative boundaries in NSW, SA, VIC and ACT be modified
> instance-by-instance if there are discrepancies that require updating.
> Where there are discrepancies, the most recent data is usually
> preferred . If there are concerns about accuracy of the most recent
> data, further consultation with other mappers or relevant government
> boundary authorities would be appropriate.
>
> In QLD, WA, TAS and NT :  LGA and Suburb/Locality boundaries be added
> one at a time, integrating with existing data where appropriate. 
> Previous data from unauthorised sources be deleted or the authorised
> source be added if the data is accurate.
>
> LGA Boundaries continue to be tagged as admin_level=6
> Suburb/Locality boundaries be tagged as admin_level=10
>
> In both instances source=PSMA_Admin_Boundaries_August_2018 
> (or whichever date is applicable to the most recent data being used)
> This source information may seem unwieldy but provides accuracy and
> completeness of information.
>
> Administrative boundaries NOT be attached to other ways such as
> creeks, rivers, roads, etcetera so that the other features can be
> modified as needed without affecting the administrative boundaries
> which are generally static.
>
> Multiple administrative boundaries (including national_parks and
> government approved protected_areas or state forests) be mapped on a
> single way, where appropriate, and with multiple relations attached to
> that single way i.e. a single way could form the boundaries for
> localities, LGAs and a national park, if appropriate in the particular
> location.
>
> Electoral boundaries NOT be added at this time.
>
> Other boundaries which have been discussed, such as regions and
> indigenous areas, NOT be added as part of this particular project
> unless they are LGAs or Suburbs/Localities (some indigenous areas may
> be identified as protected areas or defined localities or LGAs, in
> which case they are added and identified on that basis). Mapping of
> regions could be further discussed separately if required. They are
> usually not defined by legislation and therefore usually not
> "administrative" in the way that LGAs and Suburbs/Localities are
> legislated.
>
>
> Comments and feedback, please.
>
>
>
>
> On Fri, Aug 31, 2018, at 4:23 PM, Andrew Harvey wrote:
>> The Department of Industry, Innovation and Science which manages
>> data.gov.au  have just completed the OSMF CC BY
>> waiver allowing the PSMA Administrative Boundaries[1] to be used
>> within OpenStreetMap[2].
>>
>> Quoting their email, "The Australian Government received great
>> encouragement from PSMA Australia Limited to make the AB
>> [Administrative Boundaries] data more accessible. Many thanks to OSM
>> for your ongoing global open data efforts."
>>
>> Others more active in working with admin boundaries in OSM might be
>> able to comment further on my analysis, but I've taken a look at the
>> current OSM data from a planet extract[3], using osmium tool to
>> extract administrative boundaries[4]:
>>
>>     osmium tags-filter australia-latest.osm.pbf
>> nwr/boundary=administrative -o osm-admin-boundaries.osm.pbf
>>
>> Then converted to GeoPackage to open in QGIS to compare to PSMA
>> boundaries:
>>
>>     ogr2ogr -f GPKG osm-admin-boundaries.gpkg
>> osm-admin-boundaries.osm.pbf
>>
>> The admin_level values for Australia are defined as[5]:
>>
>> 3 n/a
>> 4 State or Territory
>> 5 n/a (but used in Victoria for regions[6])
>> 6 Local Government Area (eg Shire/Council)
>> 7 District or Region (e.g Perthshire, Fitzroy, Canning, Greater
>> Sydney, Greater Melbourne, etc.)
>> 8 

Re: [talk-au] PSMA Administrative Boundaries

[cleary]

A month ago, we celebrated the news that OSM now has approval to use the
PSMA Administrative Boundaries and there was some discussion, including
the need for a proper import process.  I am willing to start adding some
boundaries in areas with which I am familiar/interested but I am waiting
for the proper import process to be determined. I am not aware if
anything has been done in regard to a plan to import this data. If so,
please guide me. If not, I propose the following:

Individual mappers download most recent data from
https://data.gov.au/dataset/psma-administrative-boundaries and add/check
data in their areas of interest and/or when they have time.  (Andrew
Harvey has provided a script to assist working with PSMA Administrative
Boundaries Data and there is a link on
https://wiki.openstreetmap.org/wiki/Australian_Data_Catalogue)
Administrative boundaries in NSW, SA, VIC and ACT be modified instance-by-
instance if there are discrepancies that require updating. Where there
are discrepancies, the most recent data is usually preferred . If there
are concerns about accuracy of the most recent data, further
consultation with other mappers or relevant government boundary
authorities would be appropriate.
In QLD, WA, TAS and NT :  LGA and Suburb/Locality boundaries be added
one at a time, integrating with existing data where appropriate.
Previous data from unauthorised sources be deleted or the authorised
source be added if the data is accurate.
LGA Boundaries continue to be tagged as admin_level=6
Suburb/Locality boundaries be tagged as admin_level=10

In both instances source=PSMA_Admin_Boundaries_August_2018  
(or whichever date is applicable to the most recent data being used)
This source information may seem unwieldy but provides accuracy and
completeness of information.
Administrative boundaries NOT be attached to other ways such as creeks,
rivers, roads, etcetera so that the other features can be modified as
needed without affecting the administrative boundaries which are
generally static.
Multiple administrative boundaries (including national_parks and
government approved protected_areas or state forests) be mapped on a
single way, where appropriate, and with multiple relations attached
to that single way i.e. a single way could form the boundaries for
localities, LGAs and a national park, if appropriate in the
particular location.
Electoral boundaries NOT be added at this time.

Other boundaries which have been discussed, such as regions and
indigenous areas, NOT be added as part of this particular project unless
they are LGAs or Suburbs/Localities (some indigenous areas may be
identified as protected areas or defined localities or LGAs, in which
case they are added and identified on that basis). Mapping of regions
could be further discussed separately if required. They are usually not
defined by legislation and therefore usually not "administrative" in the
way that LGAs and Suburbs/Localities are legislated.

Comments and feedback, please.




On Fri, Aug 31, 2018, at 4:23 PM, Andrew Harvey wrote:
> The Department of Industry, Innovation and Science which manages
> data.gov.au have just completed the OSMF CC BY waiver allowing the
> PSMA Administrative Boundaries[1] to be used within OpenStreetMap[2].> 
> Quoting their email, "The Australian Government received great
> encouragement from PSMA Australia Limited to make the AB
> [Administrative Boundaries] data more accessible. Many thanks to OSM
> for your ongoing global open data efforts."> 
> Others more active in working with admin boundaries in OSM might be
> able to comment further on my analysis, but I've taken a look at the
> current OSM data from a planet extract[3], using osmium tool to
> extract administrative boundaries[4]:> 
> osmium tags-filter australia-latest.osm.pbf
> nwr/boundary=administrative -o osm-admin-boundaries.osm.pbf> 
> Then converted to GeoPackage to open in QGIS to compare to PSMA
> boundaries:> 
> ogr2ogr -f GPKG osm-admin-boundaries.gpkg osm-admin-
> boundaries.osm.pbf> 
> The admin_level values for Australia are defined as[5]:
> 
> 3 n/a
> 4 State or Territory
> 5 n/a (but used in Victoria for regions[6])
> 6 Local Government Area (eg Shire/Council)
> 7 District or Region (e.g Perthshire, Fitzroy, Canning, Greater
> Sydney, Greater Melbourne, etc.)> 8 Postcode
> 9 Suburb and Locality
> 10 Suburb and Locality
> 
> (not sure why the split across 9 and 10...? but it looks like no
> data for 9 and all suburb/localities are marked as 10, a seperate
> discussion but perhaps they should be 9 and smaller neighbourhoods
> be in 10?)> 
> PSMA Admin Boundaries provide (for Australia wide Shapefiles see [7]
> built from [8])> 
> Commonwealth Electoral Boundaries (I don't think should be
> included in OSM)> State Electoral Boundaries (I don't think should be 
> included in OSM)
> State Boundaries
> Local Government Areas
> Suburbs / Localities
> Town Points
> Wards
> 
> admin_level 6 LGAs
> It looks like NSW, VIC 

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

Good to see there is a lot of interest in this. Local knowledge is going to
be key to ensuring success if we undertake work to bring this into OSM.

On Fri, 31 Aug 2018 at 16:45, Ewen Hill  wrote:

> I would like to aski if it is possible to
> 1. add the ability to have, possibly at [*5*], the Aboriginal nations
> e.g.
> https://commons.wikimedia.org/wiki/File:Map_Victoria_Aboriginal_tribes_(colourmap).jpg
> and
>


> 2. swap  *6 Local Government Area (eg Shire/Council) *and *7 District or
> Region (e.g Perthshire, Fitzroy, Canning, Greater Sydney, Greater
> Melbourne, etc.) *as normally these areas are large than a singular LGA.
>

Despite what is on the OSM wiki, admin_level=7 is only really being used in
ACT at the moment, Greater Sydney isn't really an admin boundary. I can
only comment for NSW, but we have regions like Hunter, Central Coast,
Illawarra, South Coast, New England, Eastern Suburbs
https://en.wikipedia.org/wiki/Regions_of_New_South_Wales but I've been
tagging these using place=district.

On Fri, 31 Aug 2018 at 17:21, Andrew Davidson  wrote:

> On 31/8/18 16:23, Andrew Harvey wrote:
> > (not sure why the split across 9 and 10...? but it looks like no data
> for 9
> > and all suburb/localities are marked as 10, a separate discussion but
> > perhaps they should be 9 and smaller neighbourhoods be in 10?)
>
> Level 9 appears to have been intended for non-ABS derived suburbs and
> level 10 for ABS. What's happened is 10 has been used for suburbs.
>

Given ABS has pretty much nothing to do with how suburbs/localities are
defined, not sure that makes sense going forward.


> Do we have any data for bounded localities smaller than suburbs? I don't
> know of any jurisdiction that defines sub-suburb areas, so I don't see
> the point of moving away from 10.
>

Maybe as informal places which we can use place
https://wiki.openstreetmap.org/wiki/Key%3Aplace tags for, but I'm not aware
of any which justify using boundary=administrative.


> I know that some people don't like having the admin boundaries in OSM. I
> find that having the suburb/locality boundaries is useful (downloading
> data by area and geo-referencing in Nominatim).


I feel this too, it's not data that we are really creating or adding value
but I can see having it in OSM can make it easier to use other OSM data as
well as helping to contributing to a global dataset with consistent schema.


> The LGA boundaries less
> so as most people don't really think very much about their local council
> and it's odd to see it appear in an address for example. But that said
> if you do the suburbs you might as well do the LGAs.
>

I hear you, but they are surveyable (usually street signs are branded with
the LGA and signs telling you when you enter the more rural shires) and it
is handy to be able to use OSM to easily check the LGA boundaries and have
all the attached metadata (website, phone number etc).

On Fri, 31 Aug 2018 at 17:53, Warin <61sundow...@gmail.com> wrote:

> My 'problems' with the admin boundaries are;
>  where they use another way that is also a road/river. And then some
> mapper comes along and improves the road/river .. but buggers up the admin
> boundary.
> then I come along and 'fix it' using the LPI Base map .. and put that
> source on the way.
>

I've seen this problem too, especially for new mappers who want to change a
road, add turn restrictions etc. it's too easy to break relations like the
admin boundary. I think this is just the cost of having this data in OSM
that occasionally it will break and need fixing. Hopefully editors will
improve to reduce the risk but might be something we have to accept.


>
> 2 things on my wish list ;
> that admin boundaries don't use other 'close enough' ways already in OSM.
> This would reduce the number of times that the relationships are broken.
>
that admin boundaries have a source tag on there ways to say what the
> source is. This means when I come along and fix a broken one my source
> statement will not be confusing when looking at  the relationship as that
> relationship will have no source tag.
>

No problem with that.

Generally I wouldn't treat these boundaries as coordinates carved in stone
that absolutely must be fixed in OSM. I think it's fine if there they are
out my a few meters if it makes sense to line them up with a road or river,
especially if those boundaries are defined by the road or river.

On Fri, 31 Aug 2018 at 22:17, cleary  wrote:

> - I agree that it is inappropriate to map administrative boundaries on the
> same way as natural phenomena such as rivers. The boundaries are usually
> where the river flowed sometime in the past when an early survey was done.
> The boundaries do not change but the rivers often do. I think it best
> always to map administrative boundaries separate from natural features.
>

I'm on the fence about this, I think it depends how they are defined, if
the suburb boundary is defined to follow a road, then the admin boundary
should use that 

Re: [talk-au] PSMA Administrative Boundaries

[Lee Mason]

Yes, good work getting the waiver. I would also agree not to include electoral 
boundaries. And my primary interest would also be Tasmania.

The state (and probably more broadly) already has a comprehensive naming of 
localities and suburbs from surveys, but mostly from the GeoScience Australia 
place names dataset. I would think that when integrating PSMA boundaries, it 
would be important to preserve these place nodes which more accurately pinpoint 
the locations of smaller communities (even if it is only a cluster of a few 
homes), which would not necessarily be the centroid of the relation.

And just hand-waving some possible scenario/solution: nodes of place=hamlet or 
lower would be integrated with the PSMA locality region. The node would be take 
role:label in the relation. The relation would place=hamlet as well as 
boundary=admin?



___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[cleary]

Permission to include the PSMA Boundaries in OSM is great news.

In regard to questions and comments from other mappers:
-  I find the current boundaries in NSW and SA to be useful i.e. include
   LGA and suburb/locality. However, electoral boundaries including
   local government wards seem very specialised and are probably better
   mapped elsewhere.-  I find LGA boundaries useful. Where a road suddenly 
changes name,
   changes surface, reversals of housenumbers occur etc., these
   phenomena often become understandable when you see where LGA
   boundaries lie.- I agree that it is inappropriate to map administrative 
boundaries on
  the same way as natural phenomena such as rivers. The boundaries are
  usually where the river flowed sometime in the past when an early
  survey was done. The boundaries do not change but the rivers often do.
  I think it best always to map administrative boundaries separate from
  natural features.- It would be possible to change suburbs from level 10 to 
level 9 if
  others find that useful but I think that sub-suburb localities such as
  neighbourhoods are not "administrative" nor (to my knowledge) are they
  defined by a government authority so I am unsure what level 10 could
  be used for, if suburbs are made level 9.
I am interested in administrative boundaries and willing to help where I
can - however I am just an enthusiast not a mapping professional - I
would not be confident to attempt any large scale import, if that is
envisaged. However, if individuals are each taking an area and gradually
adding new boundaries or checking existing boundaries against PSMA data,
I would be interested in contributing, once the process is agreed.
Thanks again to Andrew Harvey - this is a great enhancement to
data in OSM.



On Fri, Aug 31, 2018, at 4:23 PM, Andrew Harvey wrote:
> The Department of Industry, Innovation and Science which manages
> data.gov.au have just completed the OSMF CC BY waiver allowing the
> PSMA Administrative Boundaries[1] to be used within OpenStreetMap[2].> 
> Quoting their email, "The Australian Government received great
> encouragement from PSMA Australia Limited to make the AB
> [Administrative Boundaries] data more accessible. Many thanks to OSM
> for your ongoing global open data efforts."> 
> Others more active in working with admin boundaries in OSM might be
> able to comment further on my analysis, but I've taken a look at the
> current OSM data from a planet extract[3], using osmium tool to
> extract administrative boundaries[4]:> 
> osmium tags-filter australia-latest.osm.pbf
> nwr/boundary=administrative -o osm-admin-boundaries.osm.pbf> 
> Then converted to GeoPackage to open in QGIS to compare to PSMA
> boundaries:> 
> ogr2ogr -f GPKG osm-admin-boundaries.gpkg osm-admin-
> boundaries.osm.pbf> 
> The admin_level values for Australia are defined as[5]:
> 
> 3 n/a
> 4 State or Territory
> 5 n/a (but used in Victoria for regions[6])
> 6 Local Government Area (eg Shire/Council)
> 7 District or Region (e.g Perthshire, Fitzroy, Canning, Greater
> Sydney, Greater Melbourne, etc.)> 8 Postcode
> 9 Suburb and Locality
> 10 Suburb and Locality
> 
> (not sure why the split across 9 and 10...? but it looks like no
> data for 9 and all suburb/localities are marked as 10, a seperate
> discussion but perhaps they should be 9 and smaller neighbourhoods
> be in 10?)> 
> PSMA Admin Boundaries provide (for Australia wide Shapefiles see [7]
> built from [8])> 
> Commonwealth Electoral Boundaries (I don't think should be
> included in OSM)> State Electoral Boundaries (I don't think should be 
> included in OSM)
> State Boundaries
> Local Government Areas
> Suburbs / Localities
> Town Points
> Wards
> 
> admin_level 6 LGAs
> It looks like NSW, VIC have complete LGA coverage already aside a few
> minor differences, but NT is patchy and TAS, QLD, WA are almost non-
> existent (ACT doesn't really have LGAs it's all managed by the
> Territory). SA is missing a few LGAs which exist in the PSMA data.> 
> admin_level 7 is a mixed bag, but the PSMA data won't help here
> 
> suburbs/localities
> NSW, VIC, SA all have pretty much complete data in OSM but there are a
> number of differences with the PSMA data which should warrent
> investigation and correction.> 
> ACT I think has good existing data (in the PSMA data in
> suburb/localities there are both levels, eg. Canberra Centre and
> Parkes), but the districts have a slightly different name format.> 
> All other states are empty in OSM.
> 
> Keeping in mind that we have state data for admin boundaries in VIC
> (VicMap), NSW (Spatial Services), ACT (ACTmapi) and SA
> (data.sa.gov.au) already.> 
> So there does appear to be a lot of data missing in OSM, which
> licensing wise we could bring in to OSM. So I guess the question is
> should we and how should we do that. Is anyone interested in working
> on this? If people are, I think it should go through the proper import
> process (discuss first, identify the 

Re: [talk-au] PSMA Administrative Boundaries

[Warin]

On 31/08/18 17:20, Andrew Davidson wrote:

On 31/8/18 16:23, Andrew Harvey wrote:
(not sure why the split across 9 and 10...? but it looks like no data 
for 9

and all suburb/localities are marked as 10, a separate discussion but
perhaps they should be 9 and smaller neighbourhoods be in 10?)


Level 9 appears to have been intended for non-ABS derived suburbs and 
level 10 for ABS. What's happened is 10 has been used for suburbs.


Do we have any data for bounded localities smaller than suburbs? I 
don't know of any jurisdiction that defines sub-suburb areas, so I 
don't see the point of moving away from 10.


Commonwealth Electoral Boundaries (I don't think should be included 
in OSM)

State Electoral Boundaries (I don't think should be included in OSM)
State Boundaries


Agree.


admin_level 6 LGAs
It looks like NSW, VIC have complete LGA coverage already aside a few 
minor
differences, but NT is patchy and TAS, QLD, WA are almost 
non-existent (ACT
doesn't really have LGAs it's all managed by the Territory). 


Given that we have previously only had data for SA/Vic/NSW any other 
data would have come from sources we were not allowed to use.



admin_level 7 is a mixed bag, but the PSMA data won't help here


That's more a problem with the rather vague definition of what level 7 
is supposed to be.


Nicaragua is using it for 'Indigenous territories'. Possibly 'we' could 
do the same?



suburbs/localities
NSW, VIC, SA all have pretty much complete data in OSM but there are a
number of differences with the PSMA data which should warrant 
investigation

and correction.


That's a bit of understatement for Vic. I checked the Vicmap suburbs 
with what's in OSM and it's not good. Currently we have ~1600 suburbs 
in OSM but there are ~3000 suburbs in Vicmaps/PSMA. It would appear 
that somewhere between ABS2011 and ABS2016 the number of Victorian 
suburbs almost doubled.


 So I guess the question is should we 


I know that some people don't like having the admin boundaries in OSM. 
I find that having the suburb/locality boundaries is useful 
(downloading data by area and geo-referencing in Nominatim). The LGA 
boundaries less so as most people don't really think very much about 
their local council and it's odd to see it appear in an address for 
example. But that said if you do the suburbs you might as well do the 
LGAs.


My 'problems' with the admin boundaries are;
 where they use another way that is also a road/river. And then some 
mapper comes along and improves the road/river .. but buggers up the 
admin boundary.
then I come along and 'fix it' using the LPI Base map .. and put that 
source on the way.


2 things on my wish list ;
that admin boundaries don't use other 'close enough' ways already in 
OSM. This would reduce the number of times that the relationships are 
broken.
that admin boundaries have a source tag on there ways to say what the 
source is. This means when I come along and fix a broken one my source 
statement will not be confusing when looking at  the relationship as 
that  relationship will have no source tag.




how should we do that. 


Import.

Import.


Is anyone interested in working on this? 


I am.

Not me.


If people are, I think it should go through the proper import process 
(discuss first,

identify the process and then execute).


You're preaching to the choir here. Amen to that.




___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Andrew Davidson]

On 31/8/18 16:23, Andrew Harvey wrote:

(not sure why the split across 9 and 10...? but it looks like no data for 9
and all suburb/localities are marked as 10, a separate discussion but
perhaps they should be 9 and smaller neighbourhoods be in 10?)


Level 9 appears to have been intended for non-ABS derived suburbs and 
level 10 for ABS. What's happened is 10 has been used for suburbs.


Do we have any data for bounded localities smaller than suburbs? I don't 
know of any jurisdiction that defines sub-suburb areas, so I don't see 
the point of moving away from 10.



Commonwealth Electoral Boundaries (I don't think should be included in OSM)
State Electoral Boundaries (I don't think should be included in OSM)
State Boundaries


Agree.


admin_level 6 LGAs
It looks like NSW, VIC have complete LGA coverage already aside a few minor
differences, but NT is patchy and TAS, QLD, WA are almost non-existent (ACT
doesn't really have LGAs it's all managed by the Territory). 


Given that we have previously only had data for SA/Vic/NSW any other 
data would have come from sources we were not allowed to use.



admin_level 7 is a mixed bag, but the PSMA data won't help here


That's more a problem with the rather vague definition of what level 7 
is supposed to be.



suburbs/localities
NSW, VIC, SA all have pretty much complete data in OSM but there are a
number of differences with the PSMA data which should warrant investigation
and correction.


That's a bit of understatement for Vic. I checked the Vicmap suburbs 
with what's in OSM and it's not good. Currently we have ~1600 suburbs in 
OSM but there are ~3000 suburbs in Vicmaps/PSMA. It would appear that 
somewhere between ABS2011 and ABS2016 the number of Victorian suburbs 
almost doubled.


 So I guess the question is should we 


I know that some people don't like having the admin boundaries in OSM. I 
find that having the suburb/locality boundaries is useful (downloading 
data by area and geo-referencing in Nominatim). The LGA boundaries less 
so as most people don't really think very much about their local council 
and it's odd to see it appear in an address for example. But that said 
if you do the suburbs you might as well do the LGAs.


how should we do that. 


Import.

Is anyone interested in working on this? 


I am.


If people are, I think it should go through the proper import process (discuss 
first,
identify the process and then execute).


You're preaching to the choir here. Amen to that.

___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Phil Wyatt]

Hi Andrew,

I am happy to help out BUT, I have never done any imports previously so it will 
be a cautious approach from me. My main interest is in Tasmania.


Cheers - Phil, 
On the road with his iPad 

> On 31 Aug 2018, at 4:23 pm, Andrew Harvey  wrote:
> 
> The Department of Industry, Innovation and Science which manages data.gov.au 
> have just completed the OSMF CC BY waiver allowing the PSMA Administrative 
> Boundaries[1] to be used within OpenStreetMap[2].
> 
> Quoting their email, "The Australian Government received great encouragement 
> from PSMA Australia Limited to make the AB [Administrative Boundaries] data 
> more accessible. Many thanks to OSM for your ongoing global open data 
> efforts."
> 
> Others more active in working with admin boundaries in OSM might be able to 
> comment further on my analysis, but I've taken a look at the current OSM data 
> from a planet extract[3], using osmium tool to extract administrative 
> boundaries[4]:
> 
> osmium tags-filter australia-latest.osm.pbf nwr/boundary=administrative 
> -o osm-admin-boundaries.osm.pbf
> 
> Then converted to GeoPackage to open in QGIS to compare to PSMA boundaries:
> 
> ogr2ogr -f GPKG osm-admin-boundaries.gpkg osm-admin-boundaries.osm.pbf
> 
> The admin_level values for Australia are defined as[5]:
> 
> 3 n/a
> 4 State or Territory
> 5 n/a (but used in Victoria for regions[6])
> 6 Local Government Area (eg Shire/Council)
> 7 District or Region (e.g Perthshire, Fitzroy, Canning, Greater Sydney, 
> Greater Melbourne, etc.)
> 8 Postcode
> 9 Suburb and Locality
> 10 Suburb and Locality
> 
> (not sure why the split across 9 and 10...? but it looks like no data for 9 
> and all suburb/localities are marked as 10, a seperate discussion but perhaps 
> they should be 9 and smaller neighbourhoods be in 10?)
> 
> PSMA Admin Boundaries provide (for Australia wide Shapefiles see [7] built 
> from [8])
> 
> Commonwealth Electoral Boundaries (I don't think should be included in OSM)
> State Electoral Boundaries (I don't think should be included in OSM)
> State Boundaries
> Local Government Areas
> Suburbs / Localities
> Town Points
> Wards
> 
> admin_level 6 LGAs
> It looks like NSW, VIC have complete LGA coverage already aside a few minor 
> differences, but NT is patchy and TAS, QLD, WA are almost non-existent (ACT 
> doesn't really have LGAs it's all managed by the Territory). SA is missing a 
> few LGAs which exist in the PSMA data.
> 
> admin_level 7 is a mixed bag, but the PSMA data won't help here
> 
> suburbs/localities
> NSW, VIC, SA all have pretty much complete data in OSM but there are a number 
> of differences with the PSMA data which should warrent investigation and 
> correction.
> 
> ACT I think has good existing data (in the PSMA data in suburb/localities 
> there are both levels, eg. Canberra Centre and Parkes), but the districts 
> have a slightly different name format.
> 
> All other states are empty in OSM.
> 
> Keeping in mind that we have state data for admin boundaries in VIC (VicMap), 
> NSW (Spatial Services), ACT (ACTmapi) and SA (data.sa.gov.au) already.
> 
> So there does appear to be a lot of data missing in OSM, which licensing wise 
> we could bring in to OSM. So I guess the question is should we and how should 
> we do that. Is anyone interested in working on this? If people are, I think 
> it should go through the proper import process (discuss first, identify the 
> process and then execute).
> 
> [1] https://data.gov.au/dataset/psma-administrative-boundaries
> [2] 
> https://wiki.openstreetmap.org/wiki/File:Department_of_Industry_Innovation_and_Science_ODbl_permission_Administrative_Boundaries.pdf
> [3] http://download.openstreetmap.fr/extracts/oceania/
> [4] https://osmcode.org/osmium-tool/manual.html#filtering-by-tags
> [5] 
> https://wiki.openstreetmap.org/wiki/Tag:boundary%3Dadministrative#10_admin_level_values_for_specific_countries
> [6] https://en.wikipedia.org/wiki/Regions_of_Victoria
> [7] https://tianjara.net/data/psma-admin-bdys/
> [8] https://github.com/andrewharvey/psma-admin-bdys-data
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

Re: [talk-au] PSMA Administrative Boundaries

[Ewen Hill]

Andrew,
   Thank you for all the hard work and achieving this key element. I would
like to look at all level numbers to standardise these and to go forward.
Also happy to have a dabble of the import process and associated doco.

I would like to aski if it is possible to
1. add the ability to have, possibly at [*5*], the Aboriginal nations e.g.
https://commons.wikimedia.org/wiki/File:Map_Victoria_Aboriginal_tribes_(colourmap).jpg
and
2. swap  *6 Local Government Area (eg Shire/Council) *and *7 District or
Region (e.g Perthshire, Fitzroy, Canning, Greater Sydney, Greater
Melbourne, etc.) *as normally these areas are large than a singular LGA.

Ewen



On Fri, 31 Aug 2018 at 16:25, Andrew Harvey 
wrote:

> The Department of Industry, Innovation and Science which manages
> data.gov.au have just completed the OSMF CC BY waiver allowing the PSMA
> Administrative Boundaries[1] to be used within OpenStreetMap[2].
>
> Quoting their email, "The Australian Government received great
> encouragement from PSMA Australia Limited to make the AB [Administrative
> Boundaries] data more accessible. Many thanks to OSM for your ongoing
> global open data efforts."
>
> Others more active in working with admin boundaries in OSM might be able
> to comment further on my analysis, but I've taken a look at the current OSM
> data from a planet extract[3], using osmium tool to extract administrative
> boundaries[4]:
>
> osmium tags-filter australia-latest.osm.pbf
> nwr/boundary=administrative -o osm-admin-boundaries.osm.pbf
>
> Then converted to GeoPackage to open in QGIS to compare to PSMA boundaries:
>
> ogr2ogr -f GPKG osm-admin-boundaries.gpkg osm-admin-boundaries.osm.pbf
>
> The admin_level values for Australia are defined as[5]:
>
> 3 n/a
> 4 State or Territory
> 5 n/a (but used in Victoria for regions[6])
> 6 Local Government Area (eg Shire/Council)
> 7 District or Region (e.g Perthshire, Fitzroy, Canning, Greater Sydney,
> Greater Melbourne, etc.)
> 8 Postcode
> 9 Suburb and Locality
> 10 Suburb and Locality
>
> (not sure why the split across 9 and 10...? but it looks like no data for
> 9 and all suburb/localities are marked as 10, a seperate discussion but
> perhaps they should be 9 and smaller neighbourhoods be in 10?)
>
> PSMA Admin Boundaries provide (for Australia wide Shapefiles see [7] built
> from [8])
>
> Commonwealth Electoral Boundaries (I don't think should be included in OSM)
> State Electoral Boundaries (I don't think should be included in OSM)
> State Boundaries
> Local Government Areas
> Suburbs / Localities
> Town Points
> Wards
>
> admin_level 6 LGAs
> It looks like NSW, VIC have complete LGA coverage already aside a few
> minor differences, but NT is patchy and TAS, QLD, WA are almost
> non-existent (ACT doesn't really have LGAs it's all managed by the
> Territory). SA is missing a few LGAs which exist in the PSMA data.
>
> admin_level 7 is a mixed bag, but the PSMA data won't help here
>
> suburbs/localities
> NSW, VIC, SA all have pretty much complete data in OSM but there are a
> number of differences with the PSMA data which should warrent investigation
> and correction.
>
> ACT I think has good existing data (in the PSMA data in suburb/localities
> there are both levels, eg. Canberra Centre and Parkes), but the districts
> have a slightly different name format.
>
> All other states are empty in OSM.
>
> Keeping in mind that we have state data for admin boundaries in VIC
> (VicMap), NSW (Spatial Services), ACT (ACTmapi) and SA (data.sa.gov.au)
> already.
>
> So there does appear to be a lot of data missing in OSM, which licensing
> wise we could bring in to OSM. So I guess the question is should we and how
> should we do that. Is anyone interested in working on this? If people are,
> I think it should go through the proper import process (discuss first,
> identify the process and then execute).
>
> [1] https://data.gov.au/dataset/psma-administrative-boundaries
> [2]
> https://wiki.openstreetmap.org/wiki/File:Department_of_Industry_Innovation_and_Science_ODbl_permission_Administrative_Boundaries.pdf
> [3] http://download.openstreetmap.fr/extracts/oceania/
> [4] https://osmcode.org/osmium-tool/manual.html#filtering-by-tags
> [5]
> https://wiki.openstreetmap.org/wiki/Tag:boundary%3Dadministrative#10_admin_level_values_for_specific_countries
> [6] https://en.wikipedia.org/wiki/Regions_of_Victoria
> [7] https://tianjara.net/data/psma-admin-bdys/
> [8] https://github.com/andrewharvey/psma-admin-bdys-data
> ___
> Talk-au mailing list
> Talk-au@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-au
>


-- 
Warm Regards

Ewen Hill
Internet Development Australia
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au

[talk-au] PSMA Administrative Boundaries

[Andrew Harvey]

The Department of Industry, Innovation and Science which manages data.gov.au
have just completed the OSMF CC BY waiver allowing the PSMA Administrative
Boundaries[1] to be used within OpenStreetMap[2].

Quoting their email, "The Australian Government received great
encouragement from PSMA Australia Limited to make the AB [Administrative
Boundaries] data more accessible. Many thanks to OSM for your ongoing
global open data efforts."

Others more active in working with admin boundaries in OSM might be able to
comment further on my analysis, but I've taken a look at the current OSM
data from a planet extract[3], using osmium tool to extract administrative
boundaries[4]:

osmium tags-filter australia-latest.osm.pbf nwr/boundary=administrative
-o osm-admin-boundaries.osm.pbf

Then converted to GeoPackage to open in QGIS to compare to PSMA boundaries:

ogr2ogr -f GPKG osm-admin-boundaries.gpkg osm-admin-boundaries.osm.pbf

The admin_level values for Australia are defined as[5]:

3 n/a
4 State or Territory
5 n/a (but used in Victoria for regions[6])
6 Local Government Area (eg Shire/Council)
7 District or Region (e.g Perthshire, Fitzroy, Canning, Greater Sydney,
Greater Melbourne, etc.)
8 Postcode
9 Suburb and Locality
10 Suburb and Locality

(not sure why the split across 9 and 10...? but it looks like no data for 9
and all suburb/localities are marked as 10, a seperate discussion but
perhaps they should be 9 and smaller neighbourhoods be in 10?)

PSMA Admin Boundaries provide (for Australia wide Shapefiles see [7] built
from [8])

Commonwealth Electoral Boundaries (I don't think should be included in OSM)
State Electoral Boundaries (I don't think should be included in OSM)
State Boundaries
Local Government Areas
Suburbs / Localities
Town Points
Wards

admin_level 6 LGAs
It looks like NSW, VIC have complete LGA coverage already aside a few minor
differences, but NT is patchy and TAS, QLD, WA are almost non-existent (ACT
doesn't really have LGAs it's all managed by the Territory). SA is missing
a few LGAs which exist in the PSMA data.

admin_level 7 is a mixed bag, but the PSMA data won't help here

suburbs/localities
NSW, VIC, SA all have pretty much complete data in OSM but there are a
number of differences with the PSMA data which should warrent investigation
and correction.

ACT I think has good existing data (in the PSMA data in suburb/localities
there are both levels, eg. Canberra Centre and Parkes), but the districts
have a slightly different name format.

All other states are empty in OSM.

Keeping in mind that we have state data for admin boundaries in VIC
(VicMap), NSW (Spatial Services), ACT (ACTmapi) and SA (data.sa.gov.au)
already.

So there does appear to be a lot of data missing in OSM, which licensing
wise we could bring in to OSM. So I guess the question is should we and how
should we do that. Is anyone interested in working on this? If people are,
I think it should go through the proper import process (discuss first,
identify the process and then execute).

[1] https://data.gov.au/dataset/psma-administrative-boundaries
[2]
https://wiki.openstreetmap.org/wiki/File:Department_of_Industry_Innovation_and_Science_ODbl_permission_Administrative_Boundaries.pdf
[3] http://download.openstreetmap.fr/extracts/oceania/
[4] https://osmcode.org/osmium-tool/manual.html#filtering-by-tags
[5]
https://wiki.openstreetmap.org/wiki/Tag:boundary%3Dadministrative#10_admin_level_values_for_specific_countries
[6] https://en.wikipedia.org/wiki/Regions_of_Victoria
[7] https://tianjara.net/data/psma-admin-bdys/
[8] https://github.com/andrewharvey/psma-admin-bdys-data
___
Talk-au mailing list
Talk-au@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-au