Re: [talk-au] Suburb boundaries
One of the things I will do when building the import scripts, is to find out how many 'excess' nodes there are for a couple of different smoothing values and eyeball why they might exist. We can then make a decision as to whether we should drop nodes and if so how many cheers On Fri, Feb 6, 2009 at 9:40 AM, BlueMM wrote: > Darrin Smith writes: > > On Thu, 05 Feb 2009 17:18:53 +1030 > > Jack Burton wrote: > [SNIP] > > > Also, consider the case of a user downloading a rectangular section > > > from OSM (since I'd imagine most of us do that, rather than deal with > > > enormous planet or country files), where a suburb boundary intersects > > > the boundary of the rectangle downloaded: > > > > > > If we use the single way method, the OSM API will give the user the > > > entire suburb boundary, even the bits that are outside the rectangle - > > > so every suburb that has any part of itself within the rectangle will > > > have its boundary fully defined within the user's osm file. > > > > > > If we use the relation method, only those segments of the boundary > > > which have nodes within the rectangle will be supplied - leaving some > > > suburbs with incomplete boundaries in the user's osm file. > > > > If the user doing the download is not prepared to handle the relation > > issue with respect to boundaries they will probably encounter far > > greater problems that just suburb boundaries. Multi-polygon relations > > for example will suffer from exactly the same problem. The issue is > > that the down-loader needs to be aware of the data structure and not > > make the data structure adjust to handle his in-competencies. For > > example in JOSM it's a matter of a 3 clicks to request all the ways of > > boundary. > > > > There are already issues of ways with too many nodes causing > > downloading problems for the OSM servers, a single area for a whole > > rural suburb (or one of the bigger boundaries like a council) is easily > > going to exceed reasonable limits of way length, and unlike a way where > > you have to download the entire way every time it's viewed, with > > relations you can choose to download only the relevant parts, and the > > whole lot if you need it. > > > > Should you happen to not have your download's bounding box cross any > > of the suburb boundaries with either method you may just end up with > > no suburb data at all anyway. Assuming you can rely on suburb data > > from a small are download is a little naive. > > I've been following this discussion, and have changed my mind each post on > whether it should be a single way or a relation :-) > > I think I now "vote" for a relation especially given the stacking issue > Darren > brought up. If used for rivers/roads etc. (which probably has much accuracy > than > consumer grade GPS), the way's are going to have to be divided up anyway > (change > in speed, oneway, surface, roundabouts etc.). So together with the mass > data > issue when viewing an area, relations seems to be the best option. > > > [SNIP] > > How excessive are the extra nodes? I wonder if those nodes have > significance, > like intersections of roads etc. Personally, if at all possible, I'd prefer > to > keep the imported ways as close to the ABS data as possible. > > Has anyone had any ideas on tagging? We have Tiger and other imports as > examples, but I imagine source_ref=ABS is a too general, there's probably > lots > of organisations with ABS as initials. Would need to tag the ABS ID's on > the > ways as well to allow for future updates. > > BlueMM > > > ___ > Talk-au mailing list > Talk-au@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk-au > -- Franc ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Suburb boundaries
Darrin Smith writes: > On Thu, 05 Feb 2009 17:18:53 +1030 > Jack Burton wrote: [SNIP] > > Also, consider the case of a user downloading a rectangular section > > from OSM (since I'd imagine most of us do that, rather than deal with > > enormous planet or country files), where a suburb boundary intersects > > the boundary of the rectangle downloaded: > > > > If we use the single way method, the OSM API will give the user the > > entire suburb boundary, even the bits that are outside the rectangle - > > so every suburb that has any part of itself within the rectangle will > > have its boundary fully defined within the user's osm file. > > > > If we use the relation method, only those segments of the boundary > > which have nodes within the rectangle will be supplied - leaving some > > suburbs with incomplete boundaries in the user's osm file. > > If the user doing the download is not prepared to handle the relation > issue with respect to boundaries they will probably encounter far > greater problems that just suburb boundaries. Multi-polygon relations > for example will suffer from exactly the same problem. The issue is > that the down-loader needs to be aware of the data structure and not > make the data structure adjust to handle his in-competencies. For > example in JOSM it's a matter of a 3 clicks to request all the ways of > boundary. > > There are already issues of ways with too many nodes causing > downloading problems for the OSM servers, a single area for a whole > rural suburb (or one of the bigger boundaries like a council) is easily > going to exceed reasonable limits of way length, and unlike a way where > you have to download the entire way every time it's viewed, with > relations you can choose to download only the relevant parts, and the > whole lot if you need it. > > Should you happen to not have your download's bounding box cross any > of the suburb boundaries with either method you may just end up with > no suburb data at all anyway. Assuming you can rely on suburb data > from a small are download is a little naive. I've been following this discussion, and have changed my mind each post on whether it should be a single way or a relation :-) I think I now "vote" for a relation especially given the stacking issue Darren brought up. If used for rivers/roads etc. (which probably has much accuracy than consumer grade GPS), the way's are going to have to be divided up anyway (change in speed, oneway, surface, roundabouts etc.). So together with the mass data issue when viewing an area, relations seems to be the best option. > [SNIP] How excessive are the extra nodes? I wonder if those nodes have significance, like intersections of roads etc. Personally, if at all possible, I'd prefer to keep the imported ways as close to the ABS data as possible. Has anyone had any ideas on tagging? We have Tiger and other imports as examples, but I imagine source_ref=ABS is a too general, there's probably lots of organisations with ABS as initials. Would need to tag the ABS ID's on the ways as well to allow for future updates. BlueMM ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Suburb boundaries
On Thu, Feb 5, 2009 at 9:23 PM, Cameron wrote: > How much do suburbs change anyway? Perhaps any changes could simply be > introduced manually. > ~Cameron I suspect this is true, changing large numbers of suburbs sounds unlikely. If we had suddenly had a new set of this data (say at the next census) then my first thought would be to just 'diff' the two sets in some way. Of course if they change the the format is supplied in or there are subtle changes in say the signifiant digits or node ordering then the whole thing gets harder. cheers > > > 2009/2/5 Darrin Smith > > On Thu, 05 Feb 2009 20:23:07 +1030 >> Jack Burton wrote: >> >> > Consider two suburbs, A & B, whose boundary is currently defined by a >> > river. Now let's say that by the time the next ABS update occurs, that >> > boundary has changed, and a small part of what used to be suburb A has >> > become part of suburb B (it can happen). Since the ABS data contains >> > only suburb boundaries (and no separate way for the river itself), and >> > we're using multiple segments per boundary, and someone has helpfully >> > merged that boundary segment with the way that forms the river (as I >> > think you suggested earlier, to avoid stacking up ways on top of each >> > other), there'd be no method for the update mechanism to know whether >> > the course of the river itself has changed (and therefore so has the >> > boundary segment, so it should move the way that defines both) or >> > whether the river has stayed where it was but the boundary no longer >> > uses that part of it (so it should split ways, create a new one, then >> > add it to the boundary relation). >> >> This is an automated process, if it can be explain logically the >> computer can be made to do it. As I said before, as soon as any points >> are moved things become complicated anyway. >> >> If I were implementing this part of it (note Franc is only talking about >> a one-time import at this stage anyway, so we are talking somewhat >> theoretically): >> I'd uniquely identify each common boundary between 2 suburbs that we >> make a way. Use a diff mechanism to detect a change on said boundary, >> and look at the data, updating and adjusting a way that hasn't been >> modified at all and removing and replacing the way if it's been >> changed beyond the ability to adjust. >> >> > With a single closed way around each suburb, the problem does not >> > arise, since the update process does not need to care about the river >> > itself (and should be clever enough to detect that another way uses >> > some of the existing nodes, so duplicate those nodes instead of >> > moving them). >> >> You fob it off so simply but there's a lot of work in your solution >> also. Following your example any time a minor change happens to a >> suburb it's likely to re-align every node on the boundary back to the >> original place, in fact it will most likely have to remove & re-add the >> entire way since it won't be sure which nodes are which any more, >> someone could have added more, removed some, etc. You could tag every >> node I guess, but seems a lot of bloat for small gain, and similar >> gains would be made to the relation model with individual tags anyway. >> >> So we have the boundary solution which when a boundary changes only has >> to modify 1 shorter way along the common boundary between the suburbs >> that change or the way solution which most likely requires the whole >> way to be replaced on an update, possibly removing other adjustments >> made on other parts of the way. From this point of view the boundary >> solution requires less far reaching changes than the area solution. >> >> Of course any unique ID is risky anyway because it can be accidentally >> removed, but that's the risk I guess :D >> >> -- >> >> =b >> >> ___ >> Talk-au mailing list >> Talk-au@openstreetmap.org >> http://lists.openstreetmap.org/listinfo/talk-au >> > > > ___ > Talk-au mailing list > Talk-au@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk-au > > -- Franc ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Suburb boundaries
On Thu, 2009-02-05 at 21:16 +1030, Darrin Smith wrote: > Futher to this I was looking back through this thread (thinking maybe > about having a look at the data myself) and I James said: > > It's described as "These boundaries have been based upon localities > gazetted by the Geographic Place name authority current at the time of > the Census." > > So it's always going to be out-of-date anyway and updated every 4 > years, it's not going to change often. But it's a much better start > than what we have now :) Very good point. I agree now that there's not much point trying to automate updates as the ABS dataset changes - that could be accomplished faster by just manually updating changes as & when gazzetted (so long as someone keeps an eye on each Gazette, but that shouldn't be a problem spread across all the Aussie mappers) [Anyone know what the status is of the copyright (if any) on the Gazette itself?]. I still prefer the idea of areas rather than relations for suburb boundaries in general, for a whole bunch of reasons states earlier (which I won't bore the list with again), but I guess that's just something we'll probably always have differing opinions on... Regards, Jack. ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Suburb boundaries
On Thu, 5 Feb 2009 21:12:31 +1030 Darrin Smith wrote: > On Thu, 5 Feb 2009 20:53:01 +1030 > Cameron wrote: > > > How much do suburbs change anyway? Perhaps any changes could simply > > be introduced manually. > > ~Cameron > > Yeah I did think that might be an easier solution, I was addressing > automatic updates because jackb brought them up :) And as soon as we > modify them in any way (align with road for example) I think it > probably comes out the more appealing however we put it in initially. > > I know for example the SA Gazette tends to provide information about > suburb boundary changes, probably before they make into the ABS > structures. I imagine the other states have similar channels. > Futher to this I was looking back through this thread (thinking maybe about having a look at the data myself) and I James said: It's described as "These boundaries have been based upon localities gazetted by the Geographic Place name authority current at the time of the Census." So it's always going to be out-of-date anyway and updated every 4 years, it's not going to change often. But it's a much better start than what we have now :) -- =b ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Suburb boundaries
On Thu, 5 Feb 2009 20:53:01 +1030 Cameron wrote: > How much do suburbs change anyway? Perhaps any changes could simply be > introduced manually. > ~Cameron Yeah I did think that might be an easier solution, I was addressing automatic updates because jackb brought them up :) And as soon as we modify them in any way (align with road for example) I think it probably comes out the more appealing however we put it in initially. I know for example the SA Gazette tends to provide information about suburb boundary changes, probably before they make into the ABS structures. I imagine the other states have similar channels. -- =b ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Suburb boundaries
How much do suburbs change anyway? Perhaps any changes could simply be introduced manually. ~Cameron 2009/2/5 Darrin Smith > On Thu, 05 Feb 2009 20:23:07 +1030 > Jack Burton wrote: > > > Consider two suburbs, A & B, whose boundary is currently defined by a > > river. Now let's say that by the time the next ABS update occurs, that > > boundary has changed, and a small part of what used to be suburb A has > > become part of suburb B (it can happen). Since the ABS data contains > > only suburb boundaries (and no separate way for the river itself), and > > we're using multiple segments per boundary, and someone has helpfully > > merged that boundary segment with the way that forms the river (as I > > think you suggested earlier, to avoid stacking up ways on top of each > > other), there'd be no method for the update mechanism to know whether > > the course of the river itself has changed (and therefore so has the > > boundary segment, so it should move the way that defines both) or > > whether the river has stayed where it was but the boundary no longer > > uses that part of it (so it should split ways, create a new one, then > > add it to the boundary relation). > > This is an automated process, if it can be explain logically the > computer can be made to do it. As I said before, as soon as any points > are moved things become complicated anyway. > > If I were implementing this part of it (note Franc is only talking about > a one-time import at this stage anyway, so we are talking somewhat > theoretically): > I'd uniquely identify each common boundary between 2 suburbs that we > make a way. Use a diff mechanism to detect a change on said boundary, > and look at the data, updating and adjusting a way that hasn't been > modified at all and removing and replacing the way if it's been > changed beyond the ability to adjust. > > > With a single closed way around each suburb, the problem does not > > arise, since the update process does not need to care about the river > > itself (and should be clever enough to detect that another way uses > > some of the existing nodes, so duplicate those nodes instead of > > moving them). > > You fob it off so simply but there's a lot of work in your solution > also. Following your example any time a minor change happens to a > suburb it's likely to re-align every node on the boundary back to the > original place, in fact it will most likely have to remove & re-add the > entire way since it won't be sure which nodes are which any more, > someone could have added more, removed some, etc. You could tag every > node I guess, but seems a lot of bloat for small gain, and similar > gains would be made to the relation model with individual tags anyway. > > So we have the boundary solution which when a boundary changes only has > to modify 1 shorter way along the common boundary between the suburbs > that change or the way solution which most likely requires the whole > way to be replaced on an update, possibly removing other adjustments > made on other parts of the way. From this point of view the boundary > solution requires less far reaching changes than the area solution. > > Of course any unique ID is risky anyway because it can be accidentally > removed, but that's the risk I guess :D > > -- > > =b > > ___ > Talk-au mailing list > Talk-au@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk-au > ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Suburb boundaries
On Thu, 05 Feb 2009 20:23:07 +1030 Jack Burton wrote: > Consider two suburbs, A & B, whose boundary is currently defined by a > river. Now let's say that by the time the next ABS update occurs, that > boundary has changed, and a small part of what used to be suburb A has > become part of suburb B (it can happen). Since the ABS data contains > only suburb boundaries (and no separate way for the river itself), and > we're using multiple segments per boundary, and someone has helpfully > merged that boundary segment with the way that forms the river (as I > think you suggested earlier, to avoid stacking up ways on top of each > other), there'd be no method for the update mechanism to know whether > the course of the river itself has changed (and therefore so has the > boundary segment, so it should move the way that defines both) or > whether the river has stayed where it was but the boundary no longer > uses that part of it (so it should split ways, create a new one, then > add it to the boundary relation). This is an automated process, if it can be explain logically the computer can be made to do it. As I said before, as soon as any points are moved things become complicated anyway. If I were implementing this part of it (note Franc is only talking about a one-time import at this stage anyway, so we are talking somewhat theoretically): I'd uniquely identify each common boundary between 2 suburbs that we make a way. Use a diff mechanism to detect a change on said boundary, and look at the data, updating and adjusting a way that hasn't been modified at all and removing and replacing the way if it's been changed beyond the ability to adjust. > With a single closed way around each suburb, the problem does not > arise, since the update process does not need to care about the river > itself (and should be clever enough to detect that another way uses > some of the existing nodes, so duplicate those nodes instead of > moving them). You fob it off so simply but there's a lot of work in your solution also. Following your example any time a minor change happens to a suburb it's likely to re-align every node on the boundary back to the original place, in fact it will most likely have to remove & re-add the entire way since it won't be sure which nodes are which any more, someone could have added more, removed some, etc. You could tag every node I guess, but seems a lot of bloat for small gain, and similar gains would be made to the relation model with individual tags anyway. So we have the boundary solution which when a boundary changes only has to modify 1 shorter way along the common boundary between the suburbs that change or the way solution which most likely requires the whole way to be replaced on an update, possibly removing other adjustments made on other parts of the way. From this point of view the boundary solution requires less far reaching changes than the area solution. Of course any unique ID is risky anyway because it can be accidentally removed, but that's the risk I guess :D -- =b ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Suburb boundaries
On Thu, 2009-02-05 at 17:04 +1030, Darrin Smith wrote: > On Thu, 05 Feb 2009 16:29:39 +1030 > Jack Burton wrote: > > > >1. What way do we want to represent the data, e.g closed ways or > > > relations consisting of borders - something else ? > > > > Closed ways (areas) - as that's how ABS define them, so it will make > > merging updated ABS data into the OSM Australia dataset (each time ABS > > update their dataset, which is presumably quite regularly) > > significantly easier. > > This isn't really relevant. Given the amount of data involved an > automated process will have to be developed to bring it all in, so this > process can just be re-utilised on any update. Very true. But a one-to-one mapping between OSM ways and ABS boundaries (accompanied by some means of identifying that mapping uniquely, like a source_ref:ABS=some_unique_id_from_ABS_dataset tag or similar) would make writing the automated update tool a significantly simpler task (e.g.: use a diff of the old cf. new ABS datasets as your input file, search for corresponding OSM ways by tag, then add/remove/reshape/rename as appropriate). Yes, you can still do that if there's a one-to-many mapping, but it's not quite so simple. And there would be lots more stuff the update process would need to check for (particularly if boundary segments are shared by other, non-place-related tags) before removing/reshaping a boundary during an update. For example: Consider two suburbs, A & B, whose boundary is currently defined by a river. Now let's say that by the time the next ABS update occurs, that boundary has changed, and a small part of what used to be suburb A has become part of suburb B (it can happen). Since the ABS data contains only suburb boundaries (and no separate way for the river itself), and we're using multiple segments per boundary, and someone has helpfully merged that boundary segment with the way that forms the river (as I think you suggested earlier, to avoid stacking up ways on top of each other), there'd be no method for the update mechanism to know whether the course of the river itself has changed (and therefore so has the boundary segment, so it should move the way that defines both) or whether the river has stayed where it was but the boundary no longer uses that part of it (so it should split ways, create a new one, then add it to the boundary relation). With a single closed way around each suburb, the problem does not arise, since the update process does not need to care about the river itself (and should be clever enough to detect that another way uses some of the existing nodes, so duplicate those nodes instead of moving them). Regards, Jack. ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Suburb boundaries
On Thu, 5 Feb 2009 08:19:57 + (UTC) BlueMM wrote: > Apparently there was a big push for unification of suburb & postcode > boundaries a few years back by the governmental spatial agencies. I > believe it hasn't been completed, parts of NT didn't correspond. Yeah I know it all lines up here in SA (dont know if it ever hasn't actually) but I wasn't sure about other states. Interesting to note some places it doesnt. > Given the electoral boundaries, it shows the limits of the > admin_level tagging system, as Australia is like the UK, where it's > more of a hierarchy (think different areas like > councils/suburbs/postcodes/parishes/electoral/state). I think there > are even examples in Australia of suburbs crossing state lines. I think the admin_level setup is more in the idea of international consistency, so similar types of areas with radically different local name can be at similar admin levels. Of course I only found out recently not only does S.A. have Hundreds as a land administration boundary but they also have Counties. Of course sourcing that information for a free source could be extremely tricky :D -- =b ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Suburb boundaries
[snip] > > Apparently there was a big push for unification of suburb & postcode > boundaries > a few years back by the governmental spatial agencies. I believe it hasn't > been > completed, parts of NT didn't correspond. > > Given the electoral boundaries, it shows the limits of the admin_level > tagging > system, as Australia is like the UK, where it's more of a hierarchy (think > different areas like councils/suburbs/postcodes/parishes/electoral/state). > I > think there are even examples in Australia of suburbs crossing state lines. Yep, this is documented in the ABS data set. > > > BTW, I love the fact we can finally use this data, but is it all ok > copyright > wise? I remember reading that is costs an arm & a leg to get postcode > boundaries > from Australia post, and here we are getting it for free from the ABS & can > be > used for commercial purposes. Wonder if it is because of some law that > states > all ABS data is free, even if it comes from behind non-open doors (like > Geoscience Australia). Hope my gut feeling is just what I had for lunch :P I am working on the basis that the ABS have ok'd the usage (in writing) and when I spoke to the lady at the ABS I was confident she had a thorough understanding of the issues. So, I think we can legitimately claim that we acted in good faith. And proper tagging should allow us to remove said data in the worst case. > > > BlueMM > > > ___ > Talk-au mailing list > Talk-au@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk-au > -- Franc ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
Re: [talk-au] Suburb boundaries
Darrin Smith writes: > On Thu, 5 Feb 2009 18:30:58 +1100 > Franc Carter wrote: > > I have added an entry to the data catalogue at > > > >http://wiki.openstreetmap.org/wiki/Import/Catalogue#One-Time_Imports > > > > and the beginnings of page about the import at > > > > Post codes and Electorals Boundaries map up nicely to a boundary > relation (especially given post codes align with suburb boundaries as > far as I know), however they will really test the limits of the area > model since they will be so much larger, and have many more points. > > I was ponder about electoral boundaries a while back, and realised they > probably fall just about the 'council' level and should therefor have a > admin_level of 5. But lets worry about the suburbs first ;) Apparently there was a big push for unification of suburb & postcode boundaries a few years back by the governmental spatial agencies. I believe it hasn't been completed, parts of NT didn't correspond. Given the electoral boundaries, it shows the limits of the admin_level tagging system, as Australia is like the UK, where it's more of a hierarchy (think different areas like councils/suburbs/postcodes/parishes/electoral/state). I think there are even examples in Australia of suburbs crossing state lines. BTW, I love the fact we can finally use this data, but is it all ok copyright wise? I remember reading that is costs an arm & a leg to get postcode boundaries from Australia post, and here we are getting it for free from the ABS & can be used for commercial purposes. Wonder if it is because of some law that states all ABS data is free, even if it comes from behind non-open doors (like Geoscience Australia). Hope my gut feeling is just what I had for lunch :P BlueMM ___ Talk-au mailing list Talk-au@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-au
