Re: [OSM-talk] [OSM-dev] Abstract on getting historic information about usage of a node
Hopefully you don't kill me right off but I re-created this Testcase in the OSM db. I only used a comment tag on each element so they won't get rendered and are identifiable in the history as Testcase later. I also created them near my home where I'm the only active mapper I now of. == Testcase == 1. In Changeset #2502021 I created Way #40896886, v1 with it's five nodes #497779713, v1 #497779710, v1 #497779707, v1 #497779704, v1 #497779695, v1 2. In Changeset #2502042 I moved node #497779713, v2 3. In Changeset #2502049 I deleted node #497779713, v3, thus modifying way #40896886, v2 4. In Changeset #2502062 I deleted the rest. == Issues == When I now lookt at the 2nd changeset I only see node #497779713 and I'm umable to find out if a way was modified at all. This is issue #1. Once I know the ID of the way (e.g. in the 1st, 3rd and 4th Changeset) and the time it was modified (also from the Changeset) The Client is able to reconstruct the geometry of the way by fetching the history of all it's nodes and comparing the timestamps in the Client. This does not scale very well - issue #2. == Solutions, Client perspective == For issue #1: a new API call: GET /api/0.6/node/#id/#version/ways For issue #2: either a new API call: GET /api/0.6/way/#id/#version/full or extending the nd-tags of following calls with version numbers: GET /api/0.6/way/#id GET /api/0.6/way/#id/#version GET /api/0.6/way/#id/history == Solutions, Server perspective == Either implement the new call with on-demand searching on a Timestamp basis, as Tom explains in [0], or extend the way_nodes table somehow, as explained by Ian in [1], thus making the needed calls less expensive. This is true for both issues. == Decisions == The Choice that is up to be made now is between the following three possibilities a) do not implement these calls b) implement them on demand c) implement them with a new column/index == Impacts == a) without these calls it's not possible to do intelligent diffs from changesets, as they would be needed to do high-quality reverts with dirty data. We need such revert-tools to encounter upcoming vandalism (which we'll face with rising popularity. b) this will cost CPU time on the API server, but regarding issue 2 it would possibly be better than a bunch of history requests. c) this will cost disk-space and memory for the index. The CPU time will be consumed only once. == further proceedings == First the maintainers of the API have to decide whih way to go. Ian already volunteered for implementing way c (see end of [1]). Peter [0] http://bit.ly/139XK1 (http://lists.openstreetmap.org/...) [1] http://bit.ly/ozrBN (http://lists.openstreetmap.org/...) ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] [OSM-dev] Abstract on getting historic information about usage of a node
On 16/09/09 15:17, Peter Körner wrote: == Decisions == The Choice that is up to be made now is between the following three possibilities a) do not implement these calls b) implement them on demand c) implement them with a new column/index [ snipped ] c) this will cost disk-space and memory for the index. The CPU time will be consumed only once. This is NOT true. As I explained yesterday there is a very good reason why we do not store a node revision in current_way_nodes and way_nodes. The problem is that if you do that then every time a node is changed you have to go and find every way which uses that node and update it to use the new version of the node. So it costs CPU time every time a node is changed. Tom -- Tom Hughes (t...@compton.nu) http://www.compton.nu/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] [OSM-dev] Abstract on getting historic information about usage of a node
On Wed, Sep 16, 2009 at 9:30 AM, Tom Hughes t...@compton.nu wrote: This is NOT true. As I explained yesterday there is a very good reason why we do not store a node revision in current_way_nodes and way_nodes. The problem is that if you do that then every time a node is changed you have to go and find every way which uses that node and update it to use the new version of the node. So it costs CPU time every time a node is changed. Can a way assume that it uses the most recent version of a node until the way is changed? Then, when a way's revision number increments, all of the rev#'s for the nodes that make up that way are saved along with the way's history. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] [OSM-dev] Abstract on getting historic information about usage of a node
c) this will cost disk-space and memory for the index. The CPU time will be consumed only once. This is NOT true. As I explained yesterday there is a very good reason why we do not store a node revision in current_way_nodes and way_nodes. I'm sorry I should have mentioned that all the points from Solutions, Server perspective down are what I've collected from the various posts. I'm not familiar with the RailsPort and so I can't really tell about the costs. But never the less we have to talk about how we could get the information (either from the API or from any other source) as without this it won't be possible to write a revert-tool that actually can revert dirty Changesets (or better: that can revert *all* dirty Changesets). I'm sorry if this sounds like an attack to you. I know it's not the APIs job to support weird requests of all kinds, but especially on issue #1 there is *no other way* to get this information. What I'm asking myself is, if serving such requests will really have such a large impact on server performance. I'm really sure there won't be too many hits on these special (history only) calls, do you? Peter ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] [OSM-dev] Abstract on getting historic information about usage of a node
Ian Dees schrieb: On Wed, Sep 16, 2009 at 9:30 AM, Tom Hughes t...@compton.nu mailto:t...@compton.nu wrote: This is NOT true. As I explained yesterday there is a very good reason why we do not store a node revision in current_way_nodes and way_nodes. The problem is that if you do that then every time a node is changed you have to go and find every way which uses that node and update it to use the new version of the node. So it costs CPU time every time a node is changed. Can a way assume that it uses the most recent version of a node until the way is changed? Then, when a way's revision number increments, all of the rev#'s for the nodes that make up that way are saved along with the way's history. But this wouldn't solve the problem of give me all ways that used node #50, v3 as the last time way #10 was updated it only used node #50, v1 and so that's the last information that's saved. Peter ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] [OSM-dev] Abstract on getting historic information about usage of a node
What's the problem with that? If way 10 did not ever use node 50v3, then it should not be included in the give me all ways that use node 50v3 call, right? (semi-ASCII-art ahead, arm your fixed-width photos) Here's way 10v1 (numbers are nodes): 50v151v152v2 ...and a history table like this: way_id way_rev node_id node_rev 10 1501 10 1511 10 1522 I come along and move node 52, causing a rev on the way and a rev on the node: And that's the point: atm. moving a node does not increase a ways version. The data stored for a way is not it's geometry but only the nodes it consists of. So changing the geometry of a way does not change it's version, removing or adding a node to it does. So when a node is moved a new node-id/node-ver-pair springs into live but its the ways are not updated. Peter ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] [OSM-dev] Abstract on getting historic information about usage of a node
On Wed, Sep 16, 2009 at 2:19 PM, Peter Körner osm-li...@mazdermind.dewrote: What's the problem with that? If way 10 did not ever use node 50v3, then it should not be included in the give me all ways that use node 50v3 call, right? (semi-ASCII-art ahead, arm your fixed-width photos) Here's way 10v1 (numbers are nodes): 50v151v152v2 ...and a history table like this: way_id way_rev node_id node_rev 10 1501 10 1511 10 1522 I come along and move node 52, causing a rev on the way and a rev on the node: And that's the point: atm. moving a node does not increase a ways version. The data stored for a way is not it's geometry but only the nodes it consists of. So changing the geometry of a way does not change it's version, removing or adding a node to it does. So when a node is moved a new node-id/node-ver-pair springs into live but its the ways are not updated. And Tom's good point is that if we were to track that information, we would have to insert W new rows into my fictitious history table where W is the number of ways that contain the node every time we change the node. How much slower would that make the node change code? I see (in my naive SQL knowledge) 1+W SQL calls (1 being a SELECT to find the way_id's and W being one INSERT into my history table for every way containing that node). That doesn't seem TOO bad... of course, node manipulation is the most prevalent call in the API I'm sure. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
