[OSM-talk] Attic data on Overpass API
Dear all, the attic feature of Overpass API should now work properly. An example how to use this feature is http://www.openstreetmap.org/user/ikonor/diary/23329 A big thank you to Ikonor at this point. In detail, the database has been rebuilt. I tried to do as much checks as possible, and this has shown further bugs, which in turn led me to fix this in the code and then re-build the database again. A thank you to Stephan and Markus who managed to obtain a temporary powerful server; this ultimately enabled to do the last database rebuild within less than a week. I will give details about the kind of bugs that kept me busy: To keep the database as small as possible (currently more than 400 GB are painful, compare this to 25 GB of a PBF planet file), we store attic data as delta to the next newer version. Unchanged details like not changed tags aren't stored at all. For example, the tags of are stored as current: "name" = "something else" current: "highway" = "bus_stop" current: "bus" = "yes" current: "shelter" = "yes" attic: before 2012: "name" = "something" attic: before 2012: "FIXME" = "check_name" attic: before 2012: "shelter" = void This allows us to save space for the repeated tags "highway" and "bus". The main traces of the bug were checksum disparities in five of the 6000 checked first augmented diffs, spanning roughly 12 September to 16 September 2012. In detail, the diffs generated from the database state as of 1st October 2012 (thus, representing deltas to the then-current state of 1st October 2012) were not consistent with the diffs generated for the same minutes based on deltas to the database state as of June 2014. A detailed re-generation of the augmented diffs of both database states has shown that an extra tag was present on the node 1700083447 in its attic state of September 2012 computed from June 2014 in comparison to the same node at the same attic state computed from the database as of October 2012. Do you have guessed what has gone wrong? Me not so far, so I had to understand the subtle details. There are millions of nodes carrying tags, so what is so special about this one? It turned out that the node has moved forth and back over a significant distance, see versions 11, 14, and 15. While the movement itself is not so large (about 2 km), it happened to change its quadtile index to a new value and then back to the old value. And in version 13, after the move forth, the tag is_capital=country has been added and not changed when the node moved back. I managed in the delta computation to set a marker on the quadtile index of the older position that the tag is present, but I forgot to set a marker on the new position that the tag isn't present on earlier versions on this quadtile index. Now: Why didn't this pop up earlier? Haven't there been tests? There have been tests, but obviously not enough, and you always only know afterwards which tests have been missing. The whole problem doesn't appear to a node when the node never existed at this place before (only 3 in a million of nodes ever get back to a quadtile index where they have been before). And it doesn't appear either when the tag had any value on this older node versions (so the total number of affected nodes is less than 100), because there is then a marker for this older version and this specific key. Both cases have been tested individually, but not both together. In total: I've worked to get the number of bugs down, and I'm confident to call the database state now consistent, but there might be other arcane bugs that affect only few objects in specific versions. So please be bold to report suspect query results to me, but be also bold in using the new attic database. Cheers, Roland ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wiki: Removing Content from .../User:xxx/Kosmos Pages
Hi Andreas, I'm the author of Kosmos. It doesn't really work with the new OSM data (64 bit IDs etc) and I haven't received any user support email for at least a couple of years, so it's safe to say no one uses it anymore. Maperitive replaced all of its functionality long time ago. So if you feel the need to clean up the wiki pages, please don't hesitate to do so. And thanks for the effort. Best regards, Igor Brejc On Fri, Aug 1, 2014 at 4:19 PM, Andreas Goss wrote: > I'm trying to clean up the Wiki and frequently use "What links here", > especially when it's not clear how a tag I used and I suspect different > definitions on different pages. > > Usually that works great and I get like 5-15 results, but for some tags I > get like 100+, because they are linked to Kosmos rules user pages. On the > Kosmos Wiki page (http://wiki.openstreetmap.org/wiki/Kosmos) it says: > > > Kosmos is no longer actively maintained, since it has been replaced > by > a new tool called Maperitive. The new tool is much more flexible > and > powerful, so the author recommends switching to it. > > Is anybody still using the Kosmos software? > > The pages in question: > https://www.google.com/#q=site:wiki.openstreetmap.org% > 2Fwiki%2FUser:+inurl:Kosmos > > Would it be alright to remove the whole content and replace it with the > same banner as on the Kosmos Wiki page? Is there a better solution? > > Here is one example where it is really annoying: > http://wiki.openstreetmap.org/w/index.php?title=Special: > WhatLinksHere/Tag:leisure%3Dcommon&limit=500 > > Andi > __ > openstreetmap.org/user/AndiG88 > wiki.openstreetmap.org/wiki/User:AndiG88 > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wiki: Removing Content from .../User:xxx/Kosmos Pages
> Is anybody still using the Kosmos software? It seems I had updated at least the mono version of the rules already, so have now deleted all my Kosmos rules from the wiki. Ed ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wiki: Removing Content from .../User:xxx/Kosmos Pages
This doesn't actually solve the problem, but you can filter the search results by searching only on certain namespaces, such as the namespace "(Main)" and "Talk". This way, at least user pages wouldn't show as a result, which is where most Kosmos rendering rules appear. > Would it be alright to remove the whole content and replace it with the > same banner as on the Kosmos Wiki page? Is there a better solution? If there was a template similar to [1], I believe that would be the best solution. But I don't know much about MediaWiki, so I'm not sure how the process of adding this template could be made easier. [1]: http://wiki.openstreetmap.org/wiki/Template:Archived_proposal [2]: http://wiki.openstreetmap.org/w/index.php?title=Special%3AWhatLinksHere&limit=500&target=Tag%3Aleisure%3Dcommon&namespace=0 [3]: http://wiki.openstreetmap.org/w/index.php?title=Special%3AWhatLinksHere&limit=500&target=Tag%3Aleisure%3Dcommon&namespace=1 2014-08-01 11:19 GMT-03:00 Andreas Goss : > I'm trying to clean up the Wiki and frequently use "What links here", > especially when it's not clear how a tag I used and I suspect different > definitions on different pages. > > Usually that works great and I get like 5-15 results, but for some tags I > get like 100+, because they are linked to Kosmos rules user pages. On the > Kosmos Wiki page (http://wiki.openstreetmap.org/wiki/Kosmos) it says: > > > Kosmos is no longer actively maintained, since it has been replaced > by > a new tool called Maperitive. The new tool is much more flexible > and > powerful, so the author recommends switching to it. > > Is anybody still using the Kosmos software? > > The pages in question: > https://www.google.com/#q=site:wiki.openstreetmap.org% > 2Fwiki%2FUser:+inurl:Kosmos > > Would it be alright to remove the whole content and replace it with the > same banner as on the Kosmos Wiki page? Is there a better solution? > > Here is one example where it is really annoying: > http://wiki.openstreetmap.org/w/index.php?title=Special: > WhatLinksHere/Tag:leisure%3Dcommon&limit=500 > > Andi > __ > openstreetmap.org/user/AndiG88 > wiki.openstreetmap.org/wiki/User:AndiG88 > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Wiki: Removing Content from .../User:xxx/Kosmos Pages
> Is anybody still using the Kosmos software? Your question prompted me to run Kosmos, as I still have it installed as well as Maperitive and I wasn't sure why. It seems I haven't yet upgraded the rules I used to create the Beer Festival map I did for station to venue which uses http://wiki.openstreetmap.org/wiki/User:EdLoach/KosmosCBF2009mono and http://wiki.openstreetmap.org/wiki/User:EdLoach/KosmosCBF2009 My third rule page is boundaries only and I'll delete it. I'll try and upgrade the above rules then delete them too (then uninstall Kosmos). I have done mono rules for another beer fest in Maperitive since, so these might do. In summary, I don't use it very often as the maps I generated with it don’t need updating very often, but at the moment it does what I need without the extra work of upgrading the rules. Ed ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Wiki: Removing Content from .../User:xxx/Kosmos Pages
I'm trying to clean up the Wiki and frequently use "What links here", especially when it's not clear how a tag I used and I suspect different definitions on different pages. Usually that works great and I get like 5-15 results, but for some tags I get like 100+, because they are linked to Kosmos rules user pages. On the Kosmos Wiki page (http://wiki.openstreetmap.org/wiki/Kosmos) it says: > Kosmos is no longer actively maintained, since it has been replaced > by a new tool called Maperitive. The new tool is much more flexible > and powerful, so the author recommends switching to it. Is anybody still using the Kosmos software? The pages in question: https://www.google.com/#q=site:wiki.openstreetmap.org%2Fwiki%2FUser:+inurl:Kosmos Would it be alright to remove the whole content and replace it with the same banner as on the Kosmos Wiki page? Is there a better solution? Here is one example where it is really annoying: http://wiki.openstreetmap.org/w/index.php?title=Special:WhatLinksHere/Tag:leisure%3Dcommon&limit=500 Andi __ openstreetmap.org/user/AndiG88 wiki.openstreetmap.org/wiki/User:AndiG88 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
