Re: [OSM-talk] minutely update DB
Le mardi 13 novembre 2012 00:11:26, ciprian niculescu a écrit : On Mon, Nov 12, 2012 at 2:21 PM, sly (sylvain letuffe) li...@letuffe.orgwrote: On lundi 12 novembre 2012, ciprian niculescu wrote: Hello, Hi, Medium size answer : It does exists for some areas as christian pointed to. And a process exists to generate those, still, what is missing is someone to run it on a (several) server(s) and provide minutely diffs extracts. nice, and the procedure is? :) I'm not the one having that in charge, but Jocelyn is (in copy) Looks like (but only looks like) code is published here : https://github.com/jocelynj/osm/tree/master/osc_modif But I don't know where the osmbin manipulation library (needed) is -- sly (sylvain letuffe) ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] minutely update DB
Hello, I'm wondering why it don't exist the possibility to have minutely updates (or hourly) for a country? I don't know what implies to make it. As far as i understand OSM to host/distribute hourly diff for country, it's something like: - do a minutely/hourly updates for the planet = result the planet.osm - do a split of the planet.osm on the polygons of the country - do a diff between the old-osm-contry and the new one = results the hourlly diff for the country Is this process wright? Can a filtering by country on the planet diff be applied? This way the split is faster to get the diff_by_country. Ciprian On Mon, Nov 5, 2012 at 9:54 AM, Christian Quest cqu...@openstreetmap.frwrote: If you want to maintain a Europe extracted updated, have a look at the Europe diffs available here: http://download.openstreetmap.fr/replication/europe/minute/ 2012/11/4 ciprian niculescu cnicu...@gmail.com: Hello, I need to have a hourly or minutely up-to date DB of OSM. I found infos on how to do it, but one thing is not clear: I must keep the hole OSM DB uptodate and after extract the region i need? I need only Europe. Thanks for clarifying this for me. Ciprian ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk -- Christian Quest - OpenStreetMap France - http://openstreetmap.fr/u/cquest ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] minutely update DB
I've asked the french tech team to have more details on how they are produced. I don't think it is done the way you describe because it takes a lot of time (more than a minute) to merge/split/diff. 2012/11/12 ciprian niculescu cnicu...@gmail.com Hello, I'm wondering why it don't exist the possibility to have minutely updates (or hourly) for a country? I don't know what implies to make it. As far as i understand OSM to host/distribute hourly diff for country, it's something like: - do a minutely/hourly updates for the planet = result the planet.osm - do a split of the planet.osm on the polygons of the country - do a diff between the old-osm-contry and the new one = results the hourlly diff for the country Is this process wright? Can a filtering by country on the planet diff be applied? This way the split is faster to get the diff_by_country. Ciprian On Mon, Nov 5, 2012 at 9:54 AM, Christian Quest cqu...@openstreetmap.frwrote: If you want to maintain a Europe extracted updated, have a look at the Europe diffs available here: http://download.openstreetmap.fr/replication/europe/minute/ 2012/11/4 ciprian niculescu cnicu...@gmail.com: Hello, I need to have a hourly or minutely up-to date DB of OSM. I found infos on how to do it, but one thing is not clear: I must keep the hole OSM DB uptodate and after extract the region i need? I need only Europe. Thanks for clarifying this for me. Ciprian ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk -- Christian Quest - OpenStreetMap France - http://openstreetmap.fr/u/cquest -- Christian Quest - OpenStreetMap France - http://openstreetmap.fr/u/cquesthttp://openstreetmap.fr/u/christian-quest ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] minutely update DB
On lundi 12 novembre 2012, ciprian niculescu wrote: Hello, Hi, I'm wondering why it don't exist the possibility to have minutely updates (or hourly) for a country? Short answer : because no one bothered to provide that for all countries. Medium size answer : It does exists for some areas as christian pointed to. And a process exists to generate those, still, what is missing is someone to run it on a (several) server(s) and provide minutely diffs extracts. Longer answer : The motivation might be low to provide such a service, low to keep pushing forward that way of doing because it isn't really flexible (You might need country X, but someone else will need country Y and some other will need an arbitrary area) while there are other ways to achieve the same result, for example using http://wiki.openstreetmap.org/wiki/Overpass_API/Augmented_Diffs to split diffs at consumer side. Tought, there isn't (AFAIK) tools to handle that now. I don't know what implies to make it. As far as i understand OSM to host/distribute hourly diff for country, it's something like: - do a minutely/hourly updates for the planet = result the planet.osm - do a split of the planet.osm on the polygons of the country - do a diff between the old-osm-contry and the new one = results the hourlly diff for the country Is this process wright? That's one possible workflow for such a process (working directly on planet.osm files however won't go anywhere due to time needed to process the planet file IMHO, but storing it into some more indexed format, why not) The other workflow would be to rely on special diffs (like Augmented_Diffs) and split those diffs according to your needs. The generation of planet Augmented Diffs beeing dedicated to some servers while consumers splits at client side. -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] minutely update DB
Hi. One can make a .osm planet extract and use osmosis to keep the area updated. It should only apply the changes which touch nodes, ways and relations which are inside that particular area. I do that for Iceland with the following commands: /path/to/binary/of/osmosis -q 5 --rri --simc --rx iceland.osm --ac --bp iceland.poly clipIncompleteEntities=yes --wx iceland2.osm; mv iceland2.osm iceland.osm - Svavar On 12/11/12 12:29, ciprian niculescu wrote: Hello, I'm wondering why it don't exist the possibility to have minutely updates (or hourly) for a country? I don't know what implies to make it. As far as i understand OSM to host/distribute hourly diff for country, it's something like: - do a minutely/hourly updates for the planet = result the planet.osm - do a split of the planet.osm on the polygons of the country - do a diff between the old-osm-contry and the new one = results the hourlly diff for the country Is this process wright? Can a filtering by country on the planet diff be applied? This way the split is faster to get the diff_by_country. Ciprian On Mon, Nov 5, 2012 at 9:54 AM, Christian Quest cqu...@openstreetmap.fr mailto:cqu...@openstreetmap.fr wrote: If you want to maintain a Europe extracted updated, have a look at the Europe diffs available here: http://download.openstreetmap.fr/replication/europe/minute/ 2012/11/4 ciprian niculescu cnicu...@gmail.com mailto:cnicu...@gmail.com: Hello, I need to have a hourly or minutely up-to date DB of OSM. I found infos on how to do it, but one thing is not clear: I must keep the hole OSM DB uptodate and after extract the region i need? I need only Europe. Thanks for clarifying this for me. Ciprian ___ talk mailing list talk@openstreetmap.org mailto:talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk -- Christian Quest - OpenStreetMap France - http://openstreetmap.fr/u/cquest ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk signature.asc Description: OpenPGP digital signature ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] minutely update DB
On Mon, Nov 12, 2012 at 5:31 PM, Svavar Kjarrval sva...@kjarrval.is wrote: Hi. Hi, One can make a .osm planet extract and use osmosis to keep the area updated. It should only apply the changes which touch nodes, ways and relations which are inside that particular area. I do that for Iceland with the following commands: /path/to/binary/of/osmosis -q 5 --rri --simc --rx iceland.osm --ac --bp iceland.poly clipIncompleteEntities=yes --wx iceland2.osm; mv iceland2.osm iceland.osm I don't full understand your command, can you explain? Thanks Ciprian - Svavar On 12/11/12 12:29, ciprian niculescu wrote: Hello, I'm wondering why it don't exist the possibility to have minutely updates (or hourly) for a country? I don't know what implies to make it. As far as i understand OSM to host/distribute hourly diff for country, it's something like: - do a minutely/hourly updates for the planet = result the planet.osm - do a split of the planet.osm on the polygons of the country - do a diff between the old-osm-contry and the new one = results the hourlly diff for the country Is this process wright? Can a filtering by country on the planet diff be applied? This way the split is faster to get the diff_by_country. Ciprian On Mon, Nov 5, 2012 at 9:54 AM, Christian Quest cqu...@openstreetmap.frwrote: If you want to maintain a Europe extracted updated, have a look at the Europe diffs available here: http://download.openstreetmap.fr/replication/europe/minute/ 2012/11/4 ciprian niculescu cnicu...@gmail.com: Hello, I need to have a hourly or minutely up-to date DB of OSM. I found infos on how to do it, but one thing is not clear: I must keep the hole OSM DB uptodate and after extract the region i need? I need only Europe. Thanks for clarifying this for me. Ciprian ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk -- Christian Quest - OpenStreetMap France - http://openstreetmap.fr/u/cquest ___ talk mailing listtalk@openstreetmap.orghttp://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] minutely update DB
On Mon, Nov 12, 2012 at 2:21 PM, sly (sylvain letuffe) li...@letuffe.orgwrote: On lundi 12 novembre 2012, ciprian niculescu wrote: Hello, Hi, Hi, I'm wondering why it don't exist the possibility to have minutely updates (or hourly) for a country? Short answer : because no one bothered to provide that for all countries. :) uuu not good Medium size answer : It does exists for some areas as christian pointed to. And a process exists to generate those, still, what is missing is someone to run it on a (several) server(s) and provide minutely diffs extracts. nice, and the procedure is? :) Longer answer : The motivation might be low to provide such a service, low to keep pushing forward that way of doing because it isn't really flexible (You might need country X, but someone else will need country Y and some other i see, so it don't scale up, it works if you want a particular area will need an arbitrary area) while there are other ways to achieve the same result, for example using http://wiki.openstreetmap.org/wiki/Overpass_API/Augmented_Diffs to split diffs at consumer side. Tought, there isn't (AFAIK) tools to handle that now. i've fast read the page and look good, i'll read all the parts, and put my questions after. I don't know what implies to make it. As far as i understand OSM to host/distribute hourly diff for country, it's something like: - do a minutely/hourly updates for the planet = result the planet.osm - do a split of the planet.osm on the polygons of the country - do a diff between the old-osm-contry and the new one = results the hourlly diff for the country Is this process wright? That's one possible workflow for such a process (working directly on planet.osm files however won't go anywhere due to time needed to process the planet file IMHO, but storing it into some more indexed format, why not) yes and/or doing some pre-filtering of things that are not needed/intended for the end use. Let me explain: For editing the OSM map, doing it directly in the main DB is ok, beacouse the rendering of main slippy map is updated in minutes, but what if i update others data (in my case the public transport), i want to see the changes fast, not after 24h. For one zone i see the way, but i wanted to do it as global as possible as maybe others need this type of service. The other workflow would be to rely on special diffs (like Augmented_Diffs) and split those diffs according to your needs. The generation of planet Augmented Diffs beeing dedicated to some servers while consumers splits at client side. -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] minutely update DB
If you want to maintain a Europe extracted updated, have a look at the Europe diffs available here: http://download.openstreetmap.fr/replication/europe/minute/ 2012/11/4 ciprian niculescu cnicu...@gmail.com: Hello, I need to have a hourly or minutely up-to date DB of OSM. I found infos on how to do it, but one thing is not clear: I must keep the hole OSM DB uptodate and after extract the region i need? I need only Europe. Thanks for clarifying this for me. Ciprian ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk -- Christian Quest - OpenStreetMap France - http://openstreetmap.fr/u/cquest ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
