Hello, I think we should clarify some terms here. The word “mirror” usually refers to a secondary copy of some very large or very heavily used resource, generally at a different physical location or network link than the primary “official” copy. Its purpose is to reduce load on the main server and avoid sending huge amounts of data across the world by keeping a recent duplicate of the main resource nearer to the final consumers.
When you say “planet osm mirror” people might assume you mean "a copy of planet.pbf closer to home”. Creating this kind of mirror is just that simple: you’d place a recent copy of planet.pbf on a web or file server of some kind, and have people within your organization or geographic area fetch planet.pbf from your server instead of the main one (generally to reduce traffic crossing out of your local network or traveling long distances). Planet.pbf is updated regularly, so you’d probably schedule some scripts to keep your mirrored copy in sync by overwriting it once in a while or applying OSM diffs. I am a bit confused by the comments saying there is only one planet mirror. There are eight of them listed at http://wiki.openstreetmap.org/wiki/Planet.osm <http://wiki.openstreetmap.org/wiki/Planet.osm> and I’m sure there are many more undocumented mirrors around the world within organizations that use OSM data. If you only want to work on one geographic region, it’s also possible to download a PBF extract of that region and keep it up to date with tools like osmupdate or osmosis. This would be much less bandwidth and resource intensive than cloning the whole planet. http://wiki.openstreetmap.org/wiki/Osmupdate <http://wiki.openstreetmap.org/wiki/Osmupdate> There is another notion which could also arguably be called “mirroring” but is commonly referred to as “replication”: creating a local database with a similar schema to the main OSM database and applying diffs to maintain a completely up-to-date copy of that main database. This allows extensive indexing and thus searching for items within OSM by location, ID, tags etc. http://wiki.openstreetmap.org/wiki/Osmosis/Replication <http://wiki.openstreetmap.org/wiki/Osmosis/Replication> Special-purpose OSM replication databases exist, which you could use instead of a general-purpose relational database depending on your use case. Overpass is one such project. Instead of using the public overpass API server, you can also install Overpass locally to mirror the whole main OSM database and keep your mirror in sync: http://wiki.openstreetmap.org/wiki/Overpass_API/Installation <http://wiki.openstreetmap.org/wiki/Overpass_API/Installation> I’m also working on a replication system called Vanilla Extract that is geared toward performing large, fast geographic extracts. It’s still under development but stable enough for us to use it on a daily basis. Our use case is routing, so it currently only indexes ways and the nodes referenced by them. No free-floating POI nodes (though we’ll eventually add that), and no author metadata. The original version (in C) loads a PBF at a single point in time and does no updates. https://github.com/conveyal/vanilla-extract <https://github.com/conveyal/vanilla-extract> The newer version (in Java using MapDB) is capable of replicating planet.osm with a one-line command and performs minutely updates, with the option of even pulling the OSM data in over the network without a local copy of planet.pbf. However, it’s still a bit slow on large extracts. We’re currently migrating it to MapDB2 and expect a performance improvement. https://github.com/conveyal/osm-lib/blob/master/src/main/java/com/conveyal/osmlib/VanillaExtract.java <https://github.com/conveyal/osm-lib/blob/master/src/main/java/com/conveyal/osmlib/VanillaExtract.java> Andrew > On 18 Nov 2015, at 18:46, Kevin Mcintyre <[email protected]> wrote: > > There's some good information here - > http://download.geofabrik.de/technical.html > <http://download.geofabrik.de/technical.html> one companies process at least. > > Appreciate the response, it's a tricky search because the results are > generally information on mirrors not how to become a mirror. > > > > On Wed, Nov 18, 2015 at 9:21 AM, Tom Hughes <[email protected] > <mailto:[email protected]>> wrote: > On 18/11/15 17:02, Kevin Mcintyre wrote: > > Hello - I'm seeking information on setting up a planet osm mirror. > > I think the reason you're not getting replies is that you're asking for > something that doesn't really exist - there isn't any general infrastructure > for mirrors in existence. > > We have one mirror that we redirect most downloads to, and that's it. > > Tom > > -- > Tom Hughes ([email protected] <mailto:[email protected]>) > http://compton.nu/ <http://compton.nu/> > > _______________________________________________ > dev mailing list > [email protected] > https://lists.openstreetmap.org/listinfo/dev
_______________________________________________ dev mailing list [email protected] https://lists.openstreetmap.org/listinfo/dev
