On 2018-11-08 6:34 AM, Nick Whitelegg wrote:
At the moment I download full planet extracts about every 6 months.
However, due to the limitations of my server, I filter out (with
osmosis) a lot of stuff I don't need so that I am basically left with
roads, footpaths, natural features, water features and selected POIs.
I'd like to move towards a system which applies diffs from geofabrik
instead, and applies them regularly (daily or weekly) with osm2pgsql.
My question is this; given that not everything in the diff will be in
my database (as I filter out what I don't need during the import
process), will osm2pgsql apply the diff successfully or will it
complain that not all features in the diff are in my database?
I can think of four ways to do this, all which have a different balance
of correctness, performance, and ease of use.
There are two "right" ways to do this. The first one is to re-import
every week. Because imports without slim tables (either --slim --drop or
no --slim) are faster, this is a good option and needs less space than a
database able to consume diffs.
The second right way involves keeping two files, one with the current
full data, and one with the filtered data. Call these "planet.pbf" and
"planet-filtered.pbf". Then when updating create "planet-new.pbf",
filter it to get "planet-filtered-new.pbf", create a diff for the
differences between "planet-filtered-new.pbf" and "planet-filtered.pbf",
and apply that diff to the database. Then replace the old files with the
new ones. This will keep the database correct.
A "wrong" way to do it is to import the filtered data, apply updates
directly, and periodically delete data from the DB. The problem with
this is that if someone adds one of the selected POI tags to a building
that you have filtered out, osm2pgsql won't have the node data to create
a geometry. This might be acceptable, depending on use case.
A less wrong way would be to modify your filtering so no nodes are
filtered out. There are still potential errors with relations, but these
are less common. If you're doing the planet or a large extract and using
flat nodes there's no storage penalty for having all the nodes.
_______________________________________________
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
