I'm beginning to implement a tile expiration solution and have run into some issues with the new tile expiry expansion.
I'm afraid I do not have precise timings, but I'm seeing what appears to be at least an order of magnitude performance penalty, probably due to memory exhaustion. Test machine is Debian stretch 4CPUs, 26GB RAM, SSD Array. Initial osm import was performed with osm2pgsql 0.92 and finished in under 48 hours. Under 0.92 I was running multiple render chains while importing changesets. Tests w/ 0.96 have been run against an otherwise idle machine. Database & flatnodes file were restored to initial state between each round of testing. WEEKLY changeset using 0.92: -=-=-=- time $OSM2PGSQL --append -s -C 3000 -G --hstore -d gis -H $PGHOST -U \ $PGUSER -r xml changes.osc \ --flat-nodes /database/postgresql/OSM-FLATNODES --slim \ --number-processes 4 --style openstreetmap-carto.style \ --tag-transform-script openstreetmap-carto.lua -e19 -o \ $WORKDIR_OSM/expired-tiles With -e19, I was able to import a weekly changeset in roughly 24 hours. Using git repository (5/27? pull): -=-=-=- time $OSM2PGSQL --append -s -C 3000 -G --hstore -d gis -H $PGHOST -U $PGUSER -r xml changes.osc \ --flat-nodes /database/postgresql/OSM-FLATNODES --slim \ --number-processes 4 --style openstreetmap-carto.style \ --tag-transform-script openstreetmap-carto.lua -e10-16 -o \ $WORKDIR_OSM/expired-tiles Consistently crashed w/ a bad alloc(). I didn't note any unusual output in the compile. Crashes even with a 24 hour changeset. DAILY changeset using Debian backport to stretch (0.96): -=-=-=- As above, using -e10-16. 22 hours spent processing a 24 hour changeset and still importing new relations. It's 35GB into swap, with osm2pgsql claiming 89% of the memory usage. ------- I realize there is a large difference between zoom level 19 and 10-16, but I assume it should take significantly less RAM/CPU for 10-16. Please feel free to point out any stupidity I've generated here and/or recommend a better way to generate a list of dirty tiles at lower zoom levels based on the 0.92 output. To be fair, I am not sure this is even related to tile expiration. I have not tried 0.96 updates without tile expiration as a baseline. j _______________________________________________ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
