Re: [OSM-dev] osm2pgsql tile expiry performance

2019-06-08 Thread j
On Tue, 4 Jun 2019 11:13:29 + (UTC) Sven Geggus wrote: > As I am about to setup two machines for this purpose right now I > reworked the scritps a bit. They are available here: > https://github.com/giggls/tileserver-scripts I'm afraid that these scripts will not work in our rendering

Re: [OSM-dev] osm2pgsql tile expiry performance

2019-06-05 Thread j
> My own test hit 22GB before I ^C'd. 21GB more than required w/o > expiration. I have no idea where that 22GB number came from. My notes actually indicate >50GB memory usage. Caffeine deficiency? j ___ dev mailing list dev@openstreetmap.org

Re: [OSM-dev] osm2pgsql tile expiry performance

2019-06-04 Thread j
On Tue, 4 Jun 2019 14:28:38 +0200 Michael Reichert wrote: > Just to ensure that all important test conditions are known. > > Do you mean the weekly changesets dump from > https://planet.openstreetmap.org/planet/changesets-latest.osm.bz2? Or > how did you create changes.osc or where did you

Re: [OSM-dev] osm2pgsql tile expiry performance

2019-06-04 Thread Michael Reichert
Hi, Am 29.05.19 um 21:40 schrieb j: > WEEKLY changeset using 0.92: > -=-=-=- > time $OSM2PGSQL --append -s -C 3000 -G --hstore -d gis -H $PGHOST -U \ > $PGUSER -r xml changes.osc \ > --flat-nodes /database/postgresql/OSM-FLATNODES --slim \ > --number-processes 4 --style openstreetmap-carto.style

Re: [OSM-dev] osm2pgsql tile expiry performance

2019-06-04 Thread Michael Reichert
Hi, Am 30.05.19 um 00:09 schrieb j: > I note that initial testing of this code used an extract of Europe: > https://github.com/openstreetmap/osm2pgsql/pull/747 > > I did not note it earlier, but I am working with the planet. I doubt > these issues would present themselves on a modestly sized

Re: [OSM-dev] osm2pgsql tile expiry performance

2019-06-04 Thread Sven Geggus
j wrote: > I'm afraid I do not have precise timings, but I'm seeing what appears > to be at least an order of magnitude performance penalty, probably due > to memory exhaustion. I can not confirm this. I have running the new tile Expiry code at least since December 2017. As I am about to setup

Re: [OSM-dev] osm2pgsql tile expiry performance

2019-05-29 Thread j
> To be fair, I am not sure this is even related to tile expiration. I > have not tried 0.96 updates without tile expiration as a baseline. After 3 hours, the same update has reached the same state of progress. osm2pgsql (debian backport) 0.96 maintains < 1GB of memory usage without expiring

[OSM-dev] osm2pgsql tile expiry performance

2019-05-29 Thread j
I'm beginning to implement a tile expiration solution and have run into some issues with the new tile expiry expansion. I'm afraid I do not have precise timings, but I'm seeing what appears to be at least an order of magnitude performance penalty, probably due to memory exhaustion. Test machine