Hi, I have a feeling that I can't test this --slim --drop with Windows. It is a much desired feature for me anyhow.
-Jukka Rahkonen- Frederik Ramm wrote: > Hi, > > Kai has made a number of interesting improvements to osm2pgsql in > the last weeks. I believe some bits are still work in progress but on > the whole osm2pgsql has become a lot more efficient - it makes better > use of cache memory and can even use multiple processes for some tasks. > Anyone who regularly spends time waiting for osm2pgsql to complete is > encouraged to check out a recent version from svn and try if that > improves things for him. > > I think it would be great to share results of osm2pgsql runs among users > - how long does it take to import X on infrastructure Y? > > I've made a start here, please add/modify as you see fit: > > http://wiki.openstreetmap.org/wiki/Osm2pgsql/Benchmarks > > There's one particular use case that osm2pgsql did not cover so well in > the past - the "I don't want to apply updates but I need to use slim > mode nonetheless because I don't have enough memory for non-slim" use > case. > > osm2pgsql is not very well suited for this because it puts all its > temporary information into the database instead of a more efficient > random-access structure. This is something I'll leave for someone else > to fix, but I did one thing to make this use case a bit better; I > introduced a "--drop" flag that makes osm2pgsql drop the temporary > tables after import, and also does not create the indexes on way id and > relation id that a --slim import normally created. So now, after > importing a data set with --drop and --slim, you should have a database > that looks almost the same as one imported without --slim. By dropping > the unnecessary tables and indexes, the database usually is only 25% of > the size of a complete --slim import (but of course it is unsuitable for > updates). > > There's one strange thing I noticed. When I dropped the creation of > indexes (more precisely, primary keys) on way id and polygon id, > suddenly osm2pgsql took ages to run - even though these indexes are > clearly not created in non-slim mode and therefore should not be required. > > I found out that the culprit is in the multipolygon code, where after > finding out that an one-way outer ring is tagged the same as the > multipolgon relation itself, a "delete_way_from_output" is issued, > presumably to remove that already-generated ring. This leads to a > "DELETE from <table> where osm_id=<id>" which requires a table scan > because of lack of primary keys. > > I have now disabled this for --slim --drop mode (the change will not > affect normal --slim mode), but have to investigate further - this will > likely create some extra areas for outer rings, but since it doesn't > have these indexes, non-slim mode should exhibit the same behaviour. > > Is anyone aware of multipolygon handling not working right when not > using --slim? We might have to (re)introduce the primary key for osm_id > at least on the polygon table to allow this deletion of duplicate areas. > > > _______________________________________________ > dev mailing list > [email protected] > http://lists.openstreetmap.org/listinfo/dev > _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
