For reference, when the disk cache has symlinking enabled: case 1: 96M
case 2: 147M On Tue, May 29, 2012 at 1:06 PM, thomas bonfort <thomas.bonf...@gmail.com> wrote: > Salut Ludovic, hi Andrew, > > I've investigated a bit more on the disk space issue with berkeley db > backends, and here are some of my findings. To summarize, the space > occupancy/efficiency is closely tied to the size of the images stored > in the cache, and the configured pagesize of the bdb backend. > > For a cache that contains many empty tiles, i.e. that are just a few > hundred to a couple thousand bytes, here are the space occupancies > (the disk cache uses 4k blocks). This is for reference only, usual > caches will not resemble this one. Also note that the disk cache was > not configured to symlink blank tiles, and that the bdb cache backend > has some code in it to reduce the tile data stored on empty tiles. > > disk: 197M > bdb pagesize=64k: 106M > bdb pagesize=4k: 96M > bdb pagesize=1k: 73M > sqlite: 103M > mbtiles (sqlite with blank tile detection): 62M > > So these numbers actually aren't so bad for bdb in all configurations, > but in this case the 4k filesytem blocksize is really inefficient. > > These are the occupancies for another tileset, where the tiles are > much heavier (tiles around 40-45k). There are also quite a few empty > tiles. > > disk:153M > bdb pagesize=64k: 209M > bdb pagesize=4k: 148M > bdb pagesize=1k: 144M > sqlite: 140M > mbtiles (sqlite with blank tile detection): 137M > > > I had initally stayed away from using small pagesizes in the bdb > backend because of some stability issues, but in the current tests I > have not run into any issues. From these tests, I think the natural > way forward is to change the default pagesize, and maybe make it > configurable in mapcache.xml as an advanced configuration option. > If you are able to confirm my findings on your instances, that would > be great. To set the pagesize you'll have to recompile/reinstall, > after having changed lib/cache_bdb.c by replacing > > #define PAGESIZE 64*1024 > by > #define PAGESIZE 1*1024 > > near line 50. > > regards, > thomas > > > > On Wed, May 23, 2012 at 3:55 PM, Ludovic Gnemmi <lgne...@rgd73-74.fr> wrote: >> Hi list, >> I would like to experiment Berkeley DB cache type but i'm surprised of the >> resulting db size compared to sqlite cache type. Here is my results for >> about 13000 tiles: >> >> Total tiles size = 44 Mb >> Sqlite = 48 Mb >> Berkeley DB = 61 Mb >> Disk = 66 Mb (with 4kb filesystem block size) >> >> I thought i could reduce storage capacity with Berkeley DB cache type as >> with Sqlite but it seems there is not much difference with disk cache type. >> Is there a way to reduce Berkeley DB cache size ? >> >> Thanks for your help >> >> Ludovic >> >> -- >> View this message in context: >> http://osgeo-org.1560.n6.nabble.com/MapCache-A-way-to-reduce-BerkeleyDB-Cache-size-tp4976403.html >> Sent from the Mapserver - User mailing list archive at Nabble.com. >> _______________________________________________ >> mapserver-users mailing list >> mapserver-users@lists.osgeo.org >> http://lists.osgeo.org/mailman/listinfo/mapserver-users _______________________________________________ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users