On Apr 6, 2009, at 10:26 AM, Marcin Rudowski wrote: > Dane Springmeyer pisze: >> A few more comments inline... >>> For areas bigger then Poland (are there bigger Countries then >>> Poland? ;) ) problem could be aligning separate rasters. I use one >>> big tiff (200MB) and gdal should allow much bigger files, but I'm >>> not sure if mapnik does. >> Mapnik currently does not support using GDAL overviews, so that may >> help with larger files: >> http://trac.mapnik.org/ticket/54 > > I didn't mean any overviews, but using "tiling" to cover large or > irregular areas. Now I use one big square tiff, but i.e. when trying > to > cover Europe, there is much of space wasted for water. > For less detailed zoom's I use separate rasters (0.5x0.5 and 0.25x0.25 > of original one).
Ah, gocha. I guess that gdal overviews may only be appropriate for certain formats, like MrSid. > > > I recently played for fun with 3-4x bigger area then my actual > settings and had no problems, so if storage isn't a problem, i.e. Cool! > Europe > hillshading could be covered with one big file. If gdal can handle > file, > mapnik too. I haven't tested aligning two adjacent rasters yet. > > My first experiment is 830MPix tiff, 790MB size after converting > srtm3 -> hillshaded raster, described in earlier email: > > Raster size: 36 410 x 22 770 > Upper Left ( -46.383, 7361946.980) ( 0d 0'1.50"W, 55d 0'1.50"N) > Lower Left ( -46.383, 5621476.490) ( 0d 0'1.50"W, 44d59'58.97"N) > Upper Right ( 2783024.787, 7361946.980) ( 25d 0'1.21"E, 55d 0'1.50"N) > Lower Right ( 2783024.787, 5621476.490) ( 25d 0'1.21"E, 44d59'58.97"N) > > Its quite big part of Europe. After demtool, output tiff is 800MB > size, > however middle step file (warped.tif from my "instruction") is 3.1GB. > > All processing, but without "nodata" holes filling, took ~1.5h on 3 > years old laptop (Mobile AMD 2.2GHz). Mapnik reads this big file > with no > difference to Poland only file, as tiff format allows reading > interesting part of the image without reading whole file. > > If any file is >4GB then gdal needs to be compiled with > BigTIFF support: libtiff with BigTiff or gdal compiled using > --with-libtiff=internal. Otherwise You get: > >> Creating output file that is 36411P x 30740L. >> ERROR 6: A 36411 pixels x 30740 lines x 1 bands Float32 image would >> be larger than 4GB but this is the largest size a TIFF can be, and >> BigTIFF is unavailable. >> Creation failed. > > To allow mapnik to open rasters >4GB it should be compiled against > gdal library with enabled BigTIFF support. But 4GB hillshading file > means 4GPix raster !!! which isn't that abstract if trying to cover > any continent as one rectangle. Not to mention srtm1 in America. Cool, really nice details! > > > Another partial solution would be adding -co "COMPRESS=LZW" or similar > to gdal_warp, but it gives only little size reduction. > > I also tried compression in demtool by adding: > papszOptions = CSLSetNameValue( papszOptions, "COMPRESS", "LZW" ); > in hillshade.cpp. Output shaded file decreased from 1070GB to 712MB > for > 36411x30740 raster, 1.1GPix (!!!) tiff: > > Raster size: 36 411 x 30 740 > Upper Left ( -46.383, 7361946.980) ( 0d 0'1.50"W, 55d 0'1.50"N) > Lower Left ( -46.383, 5012273.600) ( 0d 0'1.50"W, 40d59'58.34"N) > Upper Right ( 2783101.224, 7361946.980) ( 25d 0'3.69"E, 55d 0'1.50"N) > Lower Right ( 2783101.224, 5012273.600) ( 25d 0'3.69"E, 40d59'58.34"N) > > There was only little time increase when generating map with mapnik > using compressed tiff (<10%) > > > > I tried using two rasters, but fixing mapnik was needed: > > http://trac.mapnik.org/ticket/295 Awesome, patch applied in r1079 > > >>> BTW. as for big tiffs, I have some problems with segmentation >>> faults. >> Interesting. Marcin, could you post details of this to a trac ticket? > > http://trac.mapnik.org/ticket/294 Perfect, thats a really valuable reference on the issue, even it if is due to problems outside of Mapnik. Thanks for all your contributions Marcin! Cheers, Dane _______________________________________________ Mapnik-users mailing list [email protected] https://lists.berlios.de/mailman/listinfo/mapnik-users
