Re: [Geoserver-users] Serving SRTM1 data with GeoServer
Thank you, Ian (and Peter). I had actually considered using gdal_merge.py to combine the GeoTiffs I'd created, but thought the resulting mega-TIFF would be too large. However: gdal_merge.py -o elevation.tif -co BIGTIFF=YES -co TILED=YES -co COMPRESS=DEFLATE *.hgt.tif worked fine. GeoServer accepted this store and maps it slowly, but without generating errors. For reference, the resulting TIFF, elevation.tif is just over 100 gigabytes in size, and has a resolution of 1296001 x 417601, which is slightly less than I expected because it turns out SRTM1 doesn't cover the entire world. On Tue, 21 May 2019, Ian Turton wrote: Date: Tue, 21 May 2019 08:18:23 +0100 From: Ian Turton To: Barry Carter Cc: geoserver-users Subject: Re: [Geoserver-users] Serving SRTM1 data with GeoServer Have a look at https://www.ianturton.com/tutorials/bluemarble.html where I have example scripts and work flow to create a base image from a set of tiles and how to make a pyramid from it. Ian On Tue, 21 May 2019 at 03:16, Barry Carter wrote: I'm trying to provide SRTM1 (1 arcsecond resolution elevation) images and data using GeoServer, but can't figure out how to serve such a large dataset efficiently. Here's what I've done so far: - Downloaded the 1 degree squared HGT files. The Earth is 360*180 = 64800 square degrees, but only 14295 HGT files since the other square degrees are 100% water. The HGT files are uncompressed and 25,934,402 bytes each. - I then used commands like: gdal_translate -co TILED=YES -co COMPRESS=DEFLATE N58W122.hgt N58W122.hgt.tif to convert these HGT files into TIFFs. The TIFFs vary in size, but are much smaller. - I then imported the directory with the TIFFs as a store using ImageMosaic. This took a while, but appears to have worked, since "srtm1" (the name I gave it) now appears in my list of stores. - I then created a layer using this source, but when I use "OpenLayers" preview, the server hangs for a while and ultimately dies with "too many open files" (or similar). This seems odd, since my Linux OS had fs.file-max set to 13063650. I even tried doubling it to 26127300, but that didn't help. At this point, I suspect I need to create an image pyramid or something so GeoServer can handle the data efficiently, but I'm not sure exactly what to do. I've used 'gdal_retile.py' previously to create pyramids from one large image into tiles at various zoom levels, but I think I want to do the opposite here (merge smaller images for lower zoom levels). The gdal_retile.py option -pyramidsonly seems to say it does that, but I can't get it work (it just hangs and creates no files) ___ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users -- Ian Turton ___ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
[Geoserver-users] Tile caches
Hey you guys, I used a large gray image as a store, and then used tile caching to make it display faster. Do the tile caches apply only to the gray image? I'm guessing yes since the cached tiles are stored as PNG/GIF/JPG and not GeoTiffs themselves, but wasn't sure. For example, it's theoretically possible that GeoServer reads the gray tile and changes the colormap before sending it to the client. Are cached tiles only used if the client requests tiled mode? I'm again guessing yes, but it's possible that tile caches make it easier to assemble images at lower zoom levels. However, it's possible GeoServer uses the data directly when tiled mode is not requested. Suppose I create a tile cache using the PNG8 format. Can GeoServer tell or force the client to only requests maps in this format? I assume that, if the client requests JPG format, the tiles are of no use to me (unless GeoServer does PNG8 to JPG conversion on the fly), so I'd like the client to use the most efficient method. ___ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
[Geoserver-users] Serving SRTM1 data with GeoServer
I'm trying to provide SRTM1 (1 arcsecond resolution elevation) images and data using GeoServer, but can't figure out how to serve such a large dataset efficiently. Here's what I've done so far: - Downloaded the 1 degree squared HGT files. The Earth is 360*180 = 64800 square degrees, but only 14295 HGT files since the other square degrees are 100% water. The HGT files are uncompressed and 25,934,402 bytes each. - I then used commands like: gdal_translate -co TILED=YES -co COMPRESS=DEFLATE N58W122.hgt N58W122.hgt.tif to convert these HGT files into TIFFs. The TIFFs vary in size, but are much smaller. - I then imported the directory with the TIFFs as a store using ImageMosaic. This took a while, but appears to have worked, since "srtm1" (the name I gave it) now appears in my list of stores. - I then created a layer using this source, but when I use "OpenLayers" preview, the server hangs for a while and ultimately dies with "too many open files" (or similar). This seems odd, since my Linux OS had fs.file-max set to 13063650. I even tried doubling it to 26127300, but that didn't help. At this point, I suspect I need to create an image pyramid or something so GeoServer can handle the data efficiently, but I'm not sure exactly what to do. I've used 'gdal_retile.py' previously to create pyramids from one large image into tiles at various zoom levels, but I think I want to do the opposite here (merge smaller images for lower zoom levels). The gdal_retile.py option -pyramidsonly seems to say it does that, but I can't get it work (it just hangs and creates no files) ___ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users