Hi David, Following on from the VRT / Bigtiff comparison Jukka posted. Have you considered storing the data as a single KEA format file, which is based on HDF5?
I have the National Elevation Dataset for the US, which comprises 3,605 1 x 1 degree tiles at 1 arc sec resolution. I first created a VRT then used gdal_translate to convert to a KEA file. Total size is 421,212 x 252,012 pixels, 77 GB. I also built overviews for a fast display (this took a long time and I don't think is needed for your case). I've just tried using gdallocationinfo on the file to get pixel information and it takes < 0.5s to get the pixel value back. The KEA library and GDAL driver source are available from: https://bitbucket.org/chchrsc/kealib/ and the format is described in: Peter Bunting, Sam Gillingham, The KEA image file format, Computers & Geosciences, Volume 57, August 2013, Pages 54-58, ISSN 0098-3004, http://dx.doi.org/10.1016/j.cageo.2013.03.025. If you don't mind having one massive file (in addition to the individual tiles, which could be archived), this might work for your use case. Thanks, Dan > > Message: 10 > Date: Fri, 31 Jan 2014 16:15:53 +0000 > From: "David Baker (Geoscience)" <david.m.ba...@chk.com> > To: "'gdal-dev@lists.osgeo.org'" <gdal-dev@lists.osgeo.org> > Subject: [gdal-dev] Fast Pixel Access > Message-ID: > <2a18a4344312134b937df938d992264a0508f...@okcexhprd122.chkenergy.net> > Content-Type: text/plain; charset="us-ascii" > > Dev's, > > I have a set of 55,501 bil files in a single directory. They are DEMS data > that cover the US in 7.5 minute tiles. I would like to randomly access > elevations at a given lat/lon's from the whole dataset. I created a vrt file > from the directory of bil files, and have been able to access the elevation > at a given lat/lon using gdallocationinfo, but because of the size of the > dataset, this operation is somewhat slow. Can the vrt be indexed? Or, is > there a faster, better way to access the pixels? I would first like to do > this with the utilities before diving into code (C#). The files are > regularly named base on their location within a 1 arc-second grid. > > Thanks, > David > > David M. Baker > Senior Advisor - Geoscience Technology > Chesapeake Energy Corporation > david.m.ba...@chk.com<mailto:david.m.ba...@chk.com> > > > _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
