I want to back up Even's statement about trying to read a file
piece-by-piece. About 99% of raster processes are better handled with
"tiled" processing -- a nice balance between i/o, memory and sanity
would be to read the image one line at a time, perform your processing
on that line, write that line out, then move to the next one:
(generically)
for i = 1, number of lines
read line i
analyze line i
write results of line i to output file
end for
If you want to get more creative (or are doing spatial processing), you
might want to read a set of lines, restricted by your memory usage.
There are almost no raster processes that ever need an entire image read
into memory at one time.
--j
Even Rouault wrote:
Stephen,
My first answer was "Maybe you have not waited long enough... ;-)"
But I could also reproduce the slowness with GDAL <= 1.5.2 while reading such
a huge amount of data into one call.
There was an inefficient reading strategy in GDALRasterBand::IRasterIO that
your dataset precisely hits, that is to say raster with big raster dimensions
and tiled data. This is bug #2457 and it has been fixed recently in the SVN
branch of 1.5 (in r14863) and trunk. I can confirm that updating to r14863 of
branches/1.5 fixes the slowlyness.
So several solutions come to mind :
* try to read your file piece by piece (ideally by adjusting your requests on
block boundaries)
* update to the SVN version
* wait for the next release
* try with GDAL_CACHEMAX=54, because 54 MB = 56 623 104 bytes > (1024 * 1024 *
2) * (27552 / 1024). See #2457 for the explanation of that formula ;-)
(Side note, I see one error in your code. You ask for GDT_Byte data but you
allocate a float array. )
Best regards
Le Wednesday 16 July 2008 17:53:21 stephen tan, vous avez écrit :
I have a nitf image that I am trying to read into an array using
gdal-1.5.1. The file is about 1.5G .
gdalinfo returns the following information:
Size is 27552, 25196
...
Band 1 Block=1024x1024 Type=UInt16, ColorInterp=Undefined
I have tried to read the image into an array using the following code:
GDALDataset *poDataset;
GDALAllRegister();
poDataset = (GDALDataset *) GDALOpenShared( fname.c_str(),
GA_ReadOnly );
if( poDataset == NULL )
{
cerr << "Could not open file: " << fname << endl;
exit(1);
}
int width = poDataset->GetRasterXSize();
int height = poDataset->GetRasterYSize();
int bands = poDataset->GetRasterCount();
GDALRasterBand *poBand;
int nBlockXSize, nBlockYSize;
poBand = poDataset->GetRasterBand( 1 );
poBand->GetBlockSize( &nBlockXSize, &nBlockYSize );
//float* data;
//data = (float *) CPLMalloc(sizeof(float)*width*height*bands);
float* data;
data = new float[width*height*bands];
//GByte* data;
//data = (GByte *) CPLMalloc(sizeof(float)*width*height*bands);
//GByte data[width*height*bands];
if(poBand->RasterIO( GF_Read, 0, 0, width, height, data, width,
height, GDT_Byte, 0, 0 ))
{
cerr << "Error: Could not read image data." << endl;
exit(1);
}
GDALClose(poDataset);
No errors are returned. I let it run for several mins before I hit CTRL+C.
Any C++ suggestions on getting the image into a 1D array and/or 2D array
would be most helpful. The above code works for smaller images. I have
also tried different GDALSetCacheMax with no success.
Thanks
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
