Hi
For all interested, I have applied the first changes to master (see
http://linfiniti.com/2011/08/improvements-to-raster-performance-in-qgis-master/)and
with a unit test! :-)
I still need to apply Etienne's patch for progress bar support which I
hope to do in this week.
regards
Tim
On Thu, Aug 4, 2011 at 9:40 AM, Tim Sutton li...@linfiniti.com wrote:
Hi
On Wed, Aug 3, 2011 at 7:38 PM, Etienne etienne...@yahoo.com wrote:
Hi, I'm glad I can help!
1) regarding the histogram caching:
GDAL does this automatically, it looks for existing aux file and a
suitable histogram block. It's all in function
GDALPamRasterBand::PamFindMatchingHistogram() in file
gdalpamrasterband.cpp .
As long as the dataset has support for PAM (stored in the .aux.xml file), it
should work.
I found a bug in the floating-point comparison which was already fixed:
http://trac.osgeo.org/gdal/ticket/4183
Unlike the stats functions (GetStatistics/ComputeStatistics), there is only
1 function which looks for cached data.
Ok for me it seems to recompute the histogram every time (which takes
about a minute+ with the sample data set I sent you). I will update my
gdal svn copy to get your patch for #4183 and see if the problem goes
away.
2) regarding the caching of stats:
Actually the GDAL driver looks for stored metadata (in function
GDALRasterBand::GetStatistics).
This works for gtiff files, I don't know for other raster types.
ok
3) regarding the progress bar:
Do you have sufficient authority to ask the GDAL folks to change the
prototype for GDALGetRasterStatistics() and GetStatistics() for them to
include a progress call-back? I could make a patch for it, but I don't know
if they would accept it.
In the mean time, I can copy the GDAL code into qgsgdalprovider and submit
that patch to you.
Ok Martin addressed this in his email in this thread. I basically have
no more authority than anyone else in the community - they evauate
each request based on merit. As Martin mentions, breaking ABI
compatibility will not go down well with them so providing a wrapper
function which adds progress support (or perhaps adding the progress
callback as the last param with a default of void so that it can be
left out will be acceptable to them) would be good.
4)
There is another issue I forgot to mention, and that is the bin count of the
histogram. GDAL uses 256 bins, and recently the GDAL histogram
calculation in QgsRasterLayerProperties::refreshHistogram() has been
changed to use 255 bins. ( const int BINCOUNT = 255; ). Could this be
put back to 256 for consistency with GDAL, or is there a reason for
that?
Ok I have changed in my local copy and will commit soon.
Thanks! Etienne
Thank you!
Regards
Tim
From: Tim Sutton li...@linfiniti.com
To: Etienne etienne...@yahoo.com
Cc: Qgis-developer@lists.osgeo.org Qgis-developer@lists.osgeo.org
Sent: Wednesday, August 3, 2011 6:00:56 AM
Subject: Re: [Qgis-developer] Provider native raster band stats
Hi Again
By the way is there a handy way to get the histogram from the aux.xml
too? I see it is stored there but haven't figured out how to retrieve
it from cache again.
- Original Message -
From: Tim Sutton li...@linfiniti.com
To: Etienne etienne...@yahoo.com
Cc: Qgis-developer@lists.osgeo.org Qgis-developer@lists.osgeo.org
Sent: Wednesday, August 3, 2011 6:00:56 AM
Subject: Re: [Qgis-developer] Provider native raster band stats
Hi Again
By the way is there a handy way to get the histogram from the aux.xml
too? I see it is stored there but havent figured out how to retrieve
it from cache again.
Regards
Tim
On Wed, Aug 3, 2011 at 10:16 AM, Tim Sutton li...@linfiniti.com wrote:
Hi
On Tue, Aug 2, 2011 at 7:06 PM, Etienne etienne...@yahoo.com wrote:
Tim, my impression is that it speeds things up, but I have a suggestion to
make it faster.
Cool!
From what I have seen in your code, the QgsGdalProvider::bandStatistics
function calls GDALComputeRasterStatistics (), which computes the
statistics every time - it doesn't fetch pre-calculated data ( which is
stored in gtiff metadata, or aux.xml file).
A better way would be to use the GDALGetRasterStatistics() which
calls GDALComputeRasterStatistics() when needed (when bForce = TRUE)
double myerval = GDALGetRasterStatistics ( myGdalBand, bApproxOK, TRUE,
pdfMin, pdfMax, pdfMean, pdfStdDev);
Ah - actually I thought GDALComputeRasterStatistics was using aux.xml
cached stats already. I guess I never read the docs carefully enough.
I applied your change and my ~8min to load raster from my original
email took about a minute to load the first time (while gdal was
creating an aux.xml) and subsequent use of the same raster resulted in
it opening instantly! So thanks for your excellent advice. I have
pushed an updated version to my repo that incorporates your changes
should others like to try. I
