Hi On Tue, Aug 2, 2011 at 7:06 PM, Etienne <etienne...@yahoo.com> wrote: > Tim, my impression is that it speeds things up, but I have a suggestion to > make it faster. > Cool!
> From what I have seen in your code, the QgsGdalProvider::bandStatistics > function calls GDALComputeRasterStatistics (), which computes the statistics > every time - it doesn't fetch pre-calculated data ( which is stored in gtiff > metadata, or aux.xml file). > > A better way would be to use the GDALGetRasterStatistics() which > calls GDALComputeRasterStatistics() when needed (when bForce = TRUE) > double myerval = GDALGetRasterStatistics ( myGdalBand, bApproxOK, TRUE, > &pdfMin, &pdfMax, &pdfMean, &pdfStdDev); > Ah - actually I thought GDALComputeRasterStatistics was using aux.xml cached stats already. I guess I never read the docs carefully enough. I applied your change and my ~8min to load raster from my original email took about a minute to load the first time (while gdal was creating an aux.xml) and subsequent use of the same raster resulted in it opening instantly! So thanks for your excellent advice. I have pushed an updated version to my repo that incorporates your changes should others like to try. I will try to write some unit tests this week and then merge my branch into QGIS master over the weekend. > > > I have tested it on a few rasters, it gets the same stats. > > The downside is that GDALGetRasterStatistics() does not offer a progress > callback, rendering the progressbar useless. > If it"s really needed, I could re-implement the function in QgsGdalProvider > (about 50 lines of code), or try to have GDALGetRasterStatistics support a > progress callback. > Yes this is quite noticeable the first time the raster is loaded and you sit wondering what is happening. I guess the more elegant would be to have the change carried out in gdal itself if that is acceptible, otherwise your patch as per your former suggestion would be greatly appreciated. Best regards Tim > regards, Etienne > > ----- Original Message ----- > From: Tim Sutton <li...@linfiniti.com> > To: Marco Hugentobler <marco.hugentob...@sourcepole.ch> > Cc: qgis-developer@lists.osgeo.org > Sent: Monday, July 11, 2011 7:04:08 PM > Subject: Re: [Qgis-developer] Provider native raster band stats > > Hi > > > On Mon, Jul 11, 2011 at 5:29 PM, Marco Hugentobler > <marco.hugentob...@sourcepole.ch> wrote: >> Hi Tim >> >> I'm not a raster guru, but the changes look good to me. >> +1 for merging. > > Thanks for your feedback Marco. I guess I'll write some unit tests > (wrong way around I know :-) ) and then merge it if they all pass and > there are no other objections. :-) > > Regards > > Tim > > >> >> Regards, >> Marco >> >> Am Donnerstag, 7. Juli 2011, 00.47:25 schrieb Tim Sutton: >>> Hi >>> >>> During the Lisbon hackfest (between all those meetings!) I worked on >>> updating the raster providers to be able to generate stats themselves. >>> The raster provider base class has a default (and historically >>> inefficient as we all know) implementation for collecting stats by >>> walking over every cell in the raster twice. This method remains, but >>> the providers can now supply stats themselves (heopfully using more >>> efficient mechanisms). I have implemented gdal native stats already. >>> Over the last few weeks I did some further testing and cleanups to >>> this work. From my testing some of my worst case files opened in >>> ~1minute rather than ~8+ minutes. It would be great if interested >>> parties could test (details below) before I put it into master. >>> >>> git remote add timlinux git://github.com/timlinux/Quantum-GIS.git >>> git fetch timlinux >>> git branch --track timlinux raster-stats >>> git checkout raster-stats >>> >>> I look forward to your feedback. >>> >>> Regards >> >> >> -- >> Dr. Marco Hugentobler >> Sourcepole - Linux & Open Source Solutions >> Churerstrasse 22, CH-8808 Pfäffikon SZ, Switzerland >> marco.hugentob...@sourcepole.ch http://www.sourcepole.ch >> Technical Advisor QGIS Project Steering Committee >> > > > > -- > Tim Sutton - QGIS Project Steering Committee Member (Release Manager) > ============================================== > Please do not email me off-list with technical > support questions. Using the lists will gain > more exposure for your issues and the knowledge > surrounding your issue will be shared with all. > > Visit http://linfiniti.com to find out about: > * QGIS programming and support services > * Mapserver and PostGIS based hosting plans > * FOSS Consulting Services > Skype: timlinux > Irc: timlinux on #qgis at freenode.net > ============================================== > _______________________________________________ > Qgis-developer mailing list > Qgis-developer@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/qgis-developer > > _______________________________________________ > Qgis-developer mailing list > Qgis-developer@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/qgis-developer > -- Tim Sutton - QGIS Project Steering Committee Member (Release Manager) ============================================== Please do not email me off-list with technical support questions. Using the lists will gain more exposure for your issues and the knowledge surrounding your issue will be shared with all. Visit http://linfiniti.com to find out about: * QGIS programming and support services * Mapserver and PostGIS based hosting plans * FOSS Consulting Services Skype: timlinux Irc: timlinux on #qgis at freenode.net ============================================== _______________________________________________ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
