Re: [gdal-dev] vrt guidance: 31 band classified raster

[Stephen Woodbridge]

Matt,

Have you tried create a cascading VRT, where the first VRT composites 
all the GTiff files into a single virtual raster and then reference the 
composite in another VRT that assigns colortable. I think this should 
work nicely.


-Steve W

On 10/3/2019 7:01 PM, matt.wil...@gov.yk.ca wrote:


Hello gdal-dev, it's been a long time!  I'm happy to be digging into 
raster data building again for a change, but could use some nudges in 
the right direction(s).


A few weeks ago Nasa released Landsat-derived Annual Dominant Land 
Cover Across ABoVE Core Domain, 1984-2014 
.


It’s composed of 175 geotiff images, with each file containing 31 
bands which in turn correspond to a single year in the 1984-2014 
period. Each band is 8bit unsigned integer with values from 1 to 15 
and 255 as nodata. Each integer value corresponds to a class such as 
“Evergreen Forest”, “Herbaceous”, “Water” and so on.


I’ve been successful in manually building a VRT file using Category 
element for the classes and ColorTable entry for a palette – but only 
for a single band.


My question is: how to properly apply this to all bands? Do I need 
duplicate category and colortable elements to every single 
VRTRasterBand element or is there a smart way to define it once and 
then refer to it like a variable?  Am I even approaching this the 
right way?


It was quite a bit of work to get this far and I’m not looking forward 
to doing this 30 more times.


Sample vrt and simplified source is attached.

*Matt Wilkie*

Geomatics Analyst

Environment | Information Management & Technology

T 867-667-8133 | Yukon.ca 


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] vrt guidance: 31 band classified raster

[Matt.Wilkie]

> It was quite a bit of work to get this far and I'm not looking forward to 
> doing this 30 more times.

On this point, discovery of the day is Easy XML 
Editor which allows editing some 
attributes as if in a spreadsheet. It's alleviated much difficulty, but is 
still finicky.

-Matt
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] vrt guidance: 31 band classified raster

[Matt.Wilkie]

Hello gdal-dev, it's been a long time!  I'm happy to be digging into raster 
data building again for a change, but could use some nudges in the right 
direction(s).

A few weeks ago Nasa released Landsat-derived Annual Dominant Land Cover Across 
ABoVE Core Domain, 
1984-2014.

It's composed of 175 geotiff images, with each file containing 31 bands which 
in turn correspond to a single year in the 1984-2014 period. Each band is 8bit 
unsigned integer with values from 1 to 15 and 255 as nodata. Each integer value 
corresponds to a class such as "Evergreen Forest", "Herbaceous", "Water" and so 
on.

I've been successful in manually building a VRT file using Category element for 
the classes and ColorTable entry for a palette - but only for a single band.

My question is: how to properly apply this to all bands? Do I need duplicate 
category and colortable elements to every single VRTRasterBand element or is 
there a smart way to define it once and then refer to it like a variable?  Am I 
even approaching this the right way?

It was quite a bit of work to get this far and I'm not looking forward to doing 
this 30 more times.

Sample vrt and simplified source is attached.

Matt Wilkie
Geomatics Analyst
Environment | Information Management & Technology
T 867-667-8133 | Yukon.ca



scratch_manually_built.vrt
Description: scratch_manually_built.vrt
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] What Statistics Are Returned By GDAL.Dataset.GetRasterBand(...).GetStatistics(...) ?

[Even Rouault]

On jeudi 3 octobre 2019 10:21:58 CEST Peter Willis wrote:
> Hello,
> 
> I am using osgeo.gdal  python module  to load and get stats from image
> bands.
> 
> While working with the, python, 'GetStatistics(...)' function I noticed that
> the returned values seemed a bit off.
> A snippet of code is shown below.

GDAL statistics take into account the nodata value (more exactly pixel whose 
value equals the nodata value are ignored from the statistics). That might 
explain the differences regarding the min, if 0 was the nodata value. But you 
also get differences for the max value

Can you paste the output of gdalinfo on such a file ?

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] What Statistics Are Returned By GDAL.Dataset.GetRasterBand(...).GetStatistics(...) ?

[Peter Willis]

Hello,

I am using osgeo.gdal  python module  to load and get stats from image
bands.

While working with the, python, 'GetStatistics(...)' function I noticed that
the returned values seemed a bit off.
A snippet of code is shown below.

I compared the statistics returned in python by the function to band
statistics in ENVI image processing package for the same input file.
The results are shown below.

I am wondering if I am missing information regarding some key data filtering
feature in GDAL GetStatistics.
What statistics are GDAL returning?


#
## PYTHON CODE
#

src_ds = gdal.Open(image_path )
 
for bnd in range(1, src_ds.RasterCount+1 ):
item = {}
print("[ GETTING BAND ]: ", bnd)
srcband=None
srcband = src_ds.GetRasterBand(bnd)
if srcband is None:
continue

bnddata=srcband.ReadAsArray()

stats = srcband.GetStatistics( 0, 1 )
if stats is None:
continue
else:
print('put the values in a dictionary')
##GDAL stats##
#item['min'] = stats[0]
#item['max'] = stats[1]
#item['mean'] = stats[2]
#item['stdev'] = stats[3]

##numpy array stats##
#item['min'] = bnddata.min()
#item['max'] = bnddata.max()
#item['mean'] = bnddata.mean()
#item['stdev'] = bnddata.std()

###
#END PYTHON CODE
###


/*..

..*/

ENVI Statistics test returns these vales for each channel

ChannelMinMax   MeanStdev
Band 1  0.00  0.441641  0.135938  0.095007
Band 2  0.00  0.42  0.134556  0.096385
Band 3  0.00  0.512614  0.143145  0.108702
Band 4  0.00  0.574203  0.159381  0.128212
Band 5  0.00  1.286870  0.190917  0.159562
Band 6  0.00  1.368695  0.218191  0.191100
Band 7  0.00  1.208142  0.179098  0.158407

/*..

..*/

GDAL GetStatistics in Python 3.0 Returns

[
{'min': 0.10543035715818, 'max': 0.35029646754265, 'mean': 0.19733288299107,
'stdev': 0.03141020073449}, 
{'min': 0.087364979088306, 'max': 0.36481207609177, 'mean':
0.19531263034662, 'stdev': 0.040193729911813}, 
{'min': 0.066563792526722, 'max': 0.40562310814857, 'mean':
0.20773722116063, 'stdev': 0.060965329408999}, 
{'min': 0.057971999049187, 'max': 0.49459338188171, 'mean':
0.23126556845189, 'stdev': 0.084854378172706}, 
{'min': 0.052771702408791, 'max': 0.57949388027191, 'mean':
0.27695694029464, 'stdev': 0.11419731959916}, 
{'min': 0.033055797219276, 'max': 0.67045384645462, 'mean':
0.31645853188616, 'stdev': 0.14756091787453}
]


/*..

..*/

Using NumPy Stats Functions in Python 3.0
np.min()
np.max()
np.mean()
np.std()

Return the following values after converting the previous GDAL bands to
'NumPy.Array' and using the NumPy functions
(Values look similar to ENVI)
[
{'min': 0.0, 'max': 0.44164082, 'mean': 0.13593808, 'stdev': 0.09500719}, 
{'min': 0.0, 'max': 0.4158, 'mean': 0.13455594, 'stdev': 0.09638489}, 
{'min': 0.0, 'max': 0.51261353, 'mean': 0.14314489, 'stdev': 0.108702265}, 
{'min': 0.0, 'max': 0.57420313, 'mean': 0.15938139, 'stdev': 0.12821245}, 
{'min': 0.0, 'max': 1.2868699, 'mean': 0.19091722, 'stdev': 0.15956147}, 
{'min': 0.0, 'max': 1.3686954, 'mean': 0.21819061, 'stdev': 0.1911003}
]



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Clarification on what is not addressed by GDALFlushCache

[Even Rouault]

On jeudi 3 octobre 2019 10:14:45 CEST Sean Gillies wrote:
> In the comments above FlushCache() in gcore/gdaldataset.cpp it is said:
> 
>  * Using this method does not prevent use from calling GDALClose()
>  * to properly close a dataset and ensure that important data not addressed
>  * by FlushCache() is written in the file.

> Does it vary by
> format and driver?

Of course, wouldn't be fun otherwise. For some formats, it might result in a 
completely consistent dataset, and in others, in something that can't be 
opened at all. So what is does, beyond evicting 'dirty' blocks from the cahce, 
is mostly an implementation detail.

> What exactly is the important data that is not addressed?

In the case of GeoTIFF, FlushCache() will for example ensure that all tile/
strip data is flushed to disk, but the TileByteCount/TileOffset index arrays 
are not updated, and os a file that was just created, they will be at their 
zero default value, making the dataset appear to be empty to a reader that 
would try to open it at that point.

If generating a large dataset, you can for example call FlushCache() at 
regular intervals to make sure that there is sufficiently space on the storage 
device (but the global block cache will also flush when it is saturated). This 
might be a way of avoiding the memory to reach the GDAL_CACHEMAX threshold. 
But this can also result in suboptimal behaviour if you call it at 
inappropriate point. For example if you write to a JPEG-compressed tiled TIFF, 
and your write pattern is row per row, then flushing before you reach a row 
number that is multiple of the tile height, will flush partially written 
blocks (their top will contain real data, and the bottom zeroes). So those 
blocks will be later decompressed and recompressed, causing unnecessary 
quality loss.

FlushCache() is automatically called by dataset destructor, so my tip would 
be: "do not use FlushCache() unless you know you need it"

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Clarification on what is not addressed by GDALFlushCache

[Sean Gillies]

In the comments above FlushCache() in gcore/gdaldataset.cpp it is said:

 * Using this method does not prevent use from calling GDALClose()
 * to properly close a dataset and ensure that important data not addressed
 * by FlushCache() is written in the file.

What exactly is the important data that is not addressed? Does it vary by
format and driver?

Thanks!

-- 
Sean Gillies
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.0 Nuget packages...

[Tamas Szekeres]

Hi Morten,

There have been problems with the GDAL 3.0 nuget versions, that's why that
was unlisted.
Specifically, the way how proj6 would want to find proj.db is not
satisfactory regarding to the C# applications.
I'm still working on a solution.

Best regards,

Tamas




Morten Breiner  ezt írta (időpont: 2019. okt. 3., Cs,
10:26):

> Hi,
>
> Currently, I am using GDAL 2.4.2 for some c# development in Visual Studio.
>
> I have installed the nuget packages found under
> https://www.nuget.org/packages/GDAL/2.4.2 (many thanks to all involved
> in maintaining those).
>
>  From the download statistics
> (https://www.nuget.org/stats/packages/GDAL?groupby=Version), I can see
> that some attempt to support GDAL 3.0 has been made, but the packages
> have been unlisted unfortunately.
>
> Does any of you know if a release of nuget packages for GDAL 3.0 has
> been scheduled or as an alternative, if recommended GDAL 3.0 packages
> for c# can be downloaded elsewhere?
>
> Many thanks in advance for clarifying answers.
>
> BR
> -Morten Breiner
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL 3.0 Nuget packages...

[Mateusz Loskot]

On Thu, 3 Oct 2019 at 10:26, Morten Breiner  wrote:
>
>  From the download statistics
> (https://www.nuget.org/stats/packages/GDAL?groupby=Version), I can see
> that some attempt to support GDAL 3.0 has been made, but the packages
> have been unlisted unfortunately.
>
> Does any of you know if a release of nuget packages for GDAL 3.0 has
> been scheduled

Those are not official GDAL packages but third-party publications.
You will need to "Contact owners" at https://www.nuget.org/packages/GDAL/

> or as an alternative, if recommended GDAL 3.0 packages
> for c# can be downloaded elsewhere?

Check binaries and SDKs regularly published by Tamas,
maintainer of C# bindings, at https://www.gisinternals.com

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL 3.0 Nuget packages...

[Morten Breiner]

Hi,

Currently, I am using GDAL 2.4.2 for some c# development in Visual Studio.

I have installed the nuget packages found under 
https://www.nuget.org/packages/GDAL/2.4.2 (many thanks to all involved 
in maintaining those).


From the download statistics 
(https://www.nuget.org/stats/packages/GDAL?groupby=Version), I can see 
that some attempt to support GDAL 3.0 has been made, but the packages 
have been unlisted unfortunately.


Does any of you know if a release of nuget packages for GDAL 3.0 has 
been scheduled or as an alternative, if recommended GDAL 3.0 packages 
for c# can be downloaded elsewhere?


Many thanks in advance for clarifying answers.

BR
-Morten Breiner
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev