[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena| Owner: grass-...@… Type: defect| Status: new Priority: major | Milestone: 6.4.1 Component: Raster| Version: svn-trunk Keywords: r.out.gdal, tiff |Platform: All Cpu: All | --+- Comment(by hamish): is this one now done? aside: do grass tiff-metadata colors now appear in qgis automagically? -- Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:56 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena| Owner: grass-...@… Type: defect| Status: new Priority: major | Milestone: 6.4.1 Component: Raster| Version: svn-trunk Keywords: r.out.gdal, tiff |Platform: All Cpu: All | --+- Comment(by mmetz): Replying to [comment:56 hamish]: is this one now done? Should be, in all branches. aside: do grass tiff-metadata colors now appear in qgis automagically? I haven't checked the latest qgis version, but it did work in earlier versions of qgis. Also interesting: does r.in.gdal read grass tiff- metadata colors, preferably prefering them over regular geotiff colortables (for UINT16, import e.g. 5 instead of 65536 rules)? Markus M -- Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:57 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work -+-- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: major | Milestone: 6.4.1 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: All | Cpu: All -+-- Changes (by hamish): * milestone: 6.5.0 = 6.4.1 -- Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:55 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work -+-- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: major | Milestone: 6.5.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: All | Cpu: All -+-- Comment (by hamish): see also #788. -- Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:54 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work
-+--
Reporter: helena | Owner: grass-dev@lists.osgeo.org
Type: defect | Status: new
Priority: major | Milestone: 6.5.0
Component: Raster | Version: svn-trunk
Resolution: |Keywords: r.out.gdal, tiff
Platform: All | Cpu: All
-+--
Comment (by mmetz):
Replying to [comment:52 hamish]:
Replying to [comment:49 mmetz]:
- GDAL signed int types: (...)
- GDAL unsigned int types: (...)
it all seems a bit complicated, but ok.
that's my understanding of the discussion with Glynn:
[http://lists.osgeo.org/pipermail/grass-dev/2009-April/043464.html
http://lists.osgeo.org/pipermail/grass-dev/2009-April/043464.html] and
following.
* before actual export, in case of custom nodata make sure the
metadata nodata value and the raster nodata value are identical
why? if custom nodata then export NULLs in the map to be the custom
value and clobber any real data which had that value.
(perhaps I don't understand something here..)
What I meant here was to make sure that the value that replaces NULL cells
is identical to the metadata nodata value, otherwise nodata info gets
lost. Clobbering real data with the nodata value is a different issue
(that is taken care of).
- if (nodata != (double) (GDAL datatype) nodata) - warning and
nodata = (double) (GDAL datatype) nodata
if the user asks to use a certain nodata value and it is illegal for the
data type then exit with an error, probably giving the available range in
the error message. don't automagically correct it for them and continue
(ie override their expressed wishes). It is a recipe for pain.
ok, will change it. The first warning will tell what would happen to the
nodata value, the second warning will report the selected GDAL datatype
and its range, then raster export aborts. E.g.
{{{
WARNING: Mismatch between metadata nodata value and actual nodata value in
exported raster: specified nodata value -.00 gets
converted to 241 by selected GDAL datatype.
WARNING: GDAL datatype: Byte, valid range: 0 - 255
ERROR: Raster export aborted.
}}}
BTW, it would be the selected GDAL datatype that overrides the user's
wishes by type casting, not r.out.gdal.
Hamish, what exactly should this compatibility flag do? There
is all sorts of software with all sorts of different
deficiencies out there...
umm, I forget what that was in reference to. ??minimalistic metadata
output??
There are already GeoTIFF compatibility hints in the manual, I would like
to leave it like that.
is everyone happy with the colortable export stuff now?
(my only issue with it is that if you pass the no-metadata create
option to GDAL, GRASS adds its stuff anyway)
Please note that nothing much changed there, colortable export can be
disabled from 6.4 onwards. AFAIKT, that GRASS stuff is currently only used
by QGIS, not even GRASS i.e. r.in.gdal or r.external use that information.
this is a substantial last-minute change to a core module
Unfortunately, but I would really like to get this ticket solved, please
test!
Markus M
--
Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:53
GRASS GIS http://grass.osgeo.org
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): All changes described in [#comment:49 comment 49] are submitted in trunk r37764. The -f flag overrides the precision test before actual export and the nodata value test during actual export. Markus M -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:50 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work -+-- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: major | Milestone: 6.5.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: All | Cpu: All -+-- Changes (by hamish): * priority: critical = major * platform: Unspecified = All * cpu: Unspecified = All * milestone: 6.4.0 = 6.5.0 -- Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:51 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Glynn Clements wrote:
Markus Metz wrote:
Considering this, rather use e.g. raster_min - 1 as default nodata value
for GDAL floating point datatypes?
If fabs(raster_min) is large, raster_min - 1 won't be exactly
representable, and may be rounded to raster_min. You would need to
subtract a value which depends upon the exponent.
Also, the input data may contain -DBL_MAX. For FCELL, if the map is
large enough, it's possible that *all* finite values occur in the
data.
I was (honestly! I was listening to you:-)) thinking about something like
if (raster_min type_min) {
unsigned int i = 0;
while ((float)(raster_min - ++i) == raster_min);
nodata = raster_min - i;
}
else if (raster_max type_max) {
unsigned int i = 0;
while ((float)(raster_max + ++i) == raster_max);
nodata = raster_max + i;
}
else
nodata = 0; /* or whatever fallback */
for GDT_Float32 but did not want to go into so much detail.
The question is: is GDAL's no-data value allowed to be any FP value,
or must it be finite? I'm guessing that the GDAL documentation doesn't
say (similarly, I don't think it's documented whether GRASS
FCELL/DCELL values are allowed to be infinite).
Actually, this may depend upon the format. If a format supports FP,
the format's specification may dictate whether or not NaN and/or
inifinities are considered valid.
Hmm. I thought the point of using GDAL is that r.out.gdal does *not*
have to bother about the format's specifications. IMHO, if there is
reason to assume that NaN and/or infinities are not valid for any of the
formats supporting FP, r.out.gdal should not use NaN and/or infinities.
OTOH, NaN and infinities are described in the IEEE FP standard. I would
avoid format-specific code in r.out.gdal, considering the large number
of supported formats and the number of creation options for each format
(for each format sometimes differing between datatypes, e.g. GeoTIFF).
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Glynn Clements wrote: Markus Metz wrote: - all floating point: IEEE's NaN Problem with NaN? According to IEEE 754, x == y is always FALSE if either x or y or both are NaN. Assuming (nodata = GDALGetRasterNoDataValue()) == NaN, Sorry, should have been nodata = GDALGetRasterNoDataValue() and isnan(nodata) then going through all cells if (cell == nodata) will always be FALSE, nodata cells may not be detected? Depends on each implementation of nodata detection? Can be solved for GRASS, but how is this done in all other GIS applications? if (x == nodata || isnan(nodata) isnan(x)) isnan() is C99 and POSIX, or it can be defined as e.g.: #define ISNAN(x) ((x) != (x)) As previously in ticket #73, I created a MASK and then exported elevation from nc_spm_08 as FLOAT32, once with nodata=0 and once with default nodata value NaN for GDT_Float32. GRASS 7 r.in.gdal: all ok GRASS 7 r.external: all ok Other GIS applications: QGIS 1.0.0: all ok gvSIG 1.1.2: doesn't support the nodata concept, and NaN is displayed like 0.0 SAGA 2.0.3: display ok for NaN, but cell query gives 0.0 instead of NaN or nodata, and NoData info says 0 ESRI ArcMap 9.2: display ok for nodata=0, not ok when using NaN, NaN displayed like value 0.0, and cell query gives NaN instead of nodata Considering this, rather use e.g. raster_min - 1 as default nodata value for GDAL floating point datatypes? ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Trying to summarize the suggestions and discussions for a new design: * before actual export new precision test for each band - FCELL/DCELL to be exported as some GDAL integer type - warning - CELL exported as GDT_FLOAT32 and raster_min -2^24^ or raster_max 2^24^ - warning - DCELL exported as GDT_Float32 - warning * before actual export new range test for each band - if raster_min = type_min and raster_max = type_max - fine, proceed - else if raster_min type_max or raster_max type_min - complete data loss, error - else there is a partial data range overlap - check data range during actual export * before actual export, get a reasonable default nodata value if none given - NaN for all GDAL floating point datatypes. Is NaN constructed with 0.0/0.0 ok? - GDAL signed int types: first try minimum in potential range, if raster_min = type_min, try maximum in potential range, if raster_max = type_max, use minimum (would be (GInt32) 0x8000 for GDT_Int32). This can re-use results of range check above. - GDAL unsigned int types: first try maximum in potential range, if raster_max = type_max, try minimum in potential range, if raster_min = type_min, use maximum. This can re-use results of range check above. - no warnings or errors at this stage because it is yet unknown if there are any NULL cells in the (with g.region selected part of the) raster maps to be exported * before actual export, in case of custom nodata make sure the metadata nodata value and the raster nodata value are identical - if (nodata != (double) (GDAL datatype) nodata) - warning and nodata = (double) (GDAL datatype) nodata - This would give feedback to the user about what GDAL has to do later for the selected export datatype and nodata value. And the metadata nodata value would always be identical to the value replacing NULL cells. * during actual export, check for presence of cells with nodata value - NULL cells present: message about nodata value used to replace NULLs - if there are cells == nodata value and NULL cells were assigned that nodata value - -f flag and user nodata value: warning, else error * during actual export, check actual raster range against range of GDAL datatype if needed (depends on result of range check above) - raster_min type_min or raster_max type_max - data loss, error as in previous C versions of r.out.gdal, only write GDAL nodata to metadata if NULLs were encountered. It seems that GDAL metadata nodata is raster band specific, not globally set for the whole file; r.out.gdal behaves accordingly. All warnings and errors should clearly explain what is the problem and give tips on how it can be solved. Hamish, what exactly should this compatibility flag do? There is all sorts of software with all sorts of different deficiencies out there... I hope I didn't forget something Markus M -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:49 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Markus Metz wrote: Considering this, rather use e.g. raster_min - 1 as default nodata value for GDAL floating point datatypes? If fabs(raster_min) is large, raster_min - 1 won't be exactly representable, and may be rounded to raster_min. You would need to subtract a value which depends upon the exponent. Also, the input data may contain -DBL_MAX. For FCELL, if the map is large enough, it's possible that *all* finite values occur in the data. The question is: is GDAL's no-data value allowed to be any FP value, or must it be finite? I'm guessing that the GDAL documentation doesn't say (similarly, I don't think it's documented whether GRASS FCELL/DCELL values are allowed to be infinite). Actually, this may depend upon the format. If a format supports FP, the format's specification may dictate whether or not NaN and/or inifinities are considered valid. -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Glynn Clements wrote: Markus Metz wrote: so the user will have to understand the conversion issues even if they never use an out-of-range value. Ultimately, its either usability or flexibility. Without the -f flag I would opt for usability, with the -f flag for flexibility. Having -f affect the interpretation of nodata= also harms usability. I know. There was discussion about that, and Dylan asked for the possibility to 2. include a 'force' option to make r.out.gdal do exactly what your command line instructions suggest-- possibly leading to corruption of NULL data Maybe this referred only to the problem of input data having the requested nodata value, and not to allow a mismatch between metadata nodata and the value replacing NULL cells? The GDAL API documentation says To clear the nodata value, just set it with an out of range value. This is currently possible with the -f flag, but r.out.gdal then does not give any information on what value is assigned to NULL cells, only what nodata value is written to the metadata. Right. There's only one nodata= option, and it's performing a dual function: what to store for null cells and what to set GDAL's no-data value to. If you want to allow a mismatch, you really need two options. I thought in this case (allow mismatch) it would be enough to write it as double to metadata without prior cast to the selected GDAL datatype. Yeah, but what are you going to write for nulls in the input data? Suppose the user specifies a CELL input map (which contains at least 0-255 and null), byte (GDT_UInt8) output, -f, and nodata=. So you use .0 as GDAL's no-data value, but how will you treat null values in the input data? In this case, the current code of all C versions of r.out.gdal writes (CELL) for NULLs. This is then cast by the gdal library to GDT_Byte (255). Maybe I misunderstood the requested behaviour for the -f flag, so let's not allow a nodata mismatch, convert the nodata value first to the selected GDAL datatype if necessary and then we get the same value for nodata metadata and raster data where NULLs were replaced. In this case, there should be a warning because (GDT_Byte) .0 = 255 is present in the input data and the user is asked to select a different nodata value. Export would proceed because of -f. Anyway, now I will stop defending a feature that I was never convinced of (allowing this nodata mismatch). Can we now put together a next version of r.out.gdal based on this discussion that always interprets any user-given nodata value (with warning where appropriate) to make sure metadata info and raster data are the same? ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
On 22/04/09 09:51, Markus Metz wrote: Anyway, now I will stop defending a feature that I was never convinced of (allowing this nodata mismatch). Can we now put together a next version of r.out.gdal based on this discussion that always interprets any user-given nodata value (with warning where appropriate) to make sure metadata info and raster data are the same? +1 Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work
--+-
Reporter: helena | Owner: grass-dev@lists.osgeo.org
Type: defect | Status: new
Priority: critical | Milestone: 6.4.0
Component: Raster | Version: svn-trunk
Resolution: |Keywords: r.out.gdal, tiff
Platform: Unspecified | Cpu: Unspecified
--+-
Comment (by hamish):
my 2c o'today,
* if type is UInt8 and input map has values outside of that, just report
an error and exit. you don't have to search every cell first, just use
G_read_range()+G_get_range_min_max() or
G_read_fp_range()+G_get_fp_range_min_max(). tricks like
{{{if(valtype_min) val=(type_min||NULL);}}} are evil. the user can use
r.mapcalc or r.reclass if they really want that. better we refuse to
honour broken requests.
* exit with an error if user supplied nodata= is outside of range for the
given type. if no value is given use the following rules:
- signed int types: minimum in range
- unsigned int types: maximum in range
- all floating point: IEEE's NaN
* ?? can we wait to tell gdal the nodata value (for the metatag) until
after we have processed all rows?? if so we can just not set one with GDAL
at all if we didn't come across any NULL cells in the write loop. We must
decide what to use if we do come across one before we start the loop (see
previous point) that would allow for a 0-255 GRASS CELL map with no NULLs
to preserve all data, while allowing a CELL map with 0-255+NULL,
type=UInt8, and the user specified a nodata= value in the range of 0-255
to correctly follow the user's wishes. If we do that then we don't need a
-f flag (which I'd rather we didn't have).
* drop the complex types in grass7. we can easily re-add them if the
next-gen raster format can store multiband i.fft output maps or whatever.
has anyone ever once come across in the wild a map using these types? or
are they on the list just because they can be?
* if writing a message saying NULL cells were found in the map but the
nodata option was not specified. Using a default value of x. beware
that what is printed by the G_warning() or G_message() text may not be the
internally stored value, as it has been passed through printf %{}. so
don't trust your own eyes looking at that output + we should carefully
test that. I guess it means supplying a different %ld %u or whatever for
each individual type, but so be it.
[crossing-threads]
* 999 as NULL is dangerous, it can easily be an elevation value. - is
bad too, you just hit it less often ( data isn't always elevation). But
even though you hit it less often doesn't mean it's still not going to
bite you one day, so it's a bug. bugs which don't happen all the time
could be considered more evil, because you have come to trust the program
and are not expecting them. but alas, probably for r.out.arc we are stuck
with whatever values Arc will take for input.
Hamish
--
Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:47
GRASS GIS http://grass.osgeo.org
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:47 hamish]: my 2c o'today, * if type is UInt8 and input map has values outside of that, just report an error and exit. you don't have to search every cell first, just use G_read_range()+G_get_range_min_max() or G_read_fp_range()+G_get_fp_range_min_max(). Note that the current region is respected and the actual data range to be exported can be smaller than reported by G_get_fp_range_min_max(). OTOH, taking that into consideration may be overdoing it a bit. * exit with an error if user supplied nodata= is outside of range for the given type. Glynn suggested interpretation: if (nodata != (double) (GDAL datatype) nodata) - warning, nodata = (double) (GDAL datatype) nodata, proceed. (I hope I got that right for a change...) Result would be a valid raster, granted that subsequent checks are passed. * ?? can we wait to tell gdal the nodata value (for the metatag) until after we have processed all rows?? if so we can just not set one with GDAL at all if we didn't come across any NULL cells in the write loop. We have waited all the time :-) That was always done in the C version and AFAICT nobody wants to change that. We must decide what to use if we do come across one before we start the loop (see previous point) that would allow for a 0-255 GRASS CELL map with no NULLs to preserve all data, while allowing a CELL map with 0-255+NULL, type=UInt8, and the user specified a nodata= value in the range of 0-255 to correctly follow the user's wishes. 0-255+NULL, type=UInt8 causes data loss unless the user found a free unused slot within 0-255. IMHO, big warning if not error on data loss i.e. when nodata value encountered *and* NULLs present. I'll try to summarize the suggestions so far into a description of the new design for r.out.gdal, but I need a bit more time to read again through all contributions. Markus M -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:48 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
GRASS GIS wrote: #73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by hamish): my 2c o'today, ... * exit with an error if user supplied nodata= is outside of range for the given type. if no value is given use the following rules: - signed int types: minimum in range - unsigned int types: maximum in range - all floating point: IEEE's NaN Problem with NaN? According to IEEE 754, x == y is always FALSE if either x or y or both are NaN. Assuming (nodata = GDALGetRasterNoDataValue()) == NaN, then going through all cells if (cell == nodata) will always be FALSE, nodata cells may not be detected? Depends on each implementation of nodata detection? Can be solved for GRASS, but how is this done in all other GIS applications? Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Markus Metz wrote: - all floating point: IEEE's NaN Problem with NaN? According to IEEE 754, x == y is always FALSE if either x or y or both are NaN. Assuming (nodata = GDALGetRasterNoDataValue()) == NaN, then going through all cells if (cell == nodata) will always be FALSE, nodata cells may not be detected? Depends on each implementation of nodata detection? Can be solved for GRASS, but how is this done in all other GIS applications? if (x == nodata || isnan(nodata) isnan(x)) isnan() is C99 and POSIX, or it can be defined as e.g.: #define ISNAN(x) ((x) != (x)) -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Glynn Clements wrote: Markus Metz wrote: I think I slowly begin to understand. You suggest to use (GDT_Int32)0x8000 for both the GDAL metadata info and the GDAL raster data although GDAL metadata nodata is double? Yes; for the latter, the value would be: (double) (GDT_Int32) 0x8000 IOW, -2147483648.0. Note that this isn't the same as atof(0x8000), which would be positive. Obviously. Because of the request for a -f flag I was fixed on leaving r.out.gdal parameters as they are, also leaving default nodata as double, no prior cast to the GDAL datatype. My confusion, it's clear now what you want for the default nodata value. Further on, I understand you suggest to interpret the user given nodata option by writing to metadata (double) (GDAL datatype) nodata value. No more objections from my side, but I would print a warning if (nodata != (double) (GDAL data type) nodata), telling the user what the nodata value will be after GDAL converted it. I would suggest to keep writing (double) nodata value with the -f flag, no prior cast. In case of the -f flag and a mismatch (nodata != (double) (GDAL data type) nodata), a message/warning could be printed with both the nodata value written to metadata and the value assigned to NULL cells. I was assuming that not only CELL, but also FCELL and DCELL maps may be exported as GDT_UInt32. Then 2^31 can occur in the raster to be exported. Actually, I was assuming that any GRASS raster type may be exported as any GDAL data type. Right. But exporting FCELL or DCELL as GDT_UInt32 is inherently lossy. Just to make it clear: I don't insist on exporting to GDT_UInt32, but this is supported by r.out.gdal for all GRASS raster types, so this possibility must be considered IMHO, unless GDT_UInt32 becomes disabled. For DCELL, every possible GDT_UInt32 value may occur in the source data, so there isn't any ideal default no-data value. That's why I first want to check the range of the raster map(s) to be exported. What if the user asks for a nodata value to be out of range, i.e. really wants it to be out of range, leading to a mismatch between the nodata value in the metadata and the value assigned to NULL cells in the GDAL export? I understood that this was requested to be possible. Hmm. To allow that, you can't cast to the source or destination type, Exactly. That's why I was thinking about e.g. (double) 2147483648 to be written to metadata, and not (double) (GDT_Int32) 2147483648. BTW, I can still not see why a user would want that, but it can be allowed. so the user will have to understand the conversion issues even if they never use an out-of-range value. Ultimately, its either usability or flexibility. Without the -f flag I would opt for usability, with the -f flag for flexibility. If r.out.gdal should support out of range nodata values, and the nodata value is read with atof(), 2^31 should stay 2^31and not be cast to -2^31. The GUI description of the nodata option says that it is of type float (actually it's a double), i.e. suggesting that any value that is representable as floating point is ok. That's a limitation of the parser. If the input is CELL, then the nodata value which the input data uses isn't any kind of float. It might be more correct to have separate int/float nodata options. OTOH, GDAL's no-data value is always a double, even for integer data. I would rather stick to GDAL's method of handling nodata options and try to mimic that in r.out.gdal. I don't have any preference, although I suspect that users will regularly get bitten by the conversion issues. As above, by default r.out.gdal could write (double) (GDAL datatype) nodata value to metadata, with the -f flag no cast to GDAL datatype? The GDAL API documentation says To clear the nodata value, just set it with an out of range value. This is currently possible with the -f flag, but r.out.gdal then does not give any information on what value is assigned to NULL cells, only what nodata value is written to the metadata. Right. There's only one nodata= option, and it's performing a dual function: what to store for null cells and what to set GDAL's no-data value to. If you want to allow a mismatch, you really need two options. I thought in this case (allow mismatch) it would be enough to write it as double to metadata without prior cast to the selected GDAL datatype. I don't see the point in treating range limits differently to precision limits. Either the output type is a superset of the input type (lossless export) or it isn't (lossy export). Easy to change, with feedback to the user as warning in case of lossy export. When exporting to FP types, the default should probably be NaN (if supported), regardless of whether the source is integer or FP. Exporting CELL to GDT_Float32 should print a warning if the range of the input data exceeds +/- 2^24, as it cannot be
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Markus Metz wrote: so the user will have to understand the conversion issues even if they never use an out-of-range value. Ultimately, its either usability or flexibility. Without the -f flag I would opt for usability, with the -f flag for flexibility. Having -f affect the interpretation of nodata= also harms usability. The GDAL API documentation says To clear the nodata value, just set it with an out of range value. This is currently possible with the -f flag, but r.out.gdal then does not give any information on what value is assigned to NULL cells, only what nodata value is written to the metadata. Right. There's only one nodata= option, and it's performing a dual function: what to store for null cells and what to set GDAL's no-data value to. If you want to allow a mismatch, you really need two options. I thought in this case (allow mismatch) it would be enough to write it as double to metadata without prior cast to the selected GDAL datatype. Yeah, but what are you going to write for nulls in the input data? Suppose the user specifies a CELL input map (which contains at least 0-255 and null), byte (GDT_UInt8) output, -f, and nodata=. So you use .0 as GDAL's no-data value, but how will you treat null values in the input data? -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Glynn Clements wrote: Markus Metz wrote: OK, but I have tested r.out.gdal with type = GDT_Int32 and nodata = 0x8000. Same result: NULL cells become -2147483648 but the nodata value in the metadata says 2147483648 (gdalinfo on the export output), Yes. I know. Running r.out.gdal ... nodata=0x8000 is misinterpreting what I was saying, as r.out.gdal parses nodata= as a floating-point value. I'm talking about using (GDT_Int32)0x8000 (i.e. -2^31) I think I slowly begin to understand. You suggest to use (GDT_Int32)0x8000 for both the GDAL metadata info and the GDAL raster data although GDAL metadata nodata is double? When using a default nodata value, what I want to get in the exported map is identical values for the metadata nodata value and the value assigned to NULL cells. GDAL metadata nodata value is always double (GDALSetRasterNoDataValue()), the value assigned to NULL cells is of the selected GDAL datatype. So 0x8000 for GDT_Int32 will be 2^31 in metadata and -2^31 in the data if passed as such to the corresponding GDAL functions, (double ) 0x8000 for metadata info and (GDT_Int32) 0x8000 for raster data. or (GDT_UInt32)0x8000 (i.e. +2^31) as appropriate for the destination type. I was assuming that not only CELL, but also FCELL and DCELL maps may be exported as GDT_UInt32. Then 2^31 can occur in the raster to be exported. Actually, I was assuming that any GRASS raster type may be exported as any GDAL data type. For r.out.gdal, there was discussion about not to override user options and instead issue an error or a warning (going for error now) that the nodata value is out of range. This isn't about overriding user options, but interpreting them correctly. Close to the module knows better which is not desirable according to Dylan and Moritz. No, the opposite. Writing code which does what the user asks for rather than doing something else which is easier to code. What if the user asks for a nodata value to be out of range, i.e. really wants it to be out of range, leading to a mismatch between the nodata value in the metadata and the value assigned to NULL cells in the GDAL export? I understood that this was requested to be possible. If r.out.gdal should support out of range nodata values, and the nodata value is read with atof(), 2^31 should stay 2^31and not be cast to -2^31. The GUI description of the nodata option says that it is of type float (actually it's a double), i.e. suggesting that any value that is representable as floating point is ok. That's a limitation of the parser. If the input is CELL, then the nodata value which the input data uses isn't any kind of float. It might be more correct to have separate int/float nodata options. OTOH, GDAL's no-data value is always a double, even for integer data. I would rather stick to GDAL's method of handling nodata options and try to mimic that in r.out.gdal. OTOH, reading as FP allows you to specify +2^31 as the nodata value when exporting CELL as UInt32. Maybe it would be better to read as double then cast to the destination type. Hmm, then it would be no longer possible to have an out-of-range nodata value in the metadata. I would support that solution, but it was explicitly requested to obey all user options when the -f flag is used, also writing the nodata value as it is to the metadata. It is then up to the user to figure out what happened to the NULL cells, assuming that the user knows what happens when using the -f flag. I can see some point to that, but may be confusing for the user, who may assume that they're talking about the data's no-data value when they're actually specifying GDAL's no-data value. That's why there are warnings if there will be a mismatch between the GDAL metadata nodata value and the GDAL raster nodata value (out of range test). And having this mismatch is only possible with the -f flag, when forcing r.out.gdal to use exactly the given parameters and options. The GDAL API documentation says To clear the nodata value, just set it with an out of range value. This is currently possible with the -f flag, but r.out.gdal then does not give any information on what value is assigned to NULL cells, only what nodata value is written to the metadata. Even so, 0x8000 is still the ideal value for exporting CELL data to either UInt32 or Int32. You just have to pick the correct interpretation of it (unsigned int or signed int respectively). The current as well as previous policy of r.out.gdal is to not to interpret the nodata value. It is written as is (currently double) to metadata, same like e.g. gdal_translate -a_nodata behaves. It doesn't have any choice about interpreting it. nodataopt-answer is a char*, while GDALSetRasterNoDataValue() expects a double. There are several ways to convert a char* to a double; atof() may be the simplest to implement, but is it the most correct? No idea. If you can't think if a more correct way,
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Markus Metz wrote: OK, but I have tested r.out.gdal with type = GDT_Int32 and nodata = 0x8000. Same result: NULL cells become -2147483648 but the nodata value in the metadata says 2147483648 (gdalinfo on the export output), Yes. I know. Running r.out.gdal ... nodata=0x8000 is misinterpreting what I was saying, as r.out.gdal parses nodata= as a floating-point value. I'm talking about using (GDT_Int32)0x8000 (i.e. -2^31) I think I slowly begin to understand. You suggest to use (GDT_Int32)0x8000 for both the GDAL metadata info and the GDAL raster data although GDAL metadata nodata is double? Yes; for the latter, the value would be: (double) (GDT_Int32) 0x8000 IOW, -2147483648.0. Note that this isn't the same as atof(0x8000), which would be positive. When using a default nodata value, what I want to get in the exported map is identical values for the metadata nodata value and the value assigned to NULL cells. GDAL metadata nodata value is always double (GDALSetRasterNoDataValue()), the value assigned to NULL cells is of the selected GDAL datatype. So 0x8000 for GDT_Int32 will be 2^31 in metadata and -2^31 in the data if passed as such to the corresponding GDAL functions, (double ) 0x8000 for metadata info and (GDT_Int32) 0x8000 for raster data. That depends upon how you interpret 0x8000. (double) (GDT_Int32) 0x8000 is equal to -2147483648.0, while atof(0x8000) is equal to +2147483648.0. GDT_Int32 itself cannot represent +2^31. When writing CELL data as GDT_Int32, the default null value should be (GDT_Int32) 0x8000, as that's the only value which cannot occur in the source data. The GDAL no-data value should be that value cast to a double, i.e.: (double) (GDT_Int32) 0x8000 or: atof(-0x8000) or: -2147483648.0 or any other permutation which produces the same double value. or (GDT_UInt32)0x8000 (i.e. +2^31) as appropriate for the destination type. I was assuming that not only CELL, but also FCELL and DCELL maps may be exported as GDT_UInt32. Then 2^31 can occur in the raster to be exported. Actually, I was assuming that any GRASS raster type may be exported as any GDAL data type. Right. But exporting FCELL or DCELL as GDT_UInt32 is inherently lossy. For DCELL, every possible GDT_UInt32 value may occur in the source data, so there isn't any ideal default no-data value. For r.out.gdal, there was discussion about not to override user options and instead issue an error or a warning (going for error now) that the nodata value is out of range. This isn't about overriding user options, but interpreting them correctly. Close to the module knows better which is not desirable according to Dylan and Moritz. No, the opposite. Writing code which does what the user asks for rather than doing something else which is easier to code. What if the user asks for a nodata value to be out of range, i.e. really wants it to be out of range, leading to a mismatch between the nodata value in the metadata and the value assigned to NULL cells in the GDAL export? I understood that this was requested to be possible. Hmm. To allow that, you can't cast to the source or destination type, so the user will have to understand the conversion issues even if they never use an out-of-range value. Ultimately, its either usability or flexibility. If r.out.gdal should support out of range nodata values, and the nodata value is read with atof(), 2^31 should stay 2^31and not be cast to -2^31. The GUI description of the nodata option says that it is of type float (actually it's a double), i.e. suggesting that any value that is representable as floating point is ok. That's a limitation of the parser. If the input is CELL, then the nodata value which the input data uses isn't any kind of float. It might be more correct to have separate int/float nodata options. OTOH, GDAL's no-data value is always a double, even for integer data. I would rather stick to GDAL's method of handling nodata options and try to mimic that in r.out.gdal. I don't have any preference, although I suspect that users will regularly get bitten by the conversion issues. OTOH, reading as FP allows you to specify +2^31 as the nodata value when exporting CELL as UInt32. Maybe it would be better to read as double then cast to the destination type. Hmm, then it would be no longer possible to have an out-of-range nodata value in the metadata. I would support that solution, but it was explicitly requested to obey all user options when the -f flag is used, also writing the nodata value as it is to the metadata. It is then up to the user to figure out what happened to the NULL cells, assuming that the user knows what happens when using the -f flag. I can see some point to that, but may be confusing for the user,
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Glynn Clements wrote: Markus Metz wrote: I guess a minimum requirement would be that something exported with r.out.gdal and then imported again with r.in.gdal should be identical (taking into consideration the region settings during export) and not have any data loss, be it due to clipping data or due to nodata problems. That's only possible if the exported data is in a format which can hold the full range of the data. A UInt32 TIFF can't hold the full range of CELL data (it can't hold any negative values). A UInt32 can hold the full range of a CELL raster map if there are no negative values. What you mean is the potential range, what I mean is the data range actually present in the raster. Which still means that UInt32 doesn't really make sense as GDAL export type for GRASS rasters. That would also mean that r.out.gdal with type = GDT_Int32 and nodata = 2147483648, NULL cells become -2147483648 but the nodata value in the metadata stays 2147483648 (gdalinfo on the export output), which in turn means that other software, also QGIS, does not see any nodata cells in the export output -information loss about NULL cells. That's why I wouldn't use 0x8000 as default nodata value for GDT_Int32 in r.out.gdal even if r.in.gdal has no problems with it. You're misinterpretating my use of 0x8000 here. I'm talking about the C interpretation, which is the (signed) integer value -2147483648. OK, but I have tested r.out.gdal with type = GDT_Int32 and nodata = 0x8000. Same result: NULL cells become -2147483648 but the nodata value in the metadata says 2147483648 (gdalinfo on the export output), and other software including QGIS doesn't see any nodata cells. I haven't changed the part of the code of r.out.gdal that writes the nodata value to metadata, also not the code that writes the actual raster band. For r.out.gdal, there was discussion about not to override user options and instead issue an error or a warning (going for error now) that the nodata value is out of range. This isn't about overriding user options, but interpreting them correctly. Close to the module knows better which is not desirable according to Dylan and Moritz. It's debatable whether just using atof(nodataopt-answer) directly is actually a correct interpretation when the input map is CELL, or whether the code should use e.g. atoi(nodataopt-answer) or (CELL)atof(nodataopt-answer) for CELL inputs. Reading as int (or casting to it) will interpret 0x8000 as -2^31, while reading as FP will interpret it as 2^31. If r.out.gdal should support out of range nodata values, and the nodata value is read with atof(), 2^31 should stay 2^31and not be cast to -2^31. The GUI description of the nodata option says that it is of type float (actually it's a double), i.e. suggesting that any value that is representable as floating point is ok. OTOH, reading as FP allows you to specify +2^31 as the nodata value when exporting CELL as UInt32. Maybe it would be better to read as double then cast to the destination type. Hmm, then it would be no longer possible to have an out-of-range nodata value in the metadata. I would support that solution, but it was explicitly requested to obey all user options when the -f flag is used, also writing the nodata value as it is to the metadata. It is then up to the user to figure out what happened to the NULL cells, assuming that the user knows what happens when using the -f flag. Currently, all default nodata values are within the range of the selected GDAL data type. User-defined nodata values are tested whether they are within the range of the selected data type, if not, r.out.gdal in trunk and devbr6 exits with an error message. r.out.gdal also exits with an error message if the nodata value is present in the raster to be exported in order to avoid data loss. This is a rather strict and conservative approach because of the reported problems with nodata in r.out.gdal exports. Right. Even so, 0x8000 is still the ideal value for exporting CELL data to either UInt32 or Int32. You just have to pick the correct interpretation of it (unsigned int or signed int respectively). The current as well as previous policy of r.out.gdal is to not to interpret the nodata value. It is written as is (currently double) to metadata, same like e.g. gdal_translate -a_nodata behaves. Afraid of rewriting large parts of r.out.gdal: can't we just use -2147483648 as default nodata value for Int32 and 0 as default nodata value for UInt32? Getting tired of this UInt32 example, but again: (unsigned int) 0x8000 is 2^31, yes? Then this value would be right in the middle of the range of UInt32, and can cause data loss when exporting a FCELL raster as UInt32 . For the sake of the argument, let's assume this FCELL raster has only positive values, some of them 2^31, and any errors when casting from FCELL to UInt32 are acknowledged and
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Markus Metz wrote: That would also mean that r.out.gdal with type = GDT_Int32 and nodata = 2147483648, NULL cells become -2147483648 but the nodata value in the metadata stays 2147483648 (gdalinfo on the export output), which in turn means that other software, also QGIS, does not see any nodata cells in the export output -information loss about NULL cells. That's why I wouldn't use 0x8000 as default nodata value for GDT_Int32 in r.out.gdal even if r.in.gdal has no problems with it. You're misinterpretating my use of 0x8000 here. I'm talking about the C interpretation, which is the (signed) integer value -2147483648. OK, but I have tested r.out.gdal with type = GDT_Int32 and nodata = 0x8000. Same result: NULL cells become -2147483648 but the nodata value in the metadata says 2147483648 (gdalinfo on the export output), Yes. I know. Running r.out.gdal ... nodata=0x8000 is misinterpreting what I was saying, as r.out.gdal parses nodata= as a floating-point value. I'm talking about using (GDT_Int32)0x8000 (i.e. -2^31) or (GDT_UInt32)0x8000 (i.e. +2^31) as appropriate for the destination type. For r.out.gdal, there was discussion about not to override user options and instead issue an error or a warning (going for error now) that the nodata value is out of range. This isn't about overriding user options, but interpreting them correctly. Close to the module knows better which is not desirable according to Dylan and Moritz. No, the opposite. Writing code which does what the user asks for rather than doing something else which is easier to code. It's debatable whether just using atof(nodataopt-answer) directly is actually a correct interpretation when the input map is CELL, or whether the code should use e.g. atoi(nodataopt-answer) or (CELL)atof(nodataopt-answer) for CELL inputs. Reading as int (or casting to it) will interpret 0x8000 as -2^31, while reading as FP will interpret it as 2^31. If r.out.gdal should support out of range nodata values, and the nodata value is read with atof(), 2^31 should stay 2^31and not be cast to -2^31. The GUI description of the nodata option says that it is of type float (actually it's a double), i.e. suggesting that any value that is representable as floating point is ok. That's a limitation of the parser. If the input is CELL, then the nodata value which the input data uses isn't any kind of float. It might be more correct to have separate int/float nodata options. OTOH, GDAL's no-data value is always a double, even for integer data. OTOH, reading as FP allows you to specify +2^31 as the nodata value when exporting CELL as UInt32. Maybe it would be better to read as double then cast to the destination type. Hmm, then it would be no longer possible to have an out-of-range nodata value in the metadata. I would support that solution, but it was explicitly requested to obey all user options when the -f flag is used, also writing the nodata value as it is to the metadata. It is then up to the user to figure out what happened to the NULL cells, assuming that the user knows what happens when using the -f flag. I can see some point to that, but may be confusing for the user, who may assume that they're talking about the data's no-data value when they're actually specifying GDAL's no-data value. Even so, 0x8000 is still the ideal value for exporting CELL data to either UInt32 or Int32. You just have to pick the correct interpretation of it (unsigned int or signed int respectively). The current as well as previous policy of r.out.gdal is to not to interpret the nodata value. It is written as is (currently double) to metadata, same like e.g. gdal_translate -a_nodata behaves. It doesn't have any choice about interpreting it. nodataopt-answer is a char*, while GDALSetRasterNoDataValue() expects a double. There are several ways to convert a char* to a double; atof() may be the simplest to implement, but is it the most correct? Afraid of rewriting large parts of r.out.gdal: can't we just use -2147483648 as default nodata value for Int32 Yes. and 0 as default nodata value for UInt32? No; 0 is a valid data value. Use +2147483648, as that will never occur in the input data. Or even any value = 2147483648. Getting tired of this UInt32 example, but again: (unsigned int) 0x8000 is 2^31, yes? Yes. Then this value would be right in the middle of the range of UInt32, and can cause data loss when exporting a FCELL raster as UInt32 . In that case, use (double) 0xU, which will never exist in CELL or FCELL data. Maybe the default should depend upon the input type? -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Moritz Lennert wrote: In the current state is there a possibility of using gdal's acceptance of unvalid nodata ? And can we force a nodata value which exists in the map ? I would agree with Dylan that this kind of brute-force method should remain possible, be it through the suggested -f flag. I'm always wary of programs that assume they know better than you what you want ;-) Added -f flag to force export in trunk r36769 and devbr6 r36770. With the -f flag, r.out.gdal overrides the new safety checks, but still warns if thinks there may be problems. Scripts will work as before, but raster export will be aborted if any of the safety checks is not passed (supposed to be a bugfix, not an enhancement). I guess this ticket can only be closed after backporting to 6.4. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Markus Metz wrote:
This is now a mix of r.in.gdal and r.out.gdal. The two modules
complement each other, and I guess a minimum requirement would be that
something exported with r.out.gdal and then imported again with
r.in.gdal should be identical (taking into consideration the region
settings during export) and not have any data loss, be it due to
clipping data or due to nodata problems.
That's only possible if the exported data is in a format which can
hold the full range of the data. A UInt32 TIFF can't hold the full
range of CELL data (it can't hold any negative values).
The only fool-proof nodata value for CELL is 0x8000; all other
values may be actual data values.
For grass rasters of type CELL, not for gdal export as Int32. gdalinfo
says NoData Value=2147483648, but cells that should be nodata are
-2147483648 and are thus not recognized as nodata.
It worked as of a few days ago, in the version I have checked out:
raster/r.in.gdal/main.c:780:
if (((GInt32 *) cell)[indx] == (GInt32) dfNoData) {
(GInt32)2147483648 and (GInt32)-2147483648 are equal.
I think the problem is that gdal allows the nodata value to be out of
range with regard to the raster type,
Yes.
e.g. in gdal, a nodata value of
- with GDT_Byte is legal and can be used to effectively disable
nodata information. No idea if this conforms to GeoTIFF specs (probably
yes), but I'm pretty sure that this is the current policy of gdal
(possible with e.g. gdal_translate).
That would also mean that r.out.gdal with type = GDT_Int32 and nodata =
2147483648, NULL cells become -2147483648 but the nodata value in the
metadata stays 2147483648 (gdalinfo on the export output), which in turn
means that other software, also QGIS, does not see any nodata cells in
the export output -information loss about NULL cells. That's why I
wouldn't use 0x8000 as default nodata value for GDT_Int32 in
r.out.gdal even if r.in.gdal has no problems with it.
You're misinterpretating my use of 0x8000 here. I'm talking
about the C interpretation, which is the (signed) integer value
-2147483648.
If it's been changed, use nodata=-0x8000.
That's currently done in r.out.gdal in trunk and devbr6.
Or cast the nodata value
to the source data type then back to double.
For r.out.gdal, there was discussion about not to override user options
and instead issue an error or a warning (going for error now) that the
nodata value is out of range.
This isn't about overriding user options, but interpreting them
correctly.
It's debatable whether just using atof(nodataopt-answer) directly is
actually a correct interpretation when the input map is CELL, or
whether the code should use e.g. atoi(nodataopt-answer) or
(CELL)atof(nodataopt-answer) for CELL inputs. Reading as int (or
casting to it) will interpret 0x8000 as -2^31, while reading as FP
will interpret it as 2^31.
OTOH, reading as FP allows you to specify +2^31 as the nodata value
when exporting CELL as UInt32. Maybe it would be better to read as
double then cast to the destination type.
Currently, all default nodata values are
within the range of the selected GDAL data type. User-defined nodata
values are tested whether they are within the range of the selected data
type, if not, r.out.gdal in trunk and devbr6 exits with an error
message. r.out.gdal also exits with an error message if the nodata value
is present in the raster to be exported in order to avoid data loss.
This is a rather strict and conservative approach because of the
reported problems with nodata in r.out.gdal exports.
Right.
Even so, 0x8000 is still the ideal value for exporting CELL data
to either UInt32 or Int32. You just have to pick the correct
interpretation of it (unsigned int or signed int respectively).
--
Glynn Clements gl...@gclements.plus.com
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
This is now a mix of r.in.gdal and r.out.gdal. The two modules
complement each other, and I guess a minimum requirement would be that
something exported with r.out.gdal and then imported again with
r.in.gdal should be identical (taking into consideration the region
settings during export) and not have any data loss, be it due to
clipping data or due to nodata problems.
Glynn Clements wrote:
The only fool-proof nodata value for CELL is 0x8000; all other
values may be actual data values.
For grass rasters of type CELL, not for gdal export as Int32. gdalinfo
says NoData Value=2147483648, but cells that should be nodata are
-2147483648 and are thus not recognized as nodata.
It worked as of a few days ago, in the version I have checked out:
raster/r.in.gdal/main.c:780:
if (((GInt32 *) cell)[indx] == (GInt32) dfNoData) {
(GInt32)2147483648 and (GInt32)-2147483648 are equal.
I think the problem is that gdal allows the nodata value to be out of
range with regard to the raster type, e.g. in gdal, a nodata value of
- with GDT_Byte is legal and can be used to effectively disable
nodata information. No idea if this conforms to GeoTIFF specs (probably
yes), but I'm pretty sure that this is the current policy of gdal
(possible with e.g. gdal_translate).
That would also mean that r.out.gdal with type = GDT_Int32 and nodata =
2147483648, NULL cells become -2147483648 but the nodata value in the
metadata stays 2147483648 (gdalinfo on the export output), which in turn
means that other software, also QGIS, does not see any nodata cells in
the export output -information loss about NULL cells. That's why I
wouldn't use 0x8000 as default nodata value for GDT_Int32 in
r.out.gdal even if r.in.gdal has no problems with it.
If it's been changed, use nodata=-0x8000.
That's currently done in r.out.gdal in trunk and devbr6.
Or cast the nodata value
to the source data type then back to double.
For r.out.gdal, there was discussion about not to override user options
and instead issue an error or a warning (going for error now) that the
nodata value is out of range. Currently, all default nodata values are
within the range of the selected GDAL data type. User-defined nodata
values are tested whether they are within the range of the selected data
type, if not, r.out.gdal in trunk and devbr6 exits with an error
message. r.out.gdal also exits with an error message if the nodata value
is present in the raster to be exported in order to avoid data loss.
This is a rather strict and conservative approach because of the
reported problems with nodata in r.out.gdal exports.
Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
On 17/04/09 16:53, Markus Metz wrote: For r.out.gdal, there was discussion about not to override user options and instead issue an error or a warning (going for error now) that the nodata value is out of range. Currently, all default nodata values are within the range of the selected GDAL data type. User-defined nodata values are tested whether they are within the range of the selected data type, if not, r.out.gdal in trunk and devbr6 exits with an error message. r.out.gdal also exits with an error message if the nodata value is present in the raster to be exported in order to avoid data loss. This is a rather strict and conservative approach because of the reported problems with nodata in r.out.gdal exports. In the current state is there a possibility of using gdal's acceptance of unvalid nodata ? And can we force a nodata value which exists in the map ? I would agree with Dylan that this kind of brute-force method should remain possible, be it through the suggested -f flag. I'm always wary of programs that assume they know better than you what you want ;-) Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Glynn Clements wrote: Dylan Beaudette wrote: 3. Allow the user to specify what they would like NULL cells encoded as. Unless I am overlooking something, it would seem reasonable to export a CELL map to a signed integer format, and use some obvious negative value for NULL: If you're exporting to a 32-bit (signed or unsigned) integer format, the obvious choice for null is 0x8000. That's what GRASS uses internally, so it can never occur as a legitimate value (if a computation ends up producing that value, it will get treated as a null in all regards). 0x8000 as nodata value seems to work with GeoTIFF for Int32 but not for UInt32, here gdalinfo tells me NoData Value=2147483648 which is well within the range of UInt32. It may be a problem that nodataval in r.out.gdal is of type double, so 0x8000 is cast to double and then either back to int or left as double. In the case of UInt32, the raster to be exported is currently read as DCELL, written as GDT_Float64 and then converted to GDT_UInt32. There is a lot of type casting going on in r.out.gdal, making tests a bit complicated. In the end, the value assigned to NULL cells must match the nodata value in the metadata, and a safe default choice seems to be type min or type max. NaN seems to work for Float32 and Float64, at least gdal reports NoData Value=nan and QGIS display is ok for GTiff, HFA, PAux, EHdr. NaN is constructed with 0.0/0.0, idea from r.univar. IMHO, using 0x8000 or similar can be dangerous (not sure about NaN) because I don't know how these values are supported by all the different file formats and all the different applications. Testing welcome! I would suggest to use some fool-proof default nodata value for GDAL integer types, i.e. type min or type max, check if it works, otherwise abort and ask the user to specify a nodata value. For GDAL floating point types, it could be recommended to use the default nodata value (NaN) instead of specifying a nodata value. Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): An attempt to solve this ticket: updated r.out.gdal in trunk r36750 and develbranch_6 r36751 The manual has been updated considerably, trying to find a compromise between useful information and a lengthy manual. Amongst other additions, there is a new paragraph on GeoTIFF caveats in NOTES, I hope that helps a bit. Markus M -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:44 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:41 epatton]: I suppose there's not much we can do if other image viewers are not configured to read proper geotiffs. For the record, the only tiff output I can create that actually is readable (i.e., color table remains intact) is r.out.tiff. Maybe allow a flag in r.out.gdal that outputs r.out.tiff-style tiffs and retire r.out.tiff in Grass 7? r.out.gdal and r.out.tiff do two fundamentally different things and IMHO should be neither compared nor merged. r.out.tiff exports an image that can be viewed with any image viewer and looks like what you see on the grass display. The pixel values are most likely very different from the raster map that is exported, because with the -p flag everything is converted to Byte. Without the -p flag, the pixel values are the colour values of the color table, not the raster values. r.out.tiff output is thus not meant to be suitable for spatial analysis (like a screenshot of the display). r.out.gdal exports data, by default with a color table for data visualization if supported by the selected output format. The color table is just for convenience and not needed for analysis. Data export was successful if r.univar or some equivalent in another GIS software package gives results identical to r.univar on the grass raster. There is a difference between can read the GeoTIFF and displays the GeoTIFF like grass displays the original raster, therefore the display is not suitable for testing because there are too many reasons why the display can be different (different monitor with different settings, different OS with different gamma, file format does not support color tables, conversion from color rules to color tables can introduce slight modifications, other GIS software uses different graphics engine, other GIS software can not read embedded color table). Sorry for being so thick, but I want to make the point of distinguishing images and data. Unfortunately, a .tif file can be both. Markus M -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:45 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by martinl): Replying to [comment:45 mmetz]: r.out.gdal and r.out.tiff do two fundamentally different things and IMHO should be neither compared nor merged. r.out.tiff exports an image that can be viewed with any image viewer and looks like what you see on the grass display. The pixel values are most likely very different from the raster map that is exported, because with the -p flag everything is converted to Byte. Without the -p flag, the pixel values are the colour values of the color table, not the raster values. r.out.tiff output is thus not meant to be suitable for spatial analysis (like a screenshot of the display). r.out.gdal exports data, by default with a color table for data visualization if supported by the selected output format. The color table is just for convenience and not needed for analysis. Data export was successful if r.univar or some equivalent in another GIS software package gives results identical to r.univar on the grass raster. There is a difference between can read the GeoTIFF and displays the GeoTIFF like grass displays the original raster, therefore the display is not suitable for testing because there are too many reasons why the display can be different (different monitor with different settings, different OS with different gamma, file format does not support color tables, conversion from color rules to color tables can introduce slight modifications, other GIS software uses different graphics engine, other GIS software can not read embedded color table). Sorry for being so thick, but I want to make the point of distinguishing images and data. Unfortunately, a .tif file can be both. thanks for the clarification. I didn't know that. Probably it should be explained more deeply in the manual page. Martin -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:46 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:31 mlennert]: Is this still an issue, or have Markus' patches solved the problem ? My patch only adds safety checks to make sure that the selected GDAL data type covers the range of the raster band to be exported and that the given null value is within the range of the selected GDAL data type. That would be an enhancement, not a bugfix. AFAICT, r.out.gdal as it is now (without the patch) can produce perfectly readable output, but it is possible to produce bogus output without these two safety checks. This is now a trade-off between code complexity and additional checks of user-specified options. I would opt to keep the code simple, also because the manual has been updated. In any case, it would be great if this ticket could be solved before the final release of 6.4. -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:32 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:33 mlennert]: From the discussion, I gather that most of the problem is not a bug in r.out.gdal, but rather the incapacity of certain viewers to display certain types of data. Yes, that's one problem, but a problem of the viewers. Standard image viewers or editors like GIMP can only read type BYTE, and probably only 1 band or 3 bands in e.g. a tiff. Other types (e.g. INT16, FLOAT32) are not supported by these image viewers. Not sure about the number of bands. GIS software can usually both read, analyse, and visualize georeferenced raster data as produced by r.out.gdal. The tiff colortable problem is already solved. Some tests are in comment 13: http://trac.osgeo.org/grass/ticket/73#comment:13 -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:34 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:35 mlennert]: Well, is this still an open bug then, or can we close it ? Or at least downgrade it so that it doesn't block 6.4 release ? Wait for feedback from Helena, Eric, Dylan, and Hamish before downgrading or closing the ticket ? -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:36 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by hamish): This was and is my main concern. MarkusM's patch could address this, I haven't run exaustive tests so don't know. Replying to [comment:20 hamish]: - oops, you are quite right. nodata must be set to something valid, otherwise the module doesn't know how to write out NULL values when it comes to them. It can't just skip them and nan is not an option for int maps. Thus, as you proposed, in the case where nodata= is out of range for the data type the module should exit with an error. I was just thinking of what to do for the metadata tag; not how the module would deal with NULL values. - if a Byte map has data 0-255 *and* NULL, it seems reasonable to me to default to nodata=255 + issue a warning. *But* if a map has 0-255 with *no* NULLs, nodata should not be set* to 255, as this would lead to a loss of fidelity. (improper removal of all cells with cat=255 for no gain) [*] ie should not be set, unless the user requests it explicitly with the nodata= opt sorry to be so thick, Hamish As for the other matter, I'm not worried if other software do not respect the GeoTiff spec, as long as we do. (it's all we have control over anyway) more in next post.. -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:37 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by hamish): Replying to [comment:32 mmetz]: My patch only adds safety checks to make sure that the selected GDAL data type covers the range of the raster band to be exported and that the given null value is within the range of the selected GDAL data type. That would be an enhancement, not a bugfix. Pretend you are r.out.gdal iterating through the raster cells to output. You come across a NULL cell. You have to write down some value, and are completely constrained to options available in the output type. What do you write out? Overflow/wrap the type? type min? type max? well (as things were) whatever you do is going to be arbirtary, and thus could lead to data loss, which is grounds for a most serious blocker bug. also, because the manual has been updated. that is easily modified, so shouldn't stop us. In any case, it would be great if this ticket could be solved before the final release of 6.4. yup Hamish -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:38 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:38 hamish]: Pretend you are r.out.gdal iterating through the raster cells to output. You come across a NULL cell. You have to write down some value, and are completely constrained to options available in the output type. What do you write out? Overflow/wrap the type? type min? type max? well (as things were) whatever you do is going to be arbirtary, and thus could lead to data loss, which is grounds for a most serious blocker bug. That would require an additional check that the nodata value is not present in the raster band to be exported. This is not too difficult. What would be more difficult is to select a nodata value that is not present in the raster band to be exported if type min and type max are present. Offhand, I can only think of a helper data structure (sorted heap/binary tree/hash table) where all raster values without duplicates are kept which can consume a lot of memory for large floating point rasters. This would also require two passes over each raster band, first investigating the values and selecting a proper nodata value, second replacing NULLs with this nodata value. Then, what to do if a user- specified nodata value is within the range of the selected GDAL data type but this value is present in the raster? Give a warning but let the user override the check, causing data loss? Sorry, I haven't thought about this problem of nodata value causing data loss, somehow left this to the user. Markus M -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:39 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mlennert): Replying to [comment:34 mmetz]: Replying to [comment:33 mlennert]: From the discussion, I gather that most of the problem is not a bug in r.out.gdal, but rather the incapacity of certain viewers to display certain types of data. Yes, that's one problem, but a problem of the viewers. Standard image viewers or editors like GIMP can only read type BYTE, and probably only 1 band or 3 bands in e.g. a tiff. Other types (e.g. INT16, FLOAT32) are not supported by these image viewers. Not sure about the number of bands. GIS software can usually both read, analyse, and visualize georeferenced raster data as produced by r.out.gdal. The tiff colortable problem is already solved. Some tests are in comment 13: http://trac.osgeo.org/grass/ticket/73#comment:13 Well, is this still an open bug then, or can we close it ? Or at least downgrade it so that it doesn't block 6.4 release ? Moritz -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:35 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mlennert): Replying to [comment:39 mmetz]: Sorry, I haven't thought about this problem of nodata value causing data loss, somehow left this to the user. And I think it should be. Just give a big fat warning in the man page, and possibly as output of the module saying which value is used for null data and that if there are non-null pixels which have the value given as nodata value, then you will lose the information contained in them. Moritz -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:40 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
On 15/04/09 19:22, Dylan Beaudette wrote: On Wednesday 15 April 2009, GRASS GIS wrote: #73: r.out.gdal tiff output does not work --+ - Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+ - Comment (by hamish): Replying to [comment:32 mmetz]: My patch only adds safety checks to make sure that the selected GDAL data type covers the range of the raster band to be exported and that the given null value is within the range of the selected GDAL data type. That would be an enhancement, not a bugfix. Pretend you are r.out.gdal iterating through the raster cells to output. You come across a NULL cell. You have to write down some value, and are completely constrained to options available in the output type. What do you write out? Overflow/wrap the type? type min? type max? well (as things were) whatever you do is going to be arbirtary, and thus could lead to data loss, which is grounds for a most serious blocker bug. I haven't thought a great deal about this, but from what I have been reading and recalling past messages on the GDAL mailing list-- there just aren't any general approaches to preserving NULL values in a geotiff when you need the entire range of a given datatype to represent your information. I think that several things can be done to address this bug, some of which have been mentioned: 1. if there are any NULL cells, warn the user- possibly refuse to export the image in the current format (BYTE, etc.) 1a. if the selected output format cannot contain the entire range, or precision, of input map data (input is FCELL, output is BYTE) give a warning or fail with error message. 2. include a 'force' option to make r.out.gdal do exactly what your command line instructions suggest-- possibly leading to corruption of NULL data 3. Allow the user to specify what they would like NULL cells encoded as. Unless I am overlooking something, it would seem reasonable to export a CELL map to a signed integer format, and use some obvious negative value for NULL: # assuming that all REAL data is within 0-255 r.out.gdal in=CELLMAP out=map.tif format=SIGNEDINT null=-1 also, because the manual has been updated. that is easily modified, so shouldn't stop us. In any case, it would be great if this ticket could be solved before the final release of 6.4. yup I would like to see the above changes, however I cannot actually implement them. Thanks to everyone who has been working on this important module. AFAIK, except for number 1, all are implemented already. Currently there is an error if the null value does not fit into the data range of the selected output file type, but not if the selected value for null data is present in the data as a valid value. But as Markus explained, checking for this would be quite intense, so IMHO a warning that there are null values and that there is the possibility of loosing information if the user does not watch out should be enough. Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
On Wednesday 15 April 2009, Moritz Lennert wrote: On 15/04/09 19:22, Dylan Beaudette wrote: On Wednesday 15 April 2009, GRASS GIS wrote: #73: r.out.gdal tiff output does not work --+- --- - Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- --- - Comment (by hamish): Replying to [comment:32 mmetz]: My patch only adds safety checks to make sure that the selected GDAL data type covers the range of the raster band to be exported and that the given null value is within the range of the selected GDAL data type. That would be an enhancement, not a bugfix. Pretend you are r.out.gdal iterating through the raster cells to output. You come across a NULL cell. You have to write down some value, and are completely constrained to options available in the output type. What do you write out? Overflow/wrap the type? type min? type max? well (as things were) whatever you do is going to be arbirtary, and thus could lead to data loss, which is grounds for a most serious blocker bug. I haven't thought a great deal about this, but from what I have been reading and recalling past messages on the GDAL mailing list-- there just aren't any general approaches to preserving NULL values in a geotiff when you need the entire range of a given datatype to represent your information. I think that several things can be done to address this bug, some of which have been mentioned: 1. if there are any NULL cells, warn the user- possibly refuse to export the image in the current format (BYTE, etc.) 1a. if the selected output format cannot contain the entire range, or precision, of input map data (input is FCELL, output is BYTE) give a warning or fail with error message. 2. include a 'force' option to make r.out.gdal do exactly what your command line instructions suggest-- possibly leading to corruption of NULL data 3. Allow the user to specify what they would like NULL cells encoded as. Unless I am overlooking something, it would seem reasonable to export a CELL map to a signed integer format, and use some obvious negative value for NULL: # assuming that all REAL data is within 0-255 r.out.gdal in=CELLMAP out=map.tif format=SIGNEDINT null=-1 also, because the manual has been updated. that is easily modified, so shouldn't stop us. In any case, it would be great if this ticket could be solved before the final release of 6.4. yup I would like to see the above changes, however I cannot actually implement them. Thanks to everyone who has been working on this important module. AFAIK, except for number 1, all are implemented already. Currently there is an error if the null value does not fit into the data range of the selected output file type, but not if the selected value for null data is present in the data as a valid value. But as Markus explained, checking for this would be quite intense, so IMHO a warning that there are null values and that there is the possibility of loosing information if the user does not watch out should be enough. Moritz Thanks for the notice on 2-3. Your approach sounds good to me. Cheers, Dylan -- Dylan Beaudette Soil Resource Laboratory http://casoilresource.lawr.ucdavis.edu/ University of California at Davis 530.754.7341 ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by helena): as for me - I did not file it as critical, because I thought it was partially problem on my side (not using the correct options) - it was upgraded to critical later. I think we just need to keep around r.out.arc, r.out.tiff in case the software that you are trying to import the data to does not like the r.out.gdal geotiff as we cannot really solve other software issues with geotiff. I also agree that the warning about possible loss of data with badly chosen NULL as suggested by Moritz should take care of the null problem. All I really tried to do was convert the nc_spm raster data into something readable by other software tools and r.out.gdal worked but some rasters were not readable or came out black so it required change in color table etc. (e.g., in ArcGIS, gimp). We ended up using r.out.arc - and lost the color tables and projection info but otherwise it worked fine. I haven't tried the very latest version of r.out.gdal though. Helena -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:42 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mlennert): Replying to [comment:42 helena]: All I really tried to do was convert the nc_spm raster data into something readable by other software tools and r.out.gdal worked but some rasters were not readable or came out black so it required change in color table etc. (e.g., in ArcGIS, gimp). We ended up using r.out.arc - and lost the color tables and projection info but otherwise it worked fine. Just out of curiosity: how is losing color table and projection info (r.out.arc) better than just a lost color table (r.out.gdal) ? Moritz -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:43 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Moritz Lennert wrote: On 15/04/09 19:22, Dylan Beaudette wrote: On Wednesday 15 April 2009, GRASS GIS wrote: #73: r.out.gdal tiff output does not work --+ - Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+ - Comment (by hamish): Replying to [comment:32 mmetz]: My patch only adds safety checks to make sure that the selected GDAL data type covers the range of the raster band to be exported and that the given null value is within the range of the selected GDAL data type. That would be an enhancement, not a bugfix. Pretend you are r.out.gdal iterating through the raster cells to output. You come across a NULL cell. You have to write down some value, and are completely constrained to options available in the output type. What do you write out? Overflow/wrap the type? type min? type max? well (as things were) whatever you do is going to be arbirtary, and thus could lead to data loss, which is grounds for a most serious blocker bug. I haven't thought a great deal about this, but from what I have been reading and recalling past messages on the GDAL mailing list-- there just aren't any general approaches to preserving NULL values in a geotiff when you need the entire range of a given datatype to represent your information. I think that several things can be done to address this bug, some of which have been mentioned: 1. if there are any NULL cells, warn the user- possibly refuse to export the image in the current format (BYTE, etc.) 1a. if the selected output format cannot contain the entire range, or precision, of input map data (input is FCELL, output is BYTE) give a warning or fail with error message. 2. include a 'force' option to make r.out.gdal do exactly what your command line instructions suggest-- possibly leading to corruption of NULL data 3. Allow the user to specify what they would like NULL cells encoded as. Unless I am overlooking something, it would seem reasonable to export a CELL map to a signed integer format, and use some obvious negative value for NULL: # assuming that all REAL data is within 0-255 r.out.gdal in=CELLMAP out=map.tif format=SIGNEDINT null=-1 also, because the manual has been updated. that is easily modified, so shouldn't stop us. In any case, it would be great if this ticket could be solved before the final release of 6.4. yup I would like to see the above changes, however I cannot actually implement them. Thanks to everyone who has been working on this important module. AFAIK, except for number 1, all are implemented already. Currently there is an error if the null value does not fit into the data range of the selected output file type, but not if the selected value for null data is present in the data as a valid value. But as Markus explained, checking for this would be quite intense, so IMHO a warning that there are null values and that there is the possibility of loosing information if the user does not watch out should be enough. 1. warn the user only if no nodata value is given, otherwise (nodata value was given) assume that the user knows what she/he is doing? 1a. only in the patch which is not submitted, and no precision test. 2. suggested but nowhere implemented. why should we have a force option to override all the requested but not yet present safety checks? 3. was always possible I THINK calculating a reasonable nodata value is both time and memory expensive, maybe there is some ingenious way to avoid memory costs, but I'm pretty sure that two passes for each raster band are necessary, i.e. time intensive. r.out.gdal selects a file format, a suitable data type and a default nodata value if none of them were given, but that's more of a guess, and the user should know best what file format, data type and nodata value is appropriate. It gets all a bit more complicated because r.out.gdal can export several raster maps into one single file if supported by the selected file format. Therefore I would prefer some simple tests and abort with a helpful message if something goes wrong instead of doing fancy automated guessing that would make the code much more complex and thus more difficult to maintain. r.out.gdal supports all raster formats supported by gdal, and that's a lot, becoming more with every new version of gdal. IMHO, r.out.gdal should check if input options would cause corrupted output and otherwise refer to the gdal documentation. I'm busy working on an update for r.out.gdal and should be able to submit to trunk tomorrow. Changes are range-check
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
On Wed, 2009-04-15 at 18:30 +, GRASS GIS wrote: #73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mlennert): Replying to [comment:42 helena]: All I really tried to do was convert the nc_spm raster data into something readable by other software tools and r.out.gdal worked but some rasters were not readable or came out black so it required change in color table etc. (e.g., in ArcGIS, gimp). We ended up using r.out.arc - and lost the color tables and projection info but otherwise it worked fine. Just out of curiosity: how is losing color table and projection info (r.out.arc) better than just a lost color table (r.out.gdal) ? the file produced by r.in.arc displayed OK right away with Arc default table so people who got the file were able to work with it although they did not get the same colors as were assigned in GRASS. The r.out.gdal result displayed black and unless you knew what was going on and how to change the table it was difficult to deal with - the file just looked broken because it would display black in other viewers too. This might have been fixed though, I just cannot get around to test it, Helena Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
Dylan Beaudette wrote: 3. Allow the user to specify what they would like NULL cells encoded as. Unless I am overlooking something, it would seem reasonable to export a CELL map to a signed integer format, and use some obvious negative value for NULL: If you're exporting to a 32-bit (signed or unsigned) integer format, the obvious choice for null is 0x8000. That's what GRASS uses internally, so it can never occur as a legitimate value (if a computation ends up producing that value, it will get treated as a null in all regards). -- Glynn Clements gl...@gclements.plus.com ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
On Wednesday 15 April 2009, Glynn Clements wrote: Dylan Beaudette wrote: 3. Allow the user to specify what they would like NULL cells encoded as. Unless I am overlooking something, it would seem reasonable to export a CELL map to a signed integer format, and use some obvious negative value for NULL: If you're exporting to a 32-bit (signed or unsigned) integer format, the obvious choice for null is 0x8000. That's what GRASS uses internally, so it can never occur as a legitimate value (if a computation ends up producing that value, it will get treated as a null in all regards). Thanks Glynn. It would be good to know about 'obvious' values for any given datatype. Cheers, Dylan -- Dylan Beaudette Soil Resource Laboratory http://casoilresource.lawr.ucdavis.edu/ University of California at Davis 530.754.7341 ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mlennert): Is this still an issue, or have Markus' patches solved the problem ? -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:31 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:29 hamish]: FWIW, it was just minor whitespace on roughly 5 lines. I don't think it is necessary to run intent compulsively as long as the whitespace doesn't get too abused. The question is, keep the GRASS source code in standard format yes or no? According to the discussion on it and to SUBMITTING, the answer is something like YES, we had a long discussion on it, reformatting the whole source code was a major change, now please be so kind as to keep the code in the standard format and not Yes, but a little deviation here and there doesn't harm. No offense! My patch would introduce some new code, that's why I ran indent on it, otherwise I would have been told to do so first before proposing a patch. Justification for the patch: the TODO's in the code have been attended to; that would avoid quite a few of the problems reported in this very long ticket and IMHO the module could be regarded as working all right if these TODO's are solved. I replaced the attached patch with a new patch against the newly formatted r.out.gdal Markus M -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:30 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by hamish): Replying to [comment:28 glynn]: FWIW, I've reformatted r.out.gdal in r34914. FWIW, it was just minor whitespace on roughly 5 lines. I don't think it is necessary to run intent compulsively as long as the whitespace doesn't get too abused. Hamish -- Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:29 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by glynn): Replying to [comment:27 mmetz]: BTW, it seems that some changes in the patch are there because the original code was not indented with grass_indent.sh. It's best if you can avoid mixing substantial changes and formatting changes in the same patch. If the code wasn't formatted either preserve the original formatting (or use e.g. svn diff -x -b to ignore whitespace changes) or re-format the code first. FWIW, I've reformatted r.out.gdal in r34914. -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:28 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work
--+-
Reporter: helena | Owner: grass-dev@lists.osgeo.org
Type: defect | Status: new
Priority: critical | Milestone: 6.4.0
Component: Raster | Version: svn-trunk
Resolution: |Keywords: r.out.gdal, tiff
Platform: Unspecified | Cpu: Unspecified
--+-
Comment (by mmetz):
Replying to [comment:25 hamish]:
Replying to [comment:24 mmetz]:
- Long: Byte and UInt16 maps write out a RGB value for each 256/66536
possible int.
- Short: These are the COLOR_TABLE_RULE_RGB_n metadata created by
GRASS by copying over the contents of the colr/ file. They are typically
4-5 lines long and work by ranges not per-possibility.
I agree it is unpleasant if the user specifies {{{
createopt=PROFILE=BASELINE }}} for a bare-bones GeoTIFF and GRASS still
writes out COLOR_TABLE_RULE metadata anyway.
But typically the user will use that flag to avoid the list of 66536
RGB values, not the list of 5 rules.
Yes, but for GeoTIFF these short colortables are not colortables in the
GeoTIFF sense, but custom metadata. The equivalent in gdal_translate would
be gdal_translate -mo COLOR_TABLE_RULE_0=... Qgis reads these rules
instead of the colortable if rules are present, but qgis is the only
application doing so. Grass doesn't do that itself, i.e. it exports
COLOR_TABLE_RULE, but ignores these entries on import (enhancement for
r.in.gdal?).
I notice gdal's gcore/rasterio.cpp has its Int32 min at -2147483647 not
-2147483648. That might make a diff if we wanted to use the min as the
default nodata value. ?? (or is that not = so ok?)
Hmm, interesting. limits.h defines INT_MAX as 2147483647 and INT_MIN as
-INTMAX - 1
Also true: USHRT_MAX (UInt16 max) = SHRT_MAX - SHRT_MIN (Int16 max and
min)
UINT_MAX (UInt32 max) = INT_MAX - INT_MIN (Int32 max and min) or INT_MAX +
INT_MAX + 1
But if gdal decides Int32 min is -2147483647, then it be.
I'm no expert on when it is best to terminate a big number with UL,
L, .0, f or use the likes of (double)(unsigned int)0xu
Not that I am, I would suggest using the same like gdal or limits.h
(UInt32 max = 4294967295U)
Float23 and Float 64 limits: on my system (Linux 64 with gcc 4.1.2) there
are FLT_MAX and DBL_MAX defined and can be used for TYPE_FLOAT32_MIN/MAX
and TYPE_FLOAT64_MIN/MAX. Are FLT_MAX and DBL_MAX defined on other systems
(in float.h) too? This would be more accurate than the current
approximation in local_proto.h.
I notice with the current code that if I make a UInt16 map and don't
specify a nodata value the G_message reports it defaults to -32768, but
this may just be %d speaking, not the variable.
If a nodata value is chosen automatically in the presence of NULL cells
and absence of user-specified nodata option, this must be correct and
consistent, i.e. always max or always min of the datatype which is not the
case right now. Should be corrected when the range testing is implemented.
Markus
--
Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:26
GRASS GIS http://grass.osgeo.org
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
On 30/10/08 08:02, GRASS GIS wrote: Comment (by mmetz): Yes, but for GeoTIFF these short colortables are not colortables in the GeoTIFF sense, but custom metadata. The equivalent in gdal_translate would be gdal_translate -mo COLOR_TABLE_RULE_0=... Qgis reads these rules instead of the colortable if rules are present, but qgis is the only application doing so. Grass doesn't do that itself, i.e. it exports COLOR_TABLE_RULE, but ignores these entries on import (enhancement for r.in.gdal?). Yes, please ! IMHO, if both a short and a long colortable are present, then GRASS should use the former since with a long colortable things get really slow for raster display. Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:23 hamish]: Replying to [comment:21 mmetz]: 1. should implicit raster editing be allowed, i.e. give and respect nodata in the absence of NULL cells, AFAIK this is just the setting of a metadata tag (or not), so not really editing the raster data at all. As such I don't mind if the user decides to set it when it would otherwise not be. There's no processing involved. This is the same like r.null setnull=nodata. Is this raster editing? note added to the help page that g.region is boss. Little word missing? ... if you need to realign the region settings '''to''' the original map's before export. Also maybe remove comments on GDAL datatypes just below the ranges of datatypes, because they are not really necessary? -c flag added to turn off (long) color tables for Byte and UInt16. I think it is worth checking if the GDAL code could be changed to only write out as much of the table as needed, not up to 65535-max empty rules. Potentially misleading description: Do not export long colortable? A user might wonder whether short colortables are always exported? The type range limit testing stuff is still TODO. (both data max/min and nodata=) The min/max values for all integer types are identical to gdal and AFAIK universally valid. See min/max values in gdal-1.5.2 gcore/rasterio.cpp. As I understand, min/max values for float32 are indirectly determined in gdal using typecast from double to float. The min/max values for float32 and float64 as suggested by me correspond to IEE 754 to my best knowledge. Values farthest away from zero have all bits in the mantissa set and all but the last in the exponent which is equal to (1 - 1/2^24^) * 2^128^ for float32 and (1 - 1/2^53^) * 2^1024^ for float64, taken from http://www.cs.berkeley.edu/~wkahan/ieee754status/IEEE754.PDF. There is a cplusplus style comment in local_proto.h. OK, it's in a c style comment, but that could be uncommented soon. In the TODO line 386 rather use GDALGetDataTypeName(datatype) than type-answer because type is not required and datatype is known by then. Even looking very hard I can't find more to criticize;-) I guess once the type ranges are confirmed the TODOs can be done. Markus -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:24 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:20 hamish]: - if a Byte map has data 0-255 *and* NULL, it seems reasonable to me to default to nodata=255 + issue a warning. *But* if a map has 0-255 with *no* NULLs, nodata should not be set* to 255, as this would lead to a loss of fidelity. (improper removal of all cells with cat=255 for no gain) This is the current implementation in 6.4 (unless modified recently) and my version. [*] ie should not be set, unless the user requests it explicitly with the nodata= opt Thicker: I disagree, this is raster editing which should be done before exporting, identical to r.null setnull=val. As a summary, I see three issues of controversy: 1. should implicit raster editing be allowed, i.e. give and respect nodata in the absence of NULL cells, give out of range nodata value (NULL cells are replaced with min or max of data type, r.null null=nodataval gets secretly converted by gdal to r.null null=datatype_min or r.null null=datatype_max), respect the current region resolution 2. should the user be allowed to do strange things like giving out of range nodata values or specify a data type not covering the range of cell values 3. should r.out.gdal consider flaws in (not only) GeoTIFF support of other applications, e.g. give an option to export the colortable and don't do it by default, or should r.out.gdal (stubbornly?) stick to the specs sorry to be so thick, Being thick is necessary, r.out.gdal is important for data exchange with people using other applications Markus -- Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:21 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by glynn): Replying to [comment:18 hamish]: No color table was specified with r.colors, so what data visualization scheme is to be expected? if none is specified (no colr/ files exists for the raster) the default rainbow is used. GRASS's libgis does this automatically, AFAIK the module never knows that r.colors hasn't been run. It wasn't always this way; prior to r30551, it would only export an explicit colour table, not the default rainbow colour table. -- Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:22 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work
--+-
Reporter: helena | Owner: grass-dev@lists.osgeo.org
Type: defect | Status: new
Priority: critical | Milestone: 6.4.0
Component: Raster | Version: svn-trunk
Resolution: |Keywords: r.out.gdal, tiff
Platform: Unspecified | Cpu: Unspecified
--+-
Comment (by hamish):
I have just applied in SVN (r34015,6) some updates which should help. I
took a lot of your suggestions, but not all.
{{{
cleanups in pursuit of bug #73 in collaboration with Markus Metz
* rename move export band to its own file
* flag to disable writing long (UInt16) color tables
* add type length defines (needs completion review)
* add some TODO comments where future work is required
* help page cleanups
}}}
Replying to [comment:21 mmetz]:
As a summary, I see three issues of controversy:
1. should implicit raster editing be allowed, i.e. give and
respect nodata in the absence of NULL cells,
AFAIK this is just the setting of a metadata tag (or not), so not
really editing the raster data at all. As such I don't mind if
the user decides to set it when it would otherwise not be.
There's no processing involved.
give out of range nodata value (NULL cells are replaced with min
or max of data type,
r.null null=nodataval gets secretly converted by gdal to
r.null null=datatype_min or r.null null=datatype_max),
It should exit with an error if the user-supplied nodata= value is out of
range for the specified data type. When the module comes across a NULL
cell it needs to know what to do with it.
respect the current region resolution
note added to the help page that g.region is boss.
2. should the user be allowed to do strange things like giving
out of range nodata values or specify a data type not covering
the range of cell values
No, G_fatal_error() for both; the module needs to know what to write out
for the cell and shouldn't just make stuff up.
3. should r.out.gdal consider flaws in (not only) GeoTIFF support
of other applications, e.g. give an option to export the
colortable and don't do it by default, or should r.out.gdal
(stubbornly?) stick to the specs
-c flag added to turn off (long) color tables for Byte and UInt16. I think
it is worth checking if the GDAL code could be changed to only write out
as much of the table as needed, not up to 65535-max empty rules.
AFAIAC, upon request we should offer the user the chance to export a bare-
bones BASELINE GeoTIFF (we have r.out.tiff for that), but by default we
should keep the GeoTIFF as self-documenting and metadata up'd as
possible-- as that is what makes the data valuable and viable in the long
term and support files are quickly lost.
I'd note that INTERLEAVE=PIXEL is only the default for the very latest
versions of GDAL released in the last months. On my system it still
defaults to BAND.
The type range limit testing stuff is still TODO. (both data max/min and
nodata=)
Hamish
--
Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:23
GRASS GIS http://grass.osgeo.org
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work
--+-
Reporter: helena | Owner: grass-dev@lists.osgeo.org
Type: defect | Status: new
Priority: critical | Milestone: 6.4.0
Component: Raster | Version: svn-trunk
Resolution: |Keywords: r.out.gdal, tiff
Platform: Unspecified | Cpu: Unspecified
--+-
Comment (by hamish):
Replying to [comment:11 mmetz] (and others):
No color table was specified with r.colors, so what data visualization
scheme is to be expected?
if none is specified (no colr/ files exists for the raster) the default
rainbow is used. GRASS's libgis does this automatically, AFAIK the
module never knows that r.colors hasn't been run.
Gimp and many other image viewers only support 8 bit per channel,
cinepaint e.g. supports 32 bit.
this is a CELL map exported with type=Byte.
Most of them can not display tiff files with anything else but Byte:
only tiff images, not tiff datasets.
the spearfish example I gave was Byte-only.
Elevation is a dataset, not an image.
True, but if elev data is put in the correct range, Gimp neither knows nor
cares what the levels represent. I was just using it as random values, not
anything meaningful.
Spearfish's elevation.dted would be a better sample dataset to use here,
it is CELL 0-255. or convert spot.image into RGB bands.
Other test datasets are the spearfish imagery60 data, and the North
Carolina dataset lsat7 images.
Still, datatypes other than Byte can only be read with spatial data
viewers that could make use of this colour table, but not all of them are
actually able to do so.
Important to know when this is happening, not really our problem, and not
much we can do about it as long as we are writing metadata to spec.
Hamish:
gdalinfo confirms palette info was not written to file:
... COLOR_TABLE_RULES_COUNT=0 ... 0: 0,0,0,255 ... 255: 0,0,0,255
MM:
But the color palette was indeed written to the file.
An empty one was written while within GRASS it had real colors.
For the expected colors run:
{{{
seq 0 255 | r.what.color -i in=elev_0
0: 255:255:0
1: 254:255:0
2: 252:255:0
...
253: 0:255:161
254: 0:255:163
255: 0:255:165
}}}
It would help a lot to have a list with these data types and their
ranges in the man page so that the appropriate data type for file export
can be chosen.
Yes. Can someone in the know supply some suggested text?
Maybe the color table is not so important, different people have
different preferences
The color table will be carefully constructed/chosen by some people and
they may wish to keep it. e.g. r.colors.stddev addon script, r.cpt2grass
addon script, and NASA color tables given for MODIS sat data (see grass's
MODIS wiki page).
These data are real (calibrated into sea surface temperature or mg/mL
chlorophyll) and also have specially designed color tables (incl log-
scale) which are nice to keep.
Suggested data visualization with an embedded color table is nice to
have, but coloring schemes are a matter of taste
geotiff has a method for assigning color palettes, all I suggest is
populating that correctly, apparently we are currently (or were) failing
at.
There is still the issue of nodata: a nodata value must fall into the
range of the chosen datatype, otherwise this information gets lost.
r.out.gdal does not check if a supplied nodata value is valid.
Hopefully better in SVN now. If the user manually specifies an out of
range value, I don't mind respecting that.
For UINT16, the color table in the tiff file will always have 65535
entries, gdal does that.
perhaps this should be addressed in GDAL then; only write palette entries
which are given.
Have an option in r.out.gdal to choose whether a color table should be
written to the tiff file?
For Uint16, how to just get the COLOR_TABLE_RULE_RGB_0 part of this
without the (RGB with 66535 entries) part?
r.out.gdal geotiff from 0-255 (black white) ortho photo with grass's
sepia color table:
{{{
Metadata:
COLOR_TABLE_RULES_COUNT=10
COLOR_TABLE_RULE_RGB_0=0.00e+00 2.55e+01 0 0 0 34 23 9
COLOR_TABLE_RULE_RGB_1=2.55e+01 5.10e+01 34 23 9 68 47 19
COLOR_TABLE_RULE_RGB_2=5.10e+01 7.65e+01 68 47 19 101 72 33
COLOR_TABLE_RULE_RGB_3=7.65e+01 1.02e+02 101 72 33 130 96 49
COLOR_TABLE_RULE_RGB_4=1.02e+02 1.275000e+02 130 96 49 158 122 71
COLOR_TABLE_RULE_RGB_5=1.275000e+02 1.53e+02 158 122 71 183 147 96
COLOR_TABLE_RULE_RGB_6=1.53e+02 1.785000e+02 183 147 96 206 174
126
COLOR_TABLE_RULE_RGB_7=1.785000e+02 2.04e+02 206 174 126 225 200
162
COLOR_TABLE_RULE_RGB_8=2.04e+02 2.295000e+02
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:18 hamish]: Replying to [comment:11 mmetz] (and others): Several of my previous descriptions about the conservative version of r.out.gdal are out of date, e.g. the way statistics are calculated. I would prefer not to stick strictly to my version, instead use it as ideas for discussion. Spearfish's elevation.dted would be a better sample dataset to use here, it is CELL 0-255. or convert spot.image into RGB bands. Other test datasets are the spearfish imagery60 data, and the North Carolina dataset lsat7 images. Using sample datasets available to all is a very good idea. Still, datatypes other than Byte can only be read with spatial data viewers that could make use of this colour table, but not all of them are actually able to do so. Important to know when this is happening, not really our problem, and not much we can do about it as long as we are writing metadata to spec. I agree, GRASS can not fix or accommodate other viewers/packages. It would help a lot to have a list with these data types and their ranges in the man page so that the appropriate data type for file export can be chosen. Yes. Can someone in the know supply some suggested text? See the updated documentation in my version. Maybe the color table is not so important, different people have different preferences The color table will be carefully constructed/chosen by some people and they may wish to keep it. e.g. r.colors.stddev addon script, r.cpt2grass addon script, and NASA color tables given for MODIS sat data (see grass's MODIS wiki page). These data are real (calibrated into sea surface temperature or mg/mL chlorophyll) and also have specially designed color tables (incl log- scale) which are nice to keep. True, but if you preprocess such data for someone else who will be running his/her own analyses, a colortable is not important. Sometimes it is desirable to keep the colortable, sometimes not, therefore I would like to have the option to keep it or not. Suggested data visualization with an embedded color table is nice to have, but coloring schemes are a matter of taste geotiff has a method for assigning color palettes, all I suggest is populating that correctly, apparently we are currently (or were) failing at. I disagree, the colortable is populated correctly with the current svn version (at least in my tests), but other GIS software may not be able to make use of it. Nothing GRASS can do about that. There is still the issue of nodata: a nodata value must fall into the range of the chosen datatype, otherwise this information gets lost. r.out.gdal does not check if a supplied nodata value is valid. Hopefully better in SVN now. If the user manually specifies an out of range value, I don't mind respecting that. Users may accidentally specify an out of range nodata value. That has a similar effect like replacing NULL values in the GRASS raster. I would at least like to show a warning. For UINT16, the color table in the tiff file will always have 65535 entries, gdal does that. perhaps this should be addressed in GDAL then; only write palette entries which are given. File an enhancement for GDAL? Have an option in r.out.gdal to choose whether a color table should be written to the tiff file? For Uint16, how to just get the COLOR_TABLE_RULE_RGB_0 part of this without the (RGB with 66535 entries) part? Done in my version. basic statistics (min, max, mean, stddev) are calculated for the current region, replacing NULL cells with nodata value (default or specified) and then exported with GDALSetRasterStatistics() how is this better than gdalinfo -stats ? I thought it could be better because other software could read the metadata. But they don't, at least the ones I tested. So there is no advantage in giving stats in the metadata. I don't understand, you convert NULL to nodata value in stats? So for a CELL map with lots of NULL areas, if I make nodata=100 (well above the number of used palette entries) or leave it at 255, the stats will be skewed towards 100 or 255 instead of just considering the real data? huh? gdalinfo -stats seems to do the right thing here. Right,I changed that and ignore nodata, stats as calculated by my version are now identical to
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
GRASS GIS wrote: #73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by neteler): Markus (Metz), what about integrating your fixes from http://markus.metz.giswork.googlepages.com/r.out.gdal.conservative.tar.gz ? Markus Sure, no objections from my side, I'm using this version only. But r.out.gdal is a very important module of GRASS and maybe some more testing is required. Also, the new features may not find approval by all, e.g. I've put in rather restrictive NULL cell handling: the user has to specify a nodata value that falls within the range of the selected datatype (Byte, UInt16, Int16, ...), a nodata value is no longer chosen automatically as in the original version. If a nodata value is not given, but NULL cells are present, r.out.gdal aborts with an error message. My version also no longer uses the current region resolution, instead the current region extends are aligned to the resolution of the raster to be exported to avoid any implicit resampling. And the colortable is only exported on request, and then with a warning message. If these changes are ok with you then integrate the changes, otherwise maybe only some, but not all changes could be integrated. More discussion needed on how to change/improve r.out.gdal? Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
On 24/10/08 14:10, Markus Metz wrote: GRASS GIS wrote: #73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by neteler): Markus (Metz), what about integrating your fixes from http://markus.metz.giswork.googlepages.com/r.out.gdal.conservative.tar.gz ? Markus Sure, no objections from my side, I'm using this version only. But r.out.gdal is a very important module of GRASS and maybe some more testing is required. Also, the new features may not find approval by all, e.g. I've put in rather restrictive NULL cell handling: the user has to specify a nodata value that falls within the range of the selected datatype (Byte, UInt16, Int16, ...), a nodata value is no longer chosen automatically as in the original version. If a nodata value is not given, but NULL cells are present, r.out.gdal aborts with an error message. My version also no longer uses the current region resolution, instead the current region extends are aligned to the resolution of the raster to be exported to avoid any implicit resampling. And the colortable is only exported on request, and then with a warning message. If these changes are ok with you then integrate the changes, otherwise maybe only some, but not all changes could be integrated. More discussion needed on how to change/improve r.out.gdal? +1 for all of these changes. Let's try with those and then discuss. Moritz ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
On Oct 24, 2008, at 9:32 AM, Moritz Lennert wrote: On 24/10/08 14:10, Markus Metz wrote: GRASS GIS wrote: #73: r.out.gdal tiff output does not work -- +- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: newPriority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified -- +- Comment (by neteler): Markus (Metz), what about integrating your fixes from http://markus.metz.giswork.googlepages.com/ r.out.gdal.conservative.tar.gz ? Markus Sure, no objections from my side, I'm using this version only. But r.out.gdal is a very important module of GRASS and maybe some more testing is required. Also, the new features may not find approval by all, e.g. I've put in rather restrictive NULL cell handling: the user has to specify a nodata value that falls within the range of the selected datatype (Byte, UInt16, Int16, ...), a nodata value is no longer chosen automatically as in the original version. If a nodata value is not given, but NULL cells are present, r.out.gdal aborts with an error message. My version also no longer uses the current region resolution, instead the current region extends are aligned to the resolution of the raster to be exported to avoid any implicit resampling. And the colortable is only exported on request, and then with a warning message. If these changes are ok with you then integrate the changes, otherwise maybe only some, but not all changes could be integrated. More discussion needed on how to change/improve r.out.gdal? +1 for all of these changes. Let's try with those and then discuss. Moritz I think those are useful changes and agree with Moritz that we should try them out. maybe add it to svn as r.out.gdal2 so that we can test and compare with the old one and then replace if there are no problems? It would be a huge help to get this resolved, Helena ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
On Friday 24 October 2008, Helena Mitasova wrote: On Oct 24, 2008, at 9:32 AM, Moritz Lennert wrote: On 24/10/08 14:10, Markus Metz wrote: GRASS GIS wrote: #73: r.out.gdal tiff output does not work -- +- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: newPriority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified -- +- Comment (by neteler): Markus (Metz), what about integrating your fixes from http://markus.metz.giswork.googlepages.com/ r.out.gdal.conservative.tar.gz ? Markus Sure, no objections from my side, I'm using this version only. But r.out.gdal is a very important module of GRASS and maybe some more testing is required. Also, the new features may not find approval by all, e.g. I've put in rather restrictive NULL cell handling: the user has to specify a nodata value that falls within the range of the selected datatype (Byte, UInt16, Int16, ...), a nodata value is no longer chosen automatically as in the original version. If a nodata value is not given, but NULL cells are present, r.out.gdal aborts with an error message. My version also no longer uses the current region resolution, instead the current region extends are aligned to the resolution of the raster to be exported to avoid any implicit resampling. And the colortable is only exported on request, and then with a warning message. If these changes are ok with you then integrate the changes, otherwise maybe only some, but not all changes could be integrated. More discussion needed on how to change/improve r.out.gdal? +1 for all of these changes. Let's try with those and then discuss. Moritz I think those are useful changes and agree with Moritz that we should try them out. maybe add it to svn as r.out.gdal2 so that we can test and compare with the old one and then replace if there are no problems? It would be a huge help to get this resolved, Helena +1 from me. I once it is in the 64 branch I can do some testing. Dylan ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev -- Dylan Beaudette Soil Resource Laboratory http://casoilresource.lawr.ucdavis.edu/ University of California at Davis 530.754.7341 ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by hamish): [keep it in the bug report please!] Comment (by neteler): what about integrating your fixes from http://markus.metz.giswork.googlepages.com/r.out.gdal.conservative.tar.gz ? Markus Metz wrote: Sure, no objections from my side, I'm using this version only. But r.out.gdal is a very important module of GRASS and maybe some more testing is required. Also, the new features may not find approval by all, yep e.g. I've put in rather restrictive NULL cell handling: the user has to specify a nodata value that falls within the range of the selected datatype (Byte, UInt16, Int16, ...), a nodata value is no longer chosen automatically as in the original version. If a nodata value is not given, but NULL cells are present, r.out.gdal aborts with an error message. When possible, I think it better to have a sensible default then give the user the option to override. That's what we have in the existing code. NULL=max possible is a common usage, so not bad for a default. Your change is too conservative for my vote, and generally I don't like banning users from doing things just because my imagination is limited as to why they would want to do something funny. If they really want to set nodata= out of bounds, and explicitly give a nodata= value specifying that, I'd say let them (with a warning in case they didn't realize, of course). My version also no longer uses the current region resolution, instead the current region extends are aligned to the resolution of the raster to be exported to avoid any implicit resampling. Nope. I see your reasons, but this is in violation of GRASS's standard raster semantics, and one of the reasons for the r.out.gdal.sh - r.out.gdal (.c) was to fix this. (sort of) Maybe it makes the GRASS-R stats people happier?, but shouldn't be the default method, and I doubt offered at all. I would think this better handled with a note in the help page with a suggestion to use g.region res=.. -a if they want that. And the colortable is only exported on request, and then with a warning message. IMO that should be automatic with Byte maps (255 levels), perhaps needed with UInt16 maps (with 65535 levels) it should be optional... If these changes are ok with you then integrate the changes, otherwise maybe only some, but not all changes could be integrated. More discussion needed on how to change/improve r.out.gdal? as above. I need to reread the (long) bug history now, but fwiw I just fixed yesterday a bug in SVN where nodata= value given by the user was only used if type= was also given, otherwise the auto-type determination reset it. I also noticed that QGIS 0.8.1 (old, but latest debianGIS package for Etch/stable) has a bug where the first palette entry always is black, even when the GeoTIFF says otherwise. (loads/views fine in other software) In my case it was a 7-color palette with cat 0 being white in the color table. Hamish -- Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:17 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by neteler): Markus (Metz), what about integrating your fixes from http://markus.metz.giswork.googlepages.com/r.out.gdal.conservative.tar.gz ? Markus -- Ticket URL: https://trac.osgeo.org/grass/ticket/73#comment:16 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by neteler): Great work! Would you mind to also test back-import into GRASS? Markus -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:14 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:14 neteler]: Great work! Thanks! Would you mind to also test back-import into GRASS? All GeoTIFFs previously exported with the test version of r.out.gdal were back-imported to GRASS-6.4.svn (snapshot_2008_08_23) GeoTIFF files:[[BR]] elevation.Byte.clrtbl.tif[[BR]] elevation.Byte.noclrtbl.tif[[BR]] elevation.UInt16.clrtbl.tif[[BR]] elevation.UInt16.noclrtbl.tif[[BR]] elevation.Int32.noclrtbl.tif[[BR]] elevation.Float32.noclrtbl.tif display: all ok, very slow for the UInt16 back-import with colortable[[BR]] colortable: all ok[[BR]] cell values: all ok[[BR]] nodata: all ok Back-importing elevation.UInt16.clrtbl.tif is pretty slow. Display takes quite some time too. It is a colortable with 65535 entries, one for each potential cell value, an exact copy of the GeoTIFF colortable. Removing the colortable with r.colors -r, then copying the colortable of the back- import of elevation.Byte.clrtbl.tif results in the same display, but now fast on refresh, as expected. Back-import of all other GeoTIFF files is fast, as is display in GRASS. I would prefer to use r.out.gmt to get the GRASS color rules into an extra file (GMT .cpt). This would also be a workaround for export file types other than Byte and UInt16 where GDAL does not support writing out a colortable. But then I don't know how widely GMT .cpt files are supported. Markus -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:15 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): r.out.gdal GeoTIFF compatibility testing and new r.out.gdal test version Changes in r.out.gdal test version new flag for colortable export, default to no nodata value adjustment to range of selected data type with warning message, if user-specified nodata value is out of range data range checking against the selected data type, abort with error if range of selected data type exceeded alignment of current region extends to raster resolution (original uses current region extends and resolution), current region restored on exit raster band statistics (min, max, mean, stddev) written to metadata if supported, GDAL decides I tested with QGIS-0.9.1 on Linux 64bit, QGIS -0.11.0-1 on XP, SAGA-2.0.3 on XP (doesn't work for me on Linux 64bit), gvSIG-1.1.2 (32bit binary) on Linux 64bit, gvSIG-1.1.1 on XP, PCI Geomatica FreeView 9.1 on XP, ER Viewer 7.2 (ER Mapper) on XP, and ArcMap 9.2 (ESRI) on XP. Tests were performed with the elevation raster map layer (range within Byte, FCELL map) in the North Carolina sample dataset, creating a MASK for elevation values = 70m, needed for testing of nodata handling with nodata=0. The elevation raster was exported as Byte, UInt16, Int32, and Float32, once with colortable (Byte and UInt16 only, colortable export for other data types not supported by gdal), once without colortable, always as GeoTIFF. QGIS-0.9.1 on Linux 64bit display: all ok colortable: all ok cell values: all ok nodata: all ok QGIS-0.11.1 on XP display: all ok colortable: all ok cell values: all ok nodata: all ok gvSIG-1.1.2 (Linux 32bit binary) on Linux 64bit. Projection EPSG:32119 display: fail for UInt16 with colortable, all other ok colortable: fail for UInt16, Byte ok cell values: fail for Byte!!! (sometimes reports negative values), all other ok. Interestingly the display seems to use correct values. Bug in gvSIG-1.1.2 nodata: nodata not supported, actual cell value reported gvSIG-1.1.1 on XP, Projection EPSG:32119 display: fail for UInt16 with colortable, all other ok colortable: fail for Uint16, ok for Byte cell values: fail for Byte!!! (sometimes reports negative values), all other ok. Interestingly the display seems to use correct values. Bug in gvSIG-1.1.1 nodata: nodata not supported, actual cell value reported PCI Geomatica FreeView 9.1 on XP display: all ok colortable: all ok cell values: all ok nodata: nodata not supported, actual cell value reported ER Viewer 7.2 (ER Mapper) on XP display: fail for UInt16 with colortable, all other ok colortable: fail for UInt16, ok for Byte cell values: cell value query not supported nodata: cell value query not supported SAGA-2.0.3 on XP (I don't get it to run on Linux 64bit) display: all ok colortable: all ignored cell values: all ok nodata: all ok ESRI ArcMap 9.2 on XP display: all ok, but all without colortable and other than Byte need manual adjustment which is painfully slow, even for these small raster layers (Properties - Symbology - Stretch - Minimum-Maximum) colortable: all ok cell values: all ok nodata: all ok Colortables are not allways properly rendered. GIMP and image viewers which are not using GDAL render a colortable for Byte type properly. Data types other than Byte can not be read by these 'standard' image editing and viewing applications. If nodata specification is not supported, applications return the actual cell value, in this case 0, instead of NoData, NULL, or similar. The files are still perfectly readable, only nodata assignment doesn't happen. I suggest adding a flag for colortable export with default to no and nodata value checking as well as data range checking against the selected data type. Another issue I discovered is that r.out.gdal uses the current region settings for export. This can cause implicit resampling to a different resolution which should be avoided IMHO. Instead, r.out.gdal could use the current region extends, but align the region settings used for export to the raster to be exported. This way, a subregion can still be exported, but the resolution of the GRASS raster is preserved. Also maybe check whether the current region extends are within the raster extends? Alignment of the current region to the raster to be exported by a new r.out.gdal is not
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Comment (by mmetz): Replying to [comment:2 dylan]: The data range as reported in ArcMap 9.2 when opening a geotiff exported from r.out.gdal is (1.17549e-038,3.40282e+038). This perfectly right insofar as this is the potential data range for float32. ArcMap does not check the actual data range when loading a new layer the first time, even when real minimum and maximum values are given in the metadata. The display of a layer gets more reasonable when choosing e.g. histogram equalization as stretch because only now ArcMap is looking at the real data range, gets real min and max values and adjusts the data visualization accordingly. ArcMap behaves like this too when opening ArcInfo ASCII grid and ESRI .hdr labeled files with float32 data (same elevation raster exported into different file formats, keeping data type float32). Maybe this is not a GeoTIFF/GRASS/GDAL bug, but ArcMap is simply not very user-friendly with data visualization? Here, GRASS is much more user-friendly, because it always knows the data range of a raster and, if no color table has been specified, automatically generates a reasonable color table, usually in color and not grayscale as ArcMap does by default... The tiff file is handled by ArcMap all right, data and georeferencing information are there, but data visualization needs to be modified by hand. I have noticed that when setting the files symbology, when set to a fixed number of classes (ranges) the correct data range is displayed, and the image seems to work fine. This suggests that it may be an out of range or unusual (in terms of ArcMap logic) no-data value. [http://casoilresource.lawr.ucdavis.edu/drupal/node/337 I have updated my notes here] -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:10 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work
--+-
Reporter: helena | Owner: grass-dev@lists.osgeo.org
Type: defect | Status: new
Priority: critical | Milestone: 6.4.0
Component: Raster | Version: svn-trunk
Resolution: |Keywords: r.out.gdal, tiff
Platform: Unspecified | Cpu: Unspecified
--+-
Comment (by mmetz):
Replying to [comment:1 hamish]:
(I see the same with 6.2.1's old script version of r.out.gdal)
it works if the values are in the range of 0-255:
but not if all values are out of range:
It is equally right to say that the chosen datatype=Byte is not capable of
holding the full data range and a different datatype must be chosen.
{{{
r.info -r elevation.dem
min=1066
max=1840
r.info -t elevation.dem
datatype=CELL
r.out.gdal in=elevation.dem out=elv.tif type=Byte
r.in.gdal in=elv.tif out=elv2
r.info -r elv2
min=255
max=255
}}}
we can try with half the raster in range:
{{{
r.mapcalc elev_0 = elevation.dem - 1066
r.info -r elev_0
min=0
max=774
r.out.gdal in=elev_0 out=elv0.tif type=Byte
No color table was specified with r.colors, so what data visualization
scheme is to be expected?
If I load elv0.tif into an image viewer it comes out as all black. But
in Gimp if you use the eyedropper you can see the index number changes,
but the palette is black for all 256 index entries.
Gimp and many other image viewers only support 8 bit per channel,
cinepaint e.g. supports 32 bit. Most of them can not display tiff files
with anything else but Byte: only tiff images, not tiff datasets.
Elevation is a dataset, not an image. INT16 or UINT16 would be appropriate
to hold elevation data, in case of cm accuracy with meters as units,
FLOAT32 would be ok. Such tiff files hold a dataset. For elevation data,
GRASS offers several color tables which can be written to a tiff file as a
suggested data visualization scheme, if they have been set with r.colors.
Still, datatypes other than Byte can only be read with spatial data
viewers that could make use of this colour table, but not all of them are
actually able to do so. They could instead choose their own color schemes.
gdalinfo confirms palette info was not written to file:
But the color palette was indeed written to the file.
{{{
Band 1 Block=1899x4 Type=Byte, ColorInterp=Palette
NoData Value=255
Metadata:
COLOR_TABLE_RULES_COUNT=0
Below here. The instructions read to display each value with R:G:B = 0:0:0
= black. A data visualization scheme should have been provided with
r.colors to have control over the colortable written to the tiff file.
Color Table (RGB with 256 entries)
0: 0,0,0,255
1: 0,0,0,255
2: 0,0,0,255
3: 0,0,0,255
4: 0,0,0,255
...
255: 0,0,0,255
}}}
If you want to store integer values 255 in the Tiff, you will have to
use one of Int16,UInt16,UInt32,Int32[,CInt16,CInt32?].
It would help a lot to have a list with these data types and their ranges
in the man page so that the appropriate data type for file export can be
chosen.
If you use type=Int32 etc you should preserve the CELL data for use in
other GIS software, even if the saved palette is bogus.
Maybe the color table is not so important, different people have different
preferences e.g. for elevation data visualization.
If imagery is exported, e.g. after atmospheric correction or masking, the
datatype should probably be Byte as it was originally, e.g. to make
postprocessing in Gimp possible.
A tiff file that holds an image (e.g. GRASS display saved as tiff, a
scanned image, aerial photography, LANDSAT) does not have a color table,
usually. It does not need one, because the data in the file are already
R:G:B or grayscale (better: single band) values.
As long as both the data in the tiff file and its georeferencing are
identical to the original raster, the tiff file could be regarded as
perfectly ok. Suggested data visualization with an embedded color table is
nice to have, but coloring schemes are a matter of taste and not crucial
for further processing (e.g. terrain analysis with a DEM).
There is still the issue of nodata: a nodata value must fall into the
range of the chosen datatype, otherwise this information gets lost.
r.out.gdal does not check if a supplied nodata value is valid.
Only Byte and UInt16 will attempt to write out a color palette to the
GeoTIFF AFAIK.
For UINT16, the color table in the tiff file will always have 65535
entries, gdal does that. This slows down display. Have an option in
r.out.gdal to choose whether a color table should be written to the tiff
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --+- Changes (by martinl): * platform: = Unspecified * component: default = Raster * cpu: = Unspecified -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:9 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work ---+ Reporter: helena| Owner: grass-dev@lists.osgeo.org Type: defect| Status: new Priority: critical | Milestone: 6.4.0 Component: default | Version: svn-trunk Resolution:|Keywords: r.out.gdal, tiff ---+ Changes (by mlennert): * priority: major = critical Comment: We've been having similar problems here and I've tested a bit further with current svn. At http://geog-pc40.ulb.ac.be/grass/misc/r_out_gdal/ you can find relevant files: - test_loc.tgz: test location with one raster file - test.tif: this same raster file exported with 'r.out.gdal type=UInt16' - test_tiffinfo.txt: tiffinfo output on test.tif (note the errors at the beginning) - test _gdalinfo.txt gdalinfo output on test.tif, which shows the correct color categories and then an all-black color table from 0 - 65535. Is it normal to have both 'COLOR_TABLE_RULE_RGB_0' entries and a color table in the tiff ? The tif can be read and displayed by QGIS. SAGA gives a weird image (both with band and pixel interleave): test_saga.png. gvSIG crashes when trying to load the file. These are not some general picture viewer, but cartography/GIS programs, so I would hope that we should be able to solve this for reasons of interoperability. Final test which seems to show that there is something wrong: r.in.gdal in=test.tif out=test: import takes a very long time and creates a huge colr table file: % 0 65535 0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 [...] 65535:0 65535:0 Whereas the original colr file only contains this: % 334 460 334:0:191:191 359.2:0:255:0 359.2:0:255:0 384.4:255:255:0 384.4:255:255:0 409.6:255:127:0 409.6:255:127:0 434.8:191:127:63 434.8:191:127:63 460:20 The display of test also takes a long time, I suppose because of this long colr table. I'm moving this up to critical as I think this is a fairly basic feature which should just work... Moritz -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:8 GRASS GIS http://grass.osgeo.org ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
GRASS GIS wrote:
#73: r.out.gdal tiff output does not work
--+-
Reporter: helena | Owner: grass-dev@lists.osgeo.org
Type: defect | Status: new
Priority: major| Milestone: 6.4.0
Component: default | Version: svn-trunk
Resolution: |Keywords: r.out.gdal, tiff
--+-
Comment (by hamish):
(I see the same with 6.2.1's old script version of r.out.gdal)
it works if the values are in the range of 0-255:
{{{
#spearfish
r.out.gdal in=fields out=fields.tif type=Byte
qiv fields.tif # qiv is a quick image viewer
}}}
but not if all values are out of range:
{{{
r.info -r elevation.dem
min=1066
max=1840
r.info -t elevation.dem
datatype=CELL
r.out.gdal in=elevation.dem out=elv.tif type=Byte
gdalinfo -stats elv.tif | grep STATISTIC
STATISTICS_MINIMUM=255
STATISTICS_MAXIMUM=255
STATISTICS_MEAN=255
STATISTICS_STDDEV=0
r.in.gdal in=elv.tif out=elv2
r.info -r elv2
min=255
max=255
}}}
we can try with half the raster in range:
{{{
r.mapcalc elev_0 = elevation.dem - 1066
r.info -r elev_0
min=0
max=774
r.out.gdal in=elev_0 out=elv0.tif type=Byte
gdalinfo -stats elv0.tif | grep STATISTIC
STATISTICS_MINIMUM=0
STATISTICS_MAXIMUM=254
STATISTICS_MEAN=138.68943338231
STATISTICS_STDDEV=56.526296226099
}}}
If I load elv0.tif into an image viewer it comes out as all black. But in
Gimp if you use the eyedropper you can see the index number changes, but
the palette is black for all 256 index entries.
gdalinfo confirms palette info was not written to file:
I have noticed that r.out.gdal only generates the palette if
G_read_colors() returns 1, i.e. if the map has its own colour table.
If G_read_colors() falls back to a default rainbow colour table, you
don't get a palette.
I suggest:
--- raster/r.out.gdal/main.c(revision 30389)
+++ raster/r.out.gdal/main.c(working copy)
@@ -125,7 +125,7 @@
GDALSetRasterColorInterpretation(hBand, GPI_RGB);
CPLPopErrorHandler();
-if( G_read_colors(name, mapset, sGrassColors ) == 1 )
+if( G_read_colors(name, mapset, sGrassColors ) = 0 )
{
int maxcolor, i;
CELL min, max;
G_read_colors() returns -1 on error (e.g. the map doesn't exist), 0 if
a default rainbow colour table was generated, and 1 if the map has its
own colour table.
AFAICT, the UInt16 case actually works, insofar as the debug output
with DEBUG=3 and the output from gdalinfo look just like they do for
an 8-bit TIFF which works fine in various viewers.
I wouldn't rule out the possibility that popular image viewers can't
handle 16-bit indexed-colour images.
TIFF's strong-point is that it supports so many different formats.
It's weak point is also that it supports so many different formats. I
doubt that there is a program in existence which can correctly handle
every possible valid TIFF file.
--
Glynn Clements [EMAIL PROTECTED]
___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: major| Milestone: 6.4.0 Component: default | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff --+- Comment (by dylan): The data range as reported in ArcMap 9.2 when opening a geotiff exported from r.out.gdal is (1.17549e-038,3.40282e+038). I have noticed that when setting the files symbology, when set to a fixed number of classes (ranges) the correct data range is displayed, and the image seems to work fine. This suggests that it may be an out of range or unusual (in terms of ArcMap logic) no-data value. [http://casoilresource.lawr.ucdavis.edu/drupal/node/337 I have updated my notes here] -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:2 GRASS GIS http://grass.osgeo.org GRASS Geographic Information System (GRASS GIS) - http://grass.osgeo.org/___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: major| Milestone: 6.4.0 Component: default | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff --+- Comment (by hamish): Replying to [comment:2 dylan]: The data range as reported in ArcMap 9.2 when opening a geotiff exported from r.out.gdal is (1.17549e-038,3.40282e+038). If it were the NULL value, I'd expect to see extreme values only on one side, not both. Endian problems is another suspect, but probably unlikely if all on x86. I have noticed that when setting the files symbology, when set to a fixed number of classes (ranges) the correct data range is displayed, and the image seems to work fine. This suggests that it may be an out of range or unusual (in terms of ArcMap logic) no-data value. can you do like d.historgram or r.univar in Arc to see if the values are streched over that range or are mostly correct but with weird outliers? what if you try other output types besides Float32? Hamish -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:3 GRASS GIS http://grass.osgeo.org GRASS Geographic Information System (GRASS GIS) - http://grass.osgeo.org/___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: major| Milestone: 6.4.0 Component: default | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff --+- Comment (by helena): Eric mentioned several viewers that don't like the tiff produced by r.out.gdal and for me it won't work with Preview on Mac and gimp. You can try for yourself: here is an original tiff file that is readable by gimp and preview and imports to GRASS nc_spm_07 LOCATION using r.in.gdal (change tfwmf to tfw). http://skagit.meas.ncsu.edu/~helena/grasswork/grassbookdat07/ncexternal/IMG3720079200P20040216.tfwmf http://skagit.meas.ncsu.edu/~helena/grasswork/grassbookdat07/ncexternal/IMG3720079200P20040216.tif After exporting with r.out.gdal the image is not readable. But you are right that the numbers appear to be correct but there seem to be some out of range values (nulls?). With some other rasters where I had nulls I tried to define nulls as - and export to AAGrid using r.out.gdal but there was a problem too. Here is one example of exported tiff http://skagit.meas.ncsu.edu/~helena/grasswork/ortho_2001_t792_1m.tif http://skagit.meas.ncsu.edu/~helena/grasswork/ortho_2001_t792_1m.tfw there may be some option that exports the tiffs correctly but the only one that was readable was this http://skagit.meas.ncsu.edu/~helena/grasswork/lakes.tif http://skagit.meas.ncsu.edu/~helena/grasswork/lakes.tfw Thanks for looking into it Helena -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:5 GRASS GIS http://grass.osgeo.org GRASS Geographic Information System (GRASS GIS) - http://grass.osgeo.org/___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work --+- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: major| Milestone: 6.4.0 Component: default | Version: svn-trunk Resolution: |Keywords: r.out.gdal, tiff --+- Changes (by warmerdam): * cc: warmerdam (added) -- Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:6 GRASS GIS http://grass.osgeo.org GRASS Geographic Information System (GRASS GIS) - http://grass.osgeo.org/___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] Re: [GRASS GIS] #73: r.out.gdal tiff output does not work
#73: r.out.gdal tiff output does not work
--+-
Reporter: helena | Owner: grass-dev@lists.osgeo.org
Type: defect | Status: new
Priority: major| Milestone: 6.4.0
Component: default | Version: svn-trunk
Resolution: |Keywords: r.out.gdal, tiff
--+-
Comment (by warmerdam):
Helena,
I'm coming into this rather late, and just skimming things. I looked at
your ortho_2001_t7921_1m.tif file (no smaller files demostrating the
problem!) and I see it is 32bit integer pixel values. I'm not surprised
lots of packages can't read this.
I looked at lakes.tif and it seems very similar to the ortho image in
configuration - also one 32bit integer sample. So I'm surprised that it
is readable and the ortho image is not in some (unnamed?) application.
PS. it would be convenient to post tiffinfo reports for some of the
problem files to learn more without having to download them. For instance
the ortho report is:
{{{
TIFF Directory at offset 0x23768d2 (37185746)
Image Width: 3048 Image Length: 3048
Bits/Sample: 32
Sample Format: signed integer
Compression Scheme: None
Photometric Interpretation: min-is-black
Samples/Pixel: 1
Rows/Strip: 1
Planar Configuration: single image plane
Tag 33550: 1.00,1.00,0.00
Tag 33922:
0.00,0.00,0.00,637033.00,222504.00,0.00
Tag 34735:
1,1,0,16,1024,0,1,1,1025,0,1,1,1026,34737,24,0,2048,0,1,4269,2049,34737,6,24,2054,0,1,9102,3072,0,1,32767,3074,0,1,32767,3075,0,1,8,3076,0,1,9001,3078,34736,1,2,3079,34736,1,3,3084,34736,1,1,3085,34736,1,0,3086,34736,1,4,3087,34736,1,5
Tag 34736:
33.75,-79.00,36.17,34.33,609601.22,0.00
Tag 34737: Lambert Conformal Conic|grs80|
Tag 42112: GDALMetadata
Item name=COLOR_TABLE_RULES_COUNT sample=01/Item
Item name=COLOR_TABLE_RULE_RGB_0 sample=00.00e+00 2.55e+02
0 0 0 255 255 255/Item
/GDALMetadata
}}}
--
Ticket URL: http://trac.osgeo.org/grass/ticket/73#comment:7
GRASS GIS http://grass.osgeo.org
GRASS Geographic Information System (GRASS GIS) - http://grass.osgeo.org/___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
