#73: r.out.gdal tiff output does not work --------------------------+------------------------------------------------- Reporter: helena | Owner: grass-dev@lists.osgeo.org Type: defect | Status: new Priority: critical | Milestone: 6.4.0 Component: Raster | Version: svn-trunk Resolution: | Keywords: r.out.gdal, tiff Platform: Unspecified | Cpu: Unspecified --------------------------+------------------------------------------------- Comment (by mmetz):
Replying to [comment:18 hamish]: > Replying to [comment:11 mmetz] (and others): > Several of my previous descriptions about the conservative version of r.out.gdal are out of date, e.g. the way statistics are calculated. I would prefer not to stick strictly to my version, instead use it as ideas for discussion. > > Spearfish's elevation.dted would be a better sample dataset to use here, it is CELL 0-255. or convert spot.image into RGB bands. > Other test datasets are the spearfish imagery60 data, and the North Carolina dataset lsat7 images. > Using sample datasets available to all is a very good idea. > > > Still, datatypes other than Byte can only be read with spatial data viewers that could make use of this colour table, but not all of them are actually able to do so. > > Important to know when this is happening, not really our problem, and not much we can do about it as long as we are writing metadata to spec. > I agree, GRASS can not fix or accommodate other viewers/packages. > > > It would help a lot to have a list with these data types and their ranges in the man page so that the appropriate data type for file export can be chosen. > > Yes. Can someone in the know supply some suggested text? > See the updated documentation in my version. > > > Maybe the color table is not so important, different people have different preferences > > The color table will be carefully constructed/chosen by some people and they may wish to keep it. e.g. r.colors.stddev addon script, r.cpt2grass addon script, and NASA color tables given for MODIS sat data (see grass's MODIS wiki page). > These data are real (calibrated into sea surface temperature or mg/mL chlorophyll) and also have specially designed color tables (incl log- scale) which are nice to keep. > True, but if you preprocess such data for someone else who will be running his/her own analyses, a colortable is not important. Sometimes it is desirable to keep the colortable, sometimes not, therefore I would like to have the option to keep it or not. > > > Suggested data visualization with an embedded color table is nice to have, but coloring schemes are a matter of taste > > geotiff has a method for assigning color palettes, all I suggest is populating that correctly, apparently we are currently (or were) failing at. > I disagree, the colortable is populated correctly with the current svn version (at least in my tests), but other GIS software may not be able to make use of it. Nothing GRASS can do about that. > > > There is still the issue of nodata: a nodata value must fall into the range of the chosen datatype, otherwise this information gets lost. r.out.gdal does not check if a supplied nodata value is valid. > > Hopefully better in SVN now. If the user manually specifies an out of range value, I don't mind respecting that. > Users may accidentally specify an out of range nodata value. That has a similar effect like replacing NULL values in the GRASS raster. I would at least like to show a warning. > > > For UINT16, the color table in the tiff file will always have 65535 entries, gdal does that. > > perhaps this should be addressed in GDAL then; only write palette entries which are given. > File an enhancement for GDAL? > > > Have an option in r.out.gdal to choose whether a color table should be written to the tiff file? > > For Uint16, how to just get the "COLOR_TABLE_RULE_RGB_0" part of this without the "(RGB with 66535 entries)" part? > Done in my version. > > > basic statistics (min, max, mean, stddev) are calculated for the current region, replacing NULL cells with nodata value (default or specified) and then exported with GDALSetRasterStatistics() > > how is this better than "gdalinfo -stats" ? I thought it could be better because other software could read the metadata. But they don't, at least the ones I tested. So there is no advantage in giving stats in the metadata. > > I don't understand, you convert NULL to nodata value in stats? So for a CELL map with lots of NULL areas, if I make nodata=100 (well above the number of used palette entries) or leave it at 255, the stats will be skewed towards 100 or 255 instead of just considering the real data? huh? "gdalinfo -stats" seems to do the right thing here. > Right,I changed that and ignore nodata, stats as calculated by my version are now identical to stats calculated by gdalinfo. > > > colortable is only exported on request (new flag needs to be set) and only if there is an entry in directories colr or colr2 > > This is wrong, libgis is in charge of handling colors. Use the API not raw file access. For one thing, otherwise you miss out on the default mapping if there isn't a colr/ file. > G_read_colors() returns 0 if no entry in colr or colr2 is present, if an entry is present it returns 1. That's what I used, not raw file access. > > > Additionally, a warning would be useful if the selected datatype does not cover the range of raster values to be exported > > i.e. if the map was CELL 0-744 as in my elev_0 example earlier in this ticket, but the user asked for Byte (0-255). I think that's a very good idea. > > Start by moving range maxes into header #defines for each type from the currently scattered "nodataval = -1E37f;" etc lines in the r.out.gdal source code. > Also do that for range min's? > > If other software ignore valid GeoTIFF metadata tags, then that's their problem and their responsibility to provide an alternative for their users. All we can do is correctly implement the spec. If other software read but incorrectly handle valid GeoTIFF metadata, then that's really their bug to deal with. > Again, at least have the option. You can still tell people using such software that they should try GRASS... > OK, the general question is, should GRASS consider that some packages don't fully support all GeoTIFF or other file type features or not and then strictly stick to the specs? > > > Apparently COLOR_TABLE_RULE entries are custom metadata not read by other software. > > Any confirmation on this? It is custom to GRASS or custom to OSGeo family (GDAL dependents) or suggested by the geotiff spec or ...??? > My personal interpretation of the source code of r.out.gdal, needs confirmation. I can only say that the tested packages apart from QGIS don't make use of COLOR_TABLE_RULE_RGB_%d entries. > > > current region restored on exit > > module should not have to do that, it should set WIND_OVERRIDE variable or so, so if module is interrupted by ctrl-C or whatever the temporary change does not become permanent. (but as mentioned earlier, this violates grass's raster module conventions and the feature should not be applied in SVN) > Yes, it violates GRASS raster conventions, but in this case I can't see why it would be desirable to change the resolution on the fly with g.region, i.e. nearest neighbour for raster export. At least it should be stated clearly in the documentation that r.out.gdal respects the current region extends and resolution. > > > if supported, GDAL decides > > what does that mean? when is min/max etc not supported?? > I don't no the specs of all raster types supported by GDAL. r.out.gdal can try to export all metadata available, but some other formats support more metadata, some less. It should not be of interest for GRASS, because GDAL decides whether certain metadata are supported or not. > > > Tests were performed with the elevation raster map layer (range within Byte, FCELL map) in the North Carolina sample dataset, creating a MASK for elevation values <= 70m, needed for testing of nodata handling with nodata=0. The elevation raster was exported as Byte, UInt16, Int32, and Float32, once with colortable (Byte and UInt16 only, colortable export for other data types not supported by gdal), once without colortable, always as GeoTIFF. > > Please supply map names and exact commands so we can replicate. > I will do so in a separate reply, this one is long enough already. > > > QGIS-0.9.1 on Linux 64bit > > QGIS-0.11.1 on XP > > ... > > ESRI ArcMap 9.2 on XP > > this is handy information for users to be aware of, but (again) not much we can do about 3rd party support beyond coding to geotiff spec/recomendations. > In the end, it boils down to colortables in UInt16 files, therefore my request for an option to export colortables. That doesn't harm and can do good. > > > I would prefer to use r.out.gmt to get the GRASS color rules into an extra file (GMT .cpt). This would also be a workaround for export file types other than Byte and UInt16 where GDAL does not support writing out a colortable. But then I don't know how widely GMT .cpt files are supported. > > Since it has been some months since the original bug report, and the code has changed, could we please restate what this bug actually is (rather than fuzzy improved r.out.gdal niceness), and some spearfish or NC sample-dataset instructions for recreating it? After studying the ticket from beginning to end I can't say where the GRASS bug is, beyond: > > * provide warning if map data overflows requested data type= > * type limits to #defines in the code, and human readable to the help page > * how to make GDAL stop writing color rules for UInt16 once Maximum= is reached (instead of writing out 66535-Max empty rules) Ask Frank Warmerdam if this is necessary? This is GDAL code, AFAIK the GDAL API does not provide such an option. > > what else? > * an option for colortable export:-) * Nodata handling: more documentation, warning or error messages if out of range because this would be implicit raster editing. > > And we must remember that r.out.gdal is not just for GeoTIFF files.. > ... but for many many other formats. Too many for testing? Leave as much error handling as possible to GDAL? In the end, r.out.gdal as it is now may not be as bad as the length of this ticket suggests, and the 3 points of Hamish plus my two points would not require a lot of rewriting. Regards, Markus -- Ticket URL: <https://trac.osgeo.org/grass/ticket/73#comment:19> GRASS GIS <http://grass.osgeo.org>
_______________________________________________ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev