Hi Jim:
_FillValue/missing_value refers to the packed value, so theres no
ambiguity in the original dataset. It is best to make sure its outside
the range of real values, but even if not, one just has to search for
that exact bit pattern.
If someone rewrites the data, its their responsibility to choose a
_FillValue/missing_value that is unambiguous.
If your use case is important, you could add _UnpackedFillValue, so that
your software did the right thing. Dunno if it would be generally useful.
John
On 9/27/2012 7:00 AM, Jim Biard wrote:
Hi.
Assuming you have the luxury of specifying your _FillValue and/or
missing_value, I agree that this isn't a big deal. However, I am
working with data where the project has defined fill/missing values that
are wholly within the range of possible values (NPP satellite data).
The approach defined below fails in such cases.
Grace and peace,
Jim
Jim Biard
Research Scholar
Cooperative Institute for Climate and Satellites
Remote Sensing and Applications Division
National Climatic Data Center
151 Patton Ave, Asheville, NC 28801-5001
jim.bi...@noaa.gov <mailto:jim.bi...@noaa.gov>
828-271-4900
On Sep 25, 2012, at 11:08 PM, Russ Rew <r...@unidata.ucar.edu
<mailto:r...@unidata.ucar.edu>> wrote:
Hi Phil,
The final para of section 2.5.1 of the CF conventions document describes
the use of the _FillValue (or missing_value) attribute in the case of
data packed using the scale-and-offset method. What is not clear - at
least to me - is what the preferred application behaviour should be in
the case where the data is unpacked and then written out to a new netCDF
file. In particular, what fill value should be used for the unpacked
data variable?
I presume that one wouldn't normally want to use the original fill value
since that value (typically an 8- or 16-bit integer) is quite likely to
fall within the normal range of the unpacked data (typically a 32- or
64-bit float).
In the absence of explicitly setting a fill value attribute on the
unpacked data variable I assume that the netCDF default fill value will
be used for the data type in question. Which may not always be desirable
(certainly not for 32-bit floats, where the default fill value can give
rise to subtle precision-related problems).
With this in mind, I was wondering if there is any merit in defining a
new attribute called, say, _UnpackedFillValue (or
unpacked_missing_value)? If client software detected this attribute then
the associated value (same data type as the scale_factor and add_offset
attributes) would be used as the fill value for the unpacked data
variable.
Alternatively, the names _FillValueUnpacked (missing_value_unpacked)
might be preferable since they would then appear together pair-wise in
CDL-type listings, e.g.
short pkd_var(z, y, x) :
...
pkd_var:_FillValue =3D -32768 ;
pkd_var:_FillValueUnpacked =3D -1.0e30 ;
pkd_var:add_offset =3D 42.0 ;
pkd_var:scale_factor =3D 1234.0 ;
...
Any merit/mileage in this idea?
A more detailed recommendation for treating special values such as
_FillValue or missing_value for packed data is described, with
associated formulas, in the "Packed Data Values" section of a
best-practices document that we wrote a few years ago:
http://www.unidata.ucar.edu/netcdf/docs/BestPractices.html#Packed%20Data%20Values
It provides a recommendation for ensuring the unpacked special value is
not in the range of other unpacked data values. If that recommendation
is followed, I think there is no need for an additional
_FillValueUnpacked (or missing_value_unpacked) attribute.
If you agree that is an acceptable approach, perhaps we should add it to
CF ...
--Russ
_______________________________________________
CF-metadata mailing list
CF-metadata@cgd.ucar.edu <mailto:CF-metadata@cgd.ucar.edu>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
