Re: [CF-metadata] Quality flag values for missing data
On Aug 23, 2012, at 2:37 PM, Randy Horne wrote: > > What is the “generally accepted” approach for how a specific quality flag > value should be assigned for a corresponding data value (in the corresponding > data variable) that has a _FillValue ? > > Is it sufficient that, because the data variable value is _FillValue, the > corresponding quality flag value can be undefined ? I do want to mention one variant to consider: if the quality of a particular point is the *reason* that the data value is _FillValue, then it is useful for that corresponding quality flag to retain its value. We use this approach in our Data Quality Screening Service, which allows users to filter (some of) our satellite data based on the quality flags. In the output product, we replace data values with _FillValues if the quality flag indicates it does not meet specified quality criteria, but then we retain the quality flag so that the user will know WHY that particular point has a _FillValue (vs. not being able to get a retrieval at all). -- Dr. Christopher Lynnes, NASA/GSFC, Code 610.2, 301-614-5185 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Quality flag values for missing data
>From the QARTOD meetings I attended and the vocabularies in our repository, >I'm confident the oceanographic community uses a lot more QC flag approaches >than listed so far. So it is safe to say that there is no single approach. Still, the approaches from Randy and Roy certainly are accepted by a whole lot of folks. If that meets your definition of "the generally accepted approach", you're good to go. John On Aug 24, 2012, at 00:27, andrew walsh wrote: > Randy, > > The oceanographic community uses a set of integer QC flags (0 to 9) > See document http://www.nodc.noaa.gov/GTSPP/document/qcmans/MG22rev1.pdf > and section 2 'Quality Flagging'. > > One of the flags is for a 'missing value' = 9. Would that suit your fill > value case? > > A netCDF sample in CDL is: > > double TEMP(TIME, DEPTH, LATITUDE, LONGITUDE) ; > TEMP:standard_name = "sea_water_temperature" ; > TEMP:units = "Celsius" ; > TEMP:_FillValue = -99.99 ; > TEMP:valid_min = -2. ; > TEMP:valid_max = 40. ; > TEMP:quality_control_set = 1. ; > TEMP:ancillary_variables = "TEMP_quality_control" ; > byte TEMP_quality_control(TIME, DEPTH, LATITUDE, LONGITUDE) ; > TEMP_quality_control:long_name = "quality control flag for temperature" ; > TEMP_quality_control:standard_name = "sea_water_temperature status_flag" ; > TEMP_quality_control:quality_control_convention = "IMOS standard using IODE > flags" ; > TEMP_quality_control:_FillValue = -9b ; > TEMP_quality_control:valid_min = 0 ; > TEMP_quality_control:valid_max = 9 ; > TEMP_quality_control:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ; > TEMP_quality_control:flag_meanings = "no_qc_performed good_data > probably_good_data bad_data_that_are_potentially_correctable bad_data > value_changed not_used not_used interpolated_values missing_values" ; > > HTH, > > Andrew Walsh > > - Original Message - From: "Randy Horne" > To: > Cc: ; ; > Sent: Friday, August 24, 2012 4:37 AM > Subject: [CF-metadata] Quality flag values for missing data > > >> >> What is the "generally accepted" approach for how a specific quality flag >> value should be assigned for a corresponding data value (in the >> corresponding data variable) that has a _FillValue ? >> >> Is it sufficient that, because the data variable value is _FillValue, the >> corresponding quality flag value can be undefined ? >> >> >> >> ..End of Message ...--> >> >> >> >> >> >> ___ >> CF-metadata mailing list >> CF-metadata@cgd.ucar.edu >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata >> > > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > John Graybeal<mailto:jgrayb...@ucsd.edu> phone: 858-534-2162 Product Manager Ocean Observatories Initiative Cyberinfrastructure Project: http://ci.oceanobservatories.org Marine Metadata Interoperability Project: http://marinemetadata.org ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Quality flag values for missing data
Hi Randy, The practice recommended for SeaDataNet is to specify a fill value for the flag that is defined in the flag convention as 'missing value'. I always feel more comfortable with explicit semantics. Cheers, Roy. From: CF-metadata [cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Randy Horne [rho...@excaliburlabs.com] Sent: 23 August 2012 19:37 To: cf-metadata@cgd.ucar.edu Cc: aschu...@harris.com; rhorn...@harris.com; ekenn...@aer.com Subject: [CF-metadata] Quality flag values for missing data What is the “generally accepted” approach for how a specific quality flag value should be assigned for a corresponding data value (in the corresponding data variable) that has a _FillValue ? Is it sufficient that, because the data variable value is _FillValue, the corresponding quality flag value can be undefined ? ..End of Message ...--> ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata-- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Quality flag values for missing data
Randy, The oceanographic community uses a set of integer QC flags (0 to 9) See document http://www.nodc.noaa.gov/GTSPP/document/qcmans/MG22rev1.pdf and section 2 'Quality Flagging'. One of the flags is for a 'missing value' = 9. Would that suit your fill value case? A netCDF sample in CDL is: double TEMP(TIME, DEPTH, LATITUDE, LONGITUDE) ; TEMP:standard_name = "sea_water_temperature" ; TEMP:units = "Celsius" ; TEMP:_FillValue = -99.99 ; TEMP:valid_min = -2. ; TEMP:valid_max = 40. ; TEMP:quality_control_set = 1. ; TEMP:ancillary_variables = "TEMP_quality_control" ; byte TEMP_quality_control(TIME, DEPTH, LATITUDE, LONGITUDE) ; TEMP_quality_control:long_name = "quality control flag for temperature" ; TEMP_quality_control:standard_name = "sea_water_temperature status_flag" ; TEMP_quality_control:quality_control_convention = "IMOS standard using IODE flags" ; TEMP_quality_control:_FillValue = -9b ; TEMP_quality_control:valid_min = 0 ; TEMP_quality_control:valid_max = 9 ; TEMP_quality_control:flag_values = 0b, 1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b, 9b ; TEMP_quality_control:flag_meanings = "no_qc_performed good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used not_used interpolated_values missing_values" ; HTH, Andrew Walsh - Original Message ----- From: "Randy Horne" To: Cc: ; ; Sent: Friday, August 24, 2012 4:37 AM Subject: [CF-metadata] Quality flag values for missing data What is the "generally accepted" approach for how a specific quality flag value should be assigned for a corresponding data value (in the corresponding data variable) that has a _FillValue ? Is it sufficient that, because the data variable value is _FillValue, the corresponding quality flag value can be undefined ? ..End of Message ...--> ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] Quality flag values for missing data
What is the generally accepted approach for how a specific quality flag value should be assigned for a corresponding data value (in the corresponding data variable) that has a _FillValue ? Is it sufficient that, because the data variable value is _FillValue, the corresponding quality flag value can be undefined ? ..End of Message ...--> ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
