Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hi John,
Thanks, I think we have reached a near conclusion to our discussions
about how best to encode a 'single' profile CTD cast following CF standard v1.6.
This not been easy as CF allows more than 1 way to do it and the standard is
large.
Thanks also Jonathon, Roy, Nan, Jim, Upendra, Thierry, Karl for your valuable
inputs,
hope I haven't missed anyone else!
Jonathon, thanks for clarifying that 'axis' attribute will be allowed on
'auxilliary coordinate variables', yes the standard was confusing.
To summarise I will use a 'Scalar' method for simplicity, noting that
the 'Vector' method is also acceptable.
Attached for reference is the netCDF file and CDL (.txt) of the same.
Convention attribute has been updated to 'CF-1.6'
Andrew
- Original Message -
From: "John Caron"
To:
Sent: Wednesday, May 02, 2012 2:21 AM
Subject: Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
On 4/30/2012 8:40 PM, andrew walsh wrote:
Hi John and CF-Metadata list,
Based on your earlier advice I decided using the Scalar way to represent
the coordinate lat, long and time rather than Vector way i.e lat(lat0,
lon(lon), time(time)
mainly for reason of simplicity.
the correct other choice is lat(profile), lon(profile), time(profile). not
sure if you caught that.
the scalar choice is fine for 1 profile per file. the other is used to store
multiple profiles in one file.
I have run the Scalar sample through the BADC netCDF checker (*Note)
at http://titania.badc.rl.ac.uk/cgi-bin/cf-checker.pl and I now get this
error
on each of the lat, lon, time variables:
ERROR (4): Axis attribute is not allowed for auxillary coordinate variables.
I don't get this error with the Vector approach. It looks like the checker
thinks
my scalar lat, lon, time are 'auxilliary coordinate variables' and axis
attributes
are not allowed on these. Is the checker interpreting things correctly?
i think we recently clarified that axis is acceptable on auxiliary
coordinates. OTOH, its unecessary, as long as you follow the other rules for
identifying coordinates (chapter 4).
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
netcdf ScalarCTDFinal {
dimensions:
pressure = UNLIMITED ; // (9 currently)
variables:
double time ;
time:standard_name = "time" ;
time:units = "days since 1950-01-01 00:00:00" ;
time:axis = "T" ;
time:valid_min = 0 ;
time:valid_max = 99 ;
byte time_qc_flag ;
time_qc_flag:long_name = "quality control flag for time (primary
Level 1 flag)" ;
time_qc_flag:quality_control_convention = "Proposed IODE qc scheme
March 2012" ;
time_qc_flag:valid_min = 1 ;
time_qc_flag:valid_max = 9 ;
time_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
time_qc_flag:flag_meanings = "good not_evaluated_or_unknown suspect
bad missing" ;
double latitude ;
latitude:standard_name = "latitude" ;
latitude:units = "degrees_north" ;
latitude:axis = "Y" ;
latitude:valid_min = -90 ;
latitude:valid_max = 90 ;
double longitude ;
longitude:standard_name = "longitude" ;
longitude:units = "degrees_east" ;
longitude:axis = "X" ;
longitude:valid_min = -180 ;
longitude:valid_max = 180 ;
byte position_qc_flag ;
position_qc_flag:long_name = "quality control flag for position
(primary Level 1 flag)" ;
position_qc_flag:quality_control_convention = "Proposed IODE qc
scheme March 2012" ;
position_qc_flag:valid_min = 1 ;
position_qc_flag:valid_max = 9 ;
position_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
position_qc_flag:flag_meanings = "good not_evaluated_or_unknown
suspect bad missing" ;
double pressure(pressure) ;
pressure:standard_name = "sea_water_pressure" ;
pressure:units = "decibars" ;
pressure:axis = "Z" ;
pressure:valid_min = 0 ;
pressure:valid_max = 12000 ;
pressure:positive = "down" ;
double temperature(pressure) ;
temperature:_FillValue = -99.99 ;
temperature:standard_name = "sea_water_temperature" ;
temperature:units = "degrees_C" ;
temperature:valid_min = -2 ;
temperature:valid_max = 40 ;
temperature:ancillary_variables = "temperature_whole_profile_flag
temperature_qc_flag temperature_sd_test" ;
temperature:coordinates = "time latitude longitude pressure" ;
byte temperature_whole_profile_flag ;
temp
Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hi Jim, Yes, thats correct. If you see the attached CDL in earlier email we have used coordinates attributes on measured variables e.g temperature:coordinates = "time latitude longitude pressure" Andrew - Original Message - From: Jim Biard To: cf-metadata@cgd.ucar.edu Sent: Wednesday, May 02, 2012 4:51 AM Subject: Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6 Hi. If someone chooses the scalar approach, isn't it necessary to declare the coordinates via the coordinates attribute on the measurement variables? Grace and peace, Jim Jim Biard Research Scholar Cooperative Institute for Climate and Satellites Remote Sensing and Applications Division National Climatic Data Center 151 Patton Ave, Asheville, NC 28801-5001 jim.bi...@noaa.gov 828-271-4900 On May 1, 2012, at 12:21 PM, John Caron wrote: On 4/30/2012 8:40 PM, andrew walsh wrote: Hi John and CF-Metadata list, Based on your earlier advice I decided using the Scalar way to represent the coordinate lat, long and time rather than Vector way i.e lat(lat0, lon(lon), time(time) mainly for reason of simplicity. the correct other choice is lat(profile), lon(profile), time(profile). not sure if you caught that. the scalar choice is fine for 1 profile per file. the other is used to store multiple profiles in one file. I have run the Scalar sample through the BADC netCDF checker (*Note) at http://titania.badc.rl.ac.uk/cgi-bin/cf-checker.pl and I now get this error on each of the lat, lon, time variables: ERROR (4): Axis attribute is not allowed for auxillary coordinate variables. I don't get this error with the Vector approach. It looks like the checker thinks my scalar lat, lon, time are 'auxilliary coordinate variables' and axis attributes are not allowed on these. Is the checker interpreting things correctly? i think we recently clarified that axis is acceptable on auxiliary coordinates. OTOH, its unecessary, as long as you follow the other rules for identifying coordinates (chapter 4). ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Reverse-time trajectory
Dear Steve, > Having CF time axes that run backwards will break a lot of software. > It will be a net disruption to interoperability. It seems to me that it will only break the software if that software is used, and that CF-netCDF is being excluded from a valid use. I wonder if it is better to use CF and cherry pick your software than otherwise? Afterall, I suspect that discrete sampling geometries themselves will also break a lot of current software ... :-) All the best, David Original message from Steve Hankin (11AM 01 May 12) > Date: Tue, 1 May 2012 11:15:54 -0700 > From: Steve Hankin > User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120420 > Thunderbird/12.0 > To: Jonathan Gregory > CC: cf-metadata@cgd.ucar.edu > Subject: Re: [CF-metadata] Reverse-time trajectory > > Richard, Jonathan, et. al., > > As the famed Henning piece on CORBA stated -- in standards > committees "no" is a preferable answer to "yes" all other things > considered. More generality can often lead to less > interoperability in CF or other data standards. > > Having CF time axes that run backwards will break a lot of software. > It will be a net disruption to interoperability. So the question: > "is there a sufficiently compelling use case for admitting it?", > where we interpret "compelling" in the context of balancing the > general loss of interoperability against the gains to particular > datasets from adding this bit of flexibility. > > - Steve > > > > On 5/1/2012 8:02 AM, Jonathan Gregory wrote: > >Dear Richard > > > >>The CF definitions of discrete sampling geometries define the "trajectory" > >>feature type as "a series of data points along a path through space with > >>monotonically increasing times". This is a stricter stance than the usual > >>CF coordinate definition of "ordered monotonically". What was the reason > >>behind the addition of the "increasing" constraint, and can it be relaxed? > >>We have data sets where a model is run backwards in time. > >Good point. I would think that "monotonically" would be enough, not > >necessarily > >increasing. I can't remember a reason for "increasing"; I assume it's just > >because sect 9 was conceived for observations in the first place. However, > >John > >Caron may well have a comment. I don't think anything prevents your storing > >the data in the orthogonal multidimensional representation, which existed > >before sect 9 did and doesn't require increasing coordinates. > > > >Best wishes > > > >Jonathan > >___ > >CF-metadata mailing list > >CF-metadata@cgd.ucar.edu > >http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- David Hassell National Centre for Atmospheric Science (NCAS) Department of Meteorology, University of Reading, Earley Gate, PO Box 243, Reading RG6 6BB, U.K. Tel : 0118 3785613 Fax : 0118 3788316 E-mail: d.c.hass...@reading.ac.uk ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Reverse-time trajectory
Dear Steve > As the famed Henning piece on CORBA stated -- in standards > committees "no" is a preferable answer to "yes" all other things > considered. More generality can often lead to less > interoperability in CF or other data standards. I think that's too negative myself. CF is successful partly because it tries to accommodate what people want to do, by and large, usually with existing mechanisms, sometimes with new ones. This is more effective for encouraging take-up of a standard than it would be to tell people what they have to do. I agree that "No" is the starting-point when we have a proposal to add or change conventions; there has to be a good reason for adding more complexity, and even more for breaking backward compatibility. However, what is already permitted by the standard is surely OK, isn't it, even if it is unusual. CF has always permitted coordinate axes to run in either sense, and never said anything special about time in that respect. I see no reason why a time coordinate axis shouldn't run backwards, just as a latitude axis could be N-S or S-N. Apart from Richard's example, paleoclimate timeseries often have backward time axes. Richard asked about discrete sampling geometries in particular. Here, there is a question whether we really need to require time to run forwards in the new representations (orthogonal multidimensional and ragged representations). It would be interesting to know what John thinks, since his software is probably the most widely used implementation of sect 9. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hi. If someone chooses the scalar approach, isn't it necessary to declare the coordinates via the coordinates attribute on the measurement variables? Grace and peace, Jim Jim Biard Research Scholar Cooperative Institute for Climate and Satellites Remote Sensing and Applications Division National Climatic Data Center 151 Patton Ave, Asheville, NC 28801-5001 jim.bi...@noaa.gov 828-271-4900 On May 1, 2012, at 12:21 PM, John Caron wrote: > On 4/30/2012 8:40 PM, andrew walsh wrote: >> Hi John and CF-Metadata list, >> >> Based on your earlier advice I decided using the Scalar way to represent >> the coordinate lat, long and time rather than Vector way i.e lat(lat0, >> lon(lon), time(time) >> mainly for reason of simplicity. > > the correct other choice is lat(profile), lon(profile), time(profile). not > sure if you caught that. > > the scalar choice is fine for 1 profile per file. the other is used to store > multiple profiles in one file. > >> >> I have run the Scalar sample through the BADC netCDF checker (*Note) >> at http://titania.badc.rl.ac.uk/cgi-bin/cf-checker.pl and I now get this >> error >> on each of the lat, lon, time variables: >> >> ERROR (4): Axis attribute is not allowed for auxillary coordinate variables. >> I don't get this error with the Vector approach. It looks like the checker >> thinks >> my scalar lat, lon, time are 'auxilliary coordinate variables' and axis >> attributes >> are not allowed on these. Is the checker interpreting things correctly? > > i think we recently clarified that axis is acceptable on auxiliary > coordinates. OTOH, its unecessary, as long as you follow the other rules for > identifying coordinates (chapter 4). > > ___ > CF-metadata mailing list > CF-metadata@cgd.ucar.edu > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Reverse-time trajectory
Richard, Jonathan, et. al., As the famed Henning piece on CORBA stated -- in standards committees "no" is a preferable answer to "yes" all other things considered. More generality can often lead to less interoperability in CF or other data standards. Having CF time axes that run backwards will break a lot of software. It will be a net disruption to interoperability. So the question: "is there a sufficiently compelling use case for admitting it?", where we interpret "compelling" in the context of balancing the general loss of interoperability against the gains to particular datasets from adding this bit of flexibility. - Steve On 5/1/2012 8:02 AM, Jonathan Gregory wrote: Dear Richard The CF definitions of discrete sampling geometries define the "trajectory" feature type as "a series of data points along a path through space with monotonically increasing times". This is a stricter stance than the usual CF coordinate definition of "ordered monotonically". What was the reason behind the addition of the "increasing" constraint, and can it be relaxed? We have data sets where a model is run backwards in time. Good point. I would think that "monotonically" would be enough, not necessarily increasing. I can't remember a reason for "increasing"; I assume it's just because sect 9 was conceived for observations in the first place. However, John Caron may well have a comment. I don't think anything prevents your storing the data in the orthogonal multidimensional representation, which existed before sect 9 did and doesn't require increasing coordinates. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Reverse-time trajectory
On 5/1/2012 9:02 AM, Jonathan Gregory wrote: Dear Richard The CF definitions of discrete sampling geometries define the "trajectory" feature type as "a series of data points along a path through space with monotonically increasing times". This is a stricter stance than the usual CF coordinate definition of "ordered monotonically". What was the reason behind the addition of the "increasing" constraint, and can it be relaxed? We have data sets where a model is run backwards in time. Good point. I would think that "monotonically" would be enough, not necessarily increasing. I can't remember a reason for "increasing"; I assume it's just because sect 9 was conceived for observations in the first place. However, John Caron may well have a comment. I don't think anything prevents your storing the data in the orthogonal multidimensional representation, which existed before sect 9 did and doesn't require increasing coordinates. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata i agree, "increasing" is not needed, and could be changed with a defect ticket. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
On 4/30/2012 8:40 PM, andrew walsh wrote: Hi John and CF-Metadata list, Based on your earlier advice I decided using the Scalar way to represent the coordinate lat, long and time rather than Vector way i.e lat(lat0, lon(lon), time(time) mainly for reason of simplicity. the correct other choice is lat(profile), lon(profile), time(profile). not sure if you caught that. the scalar choice is fine for 1 profile per file. the other is used to store multiple profiles in one file. I have run the Scalar sample through the BADC netCDF checker (*Note) at http://titania.badc.rl.ac.uk/cgi-bin/cf-checker.pl and I now get this error on each of the lat, lon, time variables: ERROR (4): Axis attribute is not allowed for auxillary coordinate variables. I don't get this error with the Vector approach. It looks like the checker thinks my scalar lat, lon, time are 'auxilliary coordinate variables' and axis attributes are not allowed on these. Is the checker interpreting things correctly? i think we recently clarified that axis is acceptable on auxiliary coordinates. OTOH, its unecessary, as long as you follow the other rules for identifying coordinates (chapter 4). ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] Response to [lc-cci] proposal for new CF standard names for land cover observation and classification
Dear Dr Winsemius > I would like to acknowledge that we also are interested in similar > variable names as posted by mr. Martin Boettcher. The subject of the > previous post was: > [CF-metadata] [lc-cci] proposal for new CF standard names for land > cover observation and classification I wonder if you could have a look at my comment on this from 28 Mar http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2012/027874.html In this comment, I wondered if we could have greater precision in the definition of "occurrence". Best wishes Jonathan Gregory ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] Reverse-time trajectory
Dear Richard > The CF definitions of discrete sampling geometries define the "trajectory" > feature type as "a series of data points along a path through space with > monotonically increasing times". This is a stricter stance than the usual CF > coordinate definition of "ordered monotonically". What was the reason behind > the addition of the "increasing" constraint, and can it be relaxed? We have > data sets where a model is run backwards in time. Good point. I would think that "monotonically" would be enough, not necessarily increasing. I can't remember a reason for "increasing"; I assume it's just because sect 9 was conceived for observations in the first place. However, John Caron may well have a comment. I don't think anything prevents your storing the data in the orthogonal multidimensional representation, which existed before sect 9 did and doesn't require increasing coordinates. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Dear Andrew > ERROR (4): Axis attribute is not allowed for auxillary coordinate variables. > I don't get this error with the Vector approach. It looks like the > checker thinks > my scalar lat, lon, time are 'auxilliary coordinate variables' and > axis attributes > are not allowed on these. Is the checker interpreting things correctly? The scalar coordinate vars are formally aux coord vars, because they are named by the coordinates attribute. There has been confusion for a long time on whether the axis attribute is allowed for aux coord vars, but we eventually decided that it should be. This has been agreed on trac ticket 62 https://cf-pcmdi.llnl.gov/trac/ticket/62 which should go in the next version of CF. Therefore you could ignore these errors. The checker will be updated when CF is. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
[CF-metadata] Response to [lc-cci] proposal for new CF standard names for land cover observation and classification
Dear CF maintainers, I would like to acknowledge that we also are interested in similar variable names as posted by mr. Martin Boettcher. The subject of the previous post was: [CF-metadata] [lc-cci] proposal for new CF standard names for land cover observation and classification In particular we are interested (and I quote) in: / standard name: probability_of_water_occurrence />/ definition: probability that the observed pixel shows water />/ unit: none (a value between 0 and 1)/ and along with this a standard name, which defines the quality of this estimate, using a modifier. The suggestion by mr. Boettcher was: / standard name modifier: confidence_level />/ definition: applied to some discrete variable, denotes the probability that the assignment of the value to the variable is correct / Our variable name for the quality would then become: standard_name: probability_of_water_occurrence_confidence_level There was some inclarity as to what 'probability of water occurrence' means. In our case, we are preparing an operational automated service for flood mapping from EnviSAT ASAR satellite data (in the future, we'll use Sentinel-1). In our algorithms, each pixel is given a probability that it contains open water. We are never 100% sure that a pixel is land or water. Therefore, we need a 'probability' instead. The ability of our algorithm to distinguish between land or water is sometimes good and sometimes not. Therefore we need the 'confidence_level'. Does this clarify our case? Please let me know if any clarifications are required. Kind regards, Hessel Winsemius DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
