Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-04-03 Thread Jonathan Gregory
Dear Karl I am not quite sure if I have understood you. I have changed ticket 85 to propose the following for sect 9.6: Wherever there are unused elements in data storage, the data variable and all its auxiliary coordinate variables (spatial and time) must contain missing values. This situation

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-04-02 Thread Steve Hankin
On 4/2/2012 4:52 AM, Jim Biard wrote: I like Jonathan's suggestion. Fine w/ me. - Steve On Sat, Mar 31, 2012 at 4:19 PM, Jonathan Gregory j.m.greg...@reading.ac.uk mailto:j.m.greg...@reading.ac.uk wrote: Dear all John Caron proposed Applications should treat the data

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-04-02 Thread Hedley, Mark
. -Original Message- From: cf-metadata-boun...@cgd.ucar.edu on behalf of Jonathan Gregory Sent: Sat 31/03/2012 21:19 To: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations Dear all John Caron proposed Applications should treat the data

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-04-02 Thread Jonathan Gregory
Dear all Thanks for your postings. There seems to be a consensus on this. I have therefore changed my existing trac ticket 85 accordingly https://cf-pcmdi.llnl.gov/trac/ticket/85 to record this consensus. Note that this means we have to amend section 9.6 as well, which explicitly disallowed

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-04-02 Thread Karl Taylor
HI all, following my previous email. I think the rules for discussed in ticket 85 should apply only to discrete sampling geometries. Karl On 4/2/12 1:00 PM, Jonathan Gregory wrote: Dear all Thanks for your postings. There seems to be a consensus on this. I have therefore changed my

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-04-02 Thread Karl Taylor
My previous email might have been blocked so here it is again: Dear all, I support the first paragraph, but concerning the last paragraph below, I would suggest being more explicit about the difference between a data void and simply when there is no valid data for an element. Also do the

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-31 Thread Jonathan Gregory
Dear all John Caron proposed Applications should treat the data as missing where the auxiliary coordinates are missing and Steve proposed (an hour later, I think) Application writers should be aware that under some (rare) circumstances data auxiliary coordinate values may be missing,

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-30 Thread Egil Støren
Hi all, Even if the missing aux coordinate values never will be found, I think there is legitimate uses of data where some of the aux coordinate values are missing where data values exist. For example if you want to compute the average of data variables, you do not need the aux coordinate

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-29 Thread Jim Biard
I'm with you, Nan. On Thu, Mar 29, 2012 at 10:22 AM, Nan Galbraith ngalbra...@whoi.edu wrote: Hi All - I agree with John Caron's wording. It should be the responsibility of the application using auxiliary coordinates to respond predictably. In many cases this would mean ignoring data

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-29 Thread Jim Biard
Jonathan, The point is, the information in the variable *is* coordinate information, and there *is* a domain/range relationship between the coordinate information and one or more measurements, even when the content of the auxiliary coordinate variable does not (for more than one possible reason)

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-29 Thread Jonathan Gregory
Dear Jim We are discussing auxiliary coordinate variables. They do not have to be 1D or monotonic. Those requirements apply to coordinate variables in the Unidata sense. CF distinguishes these two concepts in Sect 1.2. The point is, the information in the variable *is* coordinate information,

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-29 Thread John Graybeal
+1 to Steve's wording (that's what I was *trying* to say, Steve's is much better). While I agree that each application should respond predictably to this situation, it is not the case that every application should respond the same way, and we should not indicate that such a monolithic response

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-28 Thread Jonathan Gregory
Dear all Brian has a good point that sect 5.3 allows that there might be missing data in aux coord variables. Appendix A - which is equally ancient in the story of CF :-) - is not consistent with this, because it didn't allow _FillValue or missing_value atts for coordinates until sect 9 was

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-28 Thread Randy Horne
Jonathan: not to belabour , but in the interest of clarity, For GOES-R level 1 and 2 hemispheric products ... there are non-existent points, at which the data vars have missing values and the aux coord vars have (syntactically) valid values. note that placing (syntactically) valid values in

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-28 Thread Jonathan Gregory
Dear Randy Thanks for the clarification. I see no problem in having missing data where there are non-missing aux coord vars. This is a usual situation. Best wishes Jonathan ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-28 Thread Jim Biard
The only thing that designates a variable as an auxiliary coordinate variable is the presence of the variable name in a coordinates attribute on another variable. There is nothing intrinsic to the variable that designates it as a coordinate variable. (Not so for true coordinate variables.) For

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-28 Thread Jonathan Gregory
Dear Jim The only thing that designates a variable as an auxiliary coordinate variable is the presence of the variable name in a coordinates attribute on another variable. Formally, that is right. By definition, a CF aux coord var is one which is named by a coordinates attribute. The

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-28 Thread John Caron
On 3/28/2012 6:26 AM, Jonathan Gregory wrote: Dear all Brian has a good point that sect 5.3 allows that there might be missing data in aux coord variables. Appendix A - which is equally ancient in the story of CF :-) - is not consistent with this, because it didn't allow _FillValue or

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-28 Thread John Caron
On 3/28/2012 10:49 AM, John Caron wrote: I think we have a number of valid use cases for missing data in aux coordinates, and i would vote to allow that. sorry i didnt really answer jonathan's concern. I would suggest this wording: Auxiliary coordinates do not have to be monotonic or have

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-28 Thread John Graybeal
suggest the more general but applications using the auxiliary coordinates to work with data may respond unpredictably when the auxiliary coordinates are missing. john On Mar 28, 2012, at 10:07, John Caron wrote: On 3/28/2012 10:49 AM, John Caron wrote: I think we have a number of valid use

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-27 Thread Bert Jagers
Hi all, Yesterday, I briefly mentioned privately another use case for missing data in auxiliary coordinates to Jonathan. I have now uploaded some text and images for that case on our wiki-server such that I can distribute it to the wider community. See:

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-27 Thread Rich Signell
Folks, There certainly are a fair number of grid featureTypes that also would benefit from FillValues being allowed on aux coordinate variables. Consider this NOAA coastal ocean model grid for Chesapeake Bay: http://screencast.com/t/Humzu7F69 and a zoom in on one of the tributaries:

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-27 Thread Brian Eaton
Hi All, I don't believe there was ever an intention to disallow missing values from auxiliary coordinate variables. First, note that the definition of an auxiliary coordinate variable in section 1.2 makes no mention of this while it is explicit in the definition of a coordinate variable that

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-26 Thread Nan Galbraith
I was unaware of this restriction on aux coordinate variables. On 3/26/12 4:24 AM, Jonathan Gregory wrote: Appendix A notes that missing data is allowed in aux coord vars only in the case of discrete sampling geometries. This means the checker could regard it as an error also if it finds any

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-26 Thread John Caron
On 3/26/2012 2:24 AM, Jonathan Gregory wrote: Dear Ros Regarding this requirement: 9.6 Where any auxiliary coordinate variable contains a missing value, all other coordinate, auxiliary coordinate and data values corresponding to that element should also contain missing values. Appendix A

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-26 Thread Jonathan Gregory
Dear Nan and John It's a good thing we're having this discussion! In my understanding, we have always prohibiting missing data in aux coord vars, and in section 9 we explicitly allowed for the first time. Evidently we should be clear, one way or the other (which was one of the intentions of the

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-26 Thread Randy Horne
Folks: Regarding the geosync request for comment . In the case of GOES-R (and also Meteosat) our coordinate variable values are N/S elevation angle and E/W scanning angle, which can be syntactically valid values albeit off the disk of the earth. However, it is very possible that there

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-26 Thread Nan Galbraith
Hi Jonathan - For underway CTD profiles (gliders and floats, too, I'd think) if the pressure sensor fails intermittently, you can approximate Z by interpolating over time, assuming there are good values along the track. In final data, we might do that, but we might like to present raw data

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-26 Thread Jim Biard
I am working with satellite data, and I, for example, have timestamps that arrive in the data stream along with sensor measurements. I can have independent missing values in both my time variable and my measurement variables. I want to preserve all the incoming data, and the restriction on true

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-26 Thread John Caron
Hi all: From a CDM developer perspective, an auxiliary coordinate is just as good as a regular coordinate variable. The extra requirements on coordinate variables are helpful in knowing when to optimize, eg monotonicity allows one to efficiently find the index given the coordinate value.

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-26 Thread Jonathan Gregory
Dear all Regarding Randy's reply: In the case of GOES-R (and also Meteosat) our coordinate variable values are N/S elevation angle and E/W scanning angle, which can be syntactically valid values albeit off the disk of the earth. In this case, are there data values, or is the data missing

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-26 Thread Nan Galbraith
Hi all - From a CDM developer perspective, an auxiliary coordinate is just as good as a regular coordinate variable. The extra requirements on coordinate variables are helpful in knowing when to optimize, eg monotonicity allows one to efficiently find the index given the coordinate value.

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-26 Thread Randy Horne
Jonathan: We are putting in fill values for these off-earth points in the data variables. very respectfully, randy Randy C. Horne (rho...@excaliburlabs.com) Principal Engineer, Excalibur Laboratories Inc. voice fax: (321) 952-5100 url: http://www.excaliburlabs.com PGP Public Keys available

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-26 Thread Nan Galbraith
On 3/26/12 1:35 PM, Jonathan Gregory wrote: ... Regarding Nan's point, I would say that we do want CF to be inclusive. It would be a mistake to impose irrelevant requirements that deterred people from using the convention. In the case you mention, the chapter 9 convention for profiles wouldn't

[CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-23 Thread Jonathan Gregory
Dear Ros Thanks a lot for working on this. I think you have correctly identified parts which are stated as requirements and recommendations, but you have to put yourself in the shoes of the CF-checker, and consider what you can actually *do* to make the checks. The CF-checker, for instance, does

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-23 Thread Jim Biard
Hi. Jonathan's reply contained the section: 9.6 Where any auxiliary coordinate variable contains a missing value, all other coordinate, auxiliary coordinate and data values corresponding to that element should also contain missing values. I thought I understood that missing values were

Re: [CF-metadata] CF-1.6 Conformance Requirements/Recommendations

2012-03-23 Thread John Caron
On 3/23/2012 1:59 PM, Jim Biard wrote: Hi. Jonathan's reply contained the section: 9.6 Where any auxiliary coordinate variable contains a missing value, all other coordinate, auxiliary coordinate and data values corresponding to that element should also contain missing