Re: [CF-metadata] Indices or Labels as Coordinate Variables

[Martin Juckes - UKRI STFC]

OK, I've raise an issue here: 
https://github.com/cf-convention/cf-conventions/issues/174


cheers,

Martin


From: CF-metadata  on behalf of Jonathan 
Gregory 
Sent: 12 July 2019 13:53:45
To: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Indices or Labels as Coordinate Variables

Dear Martin

Ah, I see. I agree that this is a defect now in our text if they have changed
what they say. I also agree that we should stick with what we've got i.e. more
restrictive, coord vars are 1D, numeric, and monotonic and called the same as
their dimension. That's our data model.

Best wishes

Jonathan

- Forwarded message from Martin Juckes - UKRI STFC 
 -

> Date: Fri, 12 Jul 2019 08:43:33 +
> From: Martin Juckes - UKRI STFC 
> To: Jonathan Gregory , "cf-metadata@cgd.ucar.edu"
>    
> Subject: Re: [CF-metadata] Indices or Labels as Coordinate Variables
>
> Hello Tom, Jonathan,
>
>
> that is interesting. It looks like a change introduced in NetCDF 4 which CF 
> has not caught up with. Do you think they really mean what they say about the 
> values having to be strictly monotonic?
>
>
> The convention does say we use this term "term precisely as it is defined in 
> the NUG section on coordinate variables", but we also state that it must be 
> numeric. Perhaps we should take "precisely" out of that sentence (I think it 
> was true in NetCDF 3)? Keeping string values out of the array dimensions and 
> preserving a clear rule about monotonicity looks like the best approach to me,
>
>
> regards,
>
> Martin
>
>
>
> 
> From: CF-metadata  on behalf of Jonathan 
> Gregory 
> Sent: 10 July 2019 17:55
> To: cf-metadata@cgd.ucar.edu
> Subject: Re: [CF-metadata] Indices or Labels as Coordinate Variables
>
> Dear Tom
>
> That's interesting. I did not know or had forgotten about that Unidata
> recommendation. CF doesn't allow string-valued coordinate variables (with
> the same name as any of their dimensions), only string-valued auxiliary
> coordinate variables.
>
> Best wishes
>
> Jonathan
>
> - Forwarded message from Tom Evans  -
>
> > Date: Tue, 9 Jul 2019 23:08:46 +
> > From: Tom Evans 
> > To: Jonathan Gregory 
> > Subject: RE: [CF-metadata] Indices or Labels as Coordinate Variables
> >
> > Dear Jonathan
> >
> > Thanks very much for your response. It really did help me understand the 
> > use of coordinate values. In particular, your suggestion that an auxiliary 
> > coordinate variable (with a different name from the dimension) could be 
> > listed in the coordinate attribute of a variable is very helpful. It 
> > produces exactly the behaviour I'd look for in Panoply, for example.
> >
> > I have a follow-up question, if you don't mind:
> >
> > The Best Practices page at the Unidata site also mentions a "string-valued 
> > coordinate variable," which has the same name as its first dimension, e.g.
> >
> >char location(location, loc_len)
> >
> > where loc_len is the length of a text label for the location. The CF 
> > specification mentions a "string-valued auxiliary coordinate variable" -- 
> > something like
> >
> >char location_name(location, loc_len)
> >
> > if I'm following this correctly -- which could also be used as a coordinate 
> > in the attribute -- but it doesn't say anything about a string-valued 
> > variable with the same name as a dimension. I also haven't found any 
> > examples that use string-valued coordinate variables named the same as a 
> > dimension. Would such a variable risk conflict or confusion with the 
> > assumptions of applicactions designed to work with CF-compliant data?
> >
> > Cheers
> > Tom Evans
> >
> >
> >
> >
> >
> > [cid:image702fcb.PNG@5e3232e3.49822ae5]<http://www.niwa.co.nz>
> >
> >
> > Dr Tom Evans
> > Software Developer
> > T +64-7-859-1832
> >
> > National Institute of Water & Atmospheric Research Ltd (NIWA)
> > Gate 10 Silverdale Road, Hillcrest, Hamilton
> > Connect with NIWA: niwa.co.nz<https://www.niwa.co.nz> 
> > Facebook<https://www.facebook.com/nzniwa> 
> > Twitter<https://twitter.com/niwa_nz> 
> > LinkedIn<https://www.linkedin.com/company/niwa> 
> > Instagram<https://www.instagram.com/niwa_science>
> >
> > To ensure compliance with legal requirements and to maintain cyber security 
> > standards, NIWA's IT systems are subject to ongoing monitoring, ac

Re: [CF-metadata] Indices or Labels as Coordinate Variables

[Jonathan Gregory]

Dear Martin

Ah, I see. I agree that this is a defect now in our text if they have changed
what they say. I also agree that we should stick with what we've got i.e. more
restrictive, coord vars are 1D, numeric, and monotonic and called the same as
their dimension. That's our data model.

Best wishes

Jonathan

- Forwarded message from Martin Juckes - UKRI STFC 
 -

> Date: Fri, 12 Jul 2019 08:43:33 +
> From: Martin Juckes - UKRI STFC 
> To: Jonathan Gregory , "cf-metadata@cgd.ucar.edu"
>       
> Subject: Re: [CF-metadata] Indices or Labels as Coordinate Variables
> 
> Hello Tom, Jonathan,
> 
> 
> that is interesting. It looks like a change introduced in NetCDF 4 which CF 
> has not caught up with. Do you think they really mean what they say about the 
> values having to be strictly monotonic?
> 
> 
> The convention does say we use this term "term precisely as it is defined in 
> the NUG section on coordinate variables", but we also state that it must be 
> numeric. Perhaps we should take "precisely" out of that sentence (I think it 
> was true in NetCDF 3)? Keeping string values out of the array dimensions and 
> preserving a clear rule about monotonicity looks like the best approach to me,
> 
> 
> regards,
> 
> Martin
> 
> 
> 
> 
> From: CF-metadata  on behalf of Jonathan 
> Gregory 
> Sent: 10 July 2019 17:55
> To: cf-metadata@cgd.ucar.edu
> Subject: Re: [CF-metadata] Indices or Labels as Coordinate Variables
> 
> Dear Tom
> 
> That's interesting. I did not know or had forgotten about that Unidata
> recommendation. CF doesn't allow string-valued coordinate variables (with
> the same name as any of their dimensions), only string-valued auxiliary
> coordinate variables.
> 
> Best wishes
> 
> Jonathan
> 
> - Forwarded message from Tom Evans  -
> 
> > Date: Tue, 9 Jul 2019 23:08:46 +
> > From: Tom Evans 
> > To: Jonathan Gregory 
> > Subject: RE: [CF-metadata] Indices or Labels as Coordinate Variables
> >
> > Dear Jonathan
> >
> > Thanks very much for your response. It really did help me understand the 
> > use of coordinate values. In particular, your suggestion that an auxiliary 
> > coordinate variable (with a different name from the dimension) could be 
> > listed in the coordinate attribute of a variable is very helpful. It 
> > produces exactly the behaviour I'd look for in Panoply, for example.
> >
> > I have a follow-up question, if you don't mind:
> >
> > The Best Practices page at the Unidata site also mentions a "string-valued 
> > coordinate variable," which has the same name as its first dimension, e.g.
> >
> >char location(location, loc_len)
> >
> > where loc_len is the length of a text label for the location. The CF 
> > specification mentions a "string-valued auxiliary coordinate variable" -- 
> > something like
> >
> >char location_name(location, loc_len)
> >
> > if I'm following this correctly -- which could also be used as a coordinate 
> > in the attribute -- but it doesn't say anything about a string-valued 
> > variable with the same name as a dimension. I also haven't found any 
> > examples that use string-valued coordinate variables named the same as a 
> > dimension. Would such a variable risk conflict or confusion with the 
> > assumptions of applicactions designed to work with CF-compliant data?
> >
> > Cheers
> > Tom Evans
> >
> >
> >
> >
> >
> > [cid:image702fcb.PNG@5e3232e3.49822ae5]<http://www.niwa.co.nz>
> >
> >
> > Dr Tom Evans
> > Software Developer
> > T +64-7-859-1832
> >
> > National Institute of Water & Atmospheric Research Ltd (NIWA)
> > Gate 10 Silverdale Road, Hillcrest, Hamilton
> > Connect with NIWA: niwa.co.nz<https://www.niwa.co.nz> 
> > Facebook<https://www.facebook.com/nzniwa> 
> > Twitter<https://twitter.com/niwa_nz> 
> > LinkedIn<https://www.linkedin.com/company/niwa> 
> > Instagram<https://www.instagram.com/niwa_science>
> >
> > To ensure compliance with legal requirements and to maintain cyber security 
> > standards, NIWA's IT systems are subject to ongoing monitoring, activity 
> > logging and auditing. This monitoring and auditing service may be provided 
> > by third parties. Such third parties can access information transmitted to, 
> > processed by and stored on NIWA's IT systems.
> >
> >
> > -Original Message-
> > From

Re: [CF-metadata] Indices or Labels as Coordinate Variables

[Martin Juckes - UKRI STFC]

Hello Tom, Jonathan,


that is interesting. It looks like a change introduced in NetCDF 4 which CF has 
not caught up with. Do you think they really mean what they say about the 
values having to be strictly monotonic?


The convention does say we use this term "term precisely as it is defined in 
the NUG section on coordinate variables", but we also state that it must be 
numeric. Perhaps we should take "precisely" out of that sentence (I think it 
was true in NetCDF 3)? Keeping string values out of the array dimensions and 
preserving a clear rule about monotonicity looks like the best approach to me,


regards,

Martin




From: CF-metadata  on behalf of Jonathan 
Gregory 
Sent: 10 July 2019 17:55
To: cf-metadata@cgd.ucar.edu
Subject: Re: [CF-metadata] Indices or Labels as Coordinate Variables

Dear Tom

That's interesting. I did not know or had forgotten about that Unidata
recommendation. CF doesn't allow string-valued coordinate variables (with
the same name as any of their dimensions), only string-valued auxiliary
coordinate variables.

Best wishes

Jonathan

- Forwarded message from Tom Evans  -

> Date: Tue, 9 Jul 2019 23:08:46 +
> From: Tom Evans 
> To: Jonathan Gregory 
> Subject: RE: [CF-metadata] Indices or Labels as Coordinate Variables
>
> Dear Jonathan
>
> Thanks very much for your response. It really did help me understand the use 
> of coordinate values. In particular, your suggestion that an auxiliary 
> coordinate variable (with a different name from the dimension) could be 
> listed in the coordinate attribute of a variable is very helpful. It produces 
> exactly the behaviour I'd look for in Panoply, for example.
>
> I have a follow-up question, if you don't mind:
>
> The Best Practices page at the Unidata site also mentions a "string-valued 
> coordinate variable," which has the same name as its first dimension, e.g.
>
>char location(location, loc_len)
>
> where loc_len is the length of a text label for the location. The CF 
> specification mentions a "string-valued auxiliary coordinate variable" -- 
> something like
>
>char location_name(location, loc_len)
>
> if I'm following this correctly -- which could also be used as a coordinate 
> in the attribute -- but it doesn't say anything about a string-valued 
> variable with the same name as a dimension. I also haven't found any examples 
> that use string-valued coordinate variables named the same as a dimension. 
> Would such a variable risk conflict or confusion with the assumptions of 
> applicactions designed to work with CF-compliant data?
>
> Cheers
> Tom Evans
>
>
>
>
>
> [cid:image702fcb.PNG@5e3232e3.49822ae5]<http://www.niwa.co.nz>
>
>
> Dr Tom Evans
> Software Developer
> T +64-7-859-1832
>
> National Institute of Water & Atmospheric Research Ltd (NIWA)
> Gate 10 Silverdale Road, Hillcrest, Hamilton
> Connect with NIWA: niwa.co.nz<https://www.niwa.co.nz> 
> Facebook<https://www.facebook.com/nzniwa> 
> Twitter<https://twitter.com/niwa_nz> 
> LinkedIn<https://www.linkedin.com/company/niwa> 
> Instagram<https://www.instagram.com/niwa_science>
>
> To ensure compliance with legal requirements and to maintain cyber security 
> standards, NIWA's IT systems are subject to ongoing monitoring, activity 
> logging and auditing. This monitoring and auditing service may be provided by 
> third parties. Such third parties can access information transmitted to, 
> processed by and stored on NIWA's IT systems.
>
>
> -Original Message-
> From: CF-metadata  On Behalf Of Jonathan 
> Gregory
> Sent: Tuesday, 9 July 2019 12:42 AM
> To: cf-metadata@cgd.ucar.edu
> Subject: [CF-metadata] Indices or Labels as Coordinate Variables
>
> Dear Tom
>
> It would be a coordinate variable if it's location(location) and in that case
> its values have to be unique and monotonic because generic applications
> may make that assumption when plotting or doing computations with them. If
> these are really labels then those operations don't really make sense anyway.
> In that case it would be more suitable to store the IDs in an auxiliary
> coordinate variable, which doesn't have the same name as the dimension, and
> is listed in the coordinates attribute of the data variable. Values in aux
> coord variables do not have to be unique or monotonic or even numeric. It is
> not mandatory to include a coordinate variable (in the Unidata sense).
>
> Best wishes
>
> Jonathan
>
> - Forwarded message from Tom Evans  -
>
> > Date: Mon, 8 Jul 2019 01:51:19 +
> > From: Tom Evans 
> > To: &qu

Re: [CF-metadata] Indices or Labels as Coordinate Variables

[Jonathan Gregory]

Dear Tom

That's interesting. I did not know or had forgotten about that Unidata
recommendation. CF doesn't allow string-valued coordinate variables (with
the same name as any of their dimensions), only string-valued auxiliary
coordinate variables.

Best wishes

Jonathan

- Forwarded message from Tom Evans  -

> Date: Tue, 9 Jul 2019 23:08:46 +
> From: Tom Evans 
> To: Jonathan Gregory 
> Subject: RE: [CF-metadata] Indices or Labels as Coordinate Variables
> 
> Dear Jonathan
> 
> Thanks very much for your response. It really did help me understand the use 
> of coordinate values. In particular, your suggestion that an auxiliary 
> coordinate variable (with a different name from the dimension) could be 
> listed in the coordinate attribute of a variable is very helpful. It produces 
> exactly the behaviour I'd look for in Panoply, for example.
> 
> I have a follow-up question, if you don't mind:
> 
> The Best Practices page at the Unidata site also mentions a "string-valued 
> coordinate variable," which has the same name as its first dimension, e.g.
> 
>char location(location, loc_len)
> 
> where loc_len is the length of a text label for the location. The CF 
> specification mentions a "string-valued auxiliary coordinate variable" -- 
> something like
> 
>char location_name(location, loc_len)
> 
> if I'm following this correctly -- which could also be used as a coordinate 
> in the attribute -- but it doesn't say anything about a string-valued 
> variable with the same name as a dimension. I also haven't found any examples 
> that use string-valued coordinate variables named the same as a dimension. 
> Would such a variable risk conflict or confusion with the assumptions of 
> applicactions designed to work with CF-compliant data?
> 
> Cheers
> Tom Evans
> 
> 
> 
> 
> 
> [cid:image702fcb.PNG@5e3232e3.49822ae5]<http://www.niwa.co.nz>
> 
> 
> Dr Tom Evans
> Software Developer
> T +64-7-859-1832
> 
> National Institute of Water & Atmospheric Research Ltd (NIWA)
> Gate 10 Silverdale Road, Hillcrest, Hamilton
> Connect with NIWA: niwa.co.nz<https://www.niwa.co.nz> 
> Facebook<https://www.facebook.com/nzniwa> 
> Twitter<https://twitter.com/niwa_nz> 
> LinkedIn<https://www.linkedin.com/company/niwa> 
> Instagram<https://www.instagram.com/niwa_science>
> 
> To ensure compliance with legal requirements and to maintain cyber security 
> standards, NIWA's IT systems are subject to ongoing monitoring, activity 
> logging and auditing. This monitoring and auditing service may be provided by 
> third parties. Such third parties can access information transmitted to, 
> processed by and stored on NIWA's IT systems.
> 
> 
> -Original Message-
> From: CF-metadata  On Behalf Of Jonathan 
> Gregory
> Sent: Tuesday, 9 July 2019 12:42 AM
> To: cf-metadata@cgd.ucar.edu
> Subject: [CF-metadata] Indices or Labels as Coordinate Variables
> 
> Dear Tom
> 
> It would be a coordinate variable if it's location(location) and in that case
> its values have to be unique and monotonic because generic applications
> may make that assumption when plotting or doing computations with them. If
> these are really labels then those operations don't really make sense anyway.
> In that case it would be more suitable to store the IDs in an auxiliary
> coordinate variable, which doesn't have the same name as the dimension, and
> is listed in the coordinates attribute of the data variable. Values in aux
> coord variables do not have to be unique or monotonic or even numeric. It is
> not mandatory to include a coordinate variable (in the Unidata sense).
> 
> Best wishes
> 
> Jonathan
> 
> - Forwarded message from Tom Evans  -
> 
> > Date: Mon, 8 Jul 2019 01:51:19 +
> > From: Tom Evans 
> > To: "cf-metadata@cgd.ucar.edu" 
> > Subject: [CF-metadata] Indices or Labels as Coordinate Variables
> >
> > I have a question regarding coordinate variables:
> >
> > I am working with time-series data representing hydrological conditions at 
> > fixed locations in a stream network. The values are generated by a model at 
> > regular time intervals, and I believe that the data will fit well into the 
> > timeSeries feature type described in the "Discrete Sampling Geometries" 
> > chapter of the CF conventions. For example, we would put all the discharge 
> > values into a single 2D array:
> >
> > double flow(time, location)
> >
> > The dimension "location" here is the "instance dimension" described in the 
&g

[CF-metadata] Indices or Labels as Coordinate Variables

[Jonathan Gregory]

Dear Tom

It would be a coordinate variable if it's location(location) and in that case
its values have to be unique and monotonic because generic applications
may make that assumption when plotting or doing computations with them. If
these are really labels then those operations don't really make sense anyway.
In that case it would be more suitable to store the IDs in an auxiliary
coordinate variable, which doesn't have the same name as the dimension, and
is listed in the coordinates attribute of the data variable. Values in aux
coord variables do not have to be unique or monotonic or even numeric. It is
not mandatory to include a coordinate variable (in the Unidata sense).

Best wishes

Jonathan

- Forwarded message from Tom Evans  -

> Date: Mon, 8 Jul 2019 01:51:19 +
> From: Tom Evans 
> To: "cf-metadata@cgd.ucar.edu" 
> Subject: [CF-metadata] Indices or Labels as Coordinate Variables
> 
> I have a question regarding coordinate variables:
> 
> I am working with time-series data representing hydrological conditions at 
> fixed locations in a stream network. The values are generated by a model at 
> regular time intervals, and I believe that the data will fit well into the 
> timeSeries feature type described in the “Discrete Sampling Geometries” 
> chapter of the CF conventions. For example, we would put all the discharge 
> values into a single 2D array:
> 
> double flow(time, location)
> 
> The dimension “location” here is the “instance dimension” described in the 
> convention.
> 
> I would like to use an integer variable named “location” as a coordinate 
> variable to go along with the location dimension. I think this would provide 
> a handy way for post-processing programs to locate a time series in our model 
> result files. The Best Practices guidance on the Unidata website, though, 
> says that coordinate variables “must be strictly monotonic” and the order of 
> the IDs in my location variable is arbitrary. All of the location values are 
> unique, but the location numbers are essentially numerical labels – location 
> 1524 is distinct from location 2817, but neither is greater than the other in 
> a way that means anything to the model. Location IDs do not consistently 
> increase or decrease traveling downstream, for example.
> 
> So, is the guidance that coordinate variable should strictly increase or 
> decrease relevant to my case? I’ve built some sample files and examined them 
> using Panoply, and in Python using xarray. I haven’t seen any problems with 
> using non-monotonic integer “ID numbers” as coordinate variables, but that 
> “must” in the guidance troubles me. If my locations are identified by 
> arbitrary numbers, do I run the risk of scrambling the links between my time 
> series and their identifiers?
> 
> Thanks
> Tom Evans
> 
> 
> 
> [cid:image9ef98c.PNG@58bab801.4bad83a3]<http://www.niwa.co.nz>
> 
> 
> Dr Tom Evans
> Software Developer
> T +64-7-859-1832
> 
> National Institute of Water & Atmospheric Research Ltd (NIWA)
> Gate 10 Silverdale Road, Hillcrest, Hamilton
> Connect with NIWA: niwa.co.nz<https://www.niwa.co.nz> 
> Facebook<https://www.facebook.com/nzniwa> 
> Twitter<https://twitter.com/niwa_nz> 
> LinkedIn<https://www.linkedin.com/company/niwa> 
> Instagram<https://www.instagram.com/niwa_science>
> 
> To ensure compliance with legal requirements and to maintain cyber security 
> standards, NIWA's IT systems are subject to ongoing monitoring, activity 
> logging and auditing. This monitoring and auditing service may be provided by 
> third parties. Such third parties can access information transmitted to, 
> processed by and stored on NIWA's IT systems.
> 
> 
> 
> 
> 



> ___
> CF-metadata mailing list
> CF-metadata@cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata


- End forwarded message -
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

[CF-metadata] Indices or Labels as Coordinate Variables

[Tom Evans]

I have a question regarding coordinate variables:

I am working with time-series data representing hydrological conditions at 
fixed locations in a stream network. The values are generated by a model at 
regular time intervals, and I believe that the data will fit well into the 
timeSeries feature type described in the “Discrete Sampling Geometries” chapter 
of the CF conventions. For example, we would put all the discharge values into 
a single 2D array:

double flow(time, location)

The dimension “location” here is the “instance dimension” described in the 
convention.

I would like to use an integer variable named “location” as a coordinate 
variable to go along with the location dimension. I think this would provide a 
handy way for post-processing programs to locate a time series in our model 
result files. The Best Practices guidance on the Unidata website, though, says 
that coordinate variables “must be strictly monotonic” and the order of the IDs 
in my location variable is arbitrary. All of the location values are unique, 
but the location numbers are essentially numerical labels – location 1524 is 
distinct from location 2817, but neither is greater than the other in a way 
that means anything to the model. Location IDs do not consistently increase or 
decrease traveling downstream, for example.

So, is the guidance that coordinate variable should strictly increase or 
decrease relevant to my case? I’ve built some sample files and examined them 
using Panoply, and in Python using xarray. I haven’t seen any problems with 
using non-monotonic integer “ID numbers” as coordinate variables, but that 
“must” in the guidance troubles me. If my locations are identified by arbitrary 
numbers, do I run the risk of scrambling the links between my time series and 
their identifiers?

Thanks
Tom Evans



[cid:image9ef98c.PNG@58bab801.4bad83a3]


Dr Tom Evans
Software Developer
T +64-7-859-1832

National Institute of Water & Atmospheric Research Ltd (NIWA)
Gate 10 Silverdale Road, Hillcrest, Hamilton
Connect with NIWA: niwa.co.nz 
Facebook Twitter 
LinkedIn 
Instagram

To ensure compliance with legal requirements and to maintain cyber security 
standards, NIWA's IT systems are subject to ongoing monitoring, activity 
logging and auditing. This monitoring and auditing service may be provided by 
third parties. Such third parties can access information transmitted to, 
processed by and stored on NIWA's IT systems.





___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata