Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
On 3/30/2013 11:08 PM, John Graybeal wrote: On Mar 28, 2013, at 22:23, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: If it's not going to be used as the time coordinate, then we don't need a standard_name or unit for it, as you don't need libraries to be able to universally auto-detect it and be able to compute with it. Perhaps true in your set of requirements, but not in mine. I want the entire data file to be computable (as much as possible), because the systems I want to build -- and allow others to build -- will not scale otherwise. Hi John, Could you state your requirements for us? ncISO is a well-established tool specifically designed to summarize the binary information contained in a CF file in an ISO metadata (text) format. How/why is it unsuitable to your needs? - Steve It may seem odd to make netCDF with CF the base of a computable architecture, given the original motivation for standard names was more narrow (as I am slowly learning). But netCDF/CF is such a strong start toward more complete interoperability, and netCDF is used as the basis of data storage so often in this community, that I keep trying to pull it in that direction (of more complete and computable interoperability). Perhaps it is a bridge too far, and another path is warranted. John --- John Graybeal Marine Metadata Interoperability Project: http://marinemetadata.org grayb...@marinemetadata.org ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
On Mar 28, 2013, at 22:23, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: If it's not going to be used as the time coordinate, then we don't need a standard_name or unit for it, as you don't need libraries to be able to universally auto-detect it and be able to compute with it. Perhaps true in your set of requirements, but not in mine. I want the entire data file to be computable (as much as possible), because the systems I want to build -- and allow others to build -- will not scale otherwise. It may seem odd to make netCDF with CF the base of a computable architecture, given the original motivation for standard names was more narrow (as I am slowly learning). But netCDF/CF is such a strong start toward more complete interoperability, and netCDF is used as the basis of data storage so often in this community, that I keep trying to pull it in that direction (of more complete and computable interoperability). Perhaps it is a bridge too far, and another path is warranted. John --- John Graybeal Marine Metadata Interoperability Project: http://marinemetadata.org grayb...@marinemetadata.org ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
On Mar 28, 2013, at 17:54, Steve Hankin steven.c.han...@noaa.gov wrote: netCDF files are in every sense binary files. They cannot be read except by custom-built utilities. (Or is there a constituency that wants to access CF using the unix strings command?) In all cases except the present discussion, it is the job of those custom-built utilities to generate formatted string representations of the information contained in the CF binary encoded variables. The entire current discussion would not be happening, if the custom-built utilities and standard code libraries supported the ability to get time information into and out of our binary files using formatted ISO 8601 strings. This is arguably not true. I gave two use cases, one was the derided equivalent of your Unix strings command (call me crazy, it fits in this case!); the other was the desire to store an ASCII string of particular structure and meaning into the binary netCDF file, and then to label the information in that binary file with what it is. No more, and no less. (Uh, unless I think of another use case. :-) Seriously, I think some use cases, partly including my first one, go directly to your point -- my tool can't print this timestamp as ISO 8601 so I'm going to duplicate the data as ASCII, in that ISO format, as a workaround -- but the second one remains a real use case regardless of existing tool support for representations. And it goes beyond time, now that we're on this topic. The fact that most use netCDF as a strict binary encoding does not mean it must exclude those who want to use it to store ASCII strings. That is perhaps the key criterion -- the community can say No ASCII string representations of anything!, or No standard names for ASCII strings, if either is a constraint they really want. So, for those who want to be able to store strings, however different that may sound, and then label them with standard names when that's appropriate -- is the tent open to that? Nothing in the standard suggested to me it was not, though it often seems to offend practitioners, so maybe I've missed something. John --- John Graybeal Marine Metadata Interoperability Project: http://marinemetadata.org grayb...@marinemetadata.org ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
CF does support using ASCII strings for enumerated lists of named objects: PI name, ship names, species names, etc. An important encoding ability. That capability is not in question. - Steve On 3/28/2013 9:36 AM, John Graybeal wrote: On Mar 28, 2013, at 17:54, Steve Hankin steven.c.han...@noaa.gov mailto:steven.c.han...@noaa.gov wrote: netCDF files are in every sense binary files. They cannot be read except by custom-built utilities. (Or is there a constituency that wants to access CF using the unix strings command?) In all cases except the present discussion, it is the job of those custom-built utilities to generate formatted string representations of the information contained in the CF binary encoded variables. The entire current discussion would not be happening, if the custom-built utilities and standard code libraries supported the ability to get time information into and out of our binary files using formatted ISO 8601 strings. This is arguably not true. I gave two use cases, one was the derided equivalent of your Unix strings command (call me crazy, it fits in this case!); the other was the desire to store an ASCII string of particular structure and meaning into the binary netCDF file, and then to label the information in that binary file with what it is. No more, and no less. (Uh, unless I think of another use case. :-) Seriously, I think some use cases, partly including my first one, go directly to your point -- my tool can't print this timestamp as ISO 8601 so I'm going to duplicate the data as ASCII, in that ISO format, as a workaround -- but the second one remains a real use case regardless of existing tool support for representations. And it goes beyond time, now that we're on this topic. The fact that most use netCDF as a strict binary encoding does not mean it must exclude those who want to use it to store ASCII strings. That is perhaps the key criterion -- the community can say No ASCII string representations of anything!, or No standard names for ASCII strings, if either is a constraint they really want. So, for those who want to be able to store strings, however different that may sound, and then label them with standard names when that's appropriate -- is the tent open to that? Nothing in the standard suggested to me it was not, though it often seems to offend practitioners, so maybe I've missed something. John --- John Graybeal Marine Metadata Interoperability Project: http://marinemetadata.org grayb...@marinemetadata.org mailto:grayb...@marinemetadata.org ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
On Thu, Mar 28, 2013 at 7:49 AM, Jim Biard jim.bi...@noaa.gov wrote: It seems to me that we are trying to figure out how to denote that a variable contains a non-arithmetic expression of time, similar to degree minute second hemisphere representations of latitude and longitude. (Non-arithmetic may be a poor way of expressing what I mean. I'm trying to say that you can't just take two values and add or subtract them in an atomic operation.) sure you can -- in both cases. you can't add or subtract the strings directly, but neither can you add or subtract anything with a computer without specifying an encoding and the operations on that encoding. It just so happens that any system we care about has basic integer and floating point types, and most don't have a datetime type, but that's an implementation issue. You can represent such values in strings, but you can also represent them by packing them into long integers (to millisecond accuracy). exactly -- these are different encodings of the same information. I see no reason to exclude the use of the units attribute to denote that the values are expressions of time in which the time since the epoch has been diced up into years, months, days, hours, minutes, and seconds (with varying precision indicated by omission of finer resolution elements). Our current use of the units attribute for time does more than just specify the units (days vs hours, etc). What are the units for such a non-arithmetic time value? They are complex. We could specify something like years months days (in the case of a variable that contained dates only), or we could specify something like datetime. When you went to the units table to find out datetime means, you would find a description. I had argued that this wasn't a unit -- and indeed, I don't think it is, but the current CF convention is to use time_unit since datetime as the unit, which is more than a unit as well. So it wouldn't be inconsistent to allow a unit like iso8601 string. However, that means specifying an alternate way to encode datetimes in CF, which I don't think we should do. If you have ISO strings associated with your data that you want to preserve, and want to be reasonably human readable, there is no reason you can't put them in a string variable, with a long_name that describes what it is. If it's not going to be used as the time coordinate, then we don't need a standard_name or unit for it, as you don't need libraries to be able to universally auto-detect it and be able to compute with it. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
My question was, Is that all it supports ASCII strings for? (Not meant to be a loaded question, but it seems to be at the heart of the discussion and opinions expressed.) John On Mar 28, 2013, at 18:56, Steve Hankin steven.c.han...@noaa.gov wrote: CF does support using ASCII strings for enumerated lists of named objects: PI name, ship names, species names, etc. An important encoding ability. That capability is not in question. - Steve On 3/28/2013 9:36 AM, John Graybeal wrote: On Mar 28, 2013, at 17:54, Steve Hankin steven.c.han...@noaa.gov wrote: netCDF files are in every sense binary files. They cannot be read except by custom-built utilities. (Or is there a constituency that wants to access CF using the unix strings command?) In all cases except the present discussion, it is the job of those custom-built utilities to generate formatted string representations of the information contained in the CF binary encoded variables. The entire current discussion would not be happening, if the custom-built utilities and standard code libraries supported the ability to get time information into and out of our binary files using formatted ISO 8601 strings. This is arguably not true. I gave two use cases, one was the derided equivalent of your Unix strings command (call me crazy, it fits in this case!); the other was the desire to store an ASCII string of particular structure and meaning into the binary netCDF file, and then to label the information in that binary file with what it is. No more, and no less. (Uh, unless I think of another use case. :-) Seriously, I think some use cases, partly including my first one, go directly to your point -- my tool can't print this timestamp as ISO 8601 so I'm going to duplicate the data as ASCII, in that ISO format, as a workaround -- but the second one remains a real use case regardless of existing tool support for representations. And it goes beyond time, now that we're on this topic. The fact that most use netCDF as a strict binary encoding does not mean it must exclude those who want to use it to store ASCII strings. That is perhaps the key criterion -- the community can say No ASCII string representations of anything!, or No standard names for ASCII strings, if either is a constraint they really want. So, for those who want to be able to store strings, however different that may sound, and then label them with standard names when that's appropriate -- is the tent open to that? Nothing in the standard suggested to me it was not, though it often seems to offend practitioners, so maybe I've missed something. John --- John Graybeal Marine Metadata Interoperability Project: http://marinemetadata.org grayb...@marinemetadata.org John Graybealmailto:grayb...@marinexplore.com John Graybeal mailto:jgrayb...@ucsd.edu phone: 858-534-2162 Product Manager Ocean Observatories Initiative Cyberinfrastructure Project: http://ci.oceanobservatories.org Marine Metadata Interoperability Project: http://marinemetadata.org ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
Hi Steve, On Wed, Mar 27, 2013 at 11:05 AM, Steve Hankin steven.c.han...@noaa.gov wrote: I think we're talking about different issues. The thought question I posed was not whether it is acceptable to have a standard_name assigned to string variable. Nothing wrong with a string variable. Rather it was to point out that ISO date-time strings are a way of encoding the physical quantity that we know as TIME. So TIME is the right standard_name for ISO date-time strings per the definition quoted above. Just to clarify this point: I interpreted your initial statement in this thread as saying that standard names are exclusively for identifying physical quantities. The five standard names I mentioned in my reply are not for physical quantities, for example: region for geographical regions, soil_type for soil types, sea_ice_classification for sea ice types, etc. -Aleksandar ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
[chris.bar...@noaa.gov] Sent: 27 March 2013 15:56 Cc: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?) On Wed, Mar 27, 2013 at 8:05 AM, Steve Hankin steven.c.han...@noaa.gov wrote: ISO date-time strings are a way of encoding the physical quantity that we know as TIME. So TIME is the right standard_name for ISO date-time strings per the definition quoted above. Now, it may be that there is a compelling argument to violating the normal definition of standard_name for the case of ISO date-time strings. Or on the other hand is it preferable to use the units attribute to indicate the use of an ISO date-time string? An ISO string for a datetime is not a name (it's still time), but it is not a unit either. What it is is a data type -- more akin to a float or integer -- i.e. a particular way to translate bytes to a value. The bytes are a char array, and the value is the datetime itself. I don't know if thinking about it this way is helpful, as we are building on netcdf, and I don't now that netcdf allows you to define new data types, but food for thought. Also, of course, all the other data types in netcdf (and CF) are direct translations to commonly used binary formats in computers, and this one is not. hmm -- a quick peak at the netcdf4 docs says: The richer enhanced model supports user-defined types and data structures So maybe this could be a user defined type? Having said that, I don't support using ISO strings to define datetimes in CF. I understand particular use-cases, like keeping the original time stamp from a data collection system and the like, but then maybe it's really just arbitrary auxiliary text information, in which case maybe we don't need a standard name or custom data types at all. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
) that didn't separate storage and representation formats because of the amount of code that was needed handle all of the cross-conversions. Grace and peace, Jim Jim Biard Research Scholar Cooperative Institute for Climate and Satellites Remote Sensing and Applications Division National Climatic Data Center 151 Patton Ave, Asheville, NC 28801-5001 jim.bi...@noaa.gov mailto:jim.bi...@noaa.gov 828-271-4900 On Mar 28, 2013, at 5:48 AM, Lowry, Roy K. r...@bodc.ac.uk mailto:r...@bodc.ac.uk wrote: Dear All, I think Chris has hit the nail on the head here. In my view neither the Standard Name nor the units of measure are the way to describe what is in essence the format of a string. So, what other options are there open to us? I can see three alternatives: 1) Use the long name to describe the string format (not just the standard used but the profile) 2) Use the existing calendar attribute 3) Specify a suitable extension to CF to do the job. These are roughly in my order of preference. Cheers, Roy. From: CF-metadata [cf-metadata-boun...@cgd.ucar.edu mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Chris Barker - NOAA Federal [chris.bar...@noaa.gov mailto:chris.bar...@noaa.gov] Sent: 27 March 2013 15:56 Cc: cf-metadata@cgd.ucar.edu mailto:cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?) On Wed, Mar 27, 2013 at 8:05 AM, Steve Hankin steven.c.han...@noaa.gov mailto:steven.c.han...@noaa.gov wrote: ISO date-time strings are a way of encoding the physical quantity that we know as TIME. So TIME is the right standard_name for ISO date-time strings per the definition quoted above. Now, it may be that there is a compelling argument to violating the normal definition of standard_name for the case of ISO date-time strings. Or on the other hand is it preferable to use the units attribute to indicate the use of an ISO date-time string? An ISO string for a datetime is not a name (it's still time), but it is not a unit either. What it is is a data type -- more akin to a float or integer -- i.e. a particular way to translate bytes to a value. The bytes are a char array, and the value is the datetime itself. I don't know if thinking about it this way is helpful, as we are building on netcdf, and I don't now that netcdf allows you to define new data types, but food for thought. Also, of course, all the other data types in netcdf (and CF) are direct translations to commonly used binary formats in computers, and this one is not. hmm -- a quick peak at the netcdf4 docs says: The richer enhanced model supports user-defined types and data structures So maybe this could be a user defined type? Having said that, I don't support using ISO strings to define datetimes in CF. I understand particular use-cases, like keeping the original time stamp from a data collection system and the like, but then maybe it's really just arbitrary auxiliary text information, in which case maybe we don't need a standard name or custom data types at all. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov mailto:chris.bar...@noaa.gov ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
Dear all, For CMIP5 we tried to construct long names, based on the standard name (omitting all underscores), but with additional key information (e.g., supplementing air_temperature with additional info. like Daily Maximum Near-Surface Air Temperature). See http://cmip-pcmdi.llnl.gov/cmip5/docs/standard_output.pdf I think long_name should be used to provide a human readable, reasonably specific description of the variable, which might be used, for example, in a title of a graph. On the other hand, in CMIP5 these titles were not fully descriptive. In the example given above, for example, there is no indication that this was a *monthly mean* of Daily Maximum Near-Surface Air Temperature, although that can be found in the other metadata. I would not favor using long name for a different purpose, if that is what's being suggested. cheers, Karl On 3/28/13 2:48 AM, Lowry, Roy K. wrote: Dear All, I think Chris has hit the nail on the head here. In my view neither the Standard Name nor the units of measure are the way to describe what is in essence the format of a string. So, what other options are there open to us? I can see three alternatives: 1) Use the long name to describe the string format (not just the standard used but the profile) 2) Use the existing calendar attribute 3) Specify a suitable extension to CF to do the job. These are roughly in my order of preference. Cheers, Roy. From: CF-metadata [cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Chris Barker - NOAA Federal [chris.bar...@noaa.gov] Sent: 27 March 2013 15:56 Cc: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?) On Wed, Mar 27, 2013 at 8:05 AM, Steve Hankin steven.c.han...@noaa.gov wrote: ISO date-time strings are a way of encoding the physical quantity that we know as TIME. So TIME is the right standard_name for ISO date-time strings per the definition quoted above. Now, it may be that there is a compelling argument to violating the normal definition of standard_name for the case of ISO date-time strings. Or on the other hand is it preferable to use the units attribute to indicate the use of an ISO date-time string? An ISO string for a datetime is not a name (it's still time), but it is not a unit either. What it is is a data type -- more akin to a float or integer -- i.e. a particular way to translate bytes to a value. The bytes are a char array, and the value is the datetime itself. I don't know if thinking about it this way is helpful, as we are building on netcdf, and I don't now that netcdf allows you to define new data types, but food for thought. Also, of course, all the other data types in netcdf (and CF) are direct translations to commonly used binary formats in computers, and this one is not. hmm -- a quick peak at the netcdf4 docs says: The richer enhanced model supports user-defined types and data structures So maybe this could be a user defined type? Having said that, I don't support using ISO strings to define datetimes in CF. I understand particular use-cases, like keeping the original time stamp from a data collection system and the like, but then maybe it's really just arbitrary auxiliary text information, in which case maybe we don't need a standard name or custom data types at all. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
with the current restriction that the storage format for times be numeric (that is, in essence, what we currently have), and leave the question of representation formats up to other layers, but I understand the desire to have a way to store human-readable dates/times that would be consistent across files. I've had many headaches maintaining a proprietary legacy software base (not netCDF-related) that didn't separate storage and representation formats because of the amount of code that was needed handle all of the cross-conversions. Grace and peace, Jim Jim Biard Research Scholar Cooperative Institute for Climate and Satellites Remote Sensing and Applications Division National Climatic Data Center 151 Patton Ave, Asheville, NC 28801-5001 jim.bi...@noaa.gov 828-271-4900 On Mar 28, 2013, at 5:48 AM, Lowry, Roy K. r...@bodc.ac.uk wrote: Dear All, I think Chris has hit the nail on the head here. In my view neither the Standard Name nor the units of measure are the way to describe what is in essence the format of a string. So, what other options are there open to us? I can see three alternatives: 1) Use the long name to describe the string format (not just the standard used but the profile) 2) Use the existing calendar attribute 3) Specify a suitable extension to CF to do the job. These are roughly in my order of preference. Cheers, Roy. From: CF-metadata [cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Chris Barker - NOAA Federal [chris.bar...@noaa.gov] Sent: 27 March 2013 15:56 Cc: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?) On Wed, Mar 27, 2013 at 8:05 AM, Steve Hankin steven.c.han...@noaa.gov wrote: ISO date-time strings are a way of encoding the physical quantity that we know as TIME. So TIME is the right standard_name for ISO date-time strings per the definition quoted above. Now, it may be that there is a compelling argument to violating the normal definition of standard_name for the case of ISO date-time strings. Or on the other hand is it preferable to use the units attribute to indicate the use of an ISO date-time string? An ISO string for a datetime is not a name (it's still time), but it is not a unit either. What it is is a data type -- more akin to a float or integer -- i.e. a particular way to translate bytes to a value. The bytes are a char array, and the value is the datetime itself. I don't know if thinking about it this way is helpful, as we are building on netcdf, and I don't now that netcdf allows you to define new data types, but food for thought. Also, of course, all the other data types in netcdf (and CF) are direct translations to commonly used binary formats in computers, and this one is not. hmm -- a quick peak at the netcdf4 docs says: The richer enhanced model supports user-defined types and data structures So maybe this could be a user defined type? Having said that, I don't support using ISO strings to define datetimes in CF. I understand particular use-cases, like keeping the original time stamp from a data collection system and the like, but then maybe it's really just arbitrary auxiliary text information, in which case maybe we don't need a standard name or custom data types at all. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
On 3/26/2013 7:20 PM, Aleksandar Jelenak - NOAA Affiliate wrote: Hi Steve, On Tue, Mar 19, 2013 at 6:19 PM, Steve Hankin steven.c.han...@noaa.gov wrote: Hi Aleksander, A question to debate in your trac ticket. Per the CF documentation, the definition of the standard_name is /The name used to identify the_physical quantity_/ I found five standard names for variables with string values. Their units are either not specified, or declared as 1 or string. Hi Aleksander, I think we're talking about different issues. The thought question I posed was not whether it is acceptable to have a standard_name assigned to string variable. Nothing wrong with a string variable. Rather it was to point out that ISO date-time strings are a way of encoding the _physical quantity_ that we know as TIME. So TIME is the right standard_name for ISO date-time strings per the definition quoted above. Now, it may be that there is a compelling argument to violating the normal definition of standard_name for the case of ISO date-time strings. Or on the other hand is it preferable to use the units attribute to indicate the use of an ISO date-time string? That is the thought question. - Steve -Aleksandar ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
I am heavily in favor of the units attribute being the way that ISO time strings are identified. Jim Biard Research Scholar Cooperative Institute for Climate and Satellites Remote Sensing and Applications Division National Climatic Data Center 151 Patton Ave, Asheville, NC 28801-5001 jim.bi...@noaa.gov 828-271-4900 On Mar 27, 2013, at 11:05 AM, Steve Hankin steven.c.han...@noaa.gov wrote: On 3/26/2013 7:20 PM, Aleksandar Jelenak - NOAA Affiliate wrote: Hi Steve, On Tue, Mar 19, 2013 at 6:19 PM, Steve Hankin steven.c.han...@noaa.gov wrote: Hi Aleksander, A question to debate in your trac ticket. Per the CF documentation, the definition of the standard_name is The name used to identify the physical quantity I found five standard names for variables with string values. Their units are either not specified, or declared as 1 or string. Hi Aleksander, I think we're talking about different issues. The thought question I posed was not whether it is acceptable to have a standard_name assigned to string variable. Nothing wrong with a string variable. Rather it was to point out that ISO date-time strings are a way of encoding the physical quantity that we know as TIME. So TIME is the right standard_name for ISO date-time strings per the definition quoted above. Now, it may be that there is a compelling argument to violating the normal definition of standard_name for the case of ISO date-time strings. Or on the other hand is it preferable to use the units attribute to indicate the use of an ISO date-time string? That is the thought question. - Steve -Aleksandar ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
On Wed, Mar 27, 2013 at 8:05 AM, Steve Hankin steven.c.han...@noaa.gov wrote: ISO date-time strings are a way of encoding the physical quantity that we know as TIME. So TIME is the right standard_name for ISO date-time strings per the definition quoted above. Now, it may be that there is a compelling argument to violating the normal definition of standard_name for the case of ISO date-time strings. Or on the other hand is it preferable to use the units attribute to indicate the use of an ISO date-time string? An ISO string for a datetime is not a name (it's still time), but it is not a unit either. What it is is a data type -- more akin to a float or integer -- i.e. a particular way to translate bytes to a value. The bytes are a char array, and the value is the datetime itself. I don't know if thinking about it this way is helpful, as we are building on netcdf, and I don't now that netcdf allows you to define new data types, but food for thought. Also, of course, all the other data types in netcdf (and CF) are direct translations to commonly used binary formats in computers, and this one is not. hmm -- a quick peak at the netcdf4 docs says: The richer enhanced model supports user-defined types and data structures So maybe this could be a user defined type? Having said that, I don't support using ISO strings to define datetimes in CF. I understand particular use-cases, like keeping the original time stamp from a data collection system and the like, but then maybe it's really just arbitrary auxiliary text information, in which case maybe we don't need a standard name or custom data types at all. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
On 3/27/2013 8:56 AM, Chris Barker - NOAA Federal wrote: On Wed, Mar 27, 2013 at 8:05 AM, Steve Hankin steven.c.han...@noaa.gov wrote: ISO date-time strings are a way of encoding the physical quantity that we know as TIME. So TIME is the right standard_name for ISO date-time strings per the definition quoted above. Now, it may be that there is a compelling argument to violating the normal definition of standard_name for the case of ISO date-time strings. Or on the other hand is it preferable to use the units attribute to indicate the use of an ISO date-time string? An ISO string for a datetime is not a name (it's still time), but it is not a unit either. What it is is a data type -- more akin to a float or integer -- i.e. a particular way to translate bytes to a value. The bytes are a char array, and the value is the datetime itself. I don't know if thinking about it this way is helpful, as we are building on netcdf, and I don't now that netcdf allows you to define new data types, but food for thought. Hi Chris, Spot on. It is indeed food for thought. Here's an analogy for what is being proposed. Suppose we proposed that CF should permit longitudes to be encoded as a special string type that displayed degrees, minutes and seconds in a friendly human-readable way. For example, 130Edeg 22min 15sec This encoding shares the essential characteristics of proposed ISO date-time strings 1. it has no definable units 2. it is not computable unless a library is found that can handle it 3. it adds no encoding capabilities to what CF already has How would we evaluate this proposal? What alternatives might we suggest for the use cases in which longitudes were required to be in this encoding? - Steve Also, of course, all the other data types in netcdf (and CF) are direct translations to commonly used binary formats in computers, and this one is not. hmm -- a quick peak at the netcdf4 docs says: The richer enhanced model supports user-defined types and data structures So maybe this could be a user defined type? Having said that, I don't support using ISO strings to define datetimes in CF. I understand particular use-cases, like keeping the original time stamp from a data collection system and the like, but then maybe it's really just arbitrary auxiliary text information, in which case maybe we don't need a standard name or custom data types at all. -Chris ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
Hello Steve, On Tue, Mar 19, 2013 at 6:19 PM, Steve Hankin steven.c.han...@noaa.gov wrote: A question to debate in your trac ticket. Per the CF documentation, the definition of the standard_name is The name used to identify the physical quantity (http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html#standard-name). It is the 'units' string (together with the netCDF data type) that describe how that physical quantity is represented as a value in the binary file. Thanks for pointing this out. I find standard names so useful that I overlooked this aspect of the convention. Why only physical quantities? It does seem a bit outdated to me now that software is increasingly doing examination of netCDF file content. A standard_name for the physical quantity time already exists, of course -- it is time. The way it is used in the CF convention duration would be a more appropriate term for me. Just saying... It is listed as such in the CF standards names table. So should datetime_iso8601 be a proposal for a new and recognized value of the time 'units' attribute, rather than a new standard_name? This isn't so clear. To start answering it ... What is the reason that standard names exist? Arguably they are to guide generic software (including metadata search engines) in their ability to locate physical quantities in a CF file. In that case it is standard_name =time that should guide the software to the variable; and units=datetime_iso8601 that will instruct the software on how to interpret the values it finds there. Yes, it makes sense to me but datetime_iso8601 as units value does not seem to be a winning proposal on this list. -Aleksandar ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
