Re: [CF-metadata] New standard names for satellite obs data (time as ISO strings)
Hi Jon, Full ISO8601 does carry time zone expressed in hours relative to UT in the syntax Zx where x is the offset from Zulu at the right-hand end of the string. Cheers, Roy. -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Jon Blower Sent: 21 October 2010 23:28 To: Benno Blumenthal Cc: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] New standard names for satellite obs data (time as ISO strings) Hi Benno, No one I know beyond the age of four thinks Sep 2009 is ambiguous Do you mean beyond the age of precisely 4.000... years or beyond the age of 4.999... years? Or is the ambiguous temporal metadata concept of the age of four sufficient? ;-) All ISO8601 dates (year, month or day resolution) are inherently ambiguous because they carry no time zone information (your precise bounds are likely to be 5 hours different from mine, or something more complex if daylight saving is involved). So with ISO8601 alone I don't think there's such a thing as the preciseSep2009 in your argument below, unless I've misunderstood what you mean, in which case apologies. Hmm... come to think of it, this might actually argue *against* using ISO8601 strings alone as indicators of time resolution. If we really *do* mean that data are representative of a 24-hour period starting at midnight UTC on the first of September, we can't represent this unambiguously as 2009-01-01 because of the time zone problem. (I think that 2009-01-01Z is illegal.) In this case we would be better off representing this period as a nominal value plus explicit bounds, or a nominal value plus time zone plus some additional information that we can discard any precision greater than 1 day. *However*, it's still very useful to know the resolution of the time axis by some means other than inspecting the coordinate bounds. An application (e.g. automatically generating a time selector widget in a GUI) will probably not want to look at all the bounds of all the time coordinates to infer the time resolution: apart from being generally tedious, it would be very difficult for monthly data (because months are different lengths, and vary between calendars). Additionally, this inference would be complicated if floating-point numbers were used to represent time coordinates, since these are usually slightly inaccurate (dangerous to compare floats for equality, etc. etc.) So, I'm starting to like the idea of an additional (and optional) CF attribute to specify time coordinate resolution. This could be specified in precise numeric terms (e.g. instrument precision of 0.12 ms) or in less-precise human calendar terms for certain kinds of data (e.g. P1M for monthly resolution). It would be additional to the coordinate bounds: data providers could specify one or the other, or both if they are consistent (or neither if appropriate?) This would not require or preclude the use of ISO8601 strings to represent time coordinate values. Best wishes, Jon -Original Message- From: bennoblument...@gmail.com [mailto:bennoblument...@gmail.com] On Behalf Of Benno Blumenthal Sent: 21 October 2010 21:44 To: Jon Blower Cc: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] New standard names for satellite obs data (time as ISO strings) Hi Jon, Sorry, I am not buying it. No one I know beyond the age of four thinks Sep 2009 is ambiguous, and I don't read your examples as needing vagueness on the time specifically. Suppose, for a moment, that you succeed beyond your wildest dreams, and it is possible to express in CF some relationship to a vague notion of Sep2009, i.e. data hasADataRelationshipWith vagueSep2009. I would say there is another relationship vagueSep2009 isAVagueVersionOf preciseSep2009 And you could have just as easily coded in CF data hasADataRelationshipWithAVaugeVersionOf preciseSep2009 i.e. there is no reason why the vaugeness cannot be coded as a dependent data property. Which is what CF is currently set up to do, with a possible extension of the cell_methods vocabulary Futhermore, you said You *could* modify CF so that to represent data that are representative of September 2010, you specify a nominal date half-way through September and set the bounds to the first and last instants of September. And perhaps use a new cell_methods of representative. But the half-way point and the bounds would be quite (very) tedious to compute in the general case (months and years are of variable length for example and depend on the calendar system). That is not a modification of CF -- that is the way it is currently encoded in CF (though there is no meaning to the nominal value, so you can set it to whatever). And yes, you have to generate the edges, which you have to do anyway if you are going to sensibly handle computations with the data. And let me repeat my main original point, so that it does not get completely buried
Re: [CF-metadata] New standard names for satellite obs data (timeas ISO strings)
On Fri, Oct 22, 2010 at 6:27 AM, Jon Blower j.d.blo...@reading.ac.uk wrote: Hi Roy, Yes, but *only* if the ISO string contains time information. If it only contains a date (e.g. 2009-09-01) you can't add a time zone as I understand it. Furthermore, there's no default to UTC - in fact the default is local time. This is true, but one can always write out the time in explicit intervals 2009-9-01T00:00Z/24:00Z One does, of course lose all the elegance of the ISO8601 abbreviation, particularly with respect to months, rendering it somewhat pointless. Given that ISO defaults to local time, it behooves CF to specify what that local time is (if CF is where we are). As it currently stands, time zone can be encoded in the units. Also the CF document states Note: if the time zone is omitted the default is UTC, and if both time and time zone are omitted the default is 00:00:00 UTC. If we take that to constrain ISO8601 as well, we have made progress. If we can do such a thing. Benno (And I'm not sure your syntax is right - I think you use +5:00, not Z5. I think Z is only used on its own to indicate Zulu/UTC. Might be wrong though.) Cheers, Jon -Original Message- From: Lowry, Roy K [mailto:r...@bodc.ac.uk] Sent: 22 October 2010 09:06 To: Jon Blower; Benno Blumenthal Cc: cf-metadata@cgd.ucar.edu Subject: RE: [CF-metadata] New standard names for satellite obs data (timeas ISO strings) Hi Jon, Full ISO8601 does carry time zone expressed in hours relative to UT in the syntax Zx where x is the offset from Zulu at the right-hand end of the string. Cheers, Roy. -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Jon Blower Sent: 21 October 2010 23:28 To: Benno Blumenthal Cc: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] New standard names for satellite obs data (time as ISO strings) Hi Benno, No one I know beyond the age of four thinks Sep 2009 is ambiguous Do you mean beyond the age of precisely 4.000... years or beyond the age of 4.999... years? Or is the ambiguous temporal metadata concept of the age of four sufficient? ;-) All ISO8601 dates (year, month or day resolution) are inherently ambiguous because they carry no time zone information (your precise bounds are likely to be 5 hours different from mine, or something more complex if daylight saving is involved). So with ISO8601 alone I don't think there's such a thing as the preciseSep2009 in your argument below, unless I've misunderstood what you mean, in which case apologies. Hmm... come to think of it, this might actually argue *against* using ISO8601 strings alone as indicators of time resolution. If we really *do* mean that data are representative of a 24-hour period starting at midnight UTC on the first of September, we can't represent this unambiguously as 2009-01-01 because of the time zone problem. (I think that 2009-01-01Z is illegal.) In this case we would be better off representing this period as a nominal value plus explicit bounds, or a nominal value plus time zone plus some additional information that we can discard any precision greater than 1 day. *However*, it's still very useful to know the resolution of the time axis by some means other than inspecting the coordinate bounds. An application (e.g. automatically generating a time selector widget in a GUI) will probably not want to look at all the bounds of all the time coordinates to infer the time resolution: apart from being generally tedious, it would be very difficult for monthly data (because months are different lengths, and vary between calendars). Additionally, this inference would be complicated if floating-point numbers were used to represent time coordinates, since these are usually slightly inaccurate (dangerous to compare floats for equality, etc. etc.) So, I'm starting to like the idea of an additional (and optional) CF attribute to specify time coordinate resolution. This could be specified in precise numeric terms (e.g. instrument precision of 0.12 ms) or in less-precise human calendar terms for certain kinds of data (e.g. P1M for monthly resolution). It would be additional to the coordinate bounds: data providers could specify one or the other, or both if they are consistent (or neither if appropriate?) This would not require or preclude the use of ISO8601 strings to represent time coordinate values. Best wishes, Jon -Original Message- From: bennoblument...@gmail.com [mailto:bennoblument...@gmail.com] On Behalf Of Benno Blumenthal Sent: 21 October 2010 21:44 To: Jon Blower Cc: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] New standard names for satellite obs data (time as ISO strings) Hi Jon, Sorry, I am not buying it. No one I know beyond the age of four thinks Sep 2009 is ambiguous, and I don't read your examples as needing
Re: [CF-metadata] Interpretation of unspecified time zone (was: New standard names for satellite obs data (timeas ISO strings))
Hi again, I must admit that your data is very different to the types of data I work with and I don't fully understand issues with composite images. Typical data for me are time series where a series of parameters, including position are logged every 30 seconds or so as a ship sails along. Years ago I had a dataset where a form of local time (ship's time) was used and they switched from daylight saving part way through the cruise. Result was that according to the data the ship was in two different places at a given time: not good. If a ship sails across the Atlantic, we don't want the time channel jumping around as time zone boundaries are crossed. We have a lot of this type of data assembled over the past 30 years - not just ships but also moored instruments - with elapsed time channels that are assumed to be UT and have put much effort in the past into the conversion of any incoming local times in both data and metadata to UT. To suddenly have this interpreted as local wouldn't im prove the accuracy of anything! Making the default CF time local might make the standard appear more robust for your data world but not for mine. Alignment with ISO standards is an essential requirement for the development of new conventions, but I don't feel retrospective adoption to be a prudent path. Cheers, Roy. From: cf-metadata-boun...@cgd.ucar.edu [cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Julia Collins [colli...@nsidc.org] Sent: 22 October 2010 17:32 To: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] Interpretation of unspecified time zone (was: New standard names for satellite obs data (timeas ISO strings)) Hi, On Fri, 22 Oct 2010, Lowry, Roy K wrote: I wonder how many existing CF data files would have the meaning of their time channel changed were this suggestion to be adopted. I am sure many. In some cases it might even make them accurate. :-) At some level, it's a tradeoff between breaking existing data descriptions and creating a more robust standard. Whether my suggestion creates a more robust standard or not is up for debate. It does align it more closely with the ISO standard, and I think there's something to be said for that. If I were Julia I would be reworking my data so that the time channel was true UT. I've had so many problems in the past with local time co-ordinates It's not data I'm generating, but rather data provided by a PI who has determined that this is how he wants to present his data. I believe the idea is to be able to view a wide geographic area over the same effective local time. It's a composite image. If you can tell me how to represent these multiple UTC times for one variable using the existing CF conventions, then I'll try to do so. I believe CF limits me to specifying one time zone for a particular time dimension. Re-working the data so that it's broken up into separate time zones would, as I understand it, mean splitting up the data variable into multiple variables, each with their own time dimension, which is not what the data provider wants to do. Splitting the variable would require the user to put the pieces back together in order to see the image as the data provider intended. As far as I can tell, following the ISO interpretation of the time zone indicator does exactly what I need without ambiguity. I'd like to describe the data in a way that's perfectly compliant with the CF convention, but in this case I don't see a way to do so. Thanks, Julia -- Julia Collins National Snow and Ice Data Center http://nsidc.org/ colli...@nsidc.org +1 303.492.6405 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] time as ISO strings
I agree completely. Using ISO date strings as an encoding for time coordinates sounds terrible to me, doubly so if you consider that they would have to be not-quite-ISO in order to accommodate non-Gregorian calendars. That said, I also agree that it would be great if netcdf tools all knew how to translate numeric time coordinates into ISO strings. (And translate from string back to number, I suppose, but for me, human-readable output is the important thing.) At the start of this thread, I would have argued that it would be nice to have an official spec for including a corresponding set of date strings as ancillary/convenience metadata, but at this point, I think that best practice can be summarized as simply match what 'ncdump -t' does. (With perhaps the addendum that people are likely to infer precision based on which fields are present, so format accordingly.) Cheers, --Seth On Thu, 21 Oct 2010 21:06:23 -0400 Steve Hankin steven.c.han...@noaa.gov wrote: Hi Jon, Benno, John and other pals, Since this email thread already contains an element of informal voting I'll cast my ballot: CF is a better standard *WITHOUT *admitting ISO date strings as an encoding for time coordinates.My opinion is is based upon this outlook: * '/To create quality software, the ability to say no is usually far more important than the ability to say yes./' (http://queue.acm.org/detail.cfm?id=1142044) Bloat and run away complexity are a continual threat to the quality of a standard as it evolves. I'd argue that the measure of whether a new feature deserves inclusion should be whether it adds useful new functionality and does so in a manner that is clean -- preserving the consistency and simplicity of the standard to the degree feasible. * Introducing ISO strings as coordinates adds no encoding power to CF. It raises issues of precision but then fails to address them adequately. (In fact it imports a host of ambiguities about interpretation of precision as Jon pointed out to us in the metadata versus positioning debate.) * It fails the test of consistency, since it is not applicable to virtually identical precision issues that exist for latitude, longitude and vertical coordinates. * the need to support two separate encodings (ISO and days since xxx) for the same date/time coordinate information would force additional complexity into the standards documentation and into clients that would hope to be generic CF applications. It wound degrade interoperability for clients that did not support both encodings * it fails to address the multiple-calendar needs of CF. (creating another inconsistency) (ISO 8601 does not standardize the interpretation of dates prior to the 1582 Julian-Gregorian discontinuity ... which likely means that library codes cannot be trusted to give consistent results.) * it does not even add a useful measure of convenience. The -t option to ncdump already provides the needed convenience for file readers. And for file creators, a new utility that would translate ISO strings into CF time step values could probably be written in less time than this email dialog has occupied. (If this statement appears exaggerated consider that the effort to develop this utility would be paid over and over in the clients that would have to interpret this encoding.) None of this is a comment on the utility of ISO date/time strings as metadata. There are appropriate uses of ISO date/time strings in CF as non-coordinate variables and attributes. The NO vote is in regard to their use as CF coordinates. - Steve === On 10/21/2010 11:57 AM, Jon Blower wrote: Hi Benno, 2010-09 is not necessarily a precise specification of a month - time zones make it a little fuzzy for one thing. Separate to this, there are parallel conversations going on in the ISO/OGC community about what time strings actually mean. A metadata person might say that 2010-09 is simply a shorthand for the fuzzy concept of September 2010 and does not represent a precise interval (i.e. a square-wave function that is 1 during September and 0 outside). Apart from the time zone issue which blurs the boundaries, this square-wave is simply not what humans mean when, for example, they tag a report as having been written in September 2010. It just distinguishes it from version 2 of the report, which was written in November. In this context, it's just a label with some temporal meaning. These metadata guys are in discussion with the positioning guys who view date/times as precisely-defined positions within a temporal CRS. You may (or may not!) like to look at the GeoAPI mailing list, in which we are trying to figure out whether we can actually use the same Java types for both of these subtly-different views of date/times (we hope we can but
Re: [CF-metadata] time as ISO strings
Hello- I don't think I have any standing to vote on CF matters, but I just wanted to say that I quite agree with Steve Hankin's viewpoint. I personally like ISO8601 date/time strings for text display, but agree that under-the-hood encodings such as CF days since T or Unix seconds since the Epoch are ideal. When CF says days since T I feel that T should be expressed in ISO8601 (it often is; I don't know if that's a requirement). Regarding the precision or accuracy of time, perhaps you already know that ISO8601 provides a means of expressing a time interval. This could be the syntax used to express temporal uncertainty in metadata. The syntax is PyYmMdDThHmMsS where: P stands for 'period'; T separates date from time components (if any); Y, M, D, H, M, S are suffixes meaning Years, Months, c (the second M(inutes) can only occur after T so it is distinguishable from M[onths]); y, m, d, h, m, are integers; s is integer or real; unneeded components can be omitted. Example: P7DT6H30M means an interval of 7 days, 6 hours, 30 minutes. -Jeff DLB Steve Hankin wrote: Since this email thread already contains an element of informal voting I'll cast my ballot: CF is a better standard *WITHOUT *admitting ISO date strings as an encoding for time coordinates. [...] None of this is a comment on the utility of ISO date/time strings as metadata. There are appropriate uses of ISO date/time strings in CF as non-coordinate variables and attributes. The NO vote is in regard to their use as CF coordinates. -- Jeff de La Beaujardière, PhD U.S. National Oceanic and Atmospheric Administration (NOAA) Sr Systems Architect, Integrated Ocean Observing System (IOOS) Program Office 1100 Wayne Ave #1225, Silver Spring MD 20910 USA +1 301 427 2427 jeff.delabeaujardi...@noaa.gov ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
