Re: [CF-metadata] Date/time and leap seconds

[John Caron]

Hi Jim:

Thanks for explaining some of the details of TAI and UTC time standards.
Obviously Im not much of an expert on them. What I am 99 44/100 % sure of
is that udunits uses 60 seconds per minute no matter what, as does
joda-time, which the netcdf-java library uses. So thats what I mean by "udunits
does not support leap seconds". Im unclear whether that means udunits and
joda-time are not really UTC.

You are right that its all about the "conversion from time/date strings to
counts of seconds and back" . I have advocated that ISO 8601 datetime
strings be an optional way to specify time coordinates in CF, with the
argument that they are unambiguous and easy enough for even rocket
scientists to understand ;) That hasnt been accepted in CF, although
netcdf-java will allow them, and one can always use them
as auxiliary coordinates, in addition to the required udunits "secs since
epoch" coordinate.

I think, like Jonathan, an additional calendar option in CF could be the
way forward here. Java 8 has a new date/time package (based on joda-time)
that will allow for correct UTC leap-seconds. So I think I can get them
into netcdf-java (eventually) if that was needed.

What do you think?

Regards,
John




On Thu, Jul 17, 2014 at 7:14 AM, Jim Biard  wrote:

>  John,
>
> I'm afraid that your response has confused me quite a lot. If you don't
> mind, please clarify what you mean by "udunits does not support leap
> seconds". TAI does not use leap seconds. It is a running count of seconds
> since Jan 1, 1958 00:00:00 UTC. UTC time messes around with leap seconds in
> order to account for the fact that a day is not exactly 864000 seconds
> long, and that the rotation rate varies over time. The leap seconds are
> used to keep time/date "strings" synchronized to the diurnal cycle, much
> the same way that leap days are used to keep date strings synchronized to
> the Vernal Equinox.
>
> I think this problem may have more to do with the conversion from
> time/date strings to counts of seconds and back than anything about the
> counts of seconds themselves.
>
> Grace and peace,
>
> Jim
>
> A true count of seconds since an epoch matches the approach of TAI, not
> UTC.
>
> On 7/16/14, 5:58 PM, John Caron wrote:
>
> Hi Maik:
>
>  Unfortunately, CF references the udunits package which is no longer
> being developed, at least for date/time coordinates.
>
>  udunits does not support leap seconds.
>
>  your best bet is to add an auxiliary time coordinate which uses leap
> seconds, eg TAI. your specialized software can use that
>
>  also add a time coordinate which uses the standard non-leap seconds from
> udunits. visualization software will use this to display calendar dates, i
> think if you are careful this will only be wrong on a leap second.
>
>  You could propose adding a leap second-aware calendar to CF.
>
>
> On Wed, Jul 16, 2014 at 2:48 PM, Maik Riechert 
> wrote:
>
>> Hi Jim
>>
>> > I see where the potential for mismatches when using time values with
>> > "unexpected" encoding means that we should provide a way to declare what
>> > kind of time-since-epoch values are used in a particular time variable.
>>
>>  And that's exactly the problem. The CDF website summarizes it quite
>> nicely: "Science data should have well-defined times to enable more
>> accurate cross-comparison between missions and as documentation for
>> future archive use." netcdf-CF is basically CDF before introduction of
>> the CDF_TIME_TT2000 type.
>>
>> In the end, people use libraries to read datasets, and I'm pretty sure
>> that the netcdf4 library in Python is used a lot, and this one assumes a
>> certain time encoding, without actually having it specified within the
>> CF conventions. And I find this all a bit worrying, but many people
>> probably don't care as their time resolution is higher than 1 second.
>> Still, I think this issue should be addressed, better sooner than later.
>> Don't you think?
>>
>> Cheers
>> Maik
>>
>> >
>> > Grace and peace,
>> >
>> > Jim
>> >
>> > On 7/16/14, 4:06 AM, Maik Riechert wrote:
>> >> Hi,
>> >>
>> >> after reading
>> >> http://cdf.gsfc.nasa.gov/html/leapseconds_requirements.html I'm a bit
>> >> confused on how CF handles times for the netcdf world. It seems as if
>> >> the 'seconds since' convention is basically POSIX time in regards to
>> >> how leap seconds are handled. Quoting wiki " Due to its handling of
>> >> leap seconds, it is neither a linear representation of time nor a true
>> >> representation of UTC". This means that any dataset with <= 1 second
>> >> resolution which covers the moment a leap second happens will have a
>> >> problem encoding the time using the existing units convention.
>> >>
>> >> I first posted this issue on the netcdf4-python tracker:
>> >> https://github.com/Unidata/netcdf4-python/issues/280
>> >>
>> >> Is this an issue which was discussed before and is there any solution?
>> >>
>> >> Cheers
>> >> Maik
>>
>> ___
>> CF-metadata mailin

[CF-metadata] Date/time and leap seconds

[Jonathan Gregory]

Dear all

I think that as far as CF is concerned the issue of leap seconds is a matter of
time encoding i.e. the translation between "unit since reference" and YMDhms.
Therefore if there are applications which critically depend on leap seconds
used in the real-world UTC calendar, I agree with John's suggestion that the
best way to deal with it would be to define a new calendar which includes the
leap seconds, and make the definition of the existing gregorian/standard/
default calendar explicitly exclude them in a way which is consistent with the
udunits treatment (if I have understood this correctly).

Best wishes

Jonathan
___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

Re: [CF-metadata] Date/time and leap seconds

[Jim Biard]

John,

I'm afraid that your response has confused me quite a lot. If you don't 
mind, please clarify what you mean by "udunits does not support leap 
seconds". TAI does not use leap seconds. It is a running count of 
seconds since Jan 1, 1958 00:00:00 UTC. UTC time messes around with leap 
seconds in order to account for the fact that a day is not exactly 
864000 seconds long, and that the rotation rate varies over time. The 
leap seconds are used to keep time/date "strings" synchronized to the 
diurnal cycle, much the same way that leap days are used to keep date 
strings synchronized to the Vernal Equinox.


I think this problem may have more to do with the conversion from 
time/date strings to counts of seconds and back than anything about the 
counts of seconds themselves.


Grace and peace,

Jim

A true count of seconds since an epoch matches the approach of TAI, not UTC.
On 7/16/14, 5:58 PM, John Caron wrote:

Hi Maik:

Unfortunately, CF references the udunits package which is no longer 
being developed, at least for date/time coordinates.


udunits does not support leap seconds.

your best bet is to add an auxiliary time coordinate which uses leap 
seconds, eg TAI. your specialized software can use that


also add a time coordinate which uses the standard non-leap seconds 
from udunits. visualization software will use this to display calendar 
dates, i think if you are careful this will only be wrong on a leap 
second.


You could propose adding a leap second-aware calendar to CF.


On Wed, Jul 16, 2014 at 2:48 PM, Maik Riechert > wrote:


Hi Jim

> I see where the potential for mismatches when using time values with
> "unexpected" encoding means that we should provide a way to
declare what
> kind of time-since-epoch values are used in a particular time
variable.

And that's exactly the problem. The CDF website summarizes it quite
nicely: "Science data should have well-defined times to enable more
accurate cross-comparison between missions and as documentation for
future archive use." netcdf-CF is basically CDF before introduction of
the CDF_TIME_TT2000 type.

In the end, people use libraries to read datasets, and I'm pretty sure
that the netcdf4 library in Python is used a lot, and this one
assumes a
certain time encoding, without actually having it specified within the
CF conventions. And I find this all a bit worrying, but many people
probably don't care as their time resolution is higher than 1 second.
Still, I think this issue should be addressed, better sooner than
later.
Don't you think?

Cheers
Maik

>
> Grace and peace,
>
> Jim
>
> On 7/16/14, 4:06 AM, Maik Riechert wrote:
>> Hi,
>>
>> after reading
>> http://cdf.gsfc.nasa.gov/html/leapseconds_requirements.html I'm
a bit
>> confused on how CF handles times for the netcdf world. It seems
as if
>> the 'seconds since' convention is basically POSIX time in
regards to
>> how leap seconds are handled. Quoting wiki " Due to its handling of
>> leap seconds, it is neither a linear representation of time nor
a true
>> representation of UTC". This means that any dataset with <= 1
second
>> resolution which covers the moment a leap second happens will
have a
>> problem encoding the time using the existing units convention.
>>
>> I first posted this issue on the netcdf4-python tracker:
>> https://github.com/Unidata/netcdf4-python/issues/280
>>
>> Is this an issue which was discussed before and is there any
solution?
>>
>> Cheers
>> Maik

___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu 
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata




--
CICS-NC  Visit us on
Facebook  *Jim Biard*
*Research Scholar*
Cooperative Institute for Climate and Satellites NC 
North Carolina State University 
NOAA's National Climatic Data Center 
151 Patton Ave, Asheville, NC 28801
e: jbi...@cicsnc.org
o: +1 828 271 4900




___
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata