Re: [CF-metadata] Date/time and leap seconds
Maik, I'm far from an expert on this, but doesn't this depend entirely on how you choose to count your seconds? If you declare your time as 'seconds since epoch', then it seems to me that it would be perfectly fine (and perhaps most appropriate) to use exact seconds as defined by International Atomic Time (TAI) since the epoch you define. I don't see where using a UTC date/time string (along the lines of ISO8601) to declare the epoch means you must use time-since-epoch values that include leap seconds. The problem that you point up in your netcdf4-python tracker issue is a problem with conversions between date/time structures (sets of years, days, hours, minutes, and seconds) and time-since-epoch values that are provided by a netCDF python module. CF makes no assumptions about any of that. In terms of CF, I think the question really comes down to an issue of how people encode and decode time variables. If you create your time-since-epoch values by subtracting two POSIX time values, then leap second effects will be included. If you subtract two TAI time values to create your time-since-epoch values, then leap seconds effects won't be included. If someone decodes either of these into date/time structures with a non-matching assumption about the type of time-since-epoch values used, then a problem can arise. It seems to me that the CF convention here is encoding agnostic. I see where the potential for mismatches when using time values with unexpected encoding means that we should provide a way to declare what kind of time-since-epoch values are used in a particular time variable. Grace and peace, Jim On 7/16/14, 4:06 AM, Maik Riechert wrote: Hi, after reading http://cdf.gsfc.nasa.gov/html/leapseconds_requirements.html I'm a bit confused on how CF handles times for the netcdf world. It seems as if the 'seconds since' convention is basically POSIX time in regards to how leap seconds are handled. Quoting wiki Due to its handling of leap seconds, it is neither a linear representation of time nor a true representation of UTC. This means that any dataset with = 1 second resolution which covers the moment a leap second happens will have a problem encoding the time using the existing units convention. I first posted this issue on the netcdf4-python tracker: https://github.com/Unidata/netcdf4-python/issues/280 Is this an issue which was discussed before and is there any solution? Cheers Maik ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- CICS-NC http://www.cicsnc.org/ Visit us on Facebook http://www.facebook.com/cicsnc *Jim Biard* *Research Scholar* Cooperative Institute for Climate and Satellites NC http://cicsnc.org/ North Carolina State University http://ncsu.edu/ NOAA's National Climatic Data Center http://ncdc.noaa.gov/ 151 Patton Ave, Asheville, NC 28801 e: jbi...@cicsnc.org o: +1 828 271 4900 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Date/time and leap seconds
Hi Jim I see where the potential for mismatches when using time values with unexpected encoding means that we should provide a way to declare what kind of time-since-epoch values are used in a particular time variable. And that's exactly the problem. The CDF website summarizes it quite nicely: Science data should have well-defined times to enable more accurate cross-comparison between missions and as documentation for future archive use. netcdf-CF is basically CDF before introduction of the CDF_TIME_TT2000 type. In the end, people use libraries to read datasets, and I'm pretty sure that the netcdf4 library in Python is used a lot, and this one assumes a certain time encoding, without actually having it specified within the CF conventions. And I find this all a bit worrying, but many people probably don't care as their time resolution is higher than 1 second. Still, I think this issue should be addressed, better sooner than later. Don't you think? Cheers Maik Grace and peace, Jim On 7/16/14, 4:06 AM, Maik Riechert wrote: Hi, after reading http://cdf.gsfc.nasa.gov/html/leapseconds_requirements.html I'm a bit confused on how CF handles times for the netcdf world. It seems as if the 'seconds since' convention is basically POSIX time in regards to how leap seconds are handled. Quoting wiki Due to its handling of leap seconds, it is neither a linear representation of time nor a true representation of UTC. This means that any dataset with = 1 second resolution which covers the moment a leap second happens will have a problem encoding the time using the existing units convention. I first posted this issue on the netcdf4-python tracker: https://github.com/Unidata/netcdf4-python/issues/280 Is this an issue which was discussed before and is there any solution? Cheers Maik ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Date/time and leap seconds
Hi Maik: Unfortunately, CF references the udunits package which is no longer being developed, at least for date/time coordinates. udunits does not support leap seconds. your best bet is to add an auxiliary time coordinate which uses leap seconds, eg TAI. your specialized software can use that also add a time coordinate which uses the standard non-leap seconds from udunits. visualization software will use this to display calendar dates, i think if you are careful this will only be wrong on a leap second. You could propose adding a leap second-aware calendar to CF. On Wed, Jul 16, 2014 at 2:48 PM, Maik Riechert maik.riech...@arcor.de wrote: Hi Jim I see where the potential for mismatches when using time values with unexpected encoding means that we should provide a way to declare what kind of time-since-epoch values are used in a particular time variable. And that's exactly the problem. The CDF website summarizes it quite nicely: Science data should have well-defined times to enable more accurate cross-comparison between missions and as documentation for future archive use. netcdf-CF is basically CDF before introduction of the CDF_TIME_TT2000 type. In the end, people use libraries to read datasets, and I'm pretty sure that the netcdf4 library in Python is used a lot, and this one assumes a certain time encoding, without actually having it specified within the CF conventions. And I find this all a bit worrying, but many people probably don't care as their time resolution is higher than 1 second. Still, I think this issue should be addressed, better sooner than later. Don't you think? Cheers Maik Grace and peace, Jim On 7/16/14, 4:06 AM, Maik Riechert wrote: Hi, after reading http://cdf.gsfc.nasa.gov/html/leapseconds_requirements.html I'm a bit confused on how CF handles times for the netcdf world. It seems as if the 'seconds since' convention is basically POSIX time in regards to how leap seconds are handled. Quoting wiki Due to its handling of leap seconds, it is neither a linear representation of time nor a true representation of UTC. This means that any dataset with = 1 second resolution which covers the moment a leap second happens will have a problem encoding the time using the existing units convention. I first posted this issue on the netcdf4-python tracker: https://github.com/Unidata/netcdf4-python/issues/280 Is this an issue which was discussed before and is there any solution? Cheers Maik ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] Date/time and leap seconds
Dear Mark: About 6 weeks ago, I contacted the udunits support team because we needed a unit of measure that was not currently supported, and they agreed to add it. They can be contacted at: support-udun...@unidata.ucar.edu. very respectfully, randy From: John Caron ca...@ucar.edu Sent: Wednesday, July 16, 2014 5:59 PM To: Maik Riechert maik.riech...@arcor.de Cc: cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] Date/time and leap seconds Hi Maik: Unfortunately, CF references the udunits package which is no longer being developed, at least for date/time coordinates. udunits does not support leap seconds. your best bet is to add an auxiliary time coordinate which uses leap seconds, eg TAI. your specialized software can use that also add a time coordinate which uses the standard non-leap seconds from udunits. visualization software will use this to display calendar dates, i think if you are careful this will only be wrong on a leap second. You could propose adding a leap second-aware calendar to CF. On Wed, Jul 16, 2014 at 2:48 PM, Maik Riechert maik.riech...@arcor.de wrote: Hi Jim I see where the potential for mismatches when using time values with unexpected encoding means that we should provide a way to declare what kind of time-since-epoch values are used in a particular time variable. And that's exactly the problem. The CDF website summarizes it quite nicely: Science data should have well-defined times to enable more accurate cross-comparison between missions and as documentation for future archive use. netcdf-CF is basically CDF before introduction of the CDF_TIME_TT2000 type. In the end, people use libraries to read datasets, and I'm pretty sure that the netcdf4 library in Python is used a lot, and this one assumes a certain time encoding, without actually having it specified within the CF conventions. And I find this all a bit worrying, but many people probably don't care as their time resolution is higher than 1 second. Still, I think this issue should be addressed, better sooner than later. Don't you think? Cheers Maik Grace and peace, Jim On 7/16/14, 4:06 AM, Maik Riechert wrote: Hi, after reading http://cdf.gsfc.nasa.gov/html/leapseconds_requirements.html I'm a bit confused on how CF handles times for the netcdf world. It seems as if the 'seconds since' convention is basically POSIX time in regards to how leap seconds are handled. Quoting wiki Due to its handling of leap seconds, it is neither a linear representation of time nor a true representation of UTC. This means that any dataset with = 1 second resolution which covers the moment a leap second happens will have a problem encoding the time using the existing units convention. I first posted this issue on the netcdf4-python tracker: https://github.com/Unidata/netcdf4-python/issues/280 Is this an issue which was discussed before and is there any solution? Cheers Maik ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
