Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 2019-01-16 1:35, Steve Allen wrote: Does it know that rubber seconds do not apply to timestamps in central Europe made using DCF77 from 1970-01-01 to 1972-01-01? All my DLL knows is what's in the UTC file. If the file indicates no rate offset in the years 1970-71 and some fractional second step adjustments, that's what you get. An existing file could be edited to match the DCF77 flavor of UTC. Then construct a UtcTable object from the file: SofaJpl.UtcTable dcf77Table = new SofaJpl.UtcTable(filename); Construct a Utc object with that table as a parameter. SofaJpl.Utc utc = new SofaJpl.Utc(year, month, day, fracday, dcf77Table); Variable "utc" will convert itself to TT based on the custom table. Then, to complete a round trip, create another Utc object from that TT and the custom table. When formatted as date/time you get the original UTC. ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] Running on TAI
Martin Burnicki wrote: > Just a few general thoughts based on an internal note... And some very good points they were. > ...The same applies to the time_t type, IMO. If you let the system kernel > run on TAI or whatever then the time() function still returns a time_t > data type, but its value represents TAI or whatever has been configured. Right... but you may or may not be skating on thin ice! I think the question of "Must time_t represent UTC?" can be a pretty illuminating one, and it cuts across several of the fundamental issues here. Strictly speaking, the Posix definition of time_t is, as Warner Losh has argued, very specifically UTC. (And of course it's also a very specifically non-UTC form of UTC.) But it's equally true, as Gary Miller and Martin have argued, that in practice people use time_t for all sorts of other things... but I think we can also agree that (some of the time, at least) this can lead to additional real confusion, precisely because it breaks the spec, precisely because doing so may clash with someone else's spec-based assumptions. But the real tragedy is that UTC is the timescale that time_t is the *worst* at representing, precisely because time_t is a uniform count of seconds since an epoch, and UTC is nonuniform. Anyone who tries to "fix" some time_t-using software to do UTC "properly" is doomed to failure. At best they'll end up reinventing TAI. To my mind, a pretty good litmus test for how good an alleged scheme for properly handling leap seconds actually is is to look at its data structure. Can it represent :60 explicitly and unambiguously? If not, it's probably a kludge that's never going to work perfectly. (But, indeed, the near universality of time_t and related representations is one of the factors that makes "solving" the leap second problem almost impossible, and will likely help lead to the abandonment of leap seconds.) This message will get too long if I start discussing the data structures that *can* represent :60 explicitly and unambiguously. Some of them are: * struct timespec (using the nonnormalized trick described under "Additional clocks" at https://www.cl.cam.ac.uk/~mgk25/posix-clocks.html) * struct tm * the pair (days-since-epoch, seconds-within-day) * a time_t *plus* a well-defined bit disambiguating the two instances of a repeated second across a leap second ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 1/16/19 12:31 AM, Poul-Henning Kamp wrote: In message <20190115205243.gb25...@ucolick.org>, Steve Allen writes: On Tue 2019-01-15T12:34:11-0800 Gary E. Miller hath writ: Yes, and no. time_t is just seconds since an epoch. Which epoch is not well defined. The epoch may well be anything. See "man difftime". That evokes a challenge for all time nuts that I can make based on reading Bulletin Horaire. What is the epoch that was used for TAI? Isn't that the same one Loran-C used ? 1958-01-01 00:00:00 GMT ? You might be thinking of CCSDS Unsegmented Time - in the header of the time message you specify what the epoch is - It's either the 1958 (beginning of "space age") or "user defined" Similar for CCSDS Day Segmented Code which is 16 or 24 bits of days after epoch, milliseconds, and (optional) submilliseconds They also clone the ISO UTC called ASCII Calendar Segmented Time Code - -MM-DDThh:mm:ss.ddZ as well as DOY versions, etc. They manage the leapsecond with a 58 or 61 second minute. It's also clear that CCSDS doesn't like non-continuous, non-monotonic time scales: " f) Standardization on the use of these time code formats for purposes OTHER than identifying an instant of calendar or time in UTC (e.g., unconventional use as a counter or tool for measuring arbitrary intervals) is not recommended. It is felt such a specialized application can best be viewed not as a time code format but rather as an engineering measurement format. Any such application of these time code formats is considered beyond the scope of this Recommended Standard. " https://public.ccsds.org/Pubs/301x0b4e1.pdf "The CCSDS-Recommended epoch is that of 1958 January 1 (TAI) and the recommended time unit is the second, using TAI as reference time scale, for use as a level 1 time code. This time code is not UTC-based and leap-second corrections do not apply." Bit1-3 =Time code identification 001 — 1958 January 1 epoch (Level 1 Time Code) 010 — Agency-defined epoch (Level 2 Time Code) Level 1 in CCSDS means "complete and unambiguous interpretation", Level 2 means "you need some other information" ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 1/15/19 11:36 PM, Warner Losh wrote: > It's a fundamental flaw of UTC. There are other ways to implement > something akin to UTC that implements some form of mean solar time > that's not an observational calendar. If either part of this is consensus, then a proposal to redefine UTC after a certain date is also fundamentally flawed. Redefinition cannot simplify a standard - any standard - it can only make it more complex, e.g.: "before the specified date, the standard behaves one way, after it behaves another way". Timekeeping has many retroactive use cases, including all those intervals people mention. What about instances that cross the redefinition boundary and the two end points have two different meanings? The engineering requirements to support the original definition never go away. The vast majority of timestamps trace back to GPS and other GNSS-derived values, even if GPS is deemed an unclean standard by the lab-coated acolytes of the atomic clocks (https://bit.ly/2MeCcGp). Or TAI itself is already available - indeed, UTC provides TAI as well as a prediction of UT1. Or simply define a new unsegmented TI timescale as was the consensus at the Torino meeting in 2003. Redefining UTC would be a colossal blunder. The time wasted over the past two decades on this fool's errand could have been better spent standing up a new TI infrastructure, or simply writing better documentation. That said, nothing about the current discussion bears on the definition of UTC, only on how best to handle conversions. Rob ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] Running on TAI
Hi all, Rob Seaman wrote: > Our Meinberg ref clocks can deliver GPS, UTC, or TAI (and thus other > derived timescales). Yes, and the same data formats are used for each of these time scales. Just a few general thoughts based on an internal note, instead of replies to individual posts here on the list: IMO you have to generally distinguish between specific data formats, and the type of information a particular data type can contains. For example, the 32:32 bit timestamp type used in NTP packets has been specified to provide UTC. But of course, if you configure your timeserver for a different time scale than UTC then it just transports "a time" according to the configured scale. I'm sure that even the UT1 NTP server operated by NIST puts a UTC timestamp into those fields. The same applies to the time_t type, IMO. If you let the system kernel run on TAI or whatever then the time() function still returns a time_t data type, but its value represents TAI or whatever has been configured. Even if you have separate API calls that either return TAI, UTC, or whatever from the kernel you can use the same data format for the returned timestamps. In fact, clock_gettime() always returns the same data type (struct timespec, which includes a time_t field), but the type of information differs depending depending on the specified clock ID, and may be a UTC time, a number of seconds and nanoseconds since boot, or whatever. IMO the basic problem is in many cases that some extended information is missing. For example, if you receive an NTP packet you are basically unable to distinguish if the time in the packet is really UTC, or something else. However, if you know that the packet originates from a UT1 server you can assume that it transports UT1 rather than UTC. Similar with time_t. If you read timestamps in a fast loop across an inserted leap second you can only distinguish times if you have a leap second status flag associated with the time, so when kernels simply step the time back to insert a leap second you don't know if the "UTC midnight" timestamp is represents the original second, or the repeated second, which is in fact the leap second. This is similar to the end of DST, where a pure hh:mm:ss time doesn't tell if it belongs to the first hour, or to the repeated hour. You know this only if you can retrieve the DST status with the time in a consistent way, so the first time has DST enabled and the repeated time has it disabled. Even if a DST flag is available (e.g. in struct tm), you don't know from the information from the structure alone if the time the structure contains is UTC, or local a time according to whichever time zone has been configured. Generally it makes a difference whether you simply need the current time which steps when a leap second is applied to UTC, or when DST changes for local time, and continues ticking normally after that event, or if you have to deal with time *intervals*. If your application deals with time *intervals* then it has to be aware that the day is one second longer than usual if a leap second has been inserted, and in local time even one hour longer at the day when DST ends. I think developers more often care about DST steps than leap second steps because DST steps simply occur more often, in shorter intervals. If you need to convert GPS time or TAI to UTC, or vice versa, then of course you need some source of information that tells the current UTC/TAI or UTC/GPS offset. This is also the case if your kernel runs on TAI. There are different possible sources which provide different amount of information. For "current" times the information that can be provided by the NTP or PTP protocol, or from a GPS receiver, is sufficient. However, if you need to deal with historic time intervals then a leap second table is required which explicitly tells at which points in time leap seconds have been inserted. This is just similar to dealing with historic local times, where the TZ DB provides information if and when a DST changeover occurred in the past. Neither GPS, nor PTP or NTP provide this historic information, but e.g. the TZ DB also includes a leap second table, or you can get the leap second file directly from the sources (IERS, NIST, USNO, ...). The problem here is that a leap second table as well as the TZ DB has to be kept up-to-date by some means, since upcoming changes can't be predicted automatically. This requirement is not only for TAI/GPS to UTC conversion. Also if you need to derive e.g. UT1 from TAI, GPS or UTC you have to provide some information (e.g. a data file) how the UT1 offset develops over time, and this information has to be updated in regular intervals. Sorry for the long post which just tries to summerize a couple of things. Martin -- Martin Burnicki Senior Software Engineer MEINBERG Funkuhren GmbH & Co. KG Email: martin.burni...@meinberg.de Phone: +49 5281 9309-414 Linkedin: https://www.linkedin.com/in/martinburnicki/ Lange Wand 9,
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Wed 2019-01-16T00:00:55-0800 Paul Hirose hath writ: > The "rubber second" era is supported. That accounted for about 90% of the > workload to create the UTC implementation! Does it know that rubber seconds do not apply to timestamps in central Europe made using DCF77 from 1970-01-01 to 1972-01-01? After the CGPM redefined the SI second in terms of cesium the German time service bureau determined that they could not use the old scheme of BIH UT2 with its rubber seconds. They had convinced themselves that German law required them to broadcast SI seconds. So starting 1970-01-01 DCF77 changed to broadcast Stepped Atomic Time (SAT) with second markers that were 1 SI second apart except on occasions when they were 1.2 SI seconds apart in order that the time markers would stay close to UT2. This was one of the issues of concern in the CCIR preparatory meetings that led to the decision to have leap seconds. So they were mostly broadcasting legal SI seconds, but too early in history to evoke comparisons with "mostly harmless" and "mostly dead". -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Wed 2019-01-16T09:03:26+ Poul-Henning Kamp hath writ: > In message <20190116085619.ga23...@ucolick.org>, Steve Allen writes: > >The epoch of TAI, and LORAN, and GPS is 1961-01-01T20:00:00 UT2. > >Or maybe 1961-01-00 UT2. > > > >That is when the atomic time scale was set to a specified value. > > But the specific value they set was not zero in 1961-01-01T20:00:00 UT2, > they set it so it would have been zero at 1958-01-01 00:00:00 GMT Because of the reset at 1961 (and other ingredients in the sausage of time) the difference between UT2 and TAI is not zero at 1958-01-01. It is close to zero, but it is not zero. One of the ingredients is that as of 1958-01-01 USNO was intentionally using a longitude that differed by 0.035 time seconds from the globally consistent value. I also have not read closely enough to say whether those offset values in the series H inception of A3 correspond to the values of UT2 before or after the longitudes of all stations changed on 1962-01-01. The global average for that shift was 1.5 ms. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
In message <20190116085619.ga23...@ucolick.org>, Steve Allen writes: >On Wed 2019-01-16T08:31:19+ Poul-Henning Kamp hath writ: >> In message <20190115205243.gb25...@ucolick.org>, Steve Allen writes: >> >That evokes a challenge for all time nuts that I can make based on >> >reading Bulletin Horaire. >> > >> >What is the epoch that was used for TAI? >> >> Isn't that the same one Loran-C used ? 1958-01-01 00:00:00 GMT ? > >Nope. > >The epoch of TAI, and LORAN, and GPS is 1961-01-01T20:00:00 UT2. >Or maybe 1961-01-00 UT2. > >That is when the atomic time scale was set to a specified value. But the specific value they set was not zero in 1961-01-01T20:00:00 UT2, they set it so it would have been zero at 1958-01-01 00:00:00 GMT -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Wed 2019-01-16T08:31:19+ Poul-Henning Kamp hath writ: > In message <20190115205243.gb25...@ucolick.org>, Steve Allen writes: > >That evokes a challenge for all time nuts that I can make based on > >reading Bulletin Horaire. > > > >What is the epoch that was used for TAI? > > Isn't that the same one Loran-C used ? 1958-01-01 00:00:00 GMT ? Nope. The epoch of TAI, and LORAN, and GPS is 1961-01-01T20:00:00 UT2. Or maybe 1961-01-00 UT2. That is when the atomic time scale was set to a specified value. See a few pieces of the way that sausage was made here https://www.ucolick.org/~sla/temporary/taiepoch/ -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
In message <20190115205243.gb25...@ucolick.org>, Steve Allen writes: >On Tue 2019-01-15T12:34:11-0800 Gary E. Miller hath writ: >> Yes, and no. time_t is just seconds since an epoch. Which epoch >> is not well defined. The epoch may well be anything. See "man difftime". > >That evokes a challenge for all time nuts that I can make based on >reading Bulletin Horaire. > >What is the epoch that was used for TAI? Isn't that the same one Loran-C used ? 1958-01-01 00:00:00 GMT ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
In message <20190115123411.44a0c...@spidey.rellim.com>, "Gary E. Miller" writes : >> But the underlying time_t cannot represent leap seconds, can it? > >Yes, and no. time_t is just seconds since an epoch. Which epoch >is not well defined. The epoch may well be anything. See "man difftime". I was lucky enough to share breakfast with Dennis Richie at the USENIX ATC in New Orleans in 1998, and we talked a lot about the history and future of /dev and timekeeping. Originally time_t, which wasn't called that yet, was only 16 bits, and the epoch was whenever you booted the system and because the machines lacked memory management hardware, "rollover was usually not a problem". >time_t dates to the first edition of "Unix Programmer's Manual, November >3, 1971". No timezone or other basis is given for time_t. They just >said "system time". The first leap second was in 1972! time_t never >caught up. >In 1973, just after the third edition, time_t was pegged to GMT. > >According to wikipedia, GMT is mean solar time, not UTC. The seconds >are not the same length, and no leap seconds: Wrong. At the time GMT was what people had always called UTC, and there were no difference between them. >Some long time after that the doc changed from GMT to UTC, but the code >did not. time() returns a time_t which pretends to be UTC, but ignores >leap seconds. From a current "man time": And that was simply how AT telephone network decided to handle leapseconds: There were far too many time keeping devices installed throughout the network for anybody to contemplate implementing leap seconds on them. Most of them didn't keep time that well anyway. I have not yet been able to pinpoint the first UNIX computer which had hardware good enough to tell if leap-seconds were mishandled, my best candidate right now is the Cray Y/MP. The first networking of UNIX computers were over serial lines inside individual labs, and the applications couldn't care less about what time it was anyway. Nobody even dreamed about using computers as authoritative clock for a long time. As far as anybody has been able to make out, the first precision time synchronization over computer networks, where leap seconds could even be relevant, is Dave Mills work on FuzzBall and NTP, in the first half of the 1980-ies on NSFnet - and the FuzzBalls didn't run UNIX and only moved packets. Even in 1988 when the UNIX API was rubberstamped into POSIX by IEEE, nobody but Dave Mills really cared about leap-seconds, and even he didn't care much about it. See for instance his last paragraph of RFC958 from 1985. When leap seconds finally appear on the radar, a freak geophysical happenstance means there are no leap seconds for the precise six years where most stuff is connected the internet. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 2019-01-15 10:50, Tom Van Baak wrote: For instance, in Python, if I do a datetime(2016,12,31,0,0,0) + timedelta(hours=30) does it come out as 1/1/2017 6:00:00 or 5:59:59 (it comes out 0600) Here's a C# code fragment which performs a similar computation. To make the demonstration more interesting I arrange for the augmented UTC to fall in the middle of a leap second. The program calls my SofaJpl fundamental astronomy DLL for the Windows .NET Framework. "Sofa" refers to the IAU SOFA (Standards of Fundamental Astronomy) library, and "Jpl" refers to the Jet Propulsion Laboratory planetary ephemerides. But there's no astronomy in this little example. // Open the leap second table. The '@' prefix on the // string causes the C# compiler to not interpret a backslash as an // escape character. SofaJpl.Utc.LoadTableFromFile(@"C:\Users\Stan\Documents\astro\IERS\leapSecTable.txt"); // Construct a Utc object at the initial epoch. SofaJpl.Utc utc1 = new SofaJpl.Utc(2016, 12, 31, 0, 0, 0); // Construct a JulianDate object at the same epoch (TT scale). SofaJpl.JulianDate tt = utc1.TerrestrialTime; // Construct a Duration object with the amount of time to add. SofaJpl.Duration timeIncrement = SofaJpl.Duration.FromSeconds(86400.5); // Add the time to TT. tt += timeIncrement; // Convert the incremented TT to a new Utc object. SofaJpl.Utc utc2 = new SofaJpl.Utc(tt); // The desired time display precision (the unit is 1 / hours). double timePrecision = 36000.0; // .1 s precision // Display the initial and final UTC epochs. Console.WriteLine(utc1.ToTimeFields(timePrecision)); Console.WriteLine(utc2.ToTimeFields(timePrecision)); Output: 2016-12-31T00:00:00.0 2016-12-31T23:59:60.5 The leap second table follows the USNO format with some extensions. 1) For easy updates by hand, the Julian date field is ignored and may be blank. 2) The final line must have the same TAI-UTC as the previous line, but a later date. This line defines the final epoch of the table. An exception occurs if you exceed it. http://maia.usno.navy.mil/ser7/tai-utc.dat The "rubber second" era is supported. That accounted for about 90% of the workload to create the UTC implementation! JulianDate and Duration objects both represent whole days and fractional days in separate double precision variables. Operators are defined if they make sense. For instance, if you subtract one JulianDate from another the result is a Duration. You can add two Durations but addition of two JulianDates is undefined. For convenience, the Duration class includes the fixed offsets TTMinusGps and TTMinusTai. Time conversions with variable offsets such as TTToTdb() are in the JulianDate class. By design, the Utc class has limited facilities. You can create a Utc object from date and time components or a JulianDate in the TT scale. The inverse operations are available too. But as shown in the example, any time math requires explicit conversion to a JulianDate object in the TT scale, then conversion back to a Utc object if the user needs to see the result in UTC. These conversions are lossless since a Utc object internally stores its value as a JulianDate in TT. (The DLL is intended for astronomy, where TT is more useful than TAI.) By not defining addition or subtraction on Utc objects I avoid ambiguity about the exact meaning of these operations in a stepped time scale. A TimeFields represents epoch as year, month, etc. (It's not called "DateTime" because the Windows .NET Framework already has such a class.) The main purpose of the class is to format JulianDate and Utc objects for output, rounded to the specified precision. For example, if a JulianDate is a few seconds from the end of the year, and precision is 60 (= one minute), the conversion to a TimeFields rounds up to January 1 00:00 in the next year. It's possible to construct named TimeFields objects and extract the components individually for output. But for simplicity my example creates unnamed objects. By default they "stringize" themselves in ISO 8601 format at the specified precision (.1 second in this case). SofaJpl is CLS compliant and therefore can be called by any CLS language, as shown by the C#, C++/CLI, and Visual Basic demonstration programs at the web page. The DLL was created primarily for personal use on my Windows system, but anyone can download it for free. A full installation is almost exactly 1 megabyte. http://home.earthlink.net/~s543t-24dst/SofaJpl_NET/index.html ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
