Re: [LEAPSECS] leap seconds in POSIX

[Brooks Harris]

On 2020-02-01 9:39 AM, Brooks Harris wrote:

On 2020-01-30 7:02 AM, Tom Van Baak wrote:

Hal,

I see some good comments; did you get the answer you wanted? My take:

> Does anybody know of a good writeup of how to fix POSIX to know 
about leap seconds

> and/or why POSIX hasn't done anything about it yet?

No write-up. No fix. It's not possible without breaking h/w, f/w, 
s/w, and time_t.




Note this is not a POSIX issue per se. All POSIX did was rubber stamp 
and formalize what various versions of UNIX did at the time. The 
method of linear timekeeping where you pick an origin and count 
regular time intervals was widely used in other systems of the era 
too: from wristwatches to wall clocks, from astronomical time to 
telephone time, from mainframes to minicomputers. They all fail to 
handle leap seconds.


If necessary, for a given application, you may be able to hack your 
way around leap seconds. But it's not POSIX then.


One practical work-around is to recognize that the words UTC or POSIX 
or time_t do not contain an accuracy specification. Thus you can 
weasel your way out and claim "one second or worse" accuracy and 
simply gloss over the existence of leap seconds.


However, this work-around fails if you are required to provide 
sub-second accuracy. Then you're stuck providing true UTC, leap 
seconds and all.


/tvb


I would add a couple observations.

You cannot fit 86401 pegs in 86400 holes.

I've come to recognize POSIX time as essentially the classic Gregorian 
calendar algorithm without leap-second compensation. So the problem is 
not just 'computer time', like POSIX time, but more broadly what civil 
time is traditionally and practically taken to be by most people as 
the calendar on the wall and time-of-day by the clock on the wall, 
with 86400 seconds in each and every day.


UTC mandates introduction of leap-seconds in the YMDhms count sequence 
as  "YMD 23:59:60" (ignoring the unlikely disaster of a negative 
leap-second). POSIX time_t is a zero-based uninterrupted incrementing 
count of seconds *without leap-seconds* since it's so-called "The 
Epoch", "1970-01-01 00:00:00". When represented as YMDhms by means of 
POSIX gmtime() it results in the classic Gregorian calendar YMDhms 
sequence, what POSIX calls "broken down time" as represented in POSIX 
struct tm. The 86401th peg, the leap-second, goes missing. There are 
only 86400 holes in POSIX time.


(As I understand it time_t is deprecated and replaced by struct 
timespec in modern POSIX systems. This does not eliminate the 
leap-second difficulty.)


Nearly all computer systems and applications operate the same way. For 
example classic Windows time uses a "1601-01-01" origin in its struct 
FILETIME and YMDhms representation in its struct SYSTEMTIME. It's 
behavior is similar to POSIX time, that is, it is a representation of 
classic Gregorian calendar without leap-second compensation. [But 
watch out! The latest Windows systems will offer the option to use 
leap-second compensated date and time. Systems so configured will 
stray from classic Windows time and from POSIX time when the next 
leap-second occurs. Fun will commence.]


The leap-second modification of the Gregorian calendar YMDhms counting 
sequence presents an irreconcilable dilemma. I think most appropriate 
term to describe the UTC v.s. computer time and civil time mismatch is 
"incommensurate". You cannot map UTC into Gregorian without losing 
something, and that something is the leap-second.


As Tom points out, the definition of the POSIX origin, "The Epoch", is 
*intentionally* vague to accommodate systems that could not accurately 
align to "1970-01-01 00:00:00". This date and time is said to be "UTC" 
but this is misleading as it is not strictly UTC because A) the count 
skips leap-seconds and B) UTC was not an integral second adjustment 
until 1972-01-01 00:00:10 (TAI) = 1972-01-01 00:00:00 (UTC) (I like to 
call this alignment point UTC1972), so the POSIX "the Epoch",  
1972-01-01 00:00:00 (POSIX), sits in the never-never world before 
UTC1972. As I understand it modern systems try to align "the Epoch" at 
exactly 63072000s (730 86400s days) before UTC1972.



-Brooks
Apologies to Tom. I may have misunderstood or misrepresented his 
meaning. Please delete "As Tom points out," from the above paragraph. 
Sorry Tom.


-Brooks





On 1/27/2020 12:59 AM, Hal Murray wrote:
Does anybody know of a good writeup of how to fix POSIX to know 
about leap

seconds and/or why POSIX hasn't done anything about it yet?

I assume the basic idea would be something like switch the kernel to 
use TAI

rather than UTC and implement conversion in some library routines.


There is a discussion on the IETF ntp list with typical S/N for this 
topic.




___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs




___
LEAPSECS mailing list

Re: [LEAPSECS] leap seconds in POSIX

[Michael Shields]

On Sat, Feb 1, 2020 at 12:01 AM Hal Murray  wrote:
> Has anybody considered having time_t and the kernel keep smeared time?
>
> The idea is that you can convert from smeared time to TAI or UTC with leaps.
> So a few new library routines should be enough for the few people who care
> about leap seconds.

Something like this library? https://github.com/google/unsmear
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Steve Allen]

On Sat 2020-02-01T00:01:22-0800 Hal Murray hath writ:
> I was hoping that there would be a good white paper or blog that discussed all
> the possibilities that have been considered and explained why they were
> rejected.

That cannot happen because of the other factor that has been in the
politics of time since the 1950s: fear

The surviving scientific and technical discussions indicate that two
time scales were considered a plausible option.  It was the regulatory
context of the CCIR where two time scales were deemed unacceptable.
The memoirs of the participants indicate that those discussions
happened at conferences where the principals gathered but which were
not actually part of the bodies who actually exercised authority.
The discussions where the decisions were made are not recorded,
and the discussion of those discussions at IAU 1970 was redacted.

The fear comes from the fact that the broadcasts of time were funded
by national governments.  The urgency to adopt leap seconds came from
the German law that which disenfranchised the German Hydrological
Institute that had been providing old rubber second UTC and declared
that only the PTB with its cesium seconds was able to provide legal
time.  If bureaucrats and lawmakers in other countries had access to
documents which described the dichotomy between SI seconds and
calendar days they might have legislated differently, and that would
destroy decades of efforts to get all radio broadcast time signals to
supply the same time scale.

The fear seen in the redaction of the 1970 IAU proceedings is still
clear early in 1972 just after the inception of leap seconds where
G.M.R.  Winkler of USNO (who was in charge of the 1970 IAU redaction)
explicitly cautioned against open discussion of legal issues.  By 1974
there had been enough recommendations and acceptance of UTC with leaps
that Winkler openly remarked "The C.C.I.R.  may have overstepped its
remit in defining UTC" and then paraphrased Spock "The process that
led to UTC may have been illogical, but it was effective."

The problem for POSIX and any technical implementations follows from
the carefully worded recommendations that were handed to bureaucrats
to get their approval.  The wording allowed the insiders to implement
what they understood to be the only politically acceptable compromise.
Anyone outside the process was thereafter condemned to conform to
specifications for a political compromise that gave no clues about its
underlying technical barrenness.

--
Steve Allen  WGS-84 (GPS)
UCO/Lick Observatory--ISB 260  Natural Sciences II, Room 165  Lat  +36.99855
1156 High Street   Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064   https://www.ucolick.org/~sla/  Hgt +250 m
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Brooks Harris]

On 2020-02-01 3:01 AM, Hal Murray wrote:

t...@leapsecond.com  said:

I see some good comments; did you get the answer you wanted?

Nothing off list, so you have seen everything that I saw.

I was hoping that there would be a good white paper or blog that discussed all
the possibilities that have been considered and explained why they were
rejected.

---

Probably crazy idea dept...

Has anybody considered having time_t and the kernel keep smeared time?

The idea is that you can convert from smeared time to TAI or UTC with leaps.
So a few new library routines should be enough for the few people who care
about leap seconds.


The inconvenient truth is that atomic time is a constant frequency 
phenomenon and mean solar time is a variable frequency phenomenon.


It would be straight forward to define a variable frequency timescale 
that represented mean solar time very closely based on UT1. Each day 
would have 86400 "seconds" in its YMDhms representation. Such a scale's 
frequency would diverge from UTC frequency by only 5 ppb, a precision 
below most systems ability to discern. However, the "seconds" of the 
YMDhms representation would not be SI seconds and would not match the 
UTC YMDhms, placing the midnight day boundary very slightly different 
from the UTC midnights. It would probably satisfy the purposes of 
computer time, civil time, celestial navigation, many astronomers, and 
it would preserve the age old tradition of timekeeping by the Sun.


But suggestion of such a variable frequency scale is anathema to the 
timing community (BIPM, IERS, etc) and unacceptable for GNSS navigation 
and many engineering purposes such as industrial control, including the 
industry I come from, the television business. Timekeeping laws all over 
the world presume adherence to UTC as defined by ITU-R Rec 460. It's not 
so much a technical problem as a political argument, the same debate 
that raged during the 1960s and resulted in the constant frequency 
definition of UTC we have today. There are two parts to the insistence 
on constant frequency dissemination; 1) there can be one and only one 
timescale and 2) this timescale must disseminate constant frequency SI 
seconds. These criteria are politically immovable objects. This leads 
the 'great leap-second debate' ongoing and unresolved since 1999.


Google Smear (and others) work around the practical problem by slewing 
the frequency of their NTP servers across a 24 hour period so the 
receiving systems never 'see' the leap-second.


Leap Smear
https://developers.google.com/time/smear#example_of_the_standard_smear

This works to avoid potential failure due to possible faulty leap-second 
implementations throughout the system but results in ambiguous 
timestamps during the smear. Note they say "The long duration keeps the 
frequency change small. The change for the smear is about 11.6 ppm. This 
is within the manufacturing and thermal errors of most machines' quartz 
oscillators, and well under NTP's 500 ppm maximum slew rate.". Note this 
is 86400 / 86401 = 0.884, 1 - 0.884 = 0.116.


The frequency of a variable frequency timescale based on UT1 would vary 
from UTC by 5 ppb worst case, currently running closer to 1 ppb, ~four 
orders of magnitude lower that Google Smear, and traceable to UTC with a 
precision <10 ns. It would be something very close to what was once 
called GMT.


But, as one colleague has put it, anyone suggesting such a solution 
would be sent home without dinner. So you didn't hear it from me.


-Brooks

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Steve Summit]

Warner Losh wrote:
> But the problem with time_t are legend. I should do a talk that
> lists them all.

Here is, I think, the fundamental one, as I like to describe it.

As Clive Feather observed years ago,

There are three desirable properties for time:
(1) A day is always 86400 seconds long.
(2) A second is a constant length.
(3) 00:00:00 is, on average, astronomical midnight.

It turns out you can have any two simultaneously, but not
all three.  So there are three official time scales:

TAI (1) and (2) are true
UT1 (1) and (3) are true
UTC (2) and (3) are true

But if you think about it, for time_t as defined by Posix, we're
trying to have (1), (2), and (3) all true.  And that just won't
ever work.

You can "solve" the computer leapsecond problem.  But you can't
solve it and keep time_t.  Also you can't get rid of time_t,
because it's far too widespread; *everyone* uses it.  So the only
possible solutions involve awkward compromises involving two,
separate-and-unequal ways of representing time.  And no one wants
that.  (We've got way too many ways of representing time already...)
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Warner Losh]

On Sat, Feb 1, 2020 at 3:39 PM Brooks Harris  wrote:

>
> (As I understand it time_t is deprecated and replaced by struct timespec
> in modern POSIX systems. This does not eliminate the leap-second
> difficulty.)
>

Kinda sorta... the timespec  struct has a time_t inside of it.


> The leap-second modification of the Gregorian calendar YMDhms counting
> sequence presents an irreconcilable dilemma. I think most appropriate
> term to describe the UTC v.s. computer time and civil time mismatch is
> "incommensurate". You cannot map UTC into Gregorian without losing
> something, and that something is the leap-second.
>

The problem is that it has an irregular radix.

:60 is the odd-duck whose quacks are the harbingers of doom :)


> As Tom points out, the definition of the POSIX origin, "The Epoch", is
> *intentionally* vague to accommodate systems that could not accurately
> align to "1970-01-01 00:00:00". This date and time is said to be "UTC"
> but this is misleading as it is not strictly UTC because A) the count
> skips leap-seconds and B) UTC was not an integral second adjustment
> until 1972-01-01 00:00:10 (TAI) = 1972-01-01 00:00:00 (UTC) (I like to
> call this alignment point UTC1972), so the POSIX "the Epoch",
> 1972-01-01 00:00:00 (POSIX), sits in the never-never world before
> UTC1972. As I understand it modern systems try to align "the Epoch" at
> exactly 63072000s (730 86400s days) before UTC1972.
>

That's because every POSIX day  is 86400 seconds long, without exception.
:) :<

POSIX doesn't define the relationship between time_t and anything else,
other than  to prescribe a mapping. Many systems these days have a
specification that like
"Number of actual seconds since UTC1972+63072000 + negative leap seconds -
positive leap seconds." to align the time scales to time_t.

But the problem with time_t are legend. I should do a talk that lists them
all.

The biggest problem is that people think they can solve the timestamp issue
with metadata. But you can't because that metadata necessarily must be
discarded to fit within external protocol requirements.  If I do an  NFS
get file attributes, one attribute is file creation time.  Should this
occur during a leap second, there's no way to encode that metadata in
response to that request. That's even assuming the stable media encodes
things such that it is knowable, or that the OS provides a way to set that
time, or that the OS ticks through the leap second properly. And as brooks
said, you can't fit 86401 pegs into  86400 holes. One hole necessarily must
hold two pegs. Smearing breaks frequency, and accepts an offset to try to
power through the problem, but it in no way solves the problem... it just
accepts certain errors as OK to paper over the whole mess.

And even if all that were right, it assumes you can get a verified,
temper-proof list of leap seconds easily, which isn't always possible. It's
the cross product of all these issues that make leap seconds an impossible
problem to solve in  a way that doesn't discard data of some sort...

Warner


> >
> >
> > On 1/27/2020 12:59 AM, Hal Murray wrote:
> >> Does anybody know of a good writeup of how to fix POSIX to know about
> >> leap
> >> seconds and/or why POSIX hasn't done anything about it yet?
> >>
> >> I assume the basic idea would be something like switch the kernel to
> >> use TAI
> >> rather than UTC and implement conversion in some library routines.
> >>
> >>
> >> There is a discussion on the IETF ntp list with typical S/N for this
> >> topic.
> >>
> >
> > ___
> > LEAPSECS mailing list
> > LEAPSECS@leapsecond.com
> > https://pairlist6.pair.net/mailman/listinfo/leapsecs
> >
> >
>
> ___
> LEAPSECS mailing list
> LEAPSECS@leapsecond.com
> https://pairlist6.pair.net/mailman/listinfo/leapsecs
>
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Brooks Harris]

On 2020-01-30 7:02 AM, Tom Van Baak wrote:

Hal,

I see some good comments; did you get the answer you wanted? My take:

> Does anybody know of a good writeup of how to fix POSIX to know 
about leap seconds

> and/or why POSIX hasn't done anything about it yet?

No write-up. No fix. It's not possible without breaking h/w, f/w, s/w, 
and time_t.




Note this is not a POSIX issue per se. All POSIX did was rubber stamp 
and formalize what various versions of UNIX did at the time. The 
method of linear timekeeping where you pick an origin and count 
regular time intervals was widely used in other systems of the era 
too: from wristwatches to wall clocks, from astronomical time to 
telephone time, from mainframes to minicomputers. They all fail to 
handle leap seconds.


If necessary, for a given application, you may be able to hack your 
way around leap seconds. But it's not POSIX then.


One practical work-around is to recognize that the words UTC or POSIX 
or time_t do not contain an accuracy specification. Thus you can 
weasel your way out and claim "one second or worse" accuracy and 
simply gloss over the existence of leap seconds.


However, this work-around fails if you are required to provide 
sub-second accuracy. Then you're stuck providing true UTC, leap 
seconds and all.


/tvb


I would add a couple observations.

You cannot fit 86401 pegs in 86400 holes.

I've come to recognize POSIX time as essentially the classic Gregorian 
calendar algorithm without leap-second compensation. So the problem is 
not just 'computer time', like POSIX time, but more broadly what civil 
time is traditionally and practically taken to be by most people as the 
calendar on the wall and time-of-day by the clock on the wall, with 
86400 seconds in each and every day.


UTC mandates introduction of leap-seconds in the YMDhms count sequence 
as  "YMD 23:59:60" (ignoring the unlikely disaster of a negative 
leap-second). POSIX time_t is a zero-based uninterrupted incrementing 
count of seconds *without leap-seconds* since it's so-called "The 
Epoch", "1970-01-01 00:00:00". When represented as YMDhms by means of 
POSIX gmtime() it results in the classic Gregorian calendar YMDhms 
sequence, what POSIX calls "broken down time" as represented in POSIX 
struct tm. The 86401th peg, the leap-second, goes missing. There are 
only 86400 holes in POSIX time.


(As I understand it time_t is deprecated and replaced by struct timespec 
in modern POSIX systems. This does not eliminate the leap-second 
difficulty.)


Nearly all computer systems and applications operate the same way. For 
example classic Windows time uses a "1601-01-01" origin in its struct 
FILETIME and YMDhms representation in its struct SYSTEMTIME. It's 
behavior is similar to POSIX time, that is, it is a representation of 
classic Gregorian calendar without leap-second compensation. [But watch 
out! The latest Windows systems will offer the option to use leap-second 
compensated date and time. Systems so configured will stray from classic 
Windows time and from POSIX time when the next leap-second occurs. Fun 
will commence.]


The leap-second modification of the Gregorian calendar YMDhms counting 
sequence presents an irreconcilable dilemma. I think most appropriate 
term to describe the UTC v.s. computer time and civil time mismatch is 
"incommensurate". You cannot map UTC into Gregorian without losing 
something, and that something is the leap-second.


As Tom points out, the definition of the POSIX origin, "The Epoch", is 
*intentionally* vague to accommodate systems that could not accurately 
align to "1970-01-01 00:00:00". This date and time is said to be "UTC" 
but this is misleading as it is not strictly UTC because A) the count 
skips leap-seconds and B) UTC was not an integral second adjustment 
until 1972-01-01 00:00:10 (TAI) = 1972-01-01 00:00:00 (UTC) (I like to 
call this alignment point UTC1972), so the POSIX "the Epoch",  
1972-01-01 00:00:00 (POSIX), sits in the never-never world before 
UTC1972. As I understand it modern systems try to align "the Epoch" at 
exactly 63072000s (730 86400s days) before UTC1972.



-Brooks




On 1/27/2020 12:59 AM, Hal Murray wrote:
Does anybody know of a good writeup of how to fix POSIX to know about 
leap

seconds and/or why POSIX hasn't done anything about it yet?

I assume the basic idea would be something like switch the kernel to 
use TAI

rather than UTC and implement conversion in some library routines.


There is a discussion on the IETF ntp list with typical S/N for this 
topic.




___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs




___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Hal Murray]

t...@leapsecond.com said:
> I see some good comments; did you get the answer you wanted?

Nothing off list, so you have seen everything that I saw.

I was hoping that there would be a good white paper or blog that discussed all 
the possibilities that have been considered and explained why they were 
rejected.

---

Probably crazy idea dept...

Has anybody considered having time_t and the kernel keep smeared time?

The idea is that you can convert from smeared time to TAI or UTC with leaps.  
So a few new library routines should be enough for the few people who care 
about leap seconds.


-- 
These are my opinions.  I hate spam.



___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Tom Van Baak]

Hal,

I see some good comments; did you get the answer you wanted? My take:

> Does anybody know of a good writeup of how to fix POSIX to know about 
leap seconds

> and/or why POSIX hasn't done anything about it yet?

No write-up. No fix. It's not possible without breaking h/w, f/w, s/w, 
and time_t.




Note this is not a POSIX issue per se. All POSIX did was rubber stamp 
and formalize what various versions of UNIX did at the time. The method 
of linear timekeeping where you pick an origin and count regular time 
intervals was widely used in other systems of the era too: from 
wristwatches to wall clocks, from astronomical time to telephone time, 
from mainframes to minicomputers. They all fail to handle leap seconds.


If necessary, for a given application, you may be able to hack your way 
around leap seconds. But it's not POSIX then.


One practical work-around is to recognize that the words UTC or POSIX or 
time_t do not contain an accuracy specification. Thus you can weasel 
your way out and claim "one second or worse" accuracy and simply gloss 
over the existence of leap seconds.


However, this work-around fails if you are required to provide 
sub-second accuracy. Then you're stuck providing true UTC, leap seconds 
and all.


/tvb


On 1/27/2020 12:59 AM, Hal Murray wrote:

Does anybody know of a good writeup of how to fix POSIX to know about leap
seconds and/or why POSIX hasn't done anything about it yet?

I assume the basic idea would be something like switch the kernel to use TAI
rather than UTC and implement conversion in some library routines.


There is a discussion on the IETF ntp list with typical S/N for this topic.



___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Brooks Harris]

On 2020-01-28 12:15 AM, Martin Burnicki wrote:

Brooks Harris wrote:

Evolution of Timekeeping in Windows
https://techcommunity.microsoft.com/t5/networking-blog/evolution-of-timekeeping-in-windows/ba-p/778020

Interesting that they call it "the (r)evolutionary changes to time
keeping in the Windows operating system" when they finally managed to
achieve a precision that other operating systems are already providing
since decades. ;-)

Martin


I hate Windows. However its the operating system I know. I certainly do 
not want to get into OS wars because the fact is we all must deal with 
all of them in one way or another, especially where timekeeping is 
concerned.


But it's a little unfair to say "Windows" cannot do tight timekeeping. 
It depends on the *version* of Windows and administrative set ups. It is 
true that the normal consumer versions default to polling NTP once a 
day, so they are not very accurate. However, it has been possible to set 
up Windows Server for high quality time services since Windows 2000. I 
was involved in an elaborate installation of Windows 2000 and Linux 
machines in the early 2000s and the timekeeping was pretty tight. It was 
not easy to accomplish because it involved all sorts of careful Registry 
tweaks to get it all to behave.


How to configure the Windows Time service against a large time offset
https://support.microsoft.com/en-us/help/884776/how-to-configure-the-windows-time-service-against-a-large-time-offset

Of course this has evolved over the years.

How the Windows Time Service Works
https://docs.microsoft.com/en-us/windows-server/networking/windows-time-service/how-the-windows-time-service-works

Windows Server is a somewhat different animal than 'consumer' and 
'professional' Windows. They do not give the Server versions away and I 
expect we can all agree that those licensing costs are exorbitant. But 
Microsoft is the Borg. Bill Gates has not been returning my calls recently.


--

On the recent leap-seconds support now enabled in Windows:

Leap Seconds for the IT Pro: What you need to know
https://techcommunity.microsoft.com/t5/networking-blog/leap-seconds-for-the-it-pro-what-you-need-to-know/ba-p/339811

Leap Seconds for the AppDev: What you should know
https://techcommunity.microsoft.com/t5/networking-blog/leap-seconds-for-the-appdev-what-you-should-know/ba-p/339813

I think this is going to cause more problems than it solves when the 
next leap-second comes around. Yes, supporting the leap-second directly 
seems a good thing to a point. (Note there are important administrative 
setups involved to enable this support on various Windows versions.) 
However, systems so enabled will produce timestamps that are inaccurate 
with respect to classic Windows and POSIX timestamps. Their objective is 
to claim traceability to UTC, especially in context of Azure cloud 
services, and that it will be. However, after the next leap-second those 
timestamps will be one second off from every other system on the planet 
and incompatible with the vast majority of applications including most 
Microsoft applications, like Office. Will they actually update Office 
timekeeping to match? BackOffice? SQL Server? .NET? What then of legacy 
timestamps and applications? How difficult will it be to compare Windows 
timestamps to Linux timestamps? Or HTML, XML, email, etc, etc. It looks 
to me this is going to cause some serious practical incompatibility 
issues. I'm not seeing how this will make things easier.


-Brooks

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Brooks Harris]

On 2020-01-27 11:52 AM, Steve Allen wrote:

On Mon 2020-01-27T19:33:37+ Tony Finch hath writ:

It looks like we're in another long gap, based on the LOD chart and the
UT1-UTC prediction. The current gap is now the second longest...

In the middle of last year the rotation of the earth accelerated
enough that LOD went below 86400 for several weeks, and DeltaT
decreased significantly.


https://datacenter.iers.org/singlePlot.php?plotname=FinalsDataIAU2000A-UT1-UTC-BULA=10

https://datacenter.iers.org/singlePlot.php?plotname=FinalsDataIAU2000A-LOD-BULA=10

The LOD effects are easier to see using the plots from the Paris
bureau of IERS by requesting to remove the predictable tidal
variations that basically look like noise on the second plot here.
I've been studying DUT1, the 1/10th second "mini leaps", mandated by 
ITU-R Recommendation 460, produced by IERS in Bulletin D 
(ftp://ftp.iers.org/products/eop/bulletind/), and disseminated by the 
time radio broadcasts (WWVB, DCF77, MSF and such). This is another way 
to look at the trends Steve has pointed out.


The Bulletin D publications in the IERS folder go back only to 
1991-06-20. I obtained the full inventory since UTC1972 from an 
authoritative source. I assembled an Excel to examine the distribution 
of intervals (called spans) between of the DUT1 dates.


Worksheet "BulD_Spans" shows the DUT1 span lengths in (UTC) days (Y) 
v.s. the nth DUT1 change (X), and Excel "trend lines". Worksheet 
"UTC_Spans" shows the leap-seconds similarly.


http://edlmax.com/Common_Calendar/BulD_and_UTC_Spans.xls

The trend lines show a continuing increase of the "span" lengths between 
DUT1 changes and leap-seconds.


The most recent Bulletin D DUT1 change, from -0.1s to -0.2s, occurred on 
2019-05-02. Examining UT1 predictions from Bulletin A 
(ftp://ftp.iers.org/products/eop/rapid/bulletina/bulletina-xxxiii-003.txt) 
indicates the next DUT1 change may not happen until 2020-11-10 when 
UT1-UTC crosses -0.25000. This will be a DUT1 span of 558 days. This is 
far longer than the longest earlier DUT1 span of 392 days that occurred 
during the 1999 to 2006 "leap-second drought".


UT1-UTC must reach -0.5s before a leap-second will be announced. If 
these trends hold it could be quite a while, perhaps years, before the 
next leap-second.


(Note that IERS does not use Bulletin A UT1-UTC prediction values for 
determining DUT1 and leap-seconds but rather from EOP C04 
(http://hpiers.obspm.fr/iers/series/opa/eopc04_IAU2000). This is the 
internal data set IERS assembles and is NOT the official 'product'. 
Bulletin A is the official publication. Note Bulletin A gives UT1-UTC to 
5 points of precision while C04 gives 7 points.)


Predicting the future is an inherently risky business.

-Brooks



--
Steve Allen   WGS-84 (GPS)
UCO/Lick Observatory--ISB 260  Natural Sciences II, Room 165  Lat  +36.99855
1156 High Street   Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064https://www.ucolick.org/~sla/   Hgt +250 m
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs




___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Tony Finch]

Steve Allen  wrote:
>
> http://hpiers.obspm.fr/eop-pc/index.php?index=C04=en
>
> ask to remove tidal variations

Super, thanks!

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Mull of Kintyre to Ardnamurchan Point: West or southwest 4 to 6, increasing 7
or gale 8, backing south 4 or 5 later. Rough or very rough, occasionally high
in west. Squally wintry showers, occasionally thundery. Moderate or good,
occasionally poor.
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Martin Burnicki]

Brooks Harris wrote:
> Evolution of Timekeeping in Windows
> https://techcommunity.microsoft.com/t5/networking-blog/evolution-of-timekeeping-in-windows/ba-p/778020

Interesting that they call it "the (r)evolutionary changes to time
keeping in the Windows operating system" when they finally managed to
achieve a precision that other operating systems are already providing
since decades. ;-)

Martin
-- 
Martin Burnicki

Senior Software Engineer

MEINBERG Funkuhren GmbH & Co. KG
Email: martin.burni...@meinberg.de
Phone: +49 5281 9309-414
Linkedin: https://www.linkedin.com/in/martinburnicki/

Lange Wand 9, 31812 Bad Pyrmont, Germany
Amtsgericht Hannover 17HRA 100322
Geschäftsführer/Managing Directors: Günter Meinberg, Werner Meinberg,
Andre Hartmann, Heiko Gerstung
Websites: https://www.meinberg.de  https://www.meinbergglobal.com
Training: https://www.meinberg.academy

___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Steve Allen]

On Mon 2020-01-27T21:17:54+ Tony Finch hath writ:
> > The LOD effects are easier to see using the plots from the Paris
> > bureau of IERS by requesting to remove the predictable tidal
> > variations that basically look like noise on the second plot here.
>
> Do you have a link and/or instructions? I can't see how to get a chart
> like that ...

http://hpiers.obspm.fr/eop-pc/index.php?index=C04=en

ask to remove tidal variations

Be nice, this web server is not robust enough to handle everyone
asking all at once.

--
Steve Allen  WGS-84 (GPS)
UCO/Lick Observatory--ISB 260  Natural Sciences II, Room 165  Lat  +36.99855
1156 High Street   Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064   https://www.ucolick.org/~sla/  Hgt +250 m
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Tony Finch]

Steve Allen  wrote:
>
> The LOD effects are easier to see using the plots from the Paris
> bureau of IERS by requesting to remove the predictable tidal
> variations that basically look like noise on the second plot here.

Do you have a link and/or instructions? I can't see how to get a chart
like that ...

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
fight poverty, oppression, hunger, ignorance, disease, and aggression
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Steve Allen]

On Mon 2020-01-27T19:33:37+ Tony Finch hath writ:
> It looks like we're in another long gap, based on the LOD chart and the
> UT1-UTC prediction. The current gap is now the second longest...

In the middle of last year the rotation of the earth accelerated
enough that LOD went below 86400 for several weeks, and DeltaT
decreased significantly.

> https://datacenter.iers.org/singlePlot.php?plotname=FinalsDataIAU2000A-UT1-UTC-BULA=10
>
> https://datacenter.iers.org/singlePlot.php?plotname=FinalsDataIAU2000A-LOD-BULA=10

The LOD effects are easier to see using the plots from the Paris
bureau of IERS by requesting to remove the predictable tidal
variations that basically look like noise on the second plot here.

--
Steve Allen  WGS-84 (GPS)
UCO/Lick Observatory--ISB 260  Natural Sciences II, Room 165  Lat  +36.99855
1156 High Street   Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064   https://www.ucolick.org/~sla/  Hgt +250 m
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Tony Finch]

Steve Summit  wrote:

> * The leap second drought of 1999-2006 rather nastily coincided
>   with a gradual change in the computing industry from "nobody's
>   clocks are synchronized that well anyway, so a second here or
>   there doesn't matter" to the opposite.  (But by the time we
>   realized we had a leapsecond problem, it was sort of too late
>   to fix it.)

It looks like we're in another long gap, based on the LOD chart and the
UT1-UTC prediction. The current gap is now the second longest...

https://datacenter.iers.org/singlePlot.php?plotname=FinalsDataIAU2000A-UT1-UTC-BULA=10

https://datacenter.iers.org/singlePlot.php?plotname=FinalsDataIAU2000A-LOD-BULA=10

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Viking: South or southwest 5 to 7, becoming variable 3 or 4 later. Moderate or
rough. Squally showers. Good.
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Steve Summit]

Warner Losh wrote:
> You've just scratched the surface of the problem.

Indeed.  (That was the third iteration of the message I sent;
the first two drafts were too long to read, due to delving into
too many details.)

> How do I touch a file on Sat, 31 Dec 2016 18:59:60 -0500 and have
> ls -l report that as its timestamp?

Sorry if this seems like a cop-out, but: you don't.  You declare
that filesystem timestamps aren't necessarily accurate to 1s
or better.  (In fact that's true today, for most filesystems,
although some have finally implemented subsecond timestamps,
and I truly don't know what to do with those.)

I had written:
>> The other basic idea is to implement new, proper-UTC-capable
>> kernel mechanisms and interfaces alongside the time_t ones.

I believe that's the best you can do today, and it ends up
involving a certain number of rather unfortunate compromises.
One is that, as I said, there are two sets of timekeeping
mechanisms and interfaces, the time_t ones and the pure-UTC ones.
Another is that these two are separate-and-unequal (on leapsecond
days, that is).  And the matrix of possibilities is sparse:

* Both possibilities necessarily exist for getting the time,
  and for converting to and from broken-down times.

* Both possibilities probably exist for interval timers.

* Both possibilities could theoretically exist for setting the
  system clock, but if it's not possible for an administrative
  utility to set the time during a leap second, if it has to wait
  for a second and set it then, I don't think anyone will complain.

* And the true-UTC alternative definitely does *not* exist
  (in my formulation, anyway) for filesystem timestamps:
  both stat(2) and utimes(2) remain time_t only.

>> Remarkably, though, *Microsoft* of all people seized the bull by
>> the horns and announced more-or-less proper leapsecond support in
>> Windows a little while back...
>
> I wonder how it deals with the externalities problems, where protocols
> specify UTC times, but then make it a simple count of seconds (ignoring
> leaps) since some epoch?

As I have written in my unpublished notes on this subject:

| If you're using absolute UTC timestamps in protocols between
| processes running on disparate systems [e.g. RTP, Chubby],
| requiring second-level (or better) synchronization, *you have a
| problem today*.  Your protocol probably breaks down at the end of
| leap second days; you probably need something like RFC 7164, too.

A kernel that is capable of faithfully delivering true-UTC
timestamps can only help the implementor of such a protocol to
get it right; I don't think it can make anything worse.

There's a subtle question about what a separate-and-unequal
kernel would deliver for time_t's on a leapsecond day.  Mostly
I think it should deliver smeared time_t's, for the benefit of
(most) programs that don't care about leap seconds, that don't
want 1s steps or other glitches, and that somehow haven't noticed
those problems yet.  But if there are programs which do care
about leap seconds, and which -- in the historical absence of
proper kernel support -- have somehow been laboriously gleaning
true UTC time from the Posix-mandated misbehavior, those programs
would be undone (until they could be rewritten to use the new,
true-UTC mechanisms, that is) by anything other than bug-compatible
Posix behavior.
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Brooks Harris]

On 2020-01-27 7:38 AM, Warner Losh wrote:



On Mon, Jan 27, 2020 at 8:17 AM Steve Summit > wrote:


Hal Murray wrote:
> Does anybody know of a good writeup of how to fix POSIX to know
about
> leap seconds and/or why POSIX hasn't done anything about it yet?

I don't know why POSIX hasn't done anything, other than that
(1) it's an excruciatingly hard problem, made even harder by
(2) the ubiquity and the specific POSIX definition of time_t.

> I assume the basic idea would be something like switch the kernel
> to use TAI rather than UTC

But that seems to end up being a non-starter, because
(1) distributing updated TAI-UTC tables is really hard
(and basically impossible for certain classes of embedded and/or
isolated systems), and (2) everyone knows how to initialize a
system clock to UTC at bootup, but almost no one knows how to do
so for TAI.

The other basic idea is to implement new, proper-UTC-capable
kernel mechanisms and interfaces alongside the time_t ones.
I'm still a big fan of the clock_gettime(CLOCK_UTC) approach
described by e.g. Markus Kuhn at

https://www.cl.cam.ac.uk/~mgk25/posix-clocks.html


, but it never seems to have caught on.  I went through the
exercise of implementing it myself in a homebrew Linux kernel,
which I used to post a message to this list a few years back
with an honest-to-goodness timestamp of

Date: Sat, 31 Dec 2016 18:59:60 -0500


You've just scratched the surface of the problem.

How do I touch a file on Sat, 31 Dec 2016 18:59:60 -0500 and have ls 
-l report that as its timestamp?


Sadly, I've never managed to properly release or publish this
work.  And that sort of gets back to the first part of the
question: Why does there not seem to be much urgency about
"solving" this problem?  Besides the points I mentioned earlier,
I can speculate on a few more:

* Leap seconds are rare, and many people don't care.

* The leap second drought of 1999-2006 rather nastily coincided
  with a gradual change in the computing industry from "nobody's
  clocks are synchronized that well anyway, so a second here or
  there doesn't matter" to the opposite.  (But by the time we
  realized we had a leapsecond problem, it was sort of too late
  to fix it.)

* These days, it seems everybody who has really thought about it
  has concluded that leap seconds are impossible or a really bad
  idea, and is sitting around waiting around for the ITU to
  abolish them, and maybe implementing leap-smeared NTP servers
  to tide us over in the meantime.

Remarkably, though, *Microsoft* of all people seized the bull by
the horns and announced more-or-less proper leapsecond support in
Windows a little while back; it might be instructive to study
that effort for lessons.


I wonder how it deals with the externalities problems, where protocols 
specify UTC times, but then make it a simple count of seconds 
(ignoring leaps) since some epoch?


Evolution of Timekeeping in Windows
https://techcommunity.microsoft.com/t5/networking-blog/evolution-of-timekeeping-in-windows/ba-p/778020

-Brooks



Warner


___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs


___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Warner Losh]

On Mon, Jan 27, 2020 at 8:17 AM Steve Summit  wrote:

> Hal Murray wrote:
> > Does anybody know of a good writeup of how to fix POSIX to know about
> > leap seconds and/or why POSIX hasn't done anything about it yet?
>
> I don't know why POSIX hasn't done anything, other than that
> (1) it's an excruciatingly hard problem, made even harder by
> (2) the ubiquity and the specific POSIX definition of time_t.
>
> > I assume the basic idea would be something like switch the kernel
> > to use TAI rather than UTC
>
> But that seems to end up being a non-starter, because
> (1) distributing updated TAI-UTC tables is really hard
> (and basically impossible for certain classes of embedded and/or
> isolated systems), and (2) everyone knows how to initialize a
> system clock to UTC at bootup, but almost no one knows how to do
> so for TAI.
>
> The other basic idea is to implement new, proper-UTC-capable
> kernel mechanisms and interfaces alongside the time_t ones.
> I'm still a big fan of the clock_gettime(CLOCK_UTC) approach
> described by e.g. Markus Kuhn at
>
> https://www.cl.cam.ac.uk/~mgk25/posix-clocks.html
>
> , but it never seems to have caught on.  I went through the
> exercise of implementing it myself in a homebrew Linux kernel,
> which I used to post a message to this list a few years back
> with an honest-to-goodness timestamp of
>
> Date: Sat, 31 Dec 2016 18:59:60 -0500
>

You've just scratched the surface of the problem.

How do I touch a file on Sat, 31 Dec 2016 18:59:60 -0500 and have ls -l
report that as its timestamp?


> Sadly, I've never managed to properly release or publish this
> work.  And that sort of gets back to the first part of the
> question: Why does there not seem to be much urgency about
> "solving" this problem?  Besides the points I mentioned earlier,
> I can speculate on a few more:
>
> * Leap seconds are rare, and many people don't care.
>
> * The leap second drought of 1999-2006 rather nastily coincided
>   with a gradual change in the computing industry from "nobody's
>   clocks are synchronized that well anyway, so a second here or
>   there doesn't matter" to the opposite.  (But by the time we
>   realized we had a leapsecond problem, it was sort of too late
>   to fix it.)
>
> * These days, it seems everybody who has really thought about it
>   has concluded that leap seconds are impossible or a really bad
>   idea, and is sitting around waiting around for the ITU to
>   abolish them, and maybe implementing leap-smeared NTP servers
>   to tide us over in the meantime.
>
> Remarkably, though, *Microsoft* of all people seized the bull by
> the horns and announced more-or-less proper leapsecond support in
> Windows a little while back; it might be instructive to study
> that effort for lessons.
>

I wonder how it deals with the externalities problems, where protocols
specify UTC times, but then make it a simple count of seconds (ignoring
leaps) since some epoch?

Warner
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Warner Losh]

On Mon, Jan 27, 2020 at 5:35 AM Tony Finch  wrote:

> Zefram via LEAPSECS  wrote:
> > Hal Murray wrote:
> > >I assume the basic idea would be something like switch the kernel to
> use TAI
> > >rather than UTC and implement conversion in some library routines.
> >
> > That's not a viable approach, because of the wide range of uses to which
> > time_t is put.
>
> In particular, the kernel needs time_t for compatibility with filesystems
> that need it, and other non-syscall interfaces that expose time_t. There's
> a lot of them.
>

Yes.  The problems are legend.

First, there are the 'return time' system calls or legacy routines. These
aren't so bad. You can make new ones: gettimeofday, settimeofday, time.
Then, you need gmtime and timegm except for TAI. The goal here being that
you use either all old, or all new routines.  The new routines are just
like the old, so that seems to work because there's no leap seconds.
clock_getime has the ability to have a CLOCK_TAI even.

Except that the gmtime routines are still not quite right. time_t <->
struct tm is still lossy. So, to really fix the problem you need to either
enhance struct tm to carry metadata of the sort of timezone (since UTC is
almost a time zone). and then you need to think hard about what to do with
gmtime/taitime wrt leap seconds. So you'd need to fix the 'feature' of
gmtime/timegm that you can do things like add to the different tm_* fields
to move time forward. But that starts to intrude into real code changes.
Otherwise, you can't represent a leap second.

And then there's functions like stat, utimes, futimes, that have time_t
burried in them as struct timeval or struct timespec. If you want to change
the interpretation of those values (which is currently UTC), you'd need new
versions of those system calls that return TAI time. And you'd have to be
careful too: nanosleep(2) and select(2) doesn't need a new version since
its timeval is an interval, not an absolute time. There's some like the
clock_* routines that don't need to change too. In FreeBSD I counted about
two dozen instances that would need to be audited.

And then you have to worry about filesystems. What happens if I'm building
software during the leap second? Regardless of how you shoe-horn the time
into place, you can wind up with a situation where the source is newer than
the object for things like generated files, which will confuse make. So now
you need to have a bunch of routines in the kernel to convert historical
UTC times to TAI times, which means you need all leap seconds that have
ever happened to do it right (remember, we added system calls to get this
data in TAI not UTC above). And that's not even getting into external
things like NFS that need to do this over the network (though largely, it's
the same problem).

And so now you're left with a data management problem. You have to update
the timezone files, because that's how you paper over UTC enough to make
the old routines work (except during a leap second). And you have to invent
mechanisms to keep them up to date in long running programs (otherwise
those slowly diverge in systems with long uptimes). And so you have to be
connected to the internet (or some other data stream) to make the updates.
And if you're making updates in place, you have to do it atomically so you
don't race.

And even if you do all that, it's impossible for me to get the TAI time for
any times > 6 months out (technically the other side Dec 31 or June 30)
because we have no clue what that time will be. We find out in January if
there's a leap in June, so future time values in June can't possibly be
right if you are doing them in December. Sure, you can pull out the 'use
struct tm or its replacement' for such times so far in the future, but
that's kinda arbitrary since times 3 months in the future are fine...
Unless the system has been disconnected from its data sources for some
reason...

So sure, you can half-ass something that solves some of the problems, but
it's a huge undertaking to do it all right. And there's huge resistance to
being this intrusive into the system just to fix a silly problem once every
couple of years. Why slow things down the other half billion seconds
between leap seconds.

It's not a simple problem at all to get completely right.

Warner
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Steve Allen]

On Mon 2020-01-27T10:15:56-0500 Steve Summit hath writ:
> Remarkably, though, *Microsoft* of all people seized the bull by
> the horns and announced more-or-less proper leapsecond support in
> Windows a little while back; it might be instructive to study
> that effort for lessons.

I think Microsoft was able to achieve this because Windows 10 is
designed to be continuously updated, including leap second
information, with new code from corporate headquarters.  This would
not have been possible with an earlier version of Windows, nor with
most any other system.

--
Steve Allen  WGS-84 (GPS)
UCO/Lick Observatory--ISB 260  Natural Sciences II, Room 165  Lat  +36.99855
1156 High Street   Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064   https://www.ucolick.org/~sla/  Hgt +250 m
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Steve Summit]

Hal Murray wrote:
> Does anybody know of a good writeup of how to fix POSIX to know about
> leap seconds and/or why POSIX hasn't done anything about it yet?

I don't know why POSIX hasn't done anything, other than that
(1) it's an excruciatingly hard problem, made even harder by
(2) the ubiquity and the specific POSIX definition of time_t.

> I assume the basic idea would be something like switch the kernel
> to use TAI rather than UTC

But that seems to end up being a non-starter, because
(1) distributing updated TAI-UTC tables is really hard
(and basically impossible for certain classes of embedded and/or
isolated systems), and (2) everyone knows how to initialize a
system clock to UTC at bootup, but almost no one knows how to do
so for TAI.

The other basic idea is to implement new, proper-UTC-capable
kernel mechanisms and interfaces alongside the time_t ones.
I'm still a big fan of the clock_gettime(CLOCK_UTC) approach
described by e.g. Markus Kuhn at

https://www.cl.cam.ac.uk/~mgk25/posix-clocks.html

, but it never seems to have caught on.  I went through the
exercise of implementing it myself in a homebrew Linux kernel,
which I used to post a message to this list a few years back
with an honest-to-goodness timestamp of

Date: Sat, 31 Dec 2016 18:59:60 -0500

Sadly, I've never managed to properly release or publish this
work.  And that sort of gets back to the first part of the
question: Why does there not seem to be much urgency about
"solving" this problem?  Besides the points I mentioned earlier,
I can speculate on a few more:

* Leap seconds are rare, and many people don't care.

* The leap second drought of 1999-2006 rather nastily coincided
  with a gradual change in the computing industry from "nobody's
  clocks are synchronized that well anyway, so a second here or
  there doesn't matter" to the opposite.  (But by the time we
  realized we had a leapsecond problem, it was sort of too late
  to fix it.)

* These days, it seems everybody who has really thought about it
  has concluded that leap seconds are impossible or a really bad
  idea, and is sitting around waiting around for the ITU to
  abolish them, and maybe implementing leap-smeared NTP servers
  to tide us over in the meantime.

Remarkably, though, *Microsoft* of all people seized the bull by
the horns and announced more-or-less proper leapsecond support in
Windows a little while back; it might be instructive to study
that effort for lessons.
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Tony Finch]

Zefram via LEAPSECS  wrote:
> Hal Murray wrote:
> >I assume the basic idea would be something like switch the kernel to use TAI
> >rather than UTC and implement conversion in some library routines.
>
> That's not a viable approach, because of the wide range of uses to which
> time_t is put.

In particular, the kernel needs time_t for compatibility with filesystems
that need it, and other non-syscall interfaces that expose time_t. There's
a lot of them.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
South Fitzroy: Westerly or southwesterly 5 to 7, occasionally 4 in south.
Rough or very rough. Rain or showers. Good, occasionally poor.
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

Re: [LEAPSECS] leap seconds in POSIX

[Martin Burnicki]

Hal Murray wrote:
> 
> Does anybody know of a good writeup of how to fix POSIX to know about leap 
> seconds and/or why POSIX hasn't done anything about it yet?

I've made a number of presentations and whitepapers about leap seconds
and problems related to them. However I'm not aware of an easy, good
solution.

The basic problem is that common API calls used to retrieve the system
time stamp from an OS don't provide a status that could be used to
distinguish between a normal second and an ongoing, inserted leap second.

Without such status a function that converts a timestamp to calendar
date and time doesn't know if the timestamp associated with an inserted
leap second should yield a second with number '60' of the current day,
or a second '00' of the next day.

I think an API that provides a timestamp and an associated status in a
*consistent* way would be too "expensive" with regard to execution time
because some locking mechanism needed to be implemented to avoid that
inconsistent timestamp and status information could be returned.

On the other hand, if an application already has a broken down date and
time (e.g. from an external time source, serial string or whatever), it
knows that the time is the leap second if the second number is '60'. So
the '60' is the required status information.

However, if you "normalize" the time of a leap second, e.g. 23:59:60,
then 60 seconds carry over to one minute, 60 minutes to 1 hour, and 24
hours to the next day. So when computing an associated timestamp, the
effective timestamp is the same as for 00:00:00 the next day, and the
information that this second is a leap second is simply lost unless
otherwise preserved in some way.

On most Unix-like systems the system time is kept as POSIX time, and
when a leap second is inserted, the kernel just steps the system time
back by 1 second.

*This* is what confuses applications that don't expect that the system
timestamp ever goes backward.

Many years ago Dave Mills has proposed a way to avoid this problem by
stopping the system clock for 1 s, and doing a system time increment by
the smallest unit whenever an application retrieves a system time stamp
while the clock is stopped.

This would be a good workaround since the time returned from the kernel
never steps back, and there are no duplicate timestamps because even
during the leap second the timestamps increment by a small amount.

Quite some time ago I asked one of the Linux kernel maintainers why they
don't implement the leap second handling this way, and the answer was
just "because it's too expensive". Whenever an application queried the
current system time, the kernel had to check if a leap second was just
being inserted, and if there was, some small amount of time had to be
added to the stopped system time.

And all this just to get it "right" for 1 second in several years. Just
stepping the time back by 1 second at a certain point in time is much
faster, and much easier to implement.

> I assume the basic idea would be something like switch the kernel to use TAI 
> rather than UTC and implement conversion in some library routines.

I think this could be a good approach, but this requires that not only
leap second announcements but also the current TAI offset is supplied to
the OS.

Runtime libraries require the correct TAI offset to be able to convert
the TAI system time to UTC. E.g., the kernel could return a TAI
timestamp, and the runtime library function like gmtime() needs to know
the current TAI offset and leap second date to be able to return a time
with a 60 in the seconds field.

I know there are leap second files from IERS or NIST, and also current
versions of the TZ database contain a copy of such file, but this
requires that this information is continuously updated, i.e. new
versions of the leap second files or the TZ data base are supplied.

This should work (I haven't tried it) if you configure the system with
one of the "right" timezones, which gets the TAI information from a
table of the TZ DB.

This may work with current versions of operating systems, where the OS
maintainers or the admin provides the required information, but may not
work reliably for systems that are out of support, or embedded systems
that never get any update. Thes would use a wrong UTC time after the
next leap second.

In the past, ntpd could provide its clients also with the TAI offset if
autokey was configured. Since autokey is now obsolete and replaced by
NTS, an extension field to the NTP packet could provide the TAI offset.

PTP/IEEE 1588 works based on TAI, and the protocol provides the TAI to
UTC offset, so either PTP or NTP with TAI offset extension field could
be used to adjust the kernel time to TAI.

Both ntpd and PTP clients should be able to write the current TAI offset
and leap second announcments to the kernel. However, AFAIK time
conversion functions only retrieve the TAI information from the TZ DB,
not from the kernel, so the TAI capabilities of NTP and PTP 

Re: [LEAPSECS] leap seconds in POSIX

[Zefram via LEAPSECS]

Hal Murray wrote:
>I assume the basic idea would be something like switch the kernel to use TAI 
>rather than UTC and implement conversion in some library routines.

That's not a viable approach, because of the wide range of uses to which
time_t is put.  One can't rely on being able to perform such conversion
for all uses.  (Though a disturbing number of libraries pretend that
one can, and then convert on the false assumption that there will be no
future leap seconds.)

It'll be necessary to distinguish explicitly between UTC-based and
TAI-based time representations.  Furthermore, the POSIX way is to leave
the existing interfaces unchanged and add new ones.  So if the kernel is
to do anything TAI, then the kernel will have to track both UTC and TAI,
and we can expect to see a new clockid_t value for the TAI version of
CLOCK_REALTIME.  The less-invasive option is to keep the kernel UTC-only,
adding a clockid_t value for the proposed leap-second-capable CLOCK_UTC,
and do all TAI work in user space.

-zefram
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs

[LEAPSECS] leap seconds in POSIX

[Hal Murray]

Does anybody know of a good writeup of how to fix POSIX to know about leap 
seconds and/or why POSIX hasn't done anything about it yet?

I assume the basic idea would be something like switch the kernel to use TAI 
rather than UTC and implement conversion in some library routines.


There is a discussion on the IETF ntp list with typical S/N for this topic.

-- 
These are my opinions.  I hate spam.



___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs