Date: Tue, 13 Dec 2022 16:52:42 +0000
From: "Geoff Clare via austin-group-l at The Open Group"
<austin-group-l@opengroup.org>
Message-ID: <Y5it2lbfu62K9oxS@localhost>
| It is too late to add timegm() in Issue 8.
I suspected that would be the case. Pity, as using UTC (or
whatever it is that POSIX time is really called, not really UTC,
as that has leap seconds) (gmtime(), modify the result, timegm())
is a way that works to adjust a time_t without doing direct arithmetic
upon it, as POSIX base time is very regular, no anomalies to deal with.
Incidentally, I'd be interested to see a quote from the C standard that
specifies time_t with the limitations you expressed in the previous
message, all I've been able to find is that it must be an arithmetic
type, and that its range and precision aren't specified. That's exactly
what is specified for a clock_t as well - however for clock_t it is also
explicit that it is possible to divide the value by a constant with
meaningful results - ie: normal arithmetic operations are possible.
If they're possible on a clock_t I see nothing there (in moderately
recent C anyway) which would suggest that they're not possible on a time_t
as well. Certainly the unspecified precision (where POSIX specifies
"seconds") means that care would need to be taken to add using the
correct units, but an implementation could provide a specification to
allow programs to discover what the precision actually is.
| You are suffering from a misconception that *timeptr somehow "specifies"
| a time since the Epoch. It does not! It specifies a broken-down time.
No, no misconception there, though sometimes I suppose (as it often is)
that my language might be a little loose.
However I do certainly hope that you agree that the broken-down time
is related to the resulting seconds since the Epoch, when I have used
"specifies" (loosely perhaps) previously, all I have ever meant is
that - that is, that mktime() is not intended to be free to return
any random time_t it likes - it must return one that corresponds to
the broken-down time passed in.
I certainly hope that you're not disagreeing with that.
| The standard describes, in detail (in the paragraph beginning "The
| relationship between ..."), how this broken-down time is *converted* to
| an integer "time since the Epoch" value.
That's not "in detail", It says (since the quote contains section and
page numbers, this extract is from Issue 8 Draft 2.1, but the substance
is unchanged from earlier versions):
The relationship between the tm structure (defined in the <time.h>
header) and the time in seconds since the Epoch is that the result
shall be as specified in the expression given in the definition of
seconds since the Epoch (see XBD Section 4.17, on page 95) corrected
for timezone and any seasonal time adjustments
For that to mean anything at all, we need to look at XBD 4.17:
A value that approximates the number of seconds that have elapsed
since the Epoch. A Coordinated Universal Time name (specified in
terms of seconds (tm_sec), minutes (tm_min), hours (tm_hour), days
since January 1 of the year (tm_yday), and calendar year minus 1900
(tm_year)) is related to a time represented as seconds since the
Epoch, according to the expression below.
The first thing to note is that this only applies to UTC times. [That's
one reason why using gmtime() and timegm() for adjusting time_t values
makes much more sense].
If the year is <1970 or the value is negative, the relationship is
undefined. If the year is 1970 and the value is non-negative, the
value is related to a Coordinated Universal Time name according
to the C-language expression, [...]
I'm not going to quote the expression here (anyone interested can look
it up for themselves) but again it is clear that this applies only to
UTC times. It says so.
XBD 4.17 goes on to say:
The relationship between the actual time of day and the current
value for seconds since the Epoch is unspecified.
This was discussed briefly before, and you claimed that all this means
(paraphrased here by me) is that the system's clock (what time() returns)
and the real world precise time of day aren't necessarily the same
(ie: there's no promise that systems are running NTP or similar).
That's certainly implied by that sentence, but that's not all that it
says - it is quite explicit that there is no specified relationship
between "actual time of day" (ie: local time) and the "seconds since
the epoch" value.
Note: not no relationship (obviously there is) just that that relationship
isn't specified by the standard.
So where exactly is this "in detail" specification of how a local time
(with all of its peculiarities) is supposed to be converted to seconds
since the epoch?
| When the standard says "shall be set to represent the specified time since
| the Epoch" it is talking about the integer value that *it* specifies to
| be calculated from the broken-down time in *timeptr. It is not in any way
| suggesting that *timeptr "specifies" a time since the Epoch.
Did you reread that before you sent it? Do you understand what you're
saying there (if read exactly as you wrote it)? That is that *timeptr
doesn't provide information from which the time since the epoch can be
deduced. That's what specifies means after all.
| In trying to treat *timeptr as "specifying" a time since the Epoch, you
| are misunderstanding the intention and misinterpreting the meaning of
| much of the mktime() text.
All I am expecting, is that the result shall reflect the broken down
time passed in. If it isn't required to do that, I see no point in
the function existing at all.
| Since I mentioned attitudes, I'll explain mine. It is that mktime()
| follows the well-known principle "be liberal in what you accept, and
| conservative in what you send" (which originated in relation to
| communication protocols but I think applies very well here).
Yes, I know Postel's principle - I knew Jon - I also know what it
really means, and how often people who don't know that misapply it,
as you are doing here.
However I'm glad that you're claiming that this principle applies to
POSIX - however you forgot to include a reference to where in the
rules of interpretation of the standard this is stated. Please
supply that reference, as I can't find it.
You can be sure that once I know where that is, I shall be applying
the principle (correctly) to other discussions we have on interpretation
of the standard, and requiring a similar "liberal in what you accept"
interpretation be applied there as well.
But here we don't really need to go into that, as it isn't needed for
your purpose - the standard is actually quite explicit that out of range
values are permitted. We don't need to "be liberal" to accept those,
doing so is explicitly required - one of the few things that is actually
specified about mktime().
What it doesn't say is what the out of range values mean, how they're
to be interpreted, and no amount of "being liberal" can supply meaning
to unknown data.
| Applying this principle to mktime() means you can give it an
| "incorrect" broken-down time and it will make sense of it and give
| you back a correct time.
Not incorrect, that's not what the standard says, what it says is that
the fields need not be within the ranges specified in <time.h>.
That's all it allows.
There's nothing there about "incorrect". There isn't either in Postel's
principle when correctly interpreted - the idea is that one should not
reject input because of some technical deficiency which doesn't affect
the results, so for example, relating to mktime() - if tm_wday isn't
initialised at all, we should not be generating undefined behaviour
by referencing it (which we might do if we just copy the struct passed
in for example). It isn't used, doesn't affect the result (it is set
in the returned struct tm, that's irrelevant here) so should be
completely ignored by the implementation.
In networking, the analogy is that when receiving a packet, with a
field which is specified to be all zeroes, but which we don't use for
anything, we should not complain if when received the field contains
something different. Doing so inhibits later enhancements.
[Aside, I did an X.25 implementation long long ago, when that
was still a useful thing to have, and to be connected to the
phone company's network, it needed to be conformance tested, to
ensure that it wouldn't break their network. That part was no
problem, but their tests also tested what my implementation would
do if their end did various bizarre things - I had adopted Postel's
principle, and was quite forgiving of nonsense that didn't affect
operations ... they were not happy with that at all, and insisted
that when they sent nonsense (which their real operational code
never did of course) I was required to act in some particular way
and reject their data - even though it was irrelevant to operation
of the protocol (actual errors which meant things could not work
were already handled of course). To get certified I had to add
lots of completely useless error checking, which would never affect
anything real, anywhere, ever...]
| * If you give it Feb 29 in a non-leap year it treats that as the day
| after Feb 28 and gives you back Mar 1.
Why that? Is that specified somewhere? An alternative
interpretation might be that if you give Feb 29, you mean the last
possible day in February, and if the year happens not to be a leap
year, that would be Feb 28, so that is a possible choice for the
corrected value. Why not?
There simply is no specification of this, and there should be, otherwise
how is a new implementation supposed to guess what it should do, and how
can an application know what will happen?
However, as Don pointed out, all the (known) implementations do it the
way you just suggested that it should be done - not because there's any
specification that says so, just because everyone copied from the first
implementation that made these adjustments. That's fine, and there's
no reason at all that we shouldn't specify this behaviour, since it is
the de-facto standard.
But it is absurd to claim that there's currently anything specified in
any standard which requires (or even anticipates) this kind of operation.
You most certainly can't get it from "be liberal" which just means not
to reject things which don't matter, certainly not to invent meanings for
data which is required, but not understood.
The next one or two of your examples are just the same kind of thing over
and over again, there's no point discussing them - there are reasonable
alternative interpretations for the input for all of them, but the
ones you picked are the ones that the implementations actually produce
(but which, as best I can tell, is undocumented everywhere - even in
the various manual pages for specific implementations of mktime(), though
you understand, I hope, I have not seen all of those that I expect exist.)
| * If you give it tm_isdst=0 for a time when DST is in effect, it gives
| you back a positive tm_isdst and alters the other fields appropriately.
While it seems that's what the implementations do, this one is even harder
to justify from what is written in the standard than the others. For this
field, the interpretation of tm_isdst is specified (a little loosely, but
still specified) and there are no (on input) out of range values (just
< 0, == 0, and > 0 ... which covers everything possible for an integer
type field). The "gives you back" part is OK, but the "alters the other
fields" is not at all what I would have expected to happen, and certainly
isn't specified anywhere.
| * If there is a DST transition where 02:00 standard time becomes 03:00
| DST and you give mktime() 02:30 (with negative tm_isdst), it treats
| that as either 30 minutes after 02:00 standard time or 30 minutes
| before 03:00 DST and gives you back a zero or positive tm_isdst,
| respectively, with the tm_hour field altered appropriately.
But this one is not the de-facto standard. It is (like the others)
clearly not in any written standard, but here, implementations (as you
know) differ. This might be what you'd like to see happen, but it is
not necessarily what does happen.
There are reasons for that - with tm_mday == 0, or -1 or something,
we can guess (probably correctly) that the application might have started
with an in range tm_mday, and subtracted some number of days, so it is
probably trying to find an earlier day. Or if it is set to 33 or
something, it has probably added some number of days, and is looking to
produce a later date. Each of those is a reasonable assumption, other
possibilities might be that in the 0 or -1 case, the program was adding
so many days that the int overflowed, and wrapped around - but that's
undefined behaviour anyway, so we can ignore that one, or that the
application simply set (or failed to set) anything rational, in which
case the GIGO principle applies, and it doesn't matter how we interpret it.
But as you stated above, with a time in the gap, there's no hint as to
why it got that way, or what the application is attempting to achieve.
That's why you're allowing "either 30 minutes after 02:00 standard time
or 30 minutes before 03:00 DST", because we have no clue at all which is
likely to be correct.
Doing that, and basing that upon Postel's principle is a total perversion
of what it means - it certainly does not mean "interpret any incorrect
data that is needed to determine the response in any way you like", that
would be ignoring the 2nd half of the principle "be conservative in what
you send" - which means to only generate results (replies in networking)
which are strictly in conformance with both what is required by the
standards, and was specified by the request.
| * If a geographical timezone changes its UTC offset [...]
This is just a repeat of the previous one, except without tm_isdst issues.
| And yes, having listed that last case along with the others, I see no
| reason that it should not follow the same principle. The "treats it as"
| wording is much the same as the DST transition case.
Certainly the reason for a gap in local time shouldn't be relevant, except
perhaps in the case where tm_isdst on input is not -1 (where in the
seasonal time transition case one might be able to guess what is happening
and return an appropriate answer, but with an offset shift, that doesn't
work).
But neither of those (the two examples you gave of times in gaps) are the
same as the earlier ones - they're not out of range values (for the UTC
times specified for the XBD 4.17 formula) and they don't produce single
unambiguous results, which certainly should be the objective here.
Further the earlier ones are all implemented the same way, seemingly,
everywhere, and those ones aren't, so if we do decide to actually specify
how mktime() is supposed to deal with out of range, and other incorrect,
input *timeptrs, then we need to take that into account.
| Returning -1 for any of these cases violates the "be liberal in what
| you accept" part of the principle.
That's utter nonsense, it does nothing of the kind, or not if you
really understand the principle, and don't just apply it whenever
it seems like it might support your argument.
Rather not returning -1 violates the "conservative in what you send"
part of the principle.
But none of that matters until we discover where in POSIX it is stated
that Postel's principle is to be used as one of the rules of interpretation.
| I agree it's not clear for pathological cases like that. It comes down
| to this statement:
|
| the tm_yday value used in the expression is the day of the year
| from 0 to 365 inclusive, calculated from the other tm structure
| members
Huh? I know that XBD 4.17 requires tm_yday to be known, so that the
formula works - and that only makes any sense to calculate once the
rest of the struct tm contains in range values, which strongly implies
that the struct tm passed in is to be normalised first, and then the
formula is applied to calculate seconds since the epoch. That is, not
as you seem to be hoping, that one simply applies that formula to whatever
random gibberish is in the struct tm, get a result, and then providing
that's within range of a time_t, it is the answer. That simply doesn't
work.
Apart from that, what does tm_yday have do do with anything at all ?
Off list I was also informed that the SysVR4.2 code for mktime() also
normalises the struct tm first, and then calculates the time_t - it is
really hard to imagine anyone able to implement it any other way.
| It may be worth trying to improve this, if implementations all do the
| tm_yday calculation the same way, but it has no real relevance in the
| matter of whether mktime() can return -1 for "incorrect" broken-down
| times.
No it doesn't, and I cannot fathom why you're bringing that one up at all.
| And I didn't claim that it does. What I said (which you trimmed) was:
|
| By a strict reading, you may be right, but it is strongly implied by
| "shall be set to represent the specified time since the Epoch". In
| any case, it is being clarified by bug 1613.
|
| My point was entirely about this "shall be set to represent" text, i.e.
| about what the returned struct tm fields must contain.
I'm also unsure why you keep returning to this point. We all agree
that when mktime() doesn't return -1, the struct tm must be set to
be what localtime() would return for time_t which is returned (which I
think is what you added in part of the resolution of bug 1613, though
that part had nothing to do with the actual issue reported - no problem
with that happening though).
There's nothing interesting to say about this any more here, please just
drop it.
If you want to propose some text for what the *timeptr should be on
return when mktime() is indicating an error, then that might be useful,
as there's currently nothing at all about that case. But beyond that
let's just forget the returned struct value - we all agree what it will
be when there's no error, repeating it over and over again achieves nothing.
| The context for
| this was Don's point that Feb 29 2023 has the tm fields in their stated
| ranges and so the standard, as written, allows the returned struct tm
| to be left as Feb 29 2023.
Yes, I know. The standard as written is useless.
| The change in bug 1613 requires them to be
| set to the values that would be returned by localtime(), so this will
| no longer be allowed.
Good. Completely irrelevant to the current issue, which is when mktime()
can return an error, as the returned struct tm is only specified when it
does not. If an implementation (which the current written standard allows)
says "Feb 29 is within the ranges specified in <time.h> so I am not allowed
to adjust that, however that day does not exist in the specified year, so
I will return an error" then what is in the struct tm on return isn't
specified anywhere, and if it still said Feb 29 in that implementation,
then fine. That's allowed (currently).
| The above quote is all that's needed, provided "the specified time
| since the Epoch" is correctly interpreted (which you are not doing).
| The time since the Epoch being referred to here is a known integer
| value which mktime() is going to return.
No, it is not. For some broken down struct tm's, there is no such integer.
| (The adjustment to bring struct tm fields
| into range is done after this value is known - see below).
But that's not what implementations do, and further, is not possible
most likely. Hence if the standard seems to be requiring that (which
I don't actually believe it does) then it would be wrong, and need fixing.
| The sort of
| adjustment you were suggesting, "if (t->tm_sec < 0) t->tm_sec = 0",
| etc. would cause the fields to no longer represent that time since the
| Epoch.
Only if that was done after seconds from the Epoch is calculated, which
in practice, it never is, and which the standard certainly doesn't require.
| No, the adjustment to bring struct tm fields into range is done after
| the time since the Epoch value has been calculated.
Cannot be. The formula doesn't work.
| This is clear just from the order in which things are described on the
| mktime() page,
Really? Where is that rule of interpretation stated? You forgot to
quote that one as well as where use of Postel's principle is mandated.
Much more likely (just like all other descriptions) the order of the
words is the order in which they make sense, to describe the result.
If it was intended to be a requirement of the implementation it would
need to say so explicitly, and use "shall" wouldn't it? Does it?
You're overreaching - seems like desperation to me.
| but also from the use of "Upon successful completion",
All that's saying is what will be in the returned *timeptr when
mktime() doesn't return an error. It says nothing at all about the
order in which things happen inside the implementation. The
implementation can make a copy of the *timeptr, adjust that, try
converting that, adjust it more, .... lots of things are possible.
It doesn't even need to make a copy - provided that when it returns
successfully, *timeptr is now correct, what that was at intermediate
stages is irrelevant (it might be observable from another thread, but
doing so is not defined to be useful). If there's an unsuccessful
return ((time_t)-1) then what's in *timeptr isn't specified, so can be
anything.
| since mktime() can't
| know whether it will complete successfully until it has calculated the
| time_t value it is going to return.
It doesn't matter, since if it doesn't complete successfully, nothing is
stated about what is in *timeptr - it might be the original struct tm,
it might be that one adjusted as best the implementation was able before
determining that an error was going to happen, or it could be almost
anything else (including a full set of uninitialised fields, to which
any reference would be UB) - the spec simply does not say.
You're presuming you know how the implementations work. You're wrong.
| > Agreed. But we need to pick tm_isdst = 0 or tm_isdst = 1, and
| > which we pick will alter what time_t value gets returned. There's
| > nothing anywhere that suggests which one should be selected.
|
| Correct, and since the standard is silent on this, either behaviour
| is allowed.
But why just those two, the standard also allows tm_isdst = -1 (anything < 0)
on input, why couldn't the implementation assume that one instead, and
in that case, we have no guidance, and no reason to pick one or the other.
| > No, that's not what that says. I can see you're presuming that the
| > implementation calculates a time_t first, and then adjusts the tm to
| > match. That's not required.
|
| Yes it is. See above.
No it isn't. See above.
| The only real potential for problems here is if an application does
| small (less than a day) additions/subtractions using the struct tm
| fields and sets tm_isdst=-1. Then it might work fine on one
| implementation but get into the kind of loop you described in an
| earlier mail on an implementation that behaves the other way. But, as
| I pointed out in that earlier discussion, no real application would
| do that.
Why not? You mean you would not write code that way, but why
wouldn't someone else. Where is the specification that says not to?
You have already pointed out that C apparently disallows arithmetic on
a time_t (except comparison, apparently) so if I want to add 5 minutes
to a time_t I need to convert it to a struct tm (and must use localtime()
as the only defined way back is mktime()) then use small (less than a day)
addition, right?
Since I don't know whether, after I have added to the struct tm, whether
the result will be summer time or not (it might have transitioned in
the period I am skipping over) I should certainly set tm_isdst to -1,
that's what the specifications says is appropriate:
A positive or 0 value for tm_isdst shall cause mktime( ) to
presume initially that Daylight Saving Time, respectively, is
or is not in effect for the specified time. A negative value for
tm_isdst shall cause mktime( ) to attempt to determine whether
Daylight Saving Time is in effect for the specified time.
After the adjustment, I don't know whether or not summer time (DST)
is or is not in effect, and I want mktime() to attempt to determine
that for me, so setting tm_isdst = -1 is exactly the correct thing to
do, isn't it? If not, from where is an application programmer
supposed to conclude differently?
Actually, from where is the application programmer supposed to determine
that just adding N to tm_min (and doing nothing else, tm_isdst perhaps
excepted) is supposed to return a time_t 5 minutes later than the one
that produced the struct tm before the addition? I can find nothing that
suggests that works (just cargo cult programming - that code does this,
so it must be OK for me to do it too).
| The fact that no such problems have come to light in the
| last 30 years also means that in practice this is a non-problem.
First, you have no way of knowing that no such problems have occurred,
you're just guessing, because no-one told you. Further, as the issue
certainly would occur rarely, and not repeat itself later when someone
wonders why this program went into a loop, that one odd time, it is the
kind of thing that your average programmer writes off as a hardware
glitch, that their code might have a very rarely seen bug is incomprehensible.
They have tested it, many times (of course, always during working hours)
and have never seen a problem.
Second, if that's to be the general philosophy for POSIX - just do
anything which doesn't seem like it causes problems, then it is time
for me to abandon POSIX completely, as that's not what I am looking for
in a specification. Most others either I'd expect.
You wouldn't tolerate this in anything else "works most of the time,
so it's OK" is not acceptable, not as a specification, ever.
kre
