Date: Tue, 13 Dec 2022 16:52:42 +0000 From: "Geoff Clare via austin-group-l at The Open Group" <austin-group-l@opengroup.org> Message-ID: <Y5it2lbfu62K9oxS@localhost>
| It is too late to add timegm() in Issue 8. I suspected that would be the case. Pity, as using UTC (or whatever it is that POSIX time is really called, not really UTC, as that has leap seconds) (gmtime(), modify the result, timegm()) is a way that works to adjust a time_t without doing direct arithmetic upon it, as POSIX base time is very regular, no anomalies to deal with. Incidentally, I'd be interested to see a quote from the C standard that specifies time_t with the limitations you expressed in the previous message, all I've been able to find is that it must be an arithmetic type, and that its range and precision aren't specified. That's exactly what is specified for a clock_t as well - however for clock_t it is also explicit that it is possible to divide the value by a constant with meaningful results - ie: normal arithmetic operations are possible. If they're possible on a clock_t I see nothing there (in moderately recent C anyway) which would suggest that they're not possible on a time_t as well. Certainly the unspecified precision (where POSIX specifies "seconds") means that care would need to be taken to add using the correct units, but an implementation could provide a specification to allow programs to discover what the precision actually is. | You are suffering from a misconception that *timeptr somehow "specifies" | a time since the Epoch. It does not! It specifies a broken-down time. No, no misconception there, though sometimes I suppose (as it often is) that my language might be a little loose. However I do certainly hope that you agree that the broken-down time is related to the resulting seconds since the Epoch, when I have used "specifies" (loosely perhaps) previously, all I have ever meant is that - that is, that mktime() is not intended to be free to return any random time_t it likes - it must return one that corresponds to the broken-down time passed in. I certainly hope that you're not disagreeing with that. | The standard describes, in detail (in the paragraph beginning "The | relationship between ..."), how this broken-down time is *converted* to | an integer "time since the Epoch" value. That's not "in detail", It says (since the quote contains section and page numbers, this extract is from Issue 8 Draft 2.1, but the substance is unchanged from earlier versions): The relationship between the tm structure (defined in the <time.h> header) and the time in seconds since the Epoch is that the result shall be as specified in the expression given in the definition of seconds since the Epoch (see XBD Section 4.17, on page 95) corrected for timezone and any seasonal time adjustments For that to mean anything at all, we need to look at XBD 4.17: A value that approximates the number of seconds that have elapsed since the Epoch. A Coordinated Universal Time name (specified in terms of seconds (tm_sec), minutes (tm_min), hours (tm_hour), days since January 1 of the year (tm_yday), and calendar year minus 1900 (tm_year)) is related to a time represented as seconds since the Epoch, according to the expression below. The first thing to note is that this only applies to UTC times. [That's one reason why using gmtime() and timegm() for adjusting time_t values makes much more sense]. If the year is <1970 or the value is negative, the relationship is undefined. If the year is 1970 and the value is non-negative, the value is related to a Coordinated Universal Time name according to the C-language expression, [...] I'm not going to quote the expression here (anyone interested can look it up for themselves) but again it is clear that this applies only to UTC times. It says so. XBD 4.17 goes on to say: The relationship between the actual time of day and the current value for seconds since the Epoch is unspecified. This was discussed briefly before, and you claimed that all this means (paraphrased here by me) is that the system's clock (what time() returns) and the real world precise time of day aren't necessarily the same (ie: there's no promise that systems are running NTP or similar). That's certainly implied by that sentence, but that's not all that it says - it is quite explicit that there is no specified relationship between "actual time of day" (ie: local time) and the "seconds since the epoch" value. Note: not no relationship (obviously there is) just that that relationship isn't specified by the standard. So where exactly is this "in detail" specification of how a local time (with all of its peculiarities) is supposed to be converted to seconds since the epoch? | When the standard says "shall be set to represent the specified time since | the Epoch" it is talking about the integer value that *it* specifies to | be calculated from the broken-down time in *timeptr. It is not in any way | suggesting that *timeptr "specifies" a time since the Epoch. Did you reread that before you sent it? Do you understand what you're saying there (if read exactly as you wrote it)? That is that *timeptr doesn't provide information from which the time since the epoch can be deduced. That's what specifies means after all. | In trying to treat *timeptr as "specifying" a time since the Epoch, you | are misunderstanding the intention and misinterpreting the meaning of | much of the mktime() text. All I am expecting, is that the result shall reflect the broken down time passed in. If it isn't required to do that, I see no point in the function existing at all. | Since I mentioned attitudes, I'll explain mine. It is that mktime() | follows the well-known principle "be liberal in what you accept, and | conservative in what you send" (which originated in relation to | communication protocols but I think applies very well here). Yes, I know Postel's principle - I knew Jon - I also know what it really means, and how often people who don't know that misapply it, as you are doing here. However I'm glad that you're claiming that this principle applies to POSIX - however you forgot to include a reference to where in the rules of interpretation of the standard this is stated. Please supply that reference, as I can't find it. You can be sure that once I know where that is, I shall be applying the principle (correctly) to other discussions we have on interpretation of the standard, and requiring a similar "liberal in what you accept" interpretation be applied there as well. But here we don't really need to go into that, as it isn't needed for your purpose - the standard is actually quite explicit that out of range values are permitted. We don't need to "be liberal" to accept those, doing so is explicitly required - one of the few things that is actually specified about mktime(). What it doesn't say is what the out of range values mean, how they're to be interpreted, and no amount of "being liberal" can supply meaning to unknown data. | Applying this principle to mktime() means you can give it an | "incorrect" broken-down time and it will make sense of it and give | you back a correct time. Not incorrect, that's not what the standard says, what it says is that the fields need not be within the ranges specified in <time.h>. That's all it allows. There's nothing there about "incorrect". There isn't either in Postel's principle when correctly interpreted - the idea is that one should not reject input because of some technical deficiency which doesn't affect the results, so for example, relating to mktime() - if tm_wday isn't initialised at all, we should not be generating undefined behaviour by referencing it (which we might do if we just copy the struct passed in for example). It isn't used, doesn't affect the result (it is set in the returned struct tm, that's irrelevant here) so should be completely ignored by the implementation. In networking, the analogy is that when receiving a packet, with a field which is specified to be all zeroes, but which we don't use for anything, we should not complain if when received the field contains something different. Doing so inhibits later enhancements. [Aside, I did an X.25 implementation long long ago, when that was still a useful thing to have, and to be connected to the phone company's network, it needed to be conformance tested, to ensure that it wouldn't break their network. That part was no problem, but their tests also tested what my implementation would do if their end did various bizarre things - I had adopted Postel's principle, and was quite forgiving of nonsense that didn't affect operations ... they were not happy with that at all, and insisted that when they sent nonsense (which their real operational code never did of course) I was required to act in some particular way and reject their data - even though it was irrelevant to operation of the protocol (actual errors which meant things could not work were already handled of course). To get certified I had to add lots of completely useless error checking, which would never affect anything real, anywhere, ever...] | * If you give it Feb 29 in a non-leap year it treats that as the day | after Feb 28 and gives you back Mar 1. Why that? Is that specified somewhere? An alternative interpretation might be that if you give Feb 29, you mean the last possible day in February, and if the year happens not to be a leap year, that would be Feb 28, so that is a possible choice for the corrected value. Why not? There simply is no specification of this, and there should be, otherwise how is a new implementation supposed to guess what it should do, and how can an application know what will happen? However, as Don pointed out, all the (known) implementations do it the way you just suggested that it should be done - not because there's any specification that says so, just because everyone copied from the first implementation that made these adjustments. That's fine, and there's no reason at all that we shouldn't specify this behaviour, since it is the de-facto standard. But it is absurd to claim that there's currently anything specified in any standard which requires (or even anticipates) this kind of operation. You most certainly can't get it from "be liberal" which just means not to reject things which don't matter, certainly not to invent meanings for data which is required, but not understood. The next one or two of your examples are just the same kind of thing over and over again, there's no point discussing them - there are reasonable alternative interpretations for the input for all of them, but the ones you picked are the ones that the implementations actually produce (but which, as best I can tell, is undocumented everywhere - even in the various manual pages for specific implementations of mktime(), though you understand, I hope, I have not seen all of those that I expect exist.) | * If you give it tm_isdst=0 for a time when DST is in effect, it gives | you back a positive tm_isdst and alters the other fields appropriately. While it seems that's what the implementations do, this one is even harder to justify from what is written in the standard than the others. For this field, the interpretation of tm_isdst is specified (a little loosely, but still specified) and there are no (on input) out of range values (just < 0, == 0, and > 0 ... which covers everything possible for an integer type field). The "gives you back" part is OK, but the "alters the other fields" is not at all what I would have expected to happen, and certainly isn't specified anywhere. | * If there is a DST transition where 02:00 standard time becomes 03:00 | DST and you give mktime() 02:30 (with negative tm_isdst), it treats | that as either 30 minutes after 02:00 standard time or 30 minutes | before 03:00 DST and gives you back a zero or positive tm_isdst, | respectively, with the tm_hour field altered appropriately. But this one is not the de-facto standard. It is (like the others) clearly not in any written standard, but here, implementations (as you know) differ. This might be what you'd like to see happen, but it is not necessarily what does happen. There are reasons for that - with tm_mday == 0, or -1 or something, we can guess (probably correctly) that the application might have started with an in range tm_mday, and subtracted some number of days, so it is probably trying to find an earlier day. Or if it is set to 33 or something, it has probably added some number of days, and is looking to produce a later date. Each of those is a reasonable assumption, other possibilities might be that in the 0 or -1 case, the program was adding so many days that the int overflowed, and wrapped around - but that's undefined behaviour anyway, so we can ignore that one, or that the application simply set (or failed to set) anything rational, in which case the GIGO principle applies, and it doesn't matter how we interpret it. But as you stated above, with a time in the gap, there's no hint as to why it got that way, or what the application is attempting to achieve. That's why you're allowing "either 30 minutes after 02:00 standard time or 30 minutes before 03:00 DST", because we have no clue at all which is likely to be correct. Doing that, and basing that upon Postel's principle is a total perversion of what it means - it certainly does not mean "interpret any incorrect data that is needed to determine the response in any way you like", that would be ignoring the 2nd half of the principle "be conservative in what you send" - which means to only generate results (replies in networking) which are strictly in conformance with both what is required by the standards, and was specified by the request. | * If a geographical timezone changes its UTC offset [...] This is just a repeat of the previous one, except without tm_isdst issues. | And yes, having listed that last case along with the others, I see no | reason that it should not follow the same principle. The "treats it as" | wording is much the same as the DST transition case. Certainly the reason for a gap in local time shouldn't be relevant, except perhaps in the case where tm_isdst on input is not -1 (where in the seasonal time transition case one might be able to guess what is happening and return an appropriate answer, but with an offset shift, that doesn't work). But neither of those (the two examples you gave of times in gaps) are the same as the earlier ones - they're not out of range values (for the UTC times specified for the XBD 4.17 formula) and they don't produce single unambiguous results, which certainly should be the objective here. Further the earlier ones are all implemented the same way, seemingly, everywhere, and those ones aren't, so if we do decide to actually specify how mktime() is supposed to deal with out of range, and other incorrect, input *timeptrs, then we need to take that into account. | Returning -1 for any of these cases violates the "be liberal in what | you accept" part of the principle. That's utter nonsense, it does nothing of the kind, or not if you really understand the principle, and don't just apply it whenever it seems like it might support your argument. Rather not returning -1 violates the "conservative in what you send" part of the principle. But none of that matters until we discover where in POSIX it is stated that Postel's principle is to be used as one of the rules of interpretation. | I agree it's not clear for pathological cases like that. It comes down | to this statement: | | the tm_yday value used in the expression is the day of the year | from 0 to 365 inclusive, calculated from the other tm structure | members Huh? I know that XBD 4.17 requires tm_yday to be known, so that the formula works - and that only makes any sense to calculate once the rest of the struct tm contains in range values, which strongly implies that the struct tm passed in is to be normalised first, and then the formula is applied to calculate seconds since the epoch. That is, not as you seem to be hoping, that one simply applies that formula to whatever random gibberish is in the struct tm, get a result, and then providing that's within range of a time_t, it is the answer. That simply doesn't work. Apart from that, what does tm_yday have do do with anything at all ? Off list I was also informed that the SysVR4.2 code for mktime() also normalises the struct tm first, and then calculates the time_t - it is really hard to imagine anyone able to implement it any other way. | It may be worth trying to improve this, if implementations all do the | tm_yday calculation the same way, but it has no real relevance in the | matter of whether mktime() can return -1 for "incorrect" broken-down | times. No it doesn't, and I cannot fathom why you're bringing that one up at all. | And I didn't claim that it does. What I said (which you trimmed) was: | | By a strict reading, you may be right, but it is strongly implied by | "shall be set to represent the specified time since the Epoch". In | any case, it is being clarified by bug 1613. | | My point was entirely about this "shall be set to represent" text, i.e. | about what the returned struct tm fields must contain. I'm also unsure why you keep returning to this point. We all agree that when mktime() doesn't return -1, the struct tm must be set to be what localtime() would return for time_t which is returned (which I think is what you added in part of the resolution of bug 1613, though that part had nothing to do with the actual issue reported - no problem with that happening though). There's nothing interesting to say about this any more here, please just drop it. If you want to propose some text for what the *timeptr should be on return when mktime() is indicating an error, then that might be useful, as there's currently nothing at all about that case. But beyond that let's just forget the returned struct value - we all agree what it will be when there's no error, repeating it over and over again achieves nothing. | The context for | this was Don's point that Feb 29 2023 has the tm fields in their stated | ranges and so the standard, as written, allows the returned struct tm | to be left as Feb 29 2023. Yes, I know. The standard as written is useless. | The change in bug 1613 requires them to be | set to the values that would be returned by localtime(), so this will | no longer be allowed. Good. Completely irrelevant to the current issue, which is when mktime() can return an error, as the returned struct tm is only specified when it does not. If an implementation (which the current written standard allows) says "Feb 29 is within the ranges specified in <time.h> so I am not allowed to adjust that, however that day does not exist in the specified year, so I will return an error" then what is in the struct tm on return isn't specified anywhere, and if it still said Feb 29 in that implementation, then fine. That's allowed (currently). | The above quote is all that's needed, provided "the specified time | since the Epoch" is correctly interpreted (which you are not doing). | The time since the Epoch being referred to here is a known integer | value which mktime() is going to return. No, it is not. For some broken down struct tm's, there is no such integer. | (The adjustment to bring struct tm fields | into range is done after this value is known - see below). But that's not what implementations do, and further, is not possible most likely. Hence if the standard seems to be requiring that (which I don't actually believe it does) then it would be wrong, and need fixing. | The sort of | adjustment you were suggesting, "if (t->tm_sec < 0) t->tm_sec = 0", | etc. would cause the fields to no longer represent that time since the | Epoch. Only if that was done after seconds from the Epoch is calculated, which in practice, it never is, and which the standard certainly doesn't require. | No, the adjustment to bring struct tm fields into range is done after | the time since the Epoch value has been calculated. Cannot be. The formula doesn't work. | This is clear just from the order in which things are described on the | mktime() page, Really? Where is that rule of interpretation stated? You forgot to quote that one as well as where use of Postel's principle is mandated. Much more likely (just like all other descriptions) the order of the words is the order in which they make sense, to describe the result. If it was intended to be a requirement of the implementation it would need to say so explicitly, and use "shall" wouldn't it? Does it? You're overreaching - seems like desperation to me. | but also from the use of "Upon successful completion", All that's saying is what will be in the returned *timeptr when mktime() doesn't return an error. It says nothing at all about the order in which things happen inside the implementation. The implementation can make a copy of the *timeptr, adjust that, try converting that, adjust it more, .... lots of things are possible. It doesn't even need to make a copy - provided that when it returns successfully, *timeptr is now correct, what that was at intermediate stages is irrelevant (it might be observable from another thread, but doing so is not defined to be useful). If there's an unsuccessful return ((time_t)-1) then what's in *timeptr isn't specified, so can be anything. | since mktime() can't | know whether it will complete successfully until it has calculated the | time_t value it is going to return. It doesn't matter, since if it doesn't complete successfully, nothing is stated about what is in *timeptr - it might be the original struct tm, it might be that one adjusted as best the implementation was able before determining that an error was going to happen, or it could be almost anything else (including a full set of uninitialised fields, to which any reference would be UB) - the spec simply does not say. You're presuming you know how the implementations work. You're wrong. | > Agreed. But we need to pick tm_isdst = 0 or tm_isdst = 1, and | > which we pick will alter what time_t value gets returned. There's | > nothing anywhere that suggests which one should be selected. | | Correct, and since the standard is silent on this, either behaviour | is allowed. But why just those two, the standard also allows tm_isdst = -1 (anything < 0) on input, why couldn't the implementation assume that one instead, and in that case, we have no guidance, and no reason to pick one or the other. | > No, that's not what that says. I can see you're presuming that the | > implementation calculates a time_t first, and then adjusts the tm to | > match. That's not required. | | Yes it is. See above. No it isn't. See above. | The only real potential for problems here is if an application does | small (less than a day) additions/subtractions using the struct tm | fields and sets tm_isdst=-1. Then it might work fine on one | implementation but get into the kind of loop you described in an | earlier mail on an implementation that behaves the other way. But, as | I pointed out in that earlier discussion, no real application would | do that. Why not? You mean you would not write code that way, but why wouldn't someone else. Where is the specification that says not to? You have already pointed out that C apparently disallows arithmetic on a time_t (except comparison, apparently) so if I want to add 5 minutes to a time_t I need to convert it to a struct tm (and must use localtime() as the only defined way back is mktime()) then use small (less than a day) addition, right? Since I don't know whether, after I have added to the struct tm, whether the result will be summer time or not (it might have transitioned in the period I am skipping over) I should certainly set tm_isdst to -1, that's what the specifications says is appropriate: A positive or 0 value for tm_isdst shall cause mktime( ) to presume initially that Daylight Saving Time, respectively, is or is not in effect for the specified time. A negative value for tm_isdst shall cause mktime( ) to attempt to determine whether Daylight Saving Time is in effect for the specified time. After the adjustment, I don't know whether or not summer time (DST) is or is not in effect, and I want mktime() to attempt to determine that for me, so setting tm_isdst = -1 is exactly the correct thing to do, isn't it? If not, from where is an application programmer supposed to conclude differently? Actually, from where is the application programmer supposed to determine that just adding N to tm_min (and doing nothing else, tm_isdst perhaps excepted) is supposed to return a time_t 5 minutes later than the one that produced the struct tm before the addition? I can find nothing that suggests that works (just cargo cult programming - that code does this, so it must be OK for me to do it too). | The fact that no such problems have come to light in the | last 30 years also means that in practice this is a non-problem. First, you have no way of knowing that no such problems have occurred, you're just guessing, because no-one told you. Further, as the issue certainly would occur rarely, and not repeat itself later when someone wonders why this program went into a loop, that one odd time, it is the kind of thing that your average programmer writes off as a hardware glitch, that their code might have a very rarely seen bug is incomprehensible. They have tested it, many times (of course, always during working hours) and have never seen a problem. Second, if that's to be the general philosophy for POSIX - just do anything which doesn't seem like it causes problems, then it is time for me to abandon POSIX completely, as that's not what I am looking for in a specification. Most others either I'd expect. You wouldn't tolerate this in anything else "works most of the time, so it's OK" is not acceptable, not as a specification, ever. kre