Re: timestamp resolution
On 9 May 2007, at 16:45, Donn Terry wrote: It is simply NOT possible to satisfy both the timestamp-for-system-events needs of the real time people (who really want attosecond resolution -- maybe only 10s or 100s, but really tiny) and the huge range needed for astronomical needs. The system timestamps need to be fast and lightweight, which means an integer or at worst a struct of integers. That doesn't work for astronomical time, which really should be floating point. Aren't you confusing a method of coordinating different time scales with the idea of finding a time scale, suitable for all purposes? I dealt with the former issue, in view of that even within astronomy, it is not possible to find a single time scale, suitable for all purposes. For example, GPS time must introduce GR corrections, and there are several different types of atomic time, depending on whether they are on planet Earth, or somewhere else n the Solar system. Floating point doesn't work right for timestamps (e.g. make) if the mantissa is too small and times that should be distinct, aren't. I'm not going to try to repeat the calculations I did back when, but to cover a range that covers the nearby times (say, +- 2000 years) at the resolution the realtime people need takes more than 64 bits. (66 or 67 if I remember correctly -- 64 is an achievable compromise, but it is a compromise on one end or the other.) So this isn't relevant. One can use several different time scales, for different purposes, but coordinating them against a TAI-JD. Otherwise, the age of the Universe, according to current theories, is about 15 billion years, or about 4.7e17 seconds, can fits within 58 bits, and I doubt, one has need for pinning it down more accurately. :-) And the atomic clocks are only accurate within 10^-7, so nanoseconds expressed as 32-bits would suffice today. But that would just be one time scale: If current UNIX time is adjusted, it would, as is now, ignore past and future leap seconds in the internal counting of seconds, but if future leap seconds or leap whatever do occur, when converted to human time stamps, there would need to be an adjustment. Hans Aberg ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: timestamp resolution
On 10 May 2007, at 15:29, Giacomo A. Catenazzi wrote: I think the specs ignore the issue, so it is only accurate within a couple of ten seconds. I figure typical system just ignore the leap seconds from the epoch, and adjusts the internal clock on the first lookup after the time server has changed. It is these jumps in the internal clock that may pose a problem: it is hard to tell which computer that have adjusted and which have not. What do you mean with internal clock? If you means the hardware clock, ignore it. Systems usually save and restore times in hw clocks only on reboots, so they use the same convention to save and restore times, so it is not a problem (but eventually with a networked file system, with the machine down between leap second and update of the leap-second file). If you just have one computer or a set of computers using the same clock, it really does not make any difference what time one is using, if it only increments. The things I discussed are only of importance in a distributed setting, with a set of independent clocks, that are expected to measure physical time (like with coordination with physical events). There, one can note that physical events are (according to GR) only causal if they are within each others light cones. Light travels about 0.3 m in a nano-second. So time stamps differing with a nano- second can only be separated as physical events at shorter cable distance. It really depend on interface of the hardware clock: if clocks support seconds (since epoch) or if you should use a MM/DD/(YY)YY HH:mm:ss interface, and I doubt POSIX could change such hardware capabilities/ It is the adjustment of the epoch so that it relates to the measurement of physical time. The clock of a typical computer isn't set to the seconds since 1 January, 1970, but to that date adjusted with the number of leap seconds that have occurred since that date, because that is what it gets from the time server. Computer are clearly restricted use the hardware they have. Hans Aberg ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
RE: timestamp resolution
I've been holding back on this one, but can't any longer. Back around the time of the Y2K hoorah, we tried to solve this same set of problems, and failed. The archive of that discussion is on the website. And there's a good reason for that failure, and this discussion is going down that same trap to the same failure. (That failure, by the way, was almost predictable from the prior attempts during the original POSIX work in the 90s to deal with the problem.) It is simply NOT possible to satisfy both the timestamp-for-system-events needs of the real time people (who really want attosecond resolution -- maybe only 10s or 100s, but really tiny) and the huge range needed for astronomical needs. The system timestamps need to be fast and lightweight, which means an integer or at worst a struct of integers. That doesn't work for astronomical time, which really should be floating point. Floating point doesn't work right for timestamps (e.g. make) if the mantissa is too small and times that should be distinct, aren't. I'm not going to try to repeat the calculations I did back when, but to cover a range that covers the nearby times (say, +- 2000 years) at the resolution the realtime people need takes more than 64 bits. (66 or 67 if I remember correctly -- 64 is an achievable compromise, but it is a compromise on one end or the other.) At the time I recommended that any future discussion in this area be explicitly limited to system timestamps (for nearby times). I suggest that that limitation be imposed on this discussion, and that a separate discussion be opened for astronomical times, which would be represented as a separate type. Another alternative is to wait until the languages and hardware have evolved enough to make a larger-than-64-bit integer viable as a timestamp. (That is, as a native type.) (96 or 128 bits doesn't matter.) (Or... what might make more sense: have POSIX require support for such a type on conforming implementations.) (Yeah, I know the politics around that one.) (Whether 96 or 128 bits is enough for astronomical time I haven't tried to figure out.) There's still a while before 2038 (when the current 32 bit timestamps roll over), but the more time we have to get all systems onto something that won't fall apart then, the less Y2K-redux we'll have. Donn -Original Message- From: Hans Aberg [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 09, 2007 4:51 AM To: [EMAIL PROTECTED] Cc: bug-make@gnu.org Subject: Re: timestamp resolution I made a post to the moderated Usenet newsgroup sci.astro.research, Julian day numbers and leap seconds, and there are so many responses, I cannot summarize them here. The question of finding a good time suitable for distributed computing as well as in astronomy and sciences, plus syncing it with legal time is very complicated, and probably requires a committee of scientists and experts from a number of fields. But one suggestion might using say TAI (International Atomic Time, with letters in French grammatical order), or possibly GPS time. It might be synced with say JD counted in seconds so that the difference becomes 0 today. Then leap seconds can be then be viewed as a feature of the UTC legal time. TAI is the mean of highly accurate Cesium atomic clocks (the SI second is defined in terms of Cesium transitions), accuracy better than 10^-7, and is broadcast, so it can be used in computers via radio-controlled clocks (which are now commonly sold). see http:// en.wikipedia.org/wiki/International_Atomic_Time. As for POSIX, I know that in some quarters, for example NASA when there is a satellite launch, one is very concerned over the problem not knowing which time the computers are set to. So it might be good to fix this gap in the POSIX standard. Hans Aberg ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: timestamp resolution
There is a document about JD, as well other time scales, at http://www.ucolick.org/~sla/leapsecs/timescales.html Apparently, JD does not take into account leap seconds (see quote below), which makes it suitable for use in distributed computer systems. If one does not like the high numbers of JD (although in 64- bit, JD seconds would be able to count more that ten times of that the currently modeled universe age), there is a MJD = JD - 240.5. It does not really make sense to discuss second or smaller divisions of time scales, unless one can handle the leap second question properly. So while at it, perhaps this question should be settled as well. :-) Hans Aberg Quote from link above: It should also be noted that the use of JD or MJD for the UTC time scale is problematic and ambiguous at the precision of one second. JD and MJD express the elapsed count of some form of ``day'' as real numbers along a presumably unsegmented, continuous number line. The UTC time scale (and, historically, GMT as used in practical situations before the advent of UTC) contains changes in rate and discontinuities. In particular, there is no obvious way to represent a leap second of UTC (or the smaller leaps present in the available forms of GMT and UTC before 1972) using JD or MJD notation. ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: timestamp resolution
On 4 May 2007, at 07:25, Larry Dwyer wrote: At 07:21 PM 5/3/2007, [EMAIL PROTECTED] wrote: Having a clock number implies that the clock can be used with clock_gettime() or even timer_create(). There is no reason to assume that there is such a clock available. However, if there is such a clock available, being able to call clock_gettime() on it is a *good* thing. In those cases (such as a network filesystem) where it is not available, clock_gettime() can always return an error. As Ulrich points out, an implementation would need a unique clock_id for each file system it supports. Even if this was considered a reasonable solution, we would have to invent a new function, or reuse pathconf(..), to associate the particular clock_id value that corresponds to a particular filesystem. I haven't followed this discussion too closely, but I think one of the best ways to count (calendar) time is using the Julian period used by astronomers: http://en.wikipedia.org/wiki/Julian_day It sets the epoch (time 0.0) to noon UT (Universal Time), January 1, -4712 (= 4713 BC, as the astronomical year 0 = 1 BC). Calendar conversions are easy. (The number of seconds since that date requires 38 bits, I got it to.) Systems that uses a different epoch, merely needs to indicate the time offset. On 4 May 2007, at 00:20, James Youngman wrote: I can't think offhand of an exception. The difference between the MS-DOS epoch and the Unix epoch is evenly divisible by 2 seconds (I think; though there were 9 leap seconds in the 1970s, hmm...). But might there be an exception I can't immediately recall? Quite probably. This would also resolve problems with leap seconds: just do what UT does. There is a proposal to scrap the leap seconds in favor of leap hours, in which case one only needs to take into account the leap seconds introduced so far, as it takes a couple of thousand years until the first leap hour comes by. Hans Aberg ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: timestamp resolution
[ CC += bug-make@gnu.org ] On 5/3/07, Ulrich Drepper [EMAIL PROTECTED] wrote: On today's call we agreed on the following proposal (names are negotiable): 1. add _PC_TIMESTAMP_RESOLUTION [ for those recently joining, this would be a pathconf() configuration option which returns the filesystem timestamp resolution for a given file. ] Is the assumption then that all filesystems record timestamps as values of the form (Epoch + N * pathconf(_PC_TIMESTAMP_RESOLUTION)), where Epoch is the Unix epoch? I can't think offhand of an exception. The difference between the MS-DOS epoch and the Unix epoch is evenly divisible by 2 seconds (I think; though there were 9 leap seconds in the 1970s, hmm...). But might there be an exception I can't immediately recall? Quite probably. 3. handle _PC_TIMESTAMP_RESOLUTION in pathconf() and fpathconf(). The two functions return the timestamp resolution for the file system in nanoseconds. What about rounding rules? For an application to do the right thing with the timestamp of two files, on different filesystems, it will need to know how both filesystems handle timestamp rounding. (hence, CC: bug-make) We don't define network filesystems in POSIX but there should be a note that the underlying filesystems resolution is returned. Presumably, -1 is returned if this cannot be determined? (For example imagine an NFS version Z server serving files from a umsdos filesystem to a modern client, for small values of Z). James. ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: timestamp resolution
Date: Thu, 3 May 2007 23:20:31 +0100 From: James Youngman [EMAIL PROTECTED] Cc: bug-make@gnu.org, [EMAIL PROTECTED] [EMAIL PROTECTED] [ CC += bug-make@gnu.org ] On 5/3/07, Ulrich Drepper [EMAIL PROTECTED] wrote: On today's call we agreed on the following proposal (names are negotiable): 1. add _PC_TIMESTAMP_RESOLUTION [ for those recently joining, this would be a pathconf() configuration option which returns the filesystem timestamp resolution for a given file. ] Is the assumption then that all filesystems record timestamps as values of the form (Epoch + N * pathconf(_PC_TIMESTAMP_RESOLUTION)), where Epoch is the Unix epoch? I can't think offhand of an exception. The difference between the MS-DOS epoch and the Unix epoch is evenly divisible by 2 seconds (I think; though there were 9 leap seconds in the 1970s, hmm...). But might there be an exception I can't immediately recall? Quite probably. Could you please explain why did you CC the bug-make list, and why you mention MS-DOS? The above addition to `pathconf', when introduced, will be only supported on systems that use glibc, is that right? So MS-DOS is not relevant, even if GNU Make uses _PC_TIMESTAMP_RESOLUTION. Btw, the Windows epoch (on NTFS volumes) is 1601. But that probably doesn't matter, either, as glibc is not used. ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
Re: timestamp resolution
(Dropping the austin-group list, since this email just explains context doubtless already known to that list) On 5/4/07, Eli Zaretskii [EMAIL PROTECTED] wrote: Could you please explain why did you CC the bug-make list, and why you mention MS-DOS? Make deals with the timestamps of files, and - I assume - needs to have some awareness of the possibility that timestamp resolution issues will give just-touched files a timestamp which is in the past, and perhaps a timestamp which is _before_ the time which was current immediately prior to the filesystem operation. The above addition to `pathconf', when introduced, will be only supported on systems that use glibc, is that right? So MS-DOS is not relevant, even if GNU Make uses _PC_TIMESTAMP_RESOLUTION. Glibc systems sometimes also support FAT filesystems. Btw, the Windows epoch (on NTFS volumes) is 1601. But that probably doesn't matter, either, as glibc is not used. James. ___ Bug-make mailing list Bug-make@gnu.org http://lists.gnu.org/mailman/listinfo/bug-make
