Re: [O] dates before 1970
On Mar 14, 2011, at 7:12 PM, Achim Gratz wrote:
Nick Dokos nicholas.do...@hp.com writes:
I was referring to the C code implementing current-time (which, btw, has been
changed in latest):
,
| DEFUN (current-time, Fcurrent_time, Scurrent_time, 0, 0, 0,
|doc: /* Return the current time, as the number of seconds since
1970-01-01 00:00:00.
| The time is returned as a list of three integers. The first has the
| most significant 16 bits of the seconds, while the second has the
| least significant 16 bits. The third integer gives the microsecond
| count.
|
| The microsecond count is zero on systems that do not provide
| resolution finer than a second. */)
| (void)
| {
| EMACS_TIME t;
|
| EMACS_GET_TIME (t);
| return list3 (make_number ((EMACS_SECS (t) 16) 0x),
|make_number ((EMACS_SECS (t) 0) 0x),
|make_number (EMACS_USECS (t)));
| }
`
I can't see how this code works correctly unless it is guaranteed that
EMACS_TIME is 32bit unsigned...
Yes, I agree. However, I am just realizing that using 32 bit unsigned
for (current-time) is fine for a loong time to come. So the issue
if *Org* can handle dates outside the safe range is is not about
`current-time' at all, but about what encode-time, decode-time,
time-subtract, time-add etc do, what kind of integers they can handle.
- Carsten
Achim.
--
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+
Waldorf MIDI Implementation additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs
- Carsten
Re: [O] dates before 1970
Hi Bastien, please let me know if you want this patch installed. - Carsten On 13.3.2011, at 21:08, Eric S Fraga wrote: Carsten Dominik carsten.domi...@gmail.com writes: [...] I am attaching a patch which tries to implement some kind of a solution for this problem. The patch introduces a new variable which will allow you to use dates outside the safe range 1970-2037 Thanks Carsten. [...] The patch also introduces a warning with a beep when Org has forced the year, which, I think, was really the main concern in this thread. Yes, at least for me this is key. I can use diary sexps, as you say, for those dates that lie outside the range supported by emacs directly but I need to know when something strange happens! Thanks again, eric -- : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 : using Org-mode version 7.5 (release_7.5.32.gdf26.dirty)
Re: [O] dates before 1970
Hi Carsten, Carsten Dominik carsten.domi...@gmail.com writes: please let me know if you want this patch installed. I won't have time to have a close look at this before tomorrow morning, as I'm still out on a week-end. But please go ahead if you think it's okay. Best, -- Bastien
Re: [O] dates before 1970
On Mar 11, 2011, at 5:30 PM, Nick Dokos wrote: Carsten Dominik carsten.domi...@gmail.com wrote: THis is exactly the point, that it depends on how Emacs was compiled, and what kind of integer is used in the date representation. Signed or unsigend, 32 or 64 bits (I think). For example, Bastien can represent dates before 1970. I cannot. I can represent dates after 2038, Bastien cannot. The work-around is to use diary sexps for dates before 1970, that seems to be safe. And then hope that by 2038, all computers will use 64 bit integers But it's even more than that, no? Emacs's time implementation (current-time, encode/decode etc) would have to change. In fact, this might be the most significant limitation right now: the values they pass around are (hi16 lo16 ms) so they assume that time values are 32 bits, no matter what the underlying implementation says. I use 64-bit Linux on an x86-64 laptop and my time_t is 64 bits (but I don't know if it's signed or unsigned). Time for some experimentation I guess... I believe that on your system you might get (hi48 lo16 ms) You can test if it is signed by trying a date before 1970 (encode-time 0 0 0 1 1 1960) - Carsten
Re: [O] dates before 1970
On Mar 14, 2011, at 4:11 PM, Nick Dokos wrote: Carsten Dominik carsten.domi...@gmail.com wrote: On Mar 11, 2011, at 5:30 PM, Nick Dokos wrote: Carsten Dominik carsten.domi...@gmail.com wrote: THis is exactly the point, that it depends on how Emacs was compiled, and what kind of integer is used in the date representation. Signed or unsigend, 32 or 64 bits (I think). For example, Bastien can represent dates before 1970. I cannot. I can represent dates after 2038, Bastien cannot. The work-around is to use diary sexps for dates before 1970, that seems to be safe. And then hope that by 2038, all computers will use 64 bit integers But it's even more than that, no? Emacs's time implementation (current-time, encode/decode etc) would have to change. In fact, this might be the most significant limitation right now: the values they pass around are (hi16 lo16 ms) so they assume that time values are 32 bits, no matter what the underlying implementation says. I use 64-bit Linux on an x86-64 laptop and my time_t is 64 bits (but I don't know if it's signed or unsigned). Time for some experimentation I guess... I believe that on your system you might get (hi48 lo16 ms) You can test if it is signed by trying a date before 1970 (encode-time 0 0 0 1 1 1960) Ah, OK - thanks! I looked in current-time, saw the 0x mask and I thought that the extra bits are truncated, but apparently not: I need to go back and look at the C rules again. I have no idea what you are talking about :) I get (encode-time 0 0 0 1 1 1960) (-4816 20176) (decode-time '(-4816 20176)) (0 0 0 1 1 1960 5 nil -18000) so it is indeed signed. I just pulled latest emacs and apparently Paul Eggert has been active in making this code more robust, presumably after your conversations on the emacs list. Great. Don't you love OpenSource projects? Thanks, Nick - Carsten
Re: [O] dates before 1970
Carsten Dominik carsten.domi...@gmail.com wrote:
Ah, OK - thanks! I looked in current-time, saw the 0x mask and I
thought that the extra bits are truncated, but apparently not: I need to
go back and look at the C rules again.
I have no idea what you are talking about :)
You think I do? :)
I was referring to the C code implementing current-time (which, btw, has been
changed in latest):
,
| DEFUN (current-time, Fcurrent_time, Scurrent_time, 0, 0, 0,
|doc: /* Return the current time, as the number of seconds since
1970-01-01 00:00:00.
| The time is returned as a list of three integers. The first has the
| most significant 16 bits of the seconds, while the second has the
| least significant 16 bits. The third integer gives the microsecond
| count.
|
| The microsecond count is zero on systems that do not provide
| resolution finer than a second. */)
| (void)
| {
| EMACS_TIME t;
|
| EMACS_GET_TIME (t);
| return list3 (make_number ((EMACS_SECS (t) 16) 0x),
| make_number ((EMACS_SECS (t) 0) 0x),
| make_number (EMACS_USECS (t)));
| }
`
Nick
Re: [O] dates before 1970
Nick Dokos nicholas.do...@hp.com writes:
I was referring to the C code implementing current-time (which, btw, has been
changed in latest):
,
| DEFUN (current-time, Fcurrent_time, Scurrent_time, 0, 0, 0,
|doc: /* Return the current time, as the number of seconds since
1970-01-01 00:00:00.
| The time is returned as a list of three integers. The first has the
| most significant 16 bits of the seconds, while the second has the
| least significant 16 bits. The third integer gives the microsecond
| count.
|
| The microsecond count is zero on systems that do not provide
| resolution finer than a second. */)
| (void)
| {
| EMACS_TIME t;
|
| EMACS_GET_TIME (t);
| return list3 (make_number ((EMACS_SECS (t) 16) 0x),
| make_number ((EMACS_SECS (t) 0) 0x),
| make_number (EMACS_USECS (t)));
| }
`
I can't see how this code works correctly unless it is guaranteed that
EMACS_TIME is 32bit unsigned...
Achim.
--
+[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+
Waldorf MIDI Implementation additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs
Re: [O] dates before 1970
Carsten Dominik carsten.domi...@gmail.com writes: [...] I am attaching a patch which tries to implement some kind of a solution for this problem. The patch introduces a new variable which will allow you to use dates outside the safe range 1970-2037 Thanks Carsten. [...] The patch also introduces a warning with a beep when Org has forced the year, which, I think, was really the main concern in this thread. Yes, at least for me this is key. I can use diary sexps, as you say, for those dates that lie outside the range supported by emacs directly but I need to know when something strange happens! Thanks again, eric -- : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 : using Org-mode version 7.5 (release_7.5.32.gdf26.dirty)
Re: [O] dates before 1970
So I am not sure what 64 bit systems do now or in the future, but it seems that we need to live with a restriction for now. Maybe this should be documented somewhere. - Carsten Most 64-bit systems use a 64-bit int. All of the 64-bit Linux systems that I've used use a signed 64-bit int. Some systems use a 64-bit unsigned int. Some use a double. The only way to know for sure is to look at their definition of time_t in time.h, as provided by the system. http://en.wikipedia.org/wiki/Time_t is as good a starting point as any. The precise words from the Open Group Base standard are: time_t and clock_t shall be integer or real-floating types. The usage of time_t in various functions is specified, but range and type is not defined. R Horn
Re: [O] dates before 1970
Hi everyone, On 11.3.2011, at 09:31, Bastien wrote: Hi, Nick Dokos nicholas.do...@hp.com writes: So I'd guess raising an exception might be the simplest way to deal with this. Here's a patch to try out: This patch has side-effects that Carsten have been recently exploring a bit. Those side-effects seem to depend on how Emacs has been compiled. For now it's best to stick to this restriction. I am attaching a patch which tries to implement some kind of a solution for this problem. The patch introduces a new variable which will allow you to use dates outside the safe range 1970-2037 once you have convinced yourself that all Emacs implementation you are likely to use now and in the future will support the extended range. To be clear, I do not recommend to make use of this, the danger that you will find yourself on a system where this issue is not resolved is not negligible. That is also why I would recommend the default `t' for this variable, and why I will continue to use diary-sexp dates for the few dates where I need a greater range. The patch also introduces a warning with a beep when Org has forced the year, which, I think, was really the main concern in this thread. Here is the docstring of the new variable - it explains what is going on. (defcustom org-read-date-force-compatible-dates t Should date/time prompt force dates that are guaranteed to work in Emacs? Depending on the system Emacs is running on, certain dates cannot be represented with the type used internally to represent time. Dates between 1970-1-1 and 2038-1-1 can always be represented correctly. Some systems allow for earlier dates, some for later, some for both. One way to find out it to insert any date into an Org buffer, putting the cursor on the year and hitting S-up and S-down to test the range. When this variable is set to t, the date/time prompt will not let you specify dates outside the 1970-2037 range, so it is certain that these dates will work in whatever version of Emacs you are running, and also that you can move a file from one Emacs implementation to another. WHenever Org is forcing the year for you, it will display a message and beep. When this variable is nil, Org will check if the date is representable in the specific Emacs implementation you are using. If not, it will force a year, usually the current year, and beep to remind you. Currently this setting is not recommended because the likelihood that you will open your Org files in an Emacs that has limited date range is not negligible. A workaround for this problem is to use diary sexp dates for time stamps outside of this range. :group 'org-time :type 'boolean) 0001-Improve-handling-of-years-with-limitations-on-repres.patch Description: Binary data
Re: [O] dates before 1970
Hi, Nick Dokos nicholas.do...@hp.com writes: So I'd guess raising an exception might be the simplest way to deal with this. Here's a patch to try out: This patch has side-effects that Carsten have been recently exploring a bit. Those side-effects seem to depend on how Emacs has been compiled. For now it's best to stick to this restriction. Best, -- Bastien
Re: [O] dates before 1970
On Mar 11, 2011, at 9:31 AM, Bastien wrote: Hi, Nick Dokos nicholas.do...@hp.com writes: So I'd guess raising an exception might be the simplest way to deal with this. Here's a patch to try out: This patch has side-effects that Carsten have been recently exploring a bit. Those side-effects seem to depend on how Emacs has been compiled. For now it's best to stick to this restriction. But beeping or so to alert the user that a date is being changed behind his back might be a good idea. I think we should ask on emacs-devel what the official position of Emacs development is regarding non-representable times. - Carsten
Re: [O] dates before 1970
Nick Dokos nicholas.do...@hp.com writes: Eric S Fraga e.fr...@ucl.ac.uk wrote: This is a sort of bug report but possibly more a curiosity... I imagine this has something to do with time 0 in Unix but I cannot seem to be able to enter any date earlier than 1 Jan 1970 using C-c! (say). However, once I have entered a date (later than that), I can use S-down on the year to get to the date I want. This seems rather inconsistent? To be precise, I get the wrong date recorded if I try: C-c ! 1968-12-10 RET (where C-c ! is =org-time-stamp-inactive=). The result is =[2011-12-10 Sat]= The bug is not so much that I cannot input dates I want but that the inactive timestamp generated is *incorrect* and yet there is no error message. Good one! The culprit is org-read-date-analyze which near the end contains this snippet of code: , | ... | (if ( year 100) (setq year (+ 2000 year))) | (if ( year 1970) (setq year (nth 5 defdecode))) ; not representable | (setq org-read-date-analyze-futurep futurep) | (list second minute hour day month year))) ` The trouble is that the caller (org-read-date) takes the result and does a round-trip through the emacs time encode/decode functions to make sure the result is sane. Dates before 1970 would break that (I get (0 9 10 26 11 2033 6 nil -18000)) so it seems it wraps around to 2033 or so). Yes, that makes sense. In addition, most callers of org-read-date call it with a non-nil to-time argument: that makes it return an emacs-encoded time (which is then manipulated as such and which I believe has to satisfy the =1970 requirement). So I'd guess raising an exception might be the simplest way to deal with this. Here's a patch to try out: This seems to work fine. Thanks. I am glad, however, that I can enter any date and then use the S-down etc. keys to get the date I want. Of course, I am not sure if anything else in org breaks as a result... org-sparse-tree with very old scheduled dates seems to work. Haven't tried much else and I would guess few would notice? Thanks again, eric -- : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 : using Org-mode version 7.5 (release_7.5.27.gefa56.dirty)
Re: [O] dates before 1970
On Mar 11, 2011, at 9:47 AM, Eric S Fraga wrote: Nick Dokos nicholas.do...@hp.com writes: Eric S Fraga e.fr...@ucl.ac.uk wrote: This is a sort of bug report but possibly more a curiosity... I imagine this has something to do with time 0 in Unix but I cannot seem to be able to enter any date earlier than 1 Jan 1970 using C-c! (say). However, once I have entered a date (later than that), I can use S-down on the year to get to the date I want. This seems rather inconsistent? To be precise, I get the wrong date recorded if I try: C-c ! 1968-12-10 RET (where C-c ! is =org-time-stamp-inactive=). The result is =[2011-12-10 Sat]= The bug is not so much that I cannot input dates I want but that the inactive timestamp generated is *incorrect* and yet there is no error message. Good one! The culprit is org-read-date-analyze which near the end contains this snippet of code: , | ... | (if ( year 100) (setq year (+ 2000 year))) | (if ( year 1970) (setq year (nth 5 defdecode))) ; not representable | (setq org-read-date-analyze-futurep futurep) | (list second minute hour day month year))) ` The trouble is that the caller (org-read-date) takes the result and does a round-trip through the emacs time encode/decode functions to make sure the result is sane. Dates before 1970 would break that (I get (0 9 10 26 11 2033 6 nil -18000)) so it seems it wraps around to 2033 or so). Yes, that makes sense. In addition, most callers of org-read-date call it with a non-nil to-time argument: that makes it return an emacs-encoded time (which is then manipulated as such and which I believe has to satisfy the =1970 requirement). So I'd guess raising an exception might be the simplest way to deal with this. Here's a patch to try out: This seems to work fine. Thanks. I am glad, however, that I can enter any date and then use the S-down etc. keys to get the date I want. Of course, I am not sure if anything else in org breaks as a result... org-sparse-tree with very old scheduled dates seems to work. Haven't tried much else and I would guess few would notice? THis is exactly the point, that it depends on how Emacs was compiled, and what kind of integer is used in the date representation. Signed or unsigend, 32 or 64 bits (I think). For example, Bastien can represent dates before 1970. I cannot. I can represent dates after 2038, Bastien cannot. The work-around is to use diary sexps for dates before 1970, that seems to be safe. And then hope that by 2038, all computers will use 64 bit integers - Carsten
Re: [O] dates before 1970
Carsten Dominik carsten.domi...@gmail.com writes: [...] THis is exactly the point, that it depends on how Emacs was compiled, and what kind of integer is used in the date representation. Signed or unsigend, 32 or 64 bits (I think). Yes, that makes sense. For example, Bastien can represent dates before 1970. I cannot. I can represent dates after 2038, Bastien cannot. I can do before 1970 but not after 2038. Interesting. -- : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 : using Org-mode version 7.5 (release_7.5.27.gefa56)
Re: [O] dates before 1970
Hi, I asked in emacs-develop and got: by Andreas Schwabon 2011-03-11T13:19:43+00:00. If your system's time-t is a signed 32-bit integer then your system is able to represent times between 1901-12-13 20:45:53 UTC and 2038-01-19 03:14:07 UTC. If your system's time-t is an unsigned 32-bit integer your system can represent times between 1970-01-01 00:00:00 UTC and 2106-02-07 06:28:15 UTC. Andreas. So I am not sure what 64 bit systems do now or in the future, but it seems that we need to live with a restriction for now. Maybe this should be documented somewhere. - Carsten On Mar 11, 2011, at 1:00 PM, Eric S Fraga wrote: Carsten Dominik carsten.domi...@gmail.com writes: [...] THis is exactly the point, that it depends on how Emacs was compiled, and what kind of integer is used in the date representation. Signed or unsigend, 32 or 64 bits (I think). Yes, that makes sense. For example, Bastien can represent dates before 1970. I cannot. I can represent dates after 2038, Bastien cannot. I can do before 1970 but not after 2038. Interesting. -- : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 : using Org-mode version 7.5 (release_7.5.27.gefa56) - Carsten
Re: [O] dates before 1970
Eric S Fraga e.fr...@ucl.ac.uk wrote: So I'd guess raising an exception might be the simplest way to deal with this. Here's a patch to try out: This seems to work fine. Thanks. Maybe not - see Bastien's mail. I am glad, however, that I can enter any date and then use the S-down etc. keys to get the date I want. Of course, I am not sure if anything else in org breaks as a result... org-sparse-tree with very old scheduled dates seems to work. Haven't tried much else and I would guess few would notice? That's the problem: one does not know whether the way from one date to another passes through the quicksand of internal emacs time. And as you say, any effects might escape notice. Nick
Re: [O] dates before 1970
Carsten Dominik carsten.domi...@gmail.com wrote: THis is exactly the point, that it depends on how Emacs was compiled, and what kind of integer is used in the date representation. Signed or unsigend, 32 or 64 bits (I think). For example, Bastien can represent dates before 1970. I cannot. I can represent dates after 2038, Bastien cannot. The work-around is to use diary sexps for dates before 1970, that seems to be safe. And then hope that by 2038, all computers will use 64 bit integers But it's even more than that, no? Emacs's time implementation (current-time, encode/decode etc) would have to change. In fact, this might be the most significant limitation right now: the values they pass around are (hi16 lo16 ms) so they assume that time values are 32 bits, no matter what the underlying implementation says. I use 64-bit Linux on an x86-64 laptop and my time_t is 64 bits (but I don't know if it's signed or unsigned). Time for some experimentation I guess... Nick
Re: [O] dates before 1970
Eric S Fraga e.fr...@ucl.ac.uk wrote: This is a sort of bug report but possibly more a curiosity... I imagine this has something to do with time 0 in Unix but I cannot seem to be able to enter any date earlier than 1 Jan 1970 using C-c! (say). However, once I have entered a date (later than that), I can use S-down on the year to get to the date I want. This seems rather inconsistent? To be precise, I get the wrong date recorded if I try: C-c ! 1968-12-10 RET (where C-c ! is =org-time-stamp-inactive=). The result is =[2011-12-10 Sat]= The bug is not so much that I cannot input dates I want but that the inactive timestamp generated is *incorrect* and yet there is no error message. Good one! The culprit is org-read-date-analyze which near the end contains this snippet of code: , | ... | (if ( year 100) (setq year (+ 2000 year))) | (if ( year 1970) (setq year (nth 5 defdecode))) ; not representable | (setq org-read-date-analyze-futurep futurep) | (list second minute hour day month year))) ` The trouble is that the caller (org-read-date) takes the result and does a round-trip through the emacs time encode/decode functions to make sure the result is sane. Dates before 1970 would break that (I get (0 9 10 26 11 2033 6 nil -18000)) so it seems it wraps around to 2033 or so). In addition, most callers of org-read-date call it with a non-nil to-time argument: that makes it return an emacs-encoded time (which is then manipulated as such and which I believe has to satisfy the =1970 requirement). So I'd guess raising an exception might be the simplest way to deal with this. Here's a patch to try out: --8---cut here---start-8--- diff --git a/lisp/org.el b/lisp/org.el index 92f2406..b9acf11 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -14718,7 +14718,8 @@ user. (nth 2 tl)) (setq org-time-was-given t)) (if ( year 100) (setq year (+ 2000 year))) -(if ( year 1970) (setq year (nth 5 defdecode))) ; not representable +;(if ( year 1970) (setq year (nth 5 defdecode))) ; not representable +(if ( year 1970) (error Year must be = 1970)) (setq org-read-date-analyze-futurep futurep) (list second minute hour day month year))) --8---cut here---end---8--- I think it does not break anything but I'm not sure I like it much. Patchwork note: this should not be applied without a lot more thought and experimentation. Nick
