Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 01/05/2022 09:32, Paul Eggert wrote:

On 4/30/22 04:22, Max Nikulin wrote:


I posted a corrected version of my `org-encode-time' macro, but I did 
not add you to Cc (I sent reply through news.gmane.io), and it has no 
special case to check whether `encode-time' supports 6 elements list 
argument:


As I understand it, org-encode-time is intended to be a compatibility 
function like org-newline-and-indent and org-string-distance. Those are 
in org-compat.el, so I assumed org-encode-time would be there too.


Maybe you are right. I believe that it should do a bit more than just 
ensure compatibility. An additional goal is to avoid pitfall with list 
vs. separate arguments result discrepancy:


(format-time-string
 "%F %T %z %Z"
 (encode-time 0 30 23 31 3 2022 nil nil "Europe/Madrid")
 "Europe/Madrid")
"2022-03-31 23:30:00 +0200 CEST"

(format-time-string
 "%F %T %z %Z"
 (encode-time '(0 30 23 31 3 2022 nil nil "Europe/Madrid"))
 "Europe/Madrid")
"2022-04-01 00:30:00 +0200 CEST"

Also, if the intent is to emulate Emacs 29 encode-time, can't we do 
something like the attached instead? The idea is to implement Emacs 29 
encode-time both on pre-29 Emacs (that don't support lists of length 6) 
and post-29 Emacs (which might drop support for the obsolescent form).



+(if (ignore-errors (encode-time '(0 0 0 1 1 1971)))
+(if (ignore-errors (encode-time 0 0 0 1 1 1971))
+(defalias 'org-encode-time #'encode-time)
+  (defun org-encode-time (time &rest args)
+(encode-time (if args (cons time args) time
+  (defun org-encode-time (time &rest args)
+(if args
+(apply #'encode-time time args)
+  (apply #'encode-time time


1. I would prefer macro since it works at compile or load time, so 
runtime impact is minimal.
2. Your variant may be fixed, but currently I do not like behavior for 
Emacs-27. Compare encode-time and org-encode-time with new calling style:


(format-time-string
 "%F %T %z %Z"
 (encode-time '(0 30 23 31 3 2022 nil nil "Europe/Madrid"))
 "Europe/Madrid")
"2022-04-01 00:30:00 +0200 CEST"

(format-time-string
 "%F %T %z %Z"
 (org-encode-time '(0 30 23 31 3 2022 nil nil "Europe/Madrid"))
 "Europe/Madrid")
"2022-03-31 23:30:00 +0200 CEST"

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/30/22 04:22, Max Nikulin wrote:


I posted a corrected version of my `org-encode-time' macro, but I did 
not add you to Cc (I sent reply through news.gmane.io), and it has no 
special case to check whether `encode-time' supports 6 elements list 
argument:


Thanks, I looked at that and have a couple of questions.

As I understand it, org-encode-time is intended to be a compatibility 
function like org-newline-and-indent and org-string-distance. Those are 
in org-compat.el, so I assumed org-encode-time would be there too.


Also, if the intent is to emulate Emacs 29 encode-time, can't we do 
something like the attached instead? The idea is to implement Emacs 29 
encode-time both on pre-29 Emacs (that don't support lists of length 6) 
and post-29 Emacs (which might drop support for the obsolescent form).
From 2f44ee7524e5b2e53f912cff1276f7817495c657 Mon Sep 17 00:00:00 2001
From: Paul Eggert 
Date: Sat, 30 Apr 2022 19:27:15 -0700
Subject: [PATCH] org-encode-time compatibility function

* lisp/org/org-compat.el (org-encode-time): New function.
---
 lisp/org/org-compat.el | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/lisp/org/org-compat.el b/lisp/org/org-compat.el
index 3e394fbab1..0a0025fa0d 100644
--- a/lisp/org/org-compat.el
+++ b/lisp/org/org-compat.el
@@ -144,6 +144,16 @@ org-file-has-changed-p--hash-table
   (defun org-time-convert-to-list (time)
 (seconds-to-time (float-time time
 
+(if (ignore-errors (encode-time '(0 0 0 1 1 1971)))
+(if (ignore-errors (encode-time 0 0 0 1 1 1971))
+(defalias 'org-encode-time #'encode-time)
+  (defun org-encode-time (time &rest args)
+(encode-time (if args (cons time args) time
+  (defun org-encode-time (time &rest args)
+(if args
+(apply #'encode-time time args)
+  (apply #'encode-time time
+
 ;; `newline-and-indent' did not take a numeric argument before 27.1.
 (if (version< emacs-version "27")
 (defsubst org-newline-and-indent (&optional _arg)
-- 
2.34.1

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 26/04/2022 02:49, Paul Eggert wrote:

On 4/25/22 08:37, Paul Eggert wrote:

I'll be happy to review the revised org-encode-time implementation, 
whenever you think it could use a review. (Sorry, I've lost track of 
what the proposal is.)


I suspended my activity due to discussions of other changes and waiting 
for commits related to your fixes of `org-parse-time-string' and 
`org-store-link' that do not require introducing of `org-encode-time-1'. 
I mean excerpts from 
https://debbugs.gnu.org/cgi/bugreport.cgi?att=1;msg=10;bug=54764;filename=0001-Improve-Org-usage-of-timestamps.patch


I posted a corrected version of my `org-encode-time' macro, but I did 
not add you to Cc (I sent reply through news.gmane.io), and it has no 
special case to check whether `encode-time' supports 6 elements list 
argument:


Max Nikulin to emacs-orgmode. [DRAFT][PATCH v2] org-encode-time 
compatibility and convenience helper. Sun, 24 Apr 2022 18:34:40 +0700. 
https://list.orgmode.org/t43cki$ct$1...@ciao.gmane.io


In my drafts I have the following changes in tests related to 
`org-parse-time-string':


diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index 6aecc3af8..551d17d64 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -268,15 +268,15 @@
 (ert-deftest test-org/org-parse-time-string ()
   "Test `org-parse-time-string'."
   (should (equal (org-parse-time-string "2012-03-29 16:40")
-'(0 40 16 29 3 2012 nil nil nil)))
+'(0 40 16 29 3 2012 nil -1 nil)))
   (should (equal (org-parse-time-string "[2012-03-29 16:40]")
-'(0 40 16 29 3 2012 nil nil nil)))
+'(0 40 16 29 3 2012 nil -1 nil)))
   (should (equal (org-parse-time-string "<2012-03-29 16:40>")
-'(0 40 16 29 3 2012 nil nil nil)))
+'(0 40 16 29 3 2012 nil -1 nil)))
   (should (equal (org-parse-time-string "<2012-03-29>")
-'(0 0 0 29 3 2012 nil nil nil)))
+'(0 0 0 29 3 2012 nil -1 nil)))
   (should (equal (org-parse-time-string "<2012-03-29>" t)
-'(0 nil nil 29 3 2012 nil nil nil
+'(0 nil nil 29 3 2012 nil -1 nil

 (ert-deftest test-org/closest-date ()
   "Test `org-closest-date' specifications."

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/25/22 08:37, Paul Eggert wrote:
Yes, I plan to omit the patches that were objected to, and install the 
rest. Once that's done you should be good to go for Org. (Alas my 
workstation died over the weekend, but I should have things up and 
running again soon...)


Got my workstation up, installed the patches into Emacs master, and am 
closing the Emacs bug report.


I'll be happy to review the revised org-encode-time implementation, 
whenever you think it could use a review. (Sorry, I've lost track of 
what the proposal is.)

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

Yes, I plan to omit the patches that were objected to, and install the 
rest. Once that's done you should be good to go for Org. (Alas my 
workstation died over the weekend, but I should have things up and 
running again soon...)

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 21/04/2022 01:29, Paul Eggert wrote:

On 4/20/22 08:07, Max Nikulin wrote:

I still believe that optional DST and ZONE values is an improvement of 
the `encode-time' interface with no real drawbacks.


Yes, that's the direction we're headed.


Paul, a week has passed since you posted the patch series. Unlike the 
changes related to timestamp representation, nobody argued concerning 6 
element list argument of `encode-time':


[PATCH 1/6] Support (encode-time (list s m h D M Y))

Unless I have missed something, this patch can be pushed without other 
ones and it will be a step forward.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Bernhard Voelker]

On 4/20/22 20:19, Paul Eggert wrote:
> diff --git a/lib/gettime-res.c b/lib/gettime-res.c
> index 611f83ad27..bb4d0b191d 100644
> --- a/lib/gettime-res.c
> +++ b/lib/gettime-res.c
> @@ -53,6 +53,8 @@ gettime_res (void)
>
>long int hz = TIMESPEC_HZ;
>long int r = hz * res.tv_sec + res.tv_nsec;
> +  struct timespec earlier;
> +  earlier.tv_nsec = -1;
>
>/* On some platforms, clock_getres (CLOCK_REALTIME, ...) yields a
>   too-large resolution, under the mistaken theory that it should
> @@ -61,9 +63,22 @@ gettime_res (void)
>   resolution.  Work around the problem with high probability by
>   trying clock_gettime several times and observing the resulting
>   bounds on resolution.  */
> -  for (int i = 0; 1 < r && i < 32; i++)
> +  int nsamples = 32;
> +  for (int i = 0; 1 < r && i < nsamples; i++)
>  {
> -  struct timespec now = current_timespec ();
> +  /* If successive timestamps disagree the clock resolution must
> + be small, so exit the inner loop to check this sample.
> + Otherwise, arrange for the outer loop to exit but continue
> + the inner-loop search for a differing timestamp sample.  */
> +  struct timespec now;
> +  for (;; i = nsamples)
> +{
> +  now = current_timespec ();
> +  if (earlier.tv_nsec != now.tv_nsec || earlier.tv_sec != now.tv_sec)
> +break;
> +}
> +  earlier = now;
> +
>r = gcd (r, now.tv_nsec ? now.tv_nsec : hz);
>  }

lib/gettime-res.c: In function 'gettime_res':
lib/gettime-res.c:77:46: error: 'earlier.tv_sec' may be used uninitialized in 
this function \
[-Werror=maybe-uninitialized]
   77 |   if (earlier.tv_nsec != now.tv_nsec || earlier.tv_sec != 
now.tv_sec)
  |   
~~~^~~


We know that earlier.tv_sec is set when tv_nsec is set, but the compiler does 
not,
obviously.  Considering the nested loops, I'd say initializing tv_sec doesn't
harm performance-wise.

Have a nice day,
Berny

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Eli Zaretskii]

> Date: Thu, 21 Apr 2022 16:56:25 -0700
> Cc: maniku...@gmail.com, emacs-orgmode@gnu.org, 54...@debbugs.gnu.org,
>  bug-gnu...@gnu.org
> From: Paul Eggert 
> 
> What appears to be happening here is that the MS-Windows native 
> timestamp resolution is 1/64th of a second, and your system's clock is 
> offset by 0.0075 s from an integer boundary. I.e., the timestamps in 
> increasing order are:
> 
>...
>1650522862 + 62/64 + 0.0075 = 1650522862.976250
>1650522862 + 63/64 + 0.0075 = 1650522862.991875
>1650522863 +  0/64 + 0.0075 = 1650522863.007500
>1650522863 +  1/64 + 0.0075 = 1650522863.023125
>1650522863 +  2/64 + 0.0075 = 1650522863.038750
>...
> 
> and the system clock never returns a timestamp on an integer boundary 
> (i.e., tv_nsec is never zero).
> 
> We have two options to express this as Emacs timestamps:
> 
> (1) We can keep information about resolution but lose information about 
> time, by using a resolution of 15.625 ms (i.e., 1/64 s) and truncating 
> timestamps to the nearest 1/64 second.  This would generate the 
> following (TICKS . HZ) timestamps:
> 
>...
>(105633463230 . 64) = 1650522862 + 62/64 = 1650522862.968750
>(105633463231 . 64) = 1650522862 + 63/64 = 1650522862.984375
>(105633463232 . 64) = 1650522863 +  0/64 = 1650522863.00
>(105633463233 . 64) = 1650522863 +  1/64 = 1650522863.015625
>(105633463234 . 64) = 1650522863 +  2/64 = 1650522863.031250
>...
> 
> (2) We can keep information about time but lose information about the 
> resolution, by using a resolution of 0.625 ms (i.e., HZ = 10 / 
> 625000 = 16000). (We use 0.625 ms because it is the coarsest resolution 
> that does not lose time info.) This would generate the following (TICKS 
> . HZ) timestamps:
> 
>...
>(2640836580762 . 1600) = 1650522862 + 1562/1600 = 1650522862.976250
>(2640836580762 . 1600) = 1650522862 + 1587/1600 = 1650522862.991875
>(2640836580762 . 1600) = 1650522863 +   12/1600 = 1650522863.007500
>(2640836580762 . 1600) = 1650522863 +   37/1600 = 1650522863.023125
>(2640836580762 . 1600) = 1650522863 +   62/1600 = 1650522863.038750
>...
> 
> The patch does (2), and this explains the "gettime_res returned 625000 
> ns" in your output.
> 
> It shouldn't be hard to change the patch to do (1), if desired. I doubt 
> whether users will care one way or the other.

These are very fine details of the implementation, which we can get
back to later.  I would like first to discuss the more general issue
of basing the design on such tests, and on the notion of "clock
resolution" as expressed by these tests.  TBH, what you propose makes
no sense to me for now, and for some reason you didn't answer my more
general questions about that, but instead preferred to respond only to
their secondary aspects.

At this point, I object to any changes in this area until we discuss
the more general aspects of this design and decide whether we agree
with it.  Such a discussion should be on emacs-devel, so I move this
there; please continue the discussion there.

Thanks.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

What appears to be happening here is that the MS-Windows native 
timestamp resolution is 1/64th of a second, and your system's clock is 
offset by 0.0075 s from an integer boundary. I.e., the timestamps in 
increasing order are:


  ...
  1650522862 + 62/64 + 0.0075 = 1650522862.976250
  1650522862 + 63/64 + 0.0075 = 1650522862.991875
  1650522863 +  0/64 + 0.0075 = 1650522863.007500
  1650522863 +  1/64 + 0.0075 = 1650522863.023125
  1650522863 +  2/64 + 0.0075 = 1650522863.038750
  ...

and the system clock never returns a timestamp on an integer boundary 
(i.e., tv_nsec is never zero).


We have two options to express this as Emacs timestamps:

(1) We can keep information about resolution but lose information about 
time, by using a resolution of 15.625 ms (i.e., 1/64 s) and truncating 
timestamps to the nearest 1/64 second.  This would generate the 
following (TICKS . HZ) timestamps:


  ...
  (105633463230 . 64) = 1650522862 + 62/64 = 1650522862.968750
  (105633463231 . 64) = 1650522862 + 63/64 = 1650522862.984375
  (105633463232 . 64) = 1650522863 +  0/64 = 1650522863.00
  (105633463233 . 64) = 1650522863 +  1/64 = 1650522863.015625
  (105633463234 . 64) = 1650522863 +  2/64 = 1650522863.031250
  ...

(2) We can keep information about time but lose information about the 
resolution, by using a resolution of 0.625 ms (i.e., HZ = 10 / 
625000 = 16000). (We use 0.625 ms because it is the coarsest resolution 
that does not lose time info.) This would generate the following (TICKS 
. HZ) timestamps:


  ...
  (2640836580762 . 1600) = 1650522862 + 1562/1600 = 1650522862.976250
  (2640836580762 . 1600) = 1650522862 + 1587/1600 = 1650522862.991875
  (2640836580762 . 1600) = 1650522863 +   12/1600 = 1650522863.007500
  (2640836580762 . 1600) = 1650522863 +   37/1600 = 1650522863.023125
  (2640836580762 . 1600) = 1650522863 +   62/1600 = 1650522863.038750
  ...

The patch does (2), and this explains the "gettime_res returned 625000 
ns" in your output.


It shouldn't be hard to change the patch to do (1), if desired. I doubt 
whether users will care one way or the other.



> don't we use time values for file timestamps?

Yes, but file timestamps should use the resolution of the file system, 
which in general is different from the resolution of the system clock. 
That's a separate matter, which would be the subject of a separate 
patch. For now we can stick with what we already have in that department.



> And for Windows, all this does is measure the "resolution" of the
> Gnulib emulation of timespec functions on MS-Windows, it tells nothing
> about the real resolution of the system time values.

If Emacs Lisp code (which currently is based on the Gnulib code) can see 
only (say) 1-microsecond timestamps, then it doesn't matter that the 
underlying system clock has (say) 1-nanosecond precision. Of course it 
would be better for Emacs to see the system timestamp resolution, and if 
we can get the time of day on MS-Windows to a precision better than 1/64 
second then I assume Emacs should do that. Once it does, the patch 
should calculate a higher HZ value to tell users about the improved 
resolution.



> if the "time resolution" determined by this procedure
> is different between two systems, does it mean that two time values
> that are 'equal' on one of them could be NOT 'equal' on another?

Sure, but the traditional (HIGH LOW MICROSEC PICOSEC) representation has 
the same issue. For example, if A's clock has 1 ms resolution and B's 
clock has 10 ms resolution, A's (time-convert nil 'list) called twice 
would return (say) the two timestamps (25184 64239 1000 0) and (25184 
64239 1001 0) at the same moments that B's calls would return (25184 
64239 1000 0) twice. A would say that the two timestamps differ; B would 
say they're the same.


This sort of disagreement is inherent to how timestamp resolution works. 
It doesn't matter whether the timestamps are represented by (HIGH LOW 
MICROSEC PICOSEC) or by (TICKS . HZ); users will run into the same 
problem in both cases.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 17/04/2022 02:23, Paul Eggert wrote:

On 4/15/22 10:23, Max Nikulin wrote:

if you are storing future events bound to wall time then namely time 
zone identifier should have precedence.


Although that would make sense for some applications it's not a good 
idea in general. For example, if you're scheduling a Zoom meeting you 
should save both, because other meeting participants may interpret 
strings like "Pacific/Apia" differently.


I would say that in such cases there is a primary time zone for such 
event and secondary time zones of other participants. Time transitions 
in the primary time zone (unless it is UTC that is the reference) affect 
participants from all other time zones. If some secondary time zone is 
changed then it affects only wall time in this particular time zone. So 
primary timezone and offsets in all time zones should be stored for user 
convenience and to figure out which notification should be sent after 
introducing new rules for some time zones.


Just identifier may be ambiguous around DST transition. So timezone 
abbreviations are ambiguous per se but when identifiers are known they 
may be still necessary to resolve uncertainties for backward time 
shifts. At certain moment the Olson DB started to use "+04" 
abbreviations instead of letters for transitions unrelated to daylight 
saving time.


Yes, timezone names like "Europe/Lisbon" are ambiguous during fallback 
transitions, or if the meaning of "Europe/Lisbon" changes. This is why 
one should also save UT offsets when generating localtime timestamps.


Before/after time transition around the date may be more meaningful for 
people. Local tradition may use other reference than Greenwich.


Around five years ago I changed TZDB to numeric use time zone 
abbreviations like "+04" instead of my inventions like "GET", because I 
wanted TZDB to follow existing practice, not invent it. A nice side 
effect is that numeric abbreviations are unambiguous. (Besides, even _I_ 
couldn't remember what "GET" meant. :-)


Numerical abbreviation broke parsers in stable linux distribution, e.g. 
a patch for Qt required in addition to tzdata update. I do not remember 
details, but removed old-style abbreviations caused some problems as 
well. I may be wrong concerning usage of such abbreviation in the 
postgres parser and earlier generated text timestamps. On the other hand 
an abbreviation for a timezone with evolved offset significantly 
contributes to uncertainties and does not help to resolve ambiguity 
around time shift.



And WET/WEST gets another bit of info in addition to numerical offset.


That info is meant only for users; I wouldn't rely on it for 
calculations because those abbreviations are ambiguous. It could well 
be, for example that the meaning of "PST" in the United States will 
change in the near future.


I agree that abbreviations are ambiguous when considered globally. When 
constrained to particular location (time zone) and moment of time, they 
may provide additional bit of information that is more convenient for 
users and enough to resolve ambiguity. It is not a general rule, 
sometimes uncertainty remains even when abbreviation is known.


I do not remember if it is possible at all to obtain using libc the 
period of constant time offset, when time shift value is valid. 
Sometimes it is necessary to recalculate offset.


Sorry, I don't understand this point. One can easily recalculate the UT 
offset in Emacs Lisp by generating the desired timestamp and calling 
decode-time on it. You surely are talking about something else, but I 
don't know what it is.


Let's assume Europe/Lisbon timezone. `decode-time' for today gives just 
+0100. It tells nothing if I need to process some thousands of 
timestamps for yesterday and past week. If some function returns "+0100 
for given timestamp and the same offset is valid for Europe/Lisbon since 
Sun Mar 27 01:00:00 2022 UT = Sun Mar 27 02:00:00 2022 WEST till Sun Oct 
30 00:59:59 2022 UT = Sun Oct 30 01:59:59 2022 WEST" then I can process 
whole bunch without any worry concerning non-existing or ambiguous time, 
extra or lost hour in time intervals. mktime should have all this 
information during intermediate calculations but it does not expose it 
to callers.


Interface of mktime is suitable for conversion of isolated timestamps. 
For bunch of related data there is enough room for optimizing.


You wrote that "2021-01-31 23:30:00 +0300" is parsed correctly. My 
opinion is that when time zone is known to be Africa/Juba (system-wide 
setting, environment variable, or an argument of the parsing function) 
then "2021-01-31 23:30:00 CAT" and "2021-01-31 23:30:00 EAT" should be 
parsed correctly (and localized date-time formats should be parsed as 
well).


That's not possible in general, since the two abbreviations can be the 
same. Traditionally in Australia, for example, "CST" meant both "Central 
Standard Time" and "Central Summer Time", and there are probably still 
apps 

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Eli Zaretskii]

> Date: Wed, 20 Apr 2022 17:11:34 -0700
> Cc: maniku...@gmail.com, emacs-orgmode@gnu.org, 54...@debbugs.gnu.org,
>  bug-gnu...@gnu.org
> From: Paul Eggert 
> 
> On 4/20/22 12:30, Eli Zaretskii wrote:
> 
> > I see the time samples change in jumps of 15 msec.
> 
> Could you give the first part of the output? I would like to see what 
> the the samples are jumping from and to, and how often they jump.

That "first part", as I understand what you wanted, would be too long
and tedious to examine, as the value changes once every 5500 lines.
So I've modified the test program to print the time only when it
changes, and here's the output:

  gettime_res returned 625000 ns
  time = 1650522863.03875
  time = 1650522863.054375000
  time = 1650522863.07000
  time = 1650522863.085625000
  time = 1650522863.10125
  time = 1650522863.116875000
  time = 1650522863.13250
  time = 1650522863.148125000
  time = 1650522863.16375
  time = 1650522863.179375000
  time = 1650522863.19500
  time = 1650522863.210625000
  time = 1650522863.22625
  time = 1650522863.241875000
  time = 1650522863.25750
  time = 1650522863.273125000
  time = 1650522863.28875
  time = 1650522863.304375000
  time = 1650522863.32000
  time = 1650522863.335625000
  time = 1650522863.35125
  time = 1650522863.366875000
  time = 1650522863.38250
  time = 1650522863.398125000
  time = 1650522863.41375

> >  Which is expected
> > on MS-Windows, given the scheduler time tick, but what does that have
> > to do with the system's time resolution?
> 
> The resolution of Elisp's (time-convert nil t) is determined by the 
> smallest nonzero gap between timestamps that are returned by C's 
> current_timespec. This is the system time resolution as far as Elisp is 
> concerned, because Elisp cannot return the current time at a finer 
> resolution than what current_timespec gives it. This resolution is not 
> necessarily the same as the time resolution of the motherboard clock, OS 
> high-res timestamp, file timestamps, etc.

Then I think I don't understand the purpose of this measurement, as
applied to Emacs Lisp.  For example, you say that this is unrelated to
file timestamps, but don't we use time values for file timestamps?
And for Windows, all this does is measure the "resolution" of the
Gnulib emulation of timespec functions on MS-Windows, it tells nothing
about the real resolution of the system time values.

More generally, if the "time resolution" determined by this procedure
is different between two systems, does it mean that two time values
that are 'equal' on one of them could be NOT 'equal' on another?  And
if so, wouldn't that mean Emacs Lisp programs will be inherently not
portable?

IOW, how do you intend to incorporate this "time resolution" into
Emacs Lisp, and what will be affected by that value?

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/20/22 12:30, Eli Zaretskii wrote:


I see the time samples change in jumps of 15 msec.


Could you give the first part of the output? I would like to see what 
the the samples are jumping from and to, and how often they jump.


Something like the following is what I'd hope to see from the first 
lines of the output of 'gllib/test-gettime-res x'. What are you seeing?


gettime_res returned 15625000 ns
time = 1650496481.515625000
time = 1650496481.53125
time = 1650496481.546875000
time = 1650496481.56250
time = 1650496481.578125000
time = 1650496481.59375
time = 1650496481.609375000
time = 1650496481.62500
time = 1650496481.640625000
time = 1650496481.65625


 Which is expected
on MS-Windows, given the scheduler time tick, but what does that have
to do with the system's time resolution?


The resolution of Elisp's (time-convert nil t) is determined by the 
smallest nonzero gap between timestamps that are returned by C's 
current_timespec. This is the system time resolution as far as Elisp is 
concerned, because Elisp cannot return the current time at a finer 
resolution than what current_timespec gives it. This resolution is not 
necessarily the same as the time resolution of the motherboard clock, OS 
high-res timestamp, file timestamps, etc.



And how is the 0.625 msec
number reported by the program obtained from those samples?


Use the largest resolution R ns consistent with the samples, such that 
10 is an integer multiple of R so that timestamp computations 
are exact.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/20/22 12:14, Eli Zaretskii wrote:

Sorry, my bad.  The result is the same, but I do get printouts.  What
do you want to know or see from there?


I want to see what the current_timespec's resolution is, which we should 
be able to tell from the debugging output. For example, on my Solaris 10 
sparc platform the command 'gltests/test-gettime-res x' outputs:


gettime_res returned 200 ns
time = 1650482432.256445600
time = 1650482432.256460600
time = 1650482432.256464400
time = 1650482432.256468200
time = 1650482432.256471400
time = 1650482432.256474600
time = 1650482432.256478000
time = 1650482432.256481200
time = 1650482432.256484800
...

and these timestamps say that with very high probability 
current_timespec's clock resolution is indeed 200 ns.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Eli Zaretskii]

> Date: Wed, 20 Apr 2022 12:23:43 -0700
> Cc: maniku...@gmail.com, emacs-orgmode@gnu.org, 54...@debbugs.gnu.org,
>  bug-gnu...@gnu.org
> From: Paul Eggert 
> 
> On 4/20/22 12:14, Eli Zaretskii wrote:
> > Sorry, my bad.  The result is the same, but I do get printouts.  What
> > do you want to know or see from there?
> 
> I want to see what the current_timespec's resolution is, which we should 
> be able to tell from the debugging output. For example, on my Solaris 10 
> sparc platform the command 'gltests/test-gettime-res x' outputs:
> 
> gettime_res returned 200 ns
> time = 1650482432.256445600
> time = 1650482432.256460600
> time = 1650482432.256464400
> time = 1650482432.256468200
> time = 1650482432.256471400
> time = 1650482432.256474600
> time = 1650482432.256478000
> time = 1650482432.256481200
> time = 1650482432.256484800
> ...
> 
> and these timestamps say that with very high probability 
> current_timespec's clock resolution is indeed 200 ns.

I see the time samples change in jumps of 15 msec.  Which is expected
on MS-Windows, given the scheduler time tick, but what does that have
to do with the system's time resolution?  And how is the 0.625 msec
number reported by the program obtained from those samples?

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/20/22 09:56, Max Nikulin wrote:


A typo: double "a".


Thanks, fixed in the attached which I installed in master.


If I get your idea correctly then "January, 31" + "1 month" should be 
more impressive as impossible date.


Thanks, good idea; also in the attached patch.


I can not figure out which elisp function can help to 
determine wall time for Aug 1 start of day in Cairo:


Africa/Cairo  Thu Jul 31 21:59:59 2014 UT = Thu Jul 31 23:59:59 2014 EET 
isdst=0 gmtoff=7200
Africa/Cairo  Thu Jul 31 22:00:00 2014 UT = Fri Aug  1 01:00:00 2014 
EEST isdst=1 gmtoff=10800


input: 2014-08-01 Africa/Cairo
(timezone may be implicit as the system one)
expected output: 01:00:00


Given mktime's limitations there's no trivial way to do this for 
arbitrary timestamps, since 00:00 doesn't exist in Cairo that day. 
Worse, in some locations near the International Date Line entire days do 
not exist, because at 00:00 they advanced the clocks forward 24 hours in 
order to move the date line.


It sounds like you're asking for a function that, given a date, yields 
the first broken-down timestamp on or after 00:00 of that date. For 
something like that, I'd use encode-time on 00:00 of that date to get a 
timestamp T, and then use time-add and decode-time to decode T-86400 
seconds, T, and T+86400 seconds, and if the decoded times all look fine 
then return (decode-time T). If not (i.e., their UTC offsets differ, or 
T's decoded time is not 00:00 on the correct date) I'd use binary search 
to find discontinuities between T-86400 and T+86400 and look next to 
those discontinuities to find timestamps closer to what you want.


Of course this is not ideal - but it's similar to what many mktime 
implementations do internally, and it's also similar to what Emacs's 
cal-dst already does (maybe you can look there for ideas), so you'd be 
in good company.From f98c3f4426fecf794f47f27aebe1f3b854fb1bfd Mon Sep 17 00:00:00 2001
From: Paul Eggert 
Date: Wed, 20 Apr 2022 12:03:19 -0700
Subject: [PATCH] More encode-time pitfall doc fixes

* doc/lispref/os.texi (Time Conversion): Improve discussion of
encode-time pitfalls based on comments by Max Nikulin (Bug#54764#63).
---
 doc/lispref/os.texi | 22 ++
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/doc/lispref/os.texi b/doc/lispref/os.texi
index cabae08970..4138dab09f 100644
--- a/doc/lispref/os.texi
+++ b/doc/lispref/os.texi
@@ -1670,7 +1670,7 @@ Time Conversion
 convention, @var{dst} is @minus{}1 and @var{zone} defaults to the
 current time zone rule (@pxref{Time Zone Rules}).
 When modernizing an obsolescent caller, ensure that the more-modern
-list equivalent contains 9 elements with a a @code{dst} element that
+list equivalent contains 9 elements with a @code{dst} element that
 is @minus{}1, not @code{nil}.
 
 Year numbers less than 100 are not treated specially.  If you want them
@@ -1695,22 +1695,28 @@ Time Conversion
 For example:
 
 @lisp
-;; Try to compute the time four years from now.
+;; Try to compute the time one month from now.
 ;; Watch out; this might not work as expected.
 (let ((time (decode-time)))
-  (setf (decoded-time-year time)
-(+ (decoded-time-year time) 4))
+  (setf (decoded-time-month time)
+(+ (decoded-time-month time) 1))
   time)
 @end lisp
 
 @noindent
 Unfortunately, this code might not work as expected if the resulting
-time is invalid due to daylight saving transitions, time zone changes,
+time is invalid due to month length differences,
+daylight saving transitions, time zone changes,
 or missing leap days or leap seconds.  For example, if executed on
-February 29, 2096 this code yields a nonexistent date because 2100 is
-not a leap year.  To avoid some (though not all) of the problem, you
+January 30 this code yields a nonexistent date February 30,
+which @code{encode-time} would adjust to early March.
+Similarly, adding four years to February 29, 2096 would yield the
+nonexistent date February 29, 2100; and adding one hour to 01:30 on
+March 13, 2022 in New York would yield a timestamp 02:30 that does not
+exist because clocks sprang forward from 02:00 to 03:00 that day.
+To avoid some (though not all) of the problem, you
 can base calculations on the middle of the affected unit, e.g., start
-at July 1 when adding years.  Alternatively, you can use the
+at the 15th of the month when adding months.  Alternatively, you can use the
 @file{calendar} and @file{time-date} libraries.
 @end defun
 
-- 
2.32.0

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Eli Zaretskii]

> Date: Wed, 20 Apr 2022 11:19:29 -0700
> Cc: maniku...@gmail.com, emacs-orgmode@gnu.org, 54...@debbugs.gnu.org,
>  Gnulib bugs 
> From: Paul Eggert 
> 
> > Thanks, the test-gettime-res test says "gettime_res returned 625000
> > ns", which is a strange number: it doesn't fit any MS-Windows system
> > time resolution figure I know about.  Do you happen to know what does
> > this number represent, and why it is the result of gettime-res.c when
> > it runs on MS-Windows?
> 
> It comes from current_timespec samples taken by gettime_res. Evidently 
> something is going wrong, either in gettime_res or in current_timespec.
> 
> I stared at the code a bit and see one possible problem, which I fixed 
> by installing the attached patch into Gnulib. I then generated a new 
> test-gettime-res.tgz compressed tarball (also attached); could you give 
> it a try?
> 
> This tarball is the result of running ./gnulib-tool as before, except I 
> added an extra print statement executed if you pass an extra argument to 
> that test program. So if you run the test and it's still outputting an 
> outlandish value for the resolution, please run the command:
> 
> gltests/test-gettime-res x
> 
> and let's take a look at its (long) debugging output.

I get the same result, and moreover I see no differences between this
and the previous tarball, and no printouts in test-gettime-res.c.  Did
you perhaps attach the wrong tarball this time?

Thanks.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/20/22 08:07, Max Nikulin wrote:

I have not checked if mktime is a part of 
POSIX and C standard.
mktime is part of both the C standard and POSIX. POSIX extends the C 
standard by saying that time_t is an integer type (the C standard allows 
time_t to be a floating-point type) and that time_t counts non-leap 
seconds since the Epoch (the C standard doesn't say what time_t counts, 
thought it implies that it counts seconds from some origin, which may 
not be 1970).
I still believe that optional DST and ZONE values is an improvement of 
the `encode-time' interface with no real drawbacks.


Yes, that's the direction we're headed.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Eli Zaretskii]

> Date: Wed, 20 Apr 2022 12:01:27 -0700
> Cc: maniku...@gmail.com, emacs-orgmode@gnu.org, 54...@debbugs.gnu.org,
>  bug-gnu...@gnu.org
> From: Paul Eggert 
> 
> > I get the same result, and moreover I see no differences between this
> > and the previous tarball, and no printouts in test-gettime-res.c.  Did
> > you perhaps attach the wrong tarball this time?
> 
> That's odd, as I get the different result (i.e., with debugging info) 
> when I use that tarball, a copy of which I got off the email archive (so 
> I know I sent it :-). Please see the attached shell transcript for 
> exactly how I got the different result, starting from the email archive.

Sorry, my bad.  The result is the same, but I do get printouts.  What
do you want to know or see from there?

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 17/04/2022 08:58, Paul Eggert wrote:

Thanks, I installed that and then installed the attached, which merges
that with some documentation improvements that I drafted based on this
thread.


Thank you for further editing of docs. Please, fix a typo.


diff --git a/doc/lispref/os.texi b/doc/lispref/os.texi
index 66689f43a9..8366689640 100644
--- a/doc/lispref/os.texi
+++ b/doc/lispref/os.texi

@@ -1687,14 +1660,18 @@ Time Conversion
  than six arguments the @emph{last} argument is used as @var{zone} and
  any other extra arguments are ignored, so that @code{(apply
  #'encode-time (decode-time ...))} works.  In this obsolescent
-convention, @var{zone} defaults to the current time zone rule
-(@pxref{Time Zone Rules}), and @var{dst} is treated as if it was
-@minus{}1.
+convention, @var{dst} is @minus{}1 and @var{zone} defaults to the
+current time zone rule (@pxref{Time Zone Rules}).
+When modernizing an obsolescent caller, ensure that the more-modern
+list equivalent contains 9 elements with a a @code{dst} element that


^^^
A typo: double "a".


+is @minus{}1, not @code{nil}.

+@lisp
+;; Try to compute the time four years from now.
+;; Watch out; this might not work as expected.
+(let ((time (decode-time)))
+  (setf (decoded-time-year time)
+(+ (decoded-time-year time) 4))
+  time)
+@end lisp



+@noindent
+Unfortunately, this code might not work as expected if the resulting
+time is invalid due to daylight saving transitions, time zone changes,
+or missing leap days or leap seconds.  For example, if executed on
+February 29, 2096 this code yields a nonexistent date because 2100 is
+not a leap year.  To avoid some (though not all) of the problem, you
+can base calculations on the middle of the affected unit, e.g., start
+at July 1 when adding years.


If I get your idea correctly then "January, 31" + "1 month" should be 
more impressive as impossible date. Year 2096 is too far in future. I am 
unsure concerning expectation. Overflow arithmetic is described above 
and e.g. JavaScript normalizes Date object in a similar fashion. The 
special point is that elisp decoded time requires explicit normalization 
however and 2100 is a good example that updating of any field may 
"break" the date.



Alternatively, you can use the
+@file{calendar} and @file{time-date} libraries.


A remark loosely related to your patch. Earlier you mentioned missed 
midnight due to time transition and suggested to use calendrical 
functions in Org. I can not figure out which elisp function can help to 
determine wall time for Aug 1 start of day in Cairo:


Africa/Cairo  Thu Jul 31 21:59:59 2014 UT = Thu Jul 31 23:59:59 2014 EET 
isdst=0 gmtoff=7200
Africa/Cairo  Thu Jul 31 22:00:00 2014 UT = Fri Aug  1 01:00:00 2014 
EEST isdst=1 gmtoff=10800


input: 2014-08-01 Africa/Cairo
(timezone may be implicit as the system one)
expected output: 01:00:00

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 19/04/2022 09:02, Paul Eggert wrote:
Here are the main points I see about making timestamps work better in 
Org mode, and related patches to how Emacs handles timestamps.


* Max would like encode-time to treat a list (SS MM HH DD MM ) as if 
it were (SS MM HH DD MM  nil -1 nil),


Thank you, Paul, it is exactly what I had in mind when I created this bug.

as that would be more 
convenient for how Org mode deals with ambiguous local timestamps. This 
is relatively easy to add for Emacs 29+, and is done in first of the 
attached proposed patches to Emacs master.


I would say that Org just does not care concerning ambiguous local time. 
Anyway there are other similar cases besides DST.


* As I understand it, Max would like a function that emulate Emacs 29 
encode-time with one argument, even if running in Emacs versions back to 
Emacs 25. I suppose such a function would also need to implement Emacs 
27+ encode-time's support for subsecond resolution. E.g., 
(org-encode-time '((44604411568 . 10) 55 0 19 4 2022 - -1 t)) 
should return (1650329744604411568 . 10) even in Emacs 25 and 26.


I am just afraid of possibility of recurrent attempts to modernize 
time-related code within Emacs including Org code in a such way that can 
not be directly ported to the Org repository. Discrepancy of the code 
increases maintenance burden. The main purpose of a compatibility 
wrapper is to prevent grep-driven refactoring. Another point of the 
helper function is to allow to remove from Emacs support confusing 
old-style `encode-time' arguments with ignored DST value. In Org 
timestamps are often built from scratch, so separate argument are still 
convenient.


Org timestamps have minute precision, even seconds are trimmed. So at 
least explicitly I did not ask for subsecond timestamps. I admit however 
that they might emerge in some code paths.


Notice that nobody from Org developers & maintainers commented the patch 
demonstrating the idea of such wrapper.


* My last topic in this email is Max's request for a feature that I'm 
not planning to put into Emacs 29 as it'll require more thought. This 
addresses the problem where your TZ is "Africa/Juba" and you want to 
encode a timestamp like "2021-01-31 23:30" which is ambiguous since at 
24:00 that day Juba moved standard time back by an hour. Unfortunately 
the underlying C mktime function does not allow disambiguation in the 
rare situation where standard time moves further west of Greenwich. 
Addressing this problem would require rewriting mktime from scratch in 
Elisp, or using heuristics that would occasionally fail, or (my 
favorite) extending glibc mktime to treat tm_isdst values other than 
-1,0,1 to support disambiguating such timestamps.


I do not urge such changes. I have not checked if mktime is a part of 
POSIX and C standard. If it is so, I am not in favor of adding more 
values for the tm_isdst field since they are not related to DST.


I started this branch of discussion to convince Paul that requirement of 
9 fields is not really encourage more correct usage of `encode-time' in 
comparison to 6 values.


More convenient interface for processing of local time moments requires 
significant amount of work, maybe some prototypes. It should be 
considered separately from this bug.


I still believe that optional DST and ZONE values is an improvement of 
the `encode-time' interface with no real drawbacks. It minimizes the 
chance of passing nil as "no DST" when actual value is unknown and 
developers are not ready to add a bunch of code to determine proper TZ 
offset for each case of time transition.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Eli Zaretskii]

> Date: Tue, 19 Apr 2022 15:22:29 -0700
> Cc: maniku...@gmail.com, emacs-orgmode@gnu.org, 54...@debbugs.gnu.org
> From: Paul Eggert 
> 
> On 4/18/22 22:50, Eli Zaretskii wrote:
> >> * admin/merge-gnulib (GNULIB_MODULES): Add gettime-res.
> >> * lib/gettime-res.c: New file, copied from Gnulib.
> >> * lib/gnulib.mk.in, m4/gnulib-comp.m4: Regenerate.
> > Is this known to support MS-Windows correctly?
> 
> I haven't tested it on that platform, though I expect it to work since 
> it relies only on current_timespec and Emacs already uses that.
> 
> I just now added some test cases to Gnulib for it; see the patch in the 
> first attachment. You can try these tests in your environment by running 
> './gnulib-tool --test gettime-res' in the Gnulib source directory. Or 
> you can save time by running './configure; make check' in the directory 
> represented by the second attachment, which is a compressed tarball 
> containing the output of './gnulib-tool --create-testdir --dir 
> test-gettime-res gettime-res'.

Thanks, the test-gettime-res test says "gettime_res returned 625000
ns", which is a strange number: it doesn't fit any MS-Windows system
time resolution figure I know about.  Do you happen to know what does
this number represent, and why it is the result of gettime-res.c when
it runs on MS-Windows?

AFAIK, the basic resolution of MS-Windows time stamps is 100 ns, so
using the above much larger number seems to hint at some significant
loss of information.  If the goal of this future changeset is to make
Emacs time stamps more fine-grained, it would be a shame not to have
the 100-ns resolution on MS-Windows.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Eli Zaretskii]

> Date: Mon, 18 Apr 2022 19:02:03 -0700
> From: Paul Eggert 
> Cc: emacs-orgmode@gnu.org, 54...@debbugs.gnu.org
> 
> From 8c25372709d256d83858be454987137dc202b725 Mon Sep 17 00:00:00 2001
> From: Paul Eggert 
> Date: Mon, 18 Apr 2022 13:08:27 -0700
> Subject: [PATCH 3/6] Add Gnulib gettime-res module
> 
> * admin/merge-gnulib (GNULIB_MODULES): Add gettime-res.
> * lib/gettime-res.c: New file, copied from Gnulib.
> * lib/gnulib.mk.in, m4/gnulib-comp.m4: Regenerate.

Is this known to support MS-Windows correctly?  If so, can you show
how to test that, or tell where I can find the results of such testing
that someone else did?

If MS-Windows isn't supported by that, I would ask to add such support
before this is installed.

Thanks.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

Here are the main points I see about making timestamps work better in 
Org mode, and related patches to how Emacs handles timestamps.


* Max would like encode-time to treat a list (SS MM HH DD MM ) as if 
it were (SS MM HH DD MM  nil -1 nil), as that would be more 
convenient for how Org mode deals with ambiguous local timestamps. This 
is relatively easy to add for Emacs 29+, and is done in first of the 
attached proposed patches to Emacs master.


* As I understand it, Max would like a function that emulate Emacs 29 
encode-time with one argument, even if running in Emacs versions back to 
Emacs 25. I suppose such a function would also need to implement Emacs 
27+ encode-time's support for subsecond resolution. E.g., 
(org-encode-time '((44604411568 . 10) 55 0 19 4 2022 - -1 t)) 
should return (1650329744604411568 . 10) even in Emacs 25 and 26.


* There are three other Emacs timestamp changes I should mention that 
might be relevant to Org mode:


** 1. Although in Emacs 28 functions like (time-convert nil t) return a 
timestamp resolution of 1 ns, in Emacs 29 I plan to fix them so that 
they return the system clock resolution, which is often coarser than 1 
ns. This is done in the 4th attached patch.


** 2. In Emacs 29 format-time-string should support a new format "%-N" 
which outputs only enough digits for the system clock resolution. (This 
is the same as GNU "date".) This is done in the 5th attached patch.


** 3. Emacs 29 current-time and related functions should generate a 
(TICKS . HZ) timestamp instead of the old (HIGH LOW MICROSECS PICOSECS) 
timestamp. This change has been planned for some time; it was announced 
in Emacs 27. As far as I know Org mode is already ready for this change 
but I thought I'd mention it again here. This is done in the last 
attached patch.


* My last topic in this email is Max's request for a feature that I'm 
not planning to put into Emacs 29 as it'll require more thought. This 
addresses the problem where your TZ is "Africa/Juba" and you want to 
encode a timestamp like "2021-01-31 23:30" which is ambiguous since at 
24:00 that day Juba moved standard time back by an hour. Unfortunately 
the underlying C mktime function does not allow disambiguation in the 
rare situation where standard time moves further west of Greenwich. 
Addressing this problem would require rewriting mktime from scratch in 
Elisp, or using heuristics that would occasionally fail, or (my 
favorite) extending glibc mktime to treat tm_isdst values other than 
-1,0,1 to support disambiguating such timestamps. In the meantime, one 
can disambiguate such timestamps in Elisp by using numeric time zones, e.g.:


  (format-time-string "%F %T %z"
  (encode-time '(0 30 23 31 1 2021 - -1 TZ))
  "Africa/Juba")

yields "2021-01-31 23:30:00 +0200" if TZ is 7200, and "2021-01-31 
23:30:00 +0300" if TZ is 10800. And perhaps this is the right way to go 
in the long run anyway.From 3d02a8e1192a782a16ffdee4940612f69a12629f Mon Sep 17 00:00:00 2001
From: Paul Eggert 
Date: Mon, 18 Apr 2022 13:08:26 -0700
Subject: [PATCH 1/6] Support (encode-time (list s m h D M Y))

* src/timefns.c (Fencode_time): Add support for a 6-elt list arg.
Requested by Max Nikulin for Org (bug#54764).
* test/src/timefns-tests.el (encode-time-alternate-apis): New test.
---
 doc/lispref/os.texi   |  5 +
 etc/NEWS  |  5 +
 src/timefns.c | 21 +++--
 test/src/timefns-tests.el |  9 +
 4 files changed, 34 insertions(+), 6 deletions(-)

diff --git a/doc/lispref/os.texi b/doc/lispref/os.texi
index cabae08970..bfcd51318e 100644
--- a/doc/lispref/os.texi
+++ b/doc/lispref/os.texi
@@ -1660,6 +1660,11 @@ Time Conversion
 handle situations like this you can use a numeric @var{zone} to
 disambiguate instead.
 
+The first argument can also be a list @code{(@var{second} @var{minute}
+@var{hour} @var{day} @var{month} @var{year})}, which is treated like
+the list @code{(@var{second} @var{minute} @var{hour} @var{day}
+@var{month} @var{year} nil -1 nil)}.
+
 As an obsolescent calling convention, this function can be given six
 or more arguments.  The first six arguments @var{second},
 @var{minute}, @var{hour}, @var{day}, @var{month}, and @var{year}
diff --git a/etc/NEWS b/etc/NEWS
index 3e7788277d..c5a136ea68 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1950,6 +1950,11 @@ For example, '(time-add nil '(1 . 1000))' no longer warns that the
 '(1 . 1000)' acts like '(1000 . 100)'.  This warning, which was a
 temporary transition aid for Emacs 27, has served its purpose.
 

+** 'encode-time' now also accepts a 6-element list with just time and date.
+(encode-time (list SECOND MINUTE HOUR DAY MONTH YEAR)) is now short for
+(encode-time (list SECOND MINUTE HOUR DAY MONTH YEAR nil -1 nil)).
+
 +++
 ** 'date-to-time' now assumes earliest values if its argument lacks
 month, day, or time.  For example, (date-to-time "2021-12-04") now
di

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/16/22 09:26, Max Nikulin wrote:
Feel free to shorten the added fragment, to change the wording, or to 
use your variant instead. See the attachment.


Thanks, I installed that and then installed the attached, which merges 
that with some documentation improvements that I drafted based on this 
thread.


It is a messy area but I hope the documentation is clearer now.From f1ba92448d1e573640547c68d9bed89fe5c43da0 Mon Sep 17 00:00:00 2001
From: Paul Eggert 
Date: Sat, 16 Apr 2022 18:48:51 -0700
Subject: [PATCH] Document encode-time caveats
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* doc/lispref/os.texi (Time of Day, Time Conversion):
Move the warnings about DST being -1 to closer to where DST is
discussed, and reword and improve the discussions and warnings.
Be more precise about years before 1969 (possible west of UTC) vs the
Epoch.  Mention some problems due to leap seconds, leap years,
daylight saving transitions, and time zone changes.  Modernize
discussion of OS timestamp range.  Prefer secular ‘BCE’ to religious
‘BC’.  Omit discussion of decoded-time-add and make-decoded-time, as
they are in a library and are not always available; instead, mention
the library.  Warn about common mistakes when doing simple date
arithmetic.
* src/timefns.c (Fencode_time): In doc string, mention date
arithmetic and tighten up the wording a bit.
---
 doc/lispref/os.texi | 153 +++-
 src/timefns.c   |  16 ++---
 2 files changed, 73 insertions(+), 96 deletions(-)

diff --git a/doc/lispref/os.texi b/doc/lispref/os.texi
index 66689f43a9..8366689640 100644
--- a/doc/lispref/os.texi
+++ b/doc/lispref/os.texi
@@ -1303,10 +1303,16 @@ Time of Day
 
 @cindex Lisp timestamp
 @cindex timestamp, Lisp
+@cindex Coordinated Universal Time
+@cindex Universal Time
+@cindex UTC
+@cindex leap seconds
   Many functions like @code{current-time} and @code{file-attributes}
 return @dfn{Lisp timestamp} values that count seconds, and that can
 represent absolute time by counting seconds since the @dfn{epoch} of
-1970-01-01 00:00:00 UTC.
+1970-01-01 00:00:00 UTC (Coordinated Universal Time).  Typically these
+counts ignore leap seconds; however, GNU and some other operating
+systems can be configured to count leap seconds.
 
   Although traditionally Lisp timestamps were integer pairs, their
 form has evolved and programs ordinarily should not depend on the
@@ -1367,8 +1373,8 @@ Time of Day
 Some of these conversions rely on operating system functions that
 limit the range of possible time values, and signal an error such as
 @samp{"Specified time is not representable"} if the
-limits are exceeded.  For instance, a system may not support years
-before 1970, or years before 1901, or years far in the future.
+limits are exceeded.  For instance, a system might not support
+timestamps before the epoch, or years far in the future.
 You can convert a time value into
 a human-readable string using @code{format-time-string}, into a Lisp
 timestamp using @code{time-convert}, and into other forms using
@@ -1434,11 +1440,11 @@ Time Zone Rules
 which is a platform-dependent default time zone.
 
 The set of supported @env{TZ} strings is system-dependent.  GNU and
-many other systems support the tzdata database, e.g.,
+many other systems support TZDB timezones, e.g.,
 @samp{"America/New_York"} specifies the time zone and daylight saving
 time history for locations near New York City.  GNU and most other
 systems support POSIX-style @env{TZ} strings, e.g.,
-@samp{"EST+5EDT,M4.1.0/2,M10.5.0/2"} specifies the rules used in New
+@samp{"EST5EDT,M4.1.0,M10.5.0"} specifies the rules used in New
 York from 1987 through 2006.  All systems support the string
 @samp{"UTC0"} meaning Universal Time.
 
@@ -1490,18 +1496,20 @@ Time Conversion
   These functions convert time values (@pxref{Time of Day}) to Lisp
 timestamps, or into calendrical information and vice versa.
 
-  Many 32-bit operating systems are limited to system times containing
-32 bits of information in their seconds component; these systems
-typically handle only the times from 1901-12-13 20:45:52 through
-2038-01-19 03:14:07 Universal Time.  However, 64-bit and some 32-bit operating
-systems have larger seconds components, and can represent times far in
-the past or future.
-
-  Calendrical conversion functions always use the Gregorian calendar, even
-for dates before the Gregorian calendar was introduced.  Year numbers
-count the number of years since the year 1 BC, and do not skip zero
+  Many operating systems use 64-bit signed integers to count seconds,
+and can represent times far in the past or future.  However, some are
+more limited.  For example, old-fashioned operating systems that use
+32-bit signed integers typically handle only times from 1901-12-13
+20:45:52 through 2038-01-19 03:14:07 Universal Time.
+
+  Calendrical conversion functions use the Gregorian calendar even for
+dates before the Gregorian 

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/15/22 10:23, Max Nikulin wrote:

if you are storing future events bound to wall time then namely 
time zone identifier should have precedence.


Although that would make sense for some applications it's not a good 
idea in general. For example, if you're scheduling a Zoom meeting you 
should save both, because other meeting participants may interpret 
strings like "Pacific/Apia" differently.



Just identifier may be ambiguous around DST transition. So timezone 
abbreviations are ambiguous per se but when identifiers are known they 
may be still necessary to resolve uncertainties for backward time 
shifts. At certain moment the Olson DB started to use "+04" 
abbreviations instead of letters for transitions unrelated to daylight 
saving time.


Yes, timezone names like "Europe/Lisbon" are ambiguous during fallback 
transitions, or if the meaning of "Europe/Lisbon" changes. This is why 
one should also save UT offsets when generating localtime timestamps.


Around five years ago I changed TZDB to numeric use time zone 
abbreviations like "+04" instead of my inventions like "GET", because I 
wanted TZDB to follow existing practice, not invent it. A nice side 
effect is that numeric abbreviations are unambiguous. (Besides, even _I_ 
couldn't remember what "GET" meant. :-)




And WET/WEST gets another bit of info in addition to numerical offset.


That info is meant only for users; I wouldn't rely on it for 
calculations because those abbreviations are ambiguous. It could well 
be, for example that the meaning of "PST" in the United States will 
change in the near future.



I do not remember if it is possible at all to obtain using libc the 
period of constant time offset, when time shift value is valid. 
Sometimes it is necessary to recalculate offset.


Sorry, I don't understand this point. One can easily recalculate the UT 
offset in Emacs Lisp by generating the desired timestamp and calling 
decode-time on it. You surely are talking about something else, but I 
don't know what it is.



You wrote that "2021-01-31 23:30:00 +0300" is parsed correctly. My 
opinion is that when time zone is known to be Africa/Juba (system-wide 
setting, environment variable, or an argument of the parsing function) 
then "2021-01-31 23:30:00 CAT" and "2021-01-31 23:30:00 EAT" should be 
parsed correctly (and localized date-time formats should be parsed as 
well).


That's not possible in general, since the two abbreviations can be the 
same. Traditionally in Australia, for example, "CST" meant both "Central 
Standard Time" and "Central Summer Time", and there are probably still 
apps that use the equivalent of TZ="CST-9:30CST,M10.1.0,M4.1.0/3" which 
does precisely that.


It's hardly ever a good idea to rely on time zone abbreviations as 
they're too often ambiguous. It's much better to use UT offsets. When 
generating a localtime timestamp, one should always output its UT offset 
(in addition to any other advisory info you might want to output). And 
if you do that, many of the abovementioned problems are easily solved.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 09/04/2022 14:52, Paul Eggert wrote:

On 4/7/22 05:37, Max Nikulin wrote:

Daylight saving time field matters only as a list component and 
ignored as a separate argument (by the way, it should be stressed in 
the docstring).


Do you have a wording suggestion? (The doc string already covers the 
topic concisely; however, conciseness is not always a virtue. :-)


Feel free to shorten the added fragment, to change the wording, or to 
use your variant instead. See the attachment.From 42a20494e9f1461f7166c922452028548796bd16 Mon Sep 17 00:00:00 2001
From: Max Nikulin 
Date: Sat, 16 Apr 2022 23:19:10 +0700
Subject: [PATCH] Stress difference of new and old ways to call `encode-time'

* doc/lispref/os.texi (Time Conversion): Add a warning that blind
changing of code calling `encode-time' to use single list instead of
multiple values may cause deferred bugs since it is common to use nil
for ignored arguments such as DST in the old calling convention.
* src/timefns.c (encode-time): Mention the warning added to the elisp
reference in the docstring.

Refactoring related to `encode-time' caused (bug#54731), so it is better
to make apparent the difference between the recommended and the
obsolescent ways to call the function.  More details concerning the
purpose and limitations of the DST field are added after discussion with
Paul Eggert in (bug#54764).
---
 doc/lispref/os.texi | 19 +++
 src/timefns.c   |  4 
 2 files changed, 23 insertions(+)

diff --git a/doc/lispref/os.texi b/doc/lispref/os.texi
index 4ee893f860..b7b41f97ab 100644
--- a/doc/lispref/os.texi
+++ b/doc/lispref/os.texi
@@ -1707,6 +1707,25 @@ the latter to the former as follows:
 You can perform simple date arithmetic by using out-of-range values for
 @var{seconds}, @var{minutes}, @var{hour}, @var{day}, and @var{month};
 for example, day 0 means the day preceding the given month.
+
+The old and the new styles to call @code{encode-time} with the same
+values of time fields may give different results.  While modernizing
+code that uses obsolescent calling convention, ensure that the list
+argument contains 9 elements.  Pay special attention that the @code{dst}
+field does not use @code{nil} expecting that actual value will be
+guessed, pass @samp{-1} instead.  During normalizing of values to
+correct state of daylight saving time users may get time shift and even
+wrong date.  It may take months to discover such problem.  When
+called with multiple arguments, the function ignores equivalent of the
+@code{dst} value and @samp{-1} is effectively used.  The new way to call
+@code{encode-time} has an advantage that it is possible to resolve
+ambiguity around backward time shift by passing @code{nil} or @code{t}.
+Unfortunately there are enough cases across the world when a particular
+area is moved to another time zone with no change of daylight saving
+time state.  @code{encode-time} may signal an error in response to
+@code{t} passed as @code{dst}.  You have to pass @code{zone} explicitly
+as time offset in such case if default ambiguity resolution is not
+acceptable.
 @end defun
 
 @node Time Parsing
diff --git a/src/timefns.c b/src/timefns.c
index b061be0a78..9af89a512d 100644
--- a/src/timefns.c
+++ b/src/timefns.c
@@ -1631,6 +1631,10 @@ convention, DST and ZONE default to -1 and nil respectively.
 Years before 1970 are not guaranteed to work.  On some systems,
 year values as low as 1901 do work.
 
+See Info node `(elisp)Time Conversion' for description of a pitfall
+that can be faced during migration from the obsolescent to the new
+calling convention due to unconscious usage of nil for the DST argument.
+
 usage: (encode-time TIME &rest OBSOLESCENT-ARGUMENTS)  */)
   (ptrdiff_t nargs, Lisp_Object *args)
 {
-- 
2.25.1

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 15/04/2022 05:46, Paul Eggert wrote:

On 4/14/22 06:19, Max Nikulin wrote:

 date-time + "America/Los_Angeles" input should not be reduced to 
timezone offset in the output.


It depends on the application. For some applications (e.g., generating 
"Date:" lines in email), it is entirely correct to output a timestamp 
like "14 Apr 2022 15:16:04 -0700", thus losing the fact that the 
timestamp was generated with TZ="America/Los_Angeles".


However if you are storing future events bound to wall time then namely 
time zone identifier should have precedence. A new rule may be issued 
between scheduling event and the time it will happen. It is terrible 
feeling when it is necessary to guess if a web site stores TZ offset or 
its identifier and in the latter case whether its administrators updated 
tzinfo. It is better to store location of event since a time zone may be 
split and time transition may apply only to a part of the original zone.


Actually I meant another case. Some representation is got for a time 
moment and it is necessary to get local time for another time moment. 
Time zone identifier or an object with internal representation allow to 
get correct offset for second moment of time. It should be possible to 
specify whether a function call is isolated conversion or further 
calculations will follow.


Zone internal object or identifier is important for calculation of 
other date-time values based on the origin value.


Again, that depends on the application. It's typically wrong to store an 
old timestamp in a form like "1950-07-01 00:00 Europe/Lisbon", because 
there is no standard for what "Europe/Lisbon" means. If you update your 
copy of TZDB, or interpret such a timestamp on another computer, that 
can change the interpretation of such a timestamp. In this particular 
case, a change in TZDB release 2021b altered the interpretation of this 
old timestamp because we discovered that DST was observed in 1950 in 
Portugal.


Just identifier may be ambiguous around DST transition. So timezone 
abbreviations are ambiguous per se but when identifiers are known they 
may be still necessary to resolve uncertainties for backward time 
shifts. At certain moment the Olson DB started to use "+04" 
abbreviations instead of letters for transitions unrelated to daylight 
saving time.


If you want to keep the TZDB identifier for advice about how to 
interpret dates relative to a timestamp, that's fine. But you should 
keep the UT offset in addition to the TZDB identifier, if you want your 
app to be fully accurate and useful. For example, you should store 
"1950-07-01 00:00:00 + Europe/Lisbon" for a timestamp generated by 
TZDB release 2021a, so that when you interpret the timestamp in release 
2021b you'll have an idea of what you're dealing with.


And WET/WEST gets another bit of info in addition to numerical offset.

I hope, they may work without explicitly providing time zone offset to 
the input that anyway requires additional calculations. 


It doesn't require additional calculations on the Emacs Lisp user's 
part. All you need to do is save the UT offset, and use it later. 
There's so little overhead to this that it's not worth worrying about.


I do not remember if it is possible at all to obtain using libc the 
period of constant time offset, when time shift value is valid. 
Sometimes it is necessary to recalculate offset.


±n hours may mean ±n*3600 seconds or time with same minutes and 
seconds values but hours value is changed by n even if a 30 min DST 
transition happens in between.


Sorry, I don't understand what this sentence is intended to mean.


Let's consider Australia/Lord_Howe with 30min backward DST shift at 
2022-04-03 02:00. 8 hours from 2022-04-02 22:00 may mean 2022-04-03 
06:00 for duration of the night shift (8:30 instead of usual 8:00). Some 
technological process requiring precisely 8 hours finishes at 05:30 in 
such case. So it is not equivalent to add 8 hours or 480 minutes. In the 
former case it is more convenient to increment particular field and 
adjust the result if it coincides with ambiguity/impossible range. In 
the latter case it is better to increment timestamp as seconds since the 
epoch and back to time fields (leaving aside leap seconds).



`parse-time-string' has another set of problems.


Sure, but that was just an example. You can write your own date parser. 
The point is that when you save a localtime timestamp, you should save 
its UT offset too, in whatever notation is appropriate.


You wrote that "2021-01-31 23:30:00 +0300" is parsed correctly. My 
opinion is that when time zone is known to be Africa/Juba (system-wide 
setting, environment variable, or an argument of the parsing function) 
then "2021-01-31 23:30:00 CAT" and "2021-01-31 23:30:00 EAT" should be 
parsed correctly (and localized date-time formats should be parsed as 
well). For transitions without DST change there is no conventional text 
representation.


UTC offset is another fea

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Tim Cross]

Paul Eggert  writes:

> On 4/14/22 06:19, Max Nikulin wrote:
>
>>  date-time + "America/Los_Angeles" input should not be reduced to timezone 
>> offset
>> in the output.
>
> It depends on the application. For some applications (e.g., generating 
> "Date:" lines
> in email), it is entirely correct to output a timestamp like "14 Apr 2022 
> 15:16:04
> -0700", thus losing the fact that the timestamp was generated with
> TZ="America/Los_Angeles".
>
>> Zone internal object or identifier is important for calculation of other 
>> date-time values based on the origin value.
>
> Again, that depends on the application. It's typically wrong to store an old
> timestamp in a form like "1950-07-01 00:00 Europe/Lisbon", because there is no
> standard for what "Europe/Lisbon" means. If you update your copy of TZDB, or
> interpret such a timestamp on another computer, that can change the 
> interpretation of
> such a timestamp. In this particular case, a change in TZDB release 2021b 
> altered the
> interpretation of this old timestamp because we discovered that DST was 
> observed in
> 1950 in Portugal.
>
> If you want to keep the TZDB identifier for advice about how to interpret 
> dates
> relative to a timestamp, that's fine. But you should keep the UT offset in 
> addition
> to the TZDB identifier, if you want your app to be fully accurate and useful. 
> For
> example, you should store "1950-07-01 00:00:00 + Europe/Lisbon" for a 
> timestamp
> generated by TZDB release 2021a, so that when you interpret the timestamp in 
> release
> 2021b you'll have an idea of what you're dealing with.
>

I think this is a very important point. Timezone data is not static. If
you only record the timezone name, offsets will be calculated using the
current definition, which may not be correct for past timestamps.

A good example of this is the DST values and the date when a TZ
transitions between DST and non-DST periods. That date can change,
either temporarily or permanently. That change can be days or even
weeks. Any date related calculations which only have knowledge about TZ
names and not the specific offset of a timestamp can therefore be out by
a significant amount.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/14/22 06:19, Max Nikulin wrote:

 date-time + 
"America/Los_Angeles" input should not be reduced to timezone offset in 
the output.


It depends on the application. For some applications (e.g., generating 
"Date:" lines in email), it is entirely correct to output a timestamp 
like "14 Apr 2022 15:16:04 -0700", thus losing the fact that the 
timestamp was generated with TZ="America/Los_Angeles".



Zone internal object or identifier is important for calculation of other 
date-time values based on the origin value.


Again, that depends on the application. It's typically wrong to store an 
old timestamp in a form like "1950-07-01 00:00 Europe/Lisbon", because 
there is no standard for what "Europe/Lisbon" means. If you update your 
copy of TZDB, or interpret such a timestamp on another computer, that 
can change the interpretation of such a timestamp. In this particular 
case, a change in TZDB release 2021b altered the interpretation of this 
old timestamp because we discovered that DST was observed in 1950 in 
Portugal.


If you want to keep the TZDB identifier for advice about how to 
interpret dates relative to a timestamp, that's fine. But you should 
keep the UT offset in addition to the TZDB identifier, if you want your 
app to be fully accurate and useful. For example, you should store 
"1950-07-01 00:00:00 + Europe/Lisbon" for a timestamp generated by 
TZDB release 2021a, so that when you interpret the timestamp in release 
2021b you'll have an idea of what you're dealing with.



I want hints like "in the case of ambiguity resolve to transition time immediately 
before/immediately after transition" or "provide suitable time prior to/after to 
transition".


Although that might be nice it's not what mktime gives us, and I doubt 
whether it's a good idea to try to implement it from scratch in Emacs.


I hope, they may work without explicitly providing time zone offset to the input that anyway requires additional calculations. 


It doesn't require additional calculations on the Emacs Lisp user's 
part. All you need to do is save the UT offset, and use it later. 
There's so little overhead to this that it's not worth worrying about.


±n hours may mean ±n*3600 seconds or time with same minutes and seconds 
values but hours value is changed by n even if a 30 min DST transition 
happens in between.


Sorry, I don't understand what this sentence is intended to mean.


`parse-time-string' has another set of problems.


Sure, but that was just an example. You can write your own date parser. 
The point is that when you save a localtime timestamp, you should save 
its UT offset too, in whatever notation is appropriate.


UTC offset is another feature and implementing the hints I have 
tried to describe may require implementing from scratch full stack of 
time handling functions.


I doubt whether that's a good idea. I've written that sort of code, and 
it's a lot more work than one might think and it's notoriously difficult 
to do it correctly. You have better things to do.
So I still do not see any point in mandatory DST and ZONE fields in new 
interface of `encode-time'.


I think we're in agreement here. As I mentioned earlier, I plan to 
modify Emacs encode-time so that you can pass it a 6-arg list as well as 
an 9-arg list. Once this change is in, the DST and ZONE fields will not 
be mandatory.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 14/04/2022 01:35, Paul Eggert wrote:

On 4/13/22 07:40, Max Nikulin wrote:


I do not see a way to get 23:30 EAT +0300.


Are you asking for a function F where you say, "I want to give F a 
possibly-ambiguous decoded local time D, and for F to return all 
timestamps that map to D"? If so, encode-time doesn't do that, because 
the underlying C API (namely, mktime) doesn't do that. All mktime and 
encode-time do is give you *one* timestamp that maps to D; it won't give 
you any other timestamps.


I am just trying to convince you that new API still can not handle 
ambiguities in response to:



Unfortunately it makes the function more convenient to
use incorrectly. This was part of the motivation for the API change.
The obsolescent calling convention has no way to deal with ambiguous 
timestamps like 2022-11-06 01:30 when TZ="America/Los_Angeles".


Completely ignoring DST in old API is a bug. Possibility to omit DST and 
ZONE is a feature for convenience and it is not related to 
correctly/incorrectly. It has no common with allowing 6-values 
SECONONDS...YEAR list.


I am aware of mktime and my opinion is that libc in such cases (dealing 
with formats for humans) is hardly usable for applications more complex 
than "hello world".


I do not want all ambiguous time values. I want to provide hints how 
they should be resolved and get in response an additional value that 
says in which way it is actually resolved.


If you're worried about possibly-ambiguous decoded local times, you 
could probe (say) one day before and one day after encode-time's result 
to see if the UTC offset changes, and let that guide you to find other 
possible timestamps that map to the decoded time. Although this is just 
a heuristic it should be good enough.


I would prefer to avoid dances with +/-1 day timestamps. I would not be 
surprised when one day they will give wrong result. During computations 
it is available to which interval of time current time offset is valid. 
Such value may be included in result of conversion. So date-time + 
"America/Los_Angeles" input should not be reduced to timezone offset in 
the output. Zone internal object or identifier is important for 
calculation of other date-time values based on the origin value.


I doubt whether you need to do that, though. Code that is not careful 
about local time offsets doesn't care how ambiguous decoded times are 
resolved. And code that does care should record UTC offsets anyway, and 
you can use those offsets to disambiguate the decoded times. Something 
like this, say:


  (defun encode-and-format-time (time tz)
    (let ((etime (encode-time (parse-time-string time
  (format-time-string "%F %T %Z %z" etime tz)))

With this definition, (encode-and-format-time "2021-01-31 23:30:00 
+0300" "Africa/Juba") yields "2021-01-31 23:30:00 EAT +0300", which is 
the timestamp you want.


I want hints like "in the case of ambiguity resolve to transition time 
immediately before/immediately after transition" or "provide suitable 
time prior to/after to transition". I hope, they may work without 
explicitly providing time zone offset to the input that anyway requires 
additional calculations.


±n hours may mean ±n*3600 seconds or time with same minutes and seconds 
values but hours value is changed by n even if a 30 min DST transition 
happens in between.


If agenda interval start falls on skipped time span in local time then 
the value immediately after transition should be used (and maybe user 
should be warned by additional entry). For agenda end interval vice 
versa local time immediately before transition is more suitable.


`parse-time-string' has another set of problems. It is impossible to 
specify particular format like for strptime(3). Actually parsing a 
string to numerical values and resolving ambiguities in numerical values 
are different tasks and it may be useful to have separate functions for 
them. I am unsure however concerning implementing constraints to parse 
free-form dates.


`encode-time' should only accept time zone as time offset and should 
not allow default or named value that may be ambiguous.


If we're talking about Org's encode-time substitute, you can of course 
do what you like. But Emacs encode-time has supported ambiguous 
timestamps for some time and I expect it's used by apps that don't care 
how ambiguous decoded times are resolved, which means we shouldn't 
remove that support without having a very good reason.


There is no reason to impose such restrictions for helpers in Org since 
Org relies on resolving ambiguities with minimal efforts.


`encode-time' supports just one kind of ambiguities. Even though it is 
the most common one, it is rather (partially false) impression than real 
support.


The truth is that most of developers are not aware of real complexity of 
dealing with time. Documentation rarely describes limitations and corner 
cases that should be handled. Even when correct time handling gets more 
priority i

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/13/22 07:40, Max Nikulin wrote:


I do not see a way to get 23:30 EAT +0300.


Are you asking for a function F where you say, "I want to give F a 
possibly-ambiguous decoded local time D, and for F to return all 
timestamps that map to D"? If so, encode-time doesn't do that, because 
the underlying C API (namely, mktime) doesn't do that. All mktime and 
encode-time do is give you *one* timestamp that maps to D; it won't give 
you any other timestamps.


If you're worried about possibly-ambiguous decoded local times, you 
could probe (say) one day before and one day after encode-time's result 
to see if the UTC offset changes, and let that guide you to find other 
possible timestamps that map to the decoded time. Although this is just 
a heuristic it should be good enough.


I doubt whether you need to do that, though. Code that is not careful 
about local time offsets doesn't care how ambiguous decoded times are 
resolved. And code that does care should record UTC offsets anyway, and 
you can use those offsets to disambiguate the decoded times. Something 
like this, say:


 (defun encode-and-format-time (time tz)
   (let ((etime (encode-time (parse-time-string time
 (format-time-string "%F %T %Z %z" etime tz)))

With this definition, (encode-and-format-time "2021-01-31 23:30:00 
+0300" "Africa/Juba") yields "2021-01-31 23:30:00 EAT +0300", which is 
the timestamp you want.




`encode-time' should only accept time zone as time offset and should not allow 
default or named value that may be ambiguous.


If we're talking about Org's encode-time substitute, you can of course 
do what you like. But Emacs encode-time has supported ambiguous 
timestamps for some time and I expect it's used by apps that don't care 
how ambiguous decoded times are resolved, which means we shouldn't 
remove that support without having a very good reason.




should be possible to provide hints to `encode-time' to get deterministic 
behavior in the case of time transitions


Yes, that feature is already there. The hint is the UTC offset, as 
illustrated above.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 09/04/2022 14:52, Paul Eggert wrote:

0001-Improve-Org-usage-of-timestamps.patch

From 094345e10ad45e06f7b32e2f8017592210f43463 Mon Sep 17 00:00:00 2001
From: Paul Eggert 
Date: Sat, 9 Apr 2022 00:17:09 -0700
Subject: [PATCH] Improve Org usage of timestamps
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The main thing is to follow the (encode-time X) convention where
X’s DST component of nil means standard time, -1 means unknown.


I am marking this thread as a patch for https://updates.orgmode.org
"Use -1 (guess) instead of nil (no) for DST value in encode-time arguments"

From my point of view the changes quoted below are most important ones 
in the patch and they should be committed to the Org repository. 
Preferably a unit test for time zones should be added for 
`org-parse-time-string'.


I omitted changes to improve code style, they should be applied as well, 
I just consider them as having less priority.


I am in doubt concerning `org-encode-time-1' part.


diff --git a/lisp/org/ol.el b/lisp/org/ol.el
index a03d85f618..fe6e97e928 100644
--- a/lisp/org/ol.el
+++ b/lisp/org/ol.el
@@ -1575,9 +1575,7 @@ org-store-link
  (setq link
(format-time-string
 (car org-time-stamp-formats)
-(apply 'encode-time
-   (list 0 0 0 (nth 1 cd) (nth 0 cd) (nth 2 cd)
- nil nil nil
+(encode-time 0 0 0 (nth 1 cd) (nth 0 cd) (nth 2 cd
  (org-link-store-props :type "calendar" :date cd)))
 
((eq major-mode 'w3-mode)

It allows to avoid a pitfall with nil as DST value.


diff --git a/lisp/org/org-macs.el b/lisp/org/org-macs.el
index b10725bd52..0916da89ac 100644
--- a/lisp/org/org-macs.el
+++ b/lisp/org/org-macs.el
@@ -1242,7 +1242,7 @@ org-parse-time-string
(string-to-number (match-string 4 s))
(string-to-number (match-string 3 s))
(string-to-number (match-string 2 s))
-   nil nil nil))
+   nil -1 nil))


It definitely must be changed in the Org code.


diff --git a/lisp/org/org.el b/lisp/org/org.el
index d656a51591..1bceb0f53a 100644
--- a/lisp/org/org.el
+++ b/lisp/org/org.el
@@ -14334,7 +14334,7 @@ org-read-date-analyze
 (setq year (nth 5 org-defdecode))
 (setq org-read-date-analyze-forced-year t
 (setq org-read-date-analyze-futurep futurep)
-(list second minute hour day month year)))
+(list second minute hour day month year nil -1 nil)))
 
 (defvar parse-time-weekdays)

 (defun org-read-date-get-relative (s today default)


Unsure concerning adding extra elements. I would prefer change of 
`encode-time' in future and a compatibility macro as in the following 
for a while.


Max Nikulin [DRAFT][PATCH] org-encode-time compatibility and convenience 
helper. Mon, 11 Apr 2022 22:22:48 +0700.

https://lsit.orgmode.org/7f4ea652-7d22-fb61-f873-5e92f078c...@gmail.com

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 09/04/2022 14:52, Paul Eggert wrote:

On 4/7/22 05:37, Max Nikulin wrote:

 From my point of view it is better to change implementation of 
`encode-time' so that it may accept 6-component list SECOND...YEAR. It 
should not add noticeable performance penalty but makes the function 
more convenient in use.


Unfortunately it makes the function more convenient to use incorrectly. 
This was part of the motivation for the API change. The obsolescent 
calling convention has no way to deal with ambiguous timestamps like 
2022-11-06 01:30 when TZ="America/Los_Angeles". Org mode surely has bugs 
in this area, although I don't have time to scout them out.


Handling DST is a step forward, it is an important case. Unfortunately 
there are enough peculiarities in the time zoo. I do not think new 
interface can be used correctly in the following case of time transition 
with no change of DST:


#+name: encode-time-and-format
#+begin_src elisp :var time=nil :var tz=() :var dst=-1
  (let* ((seconds-year
  (reverse (mapcar #'string-to-number (split-string time "[- :]"
 (time-list (append seconds-year (list nil dst tz
(format-time-string "%F %T %Z %z" (encode-time time-list) tz))
#+end_src

zdump -v Africa/Juba
...
Africa/Juba  Sun Jan 31 20:59:59 2021 UT = Sun Jan 31 23:59:59 2021 EAT 
isdst=0 gmtoff=10800
Africa/Juba  Sun Jan 31 21:00:00 2021 UT = Sun Jan 31 23:00:00 2021 CAT 
isdst=0 gmtoff=7200


#+call: encode-time-and-format(time="2021-01-31 23:30:00", 
tz="Africa/Juba", dst=-1)


#+RESULTS:
: 2021-01-31 23:30:00 CAT +0200

#+call: encode-time-and-format(time="2021-01-31 23:30:00", 
tz="Africa/Juba", dst=())


#+RESULTS:
: 2021-01-31 23:30:00 CAT +0200

#+call: encode-time-and-format(time="2021-01-31 23:30:00", 
tz="Africa/Juba", dst='t)
: Debugger entered--Lisp error: (error "Specified time is not 
representable")



I do not see a way to get 23:30 EAT +0300. For you example with regular 
DST transition there is no any problem:



#+call: encode-time-and-format(time="2022-11-06 01:30:00", 
tz="America/Los_Angeles", dst=-1)


#+RESULTS:
: 2022-11-06 01:30:00 PDT -0700

#+call: encode-time-and-format(time="2022-11-06 01:30:00", 
tz="America/Los_Angeles", dst=())


#+RESULTS:
: 2022-11-06 01:30:00 PST -0800

#+call: encode-time-and-format(time="2022-11-06 01:30:00", 
tz="America/Los_Angeles", dst='t)


#+RESULTS:
: 2022-11-06 01:30:00 PDT -0700

PS. Org mode usually uses encode-time for calendrical calculations. This 
is dicey, as some days don't exist (for example, December 30, 2011 does 
not exist if TZ="Pacific/Apia", because Samoa moved across the 
International Date Line that day). And it's also dicey when Org mode 
uses 00:00:00 (midnight at the start of the day) as a timestamp 
representing the entire day, as it's all too common for midnight to not 
exist (or to be duplicated) due to a DST transition. Generally speaking, 
when Org mode is doing calendrical calculations it should use 
calendrical functions rather than encode-time+decode-time, which are 
best used for time calculations not calendar calculations. (I realize 
that fixing this in Org would be nontrivial; perhaps I should file this 
"PS" as an Org bug report for whoever has time to fix it)


Then `encode-time' should only accept time zone as time offset and 
should not allow default or named value that may be ambiguous. However 
my opinion is that is should be possible to provide hints to 
`encode-time' to get deterministic behavior in the case of time transitions.



diff --git a/lisp/org/org-compat.el b/lisp/org/org-compat.el
index 819ce74d93..247373d6b9 100644
--- a/lisp/org/org-compat.el
+++ b/lisp/org/org-compat.el
@@ -115,6 +115,27 @@ org-table1-hline-regexp
   (defun org-time-convert-to-list (time)
 (seconds-to-time (float-time time
 
+;; Like Emacs 27+ `encode-time' with one argument.

+(if (ignore-errors (encode-time (decode-time)))
+(defsubst org-encode-time-1 (time)
+  (encode-time time))
+  (defun org-encode-time-1 (time)
+(let ((dst-zone (nthcdr 7 time)))
+  (unless (consp (cdr dst-zone))
+   (signal wrong-type-argument (list time)))
+  (let ((etime (apply #'encode-time time))
+   (dst (car dst-zone))
+   (zone (cadr dst-zone)))
+   (when (and (symbolp dst) (not (integerp zone)) (not (consp zone)))
+ (let* ((detime (decode-time etime))
+(dedst (nth 7 detime)))
+   (when (and (not (eq dedst dst)) (symbolp dedst))
+ ;; Assume one-hour DST and adjust the timestamp.
+ (setq etime (time-add etime (seconds-to-time
+  (- (if dedst 3600 0)
+ (if dst 3600 0


I am against this workaround. It fixes (to some extent) usual DST 
transitions, but it adds another bug


Australia/Lord_Howe  Sat Apr  2 14:59:59 2022 UT = Sun Apr  3 01:59:59 
2022 +11 isdst=1 gmtoff=39600
Australia/Lord_Howe  Sat Apr  2 15:00:00 2022 UT = Su

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

I am sorry, I have realized that debbugs.gnu.org does not send copies to 
the addresses specified in the X-Debbugs-CC header of the original 
report. Now I am sending a copy of my earlier message to emacs-orgmode.


Please, add 54...@debbugs.gnu.org to replies to this message.

Paul,

I am sorry that I forced you to make more changes in the Org code. While 
I was filing this bug, my intention was to add something like "if 
(!NILP(...)) {  }" around several lines in src/timefns.c.


Now we should decide whether we leave this bug for possible changes in 
the `encode-time' implementation and moving the discussion related to 
further changes in Org to the emacs-orgmode mail list or we change the 
title of this bug and maybe create another one for `encode-time'.


On 09/04/2022 14:52, Paul Eggert wrote:

On 4/7/22 05:37, Max Nikulin wrote:

 (encode-time '(0 30 20 07 04 2022 nil nil nil)) ; wrong! 


Yes, and I see a couple of places (org-parse-time-string, 
org-read-date-analyze) where Org mode returns the wrong decoded 
timestamps, ending in (nil nil nil)


I see you even change `org-store-link' that has the same problem.

Obsolescent, not obsolete. The old form still works and it's not going 
away any time soon. If the efficiency concerns you mention are 
significant, we should keep the old form indefinitely.


I am against preserving the old form because it ignores the DST field. 
It is confusing and error prone to be a part of a well designed interface.


Actually I have a draft of `org-encode-time' macro that transforms 6 
elements list to 9 elements one at load or compile time, so it should 
not hurt runtime performance. I have not tried to replace all calls of 
the `encode-time' function yet however. But I still prefer to drop 6/9 
elements branch even if it may happen a decade later.
 From my point of view it is better to change implementation of 
`encode-time' so that it may accept 6-component list SECOND...YEAR. It 
should not add noticeable performance penalty but makes the function 
more convenient in use.


Unfortunately it makes the function more convenient to use incorrectly. 
This was part of the motivation for the API change. The obsolescent 
calling convention has no way to deal with ambiguous timestamps like 
2022-11-06 01:30 when TZ="America/Los_Angeles". Org mode surely has bugs 
in this area, although I don't have time to scout them out.


I do not see your point here. Old calling convention did not allow to 
specify DST flag at all and it was a problem. With the list argument 
even if last 3 fields are optional, it will not prevent adding DST value 
when it is known from some source. I think, requiring 3 extra fields 
when DST value is unknown is too hing price just to make a developer 
aware that it may be ambiguous (moreover it will not work without a 
clear statement in the docstring).


Org timestamps does not allow to specify timezone abbreviation such as 
PDT/PST to distinguish DST. If it were added then users would have false 
impression of full time zones support in Org. It would require huge 
amount of work. So guessed DST is the best that often can be offered.


Daylight saving time field matters only as a list component and 
ignored as a separate argument (by the way, it should be stressed in 
the docstring).


Do you have a wording suggestion? (The doc string already covers the 
topic concisely; however, conciseness is not always a virtue. :-)


My point is that subtle breaking changes must be prominent and hard to 
ignore. I do not have yet ready phases and you highlighted another issue 
missed in the docstring.


So my proposal is to not force Org mode to use new calling convention 
for `encode-time' till DST and ZONE list components will became 
optional ones in a released Emacs version.


This would delay things for ten years or so, no? We'd have to wait until 
Org mode supported only Emacs 29 and later.


I do not think it is a problem.

Instead, I suggest that we stick with what we have when that's cleaner. 
That is, Org mode can use the obsolescent encode-time API when it's 
cleaner to do that.


I considered such approach to defense against "aggressive" modernizing 
of Emacs code. Then I decided it is better to allow to deprecate one of 
the styles of calling `encode-time'. I tried to express it in the 
original report and above.



I haven't installed the patch, or tested it other than via 'make check'.


Org has its own repository and changes should be committed there at first.

Org has unit tests, see
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/testing/README
you changes should be accompanied by some adjustments in test expectations.

I believe that at least in some cases Org test suite should be run for 
the bundled version of Org after Emacs builds. There are may be 
copyright issues with including the tests into the Emacs source tree. 
Maybe some changes in tests were accepted without paperwork.


PS. Org mode usually uses encode-time for c

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/7/22 05:37, Max Nikulin wrote:

 (encode-time '(0 30 20 07 04 2022 nil nil nil)) ; wrong! 


Yes, and I see a couple of places (org-parse-time-string, 
org-read-date-analyze) where Org mode returns the wrong decoded 
timestamps, ending in (nil nil nil) even though the DST flag is unknown 
so they should end in (nil -1 nil). Please see attached proposed patch 
which fixes this (also see below).



Since Emacs-27 time fields as separated arguments are considered 
obsolete for calls of `encode-time'.


Obsolescent, not obsolete. The old form still works and it's not going 
away any time soon. If the efficiency concerns you mention are 
significant, we should keep the old form indefinitely.



t is inconvenient to add 3 extra mandatory components at 
the each place.


Then let's keep using the obsolescent calling convention for places 
where that's convenient. Perhaps we should change the documentation to 
say "older" instead of "obsolescent".



 From my point of view it is better to change implementation of 
`encode-time' so that it may accept 6-component list SECOND...YEAR. It 
should not add noticeable performance penalty but makes the function 
more convenient in use.


Unfortunately it makes the function more convenient to use incorrectly. 
This was part of the motivation for the API change. The obsolescent 
calling convention has no way to deal with ambiguous timestamps like 
2022-11-06 01:30 when TZ="America/Los_Angeles". Org mode surely has bugs 
in this area, although I don't have time to scout them out.



Daylight saving 
time field matters only as a list component and ignored as a separate 
argument (by the way, it should be stressed in the docstring).


Do you have a wording suggestion? (The doc string already covers the 
topic concisely; however, conciseness is not always a virtue. :-)


The reason -1 is the default in the obsolete API is backward 
compatibility. If we could have designed the API from scratch it would 
have been different.



In the Org code it is unsure which way to call `encode-time' is more 
convenient. In a half of the cases a list is obtained from another 
function, but another half is timestamp built from computed components. 
Unless the inconsistency with DST I would say that both ways to call the 
function should be supported.


Yes, that's the idea.


So my proposal is to not force Org mode to use new calling convention 
for `encode-time' till DST and ZONE list components will became optional 
ones in a released Emacs version.


This would delay things for ten years or so, no? We'd have to wait until 
Org mode supported only Emacs 29 and later.


Instead, I suggest that we stick with what we have when that's cleaner. 
That is, Org mode can use the obsolescent encode-time API when it's 
cleaner to do that.


It would be helpful for Org mode to use the new encode-time form in some 
cases, when the new form is cleaner. It's easy to support the new form 
efficiently even in older Emacs, using a compatibility shim. This is 
also in the attached proposed patch.


This patch has a few other minor cleanups in the area.

I haven't installed the patch, or tested it other than via 'make check'.


PS. Org mode usually uses encode-time for calendrical calculations. This 
is dicey, as some days don't exist (for example, December 30, 2011 does 
not exist if TZ="Pacific/Apia", because Samoa moved across the 
International Date Line that day). And it's also dicey when Org mode 
uses 00:00:00 (midnight at the start of the day) as a timestamp 
representing the entire day, as it's all too common for midnight to not 
exist (or to be duplicated) due to a DST transition. Generally speaking, 
when Org mode is doing calendrical calculations it should use 
calendrical functions rather than encode-time+decode-time, which are 
best used for time calculations not calendar calculations. (I realize 
that fixing this in Org would be nontrivial; perhaps I should file this 
"PS" as an Org bug report for whoever has time to fix it)From 094345e10ad45e06f7b32e2f8017592210f43463 Mon Sep 17 00:00:00 2001
From: Paul Eggert 
Date: Sat, 9 Apr 2022 00:17:09 -0700
Subject: [PATCH] Improve Org usage of timestamps
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The main thing is to follow the (encode-time X) convention where
X’s DST component of nil means standard time, -1 means unknown.
* lisp/org/ol.el (org-store-link): Prefer plain (encode-time ...)
to (apply 'encode-time ...), for speed.
* lisp/org/org-clock.el (org-clock-sum)
(org-clock-update-time-maybe):
Prefer org-time-string-to-seconds to doing it by hand.
* lisp/org/org-colview.el (org-colview-construct-allowed-dates):
* lisp/org/org.el (org-time-string-to-time, org-timestamp-change):
Prefer (org-encode-time-1 X) to (apply #'encode-time X) when X is
the output of org-parse-time-string.  They're equivalent and
the former is a bit cleaner.
* lisp/org/org-compat.el (org-encode-time-1): New function.
*