Re: [BUG/FR] org-plot: replot on resize [9.7-pre (N/A @ /home/viz/lib/emacs/straight/build/org/)]

[Visuwesh]

[சனி ஜூன் 15, 2024] Ihor Radchenko wrote:

> Visuwesh  writes:
>
>>> So, you can set DATA to (list (or dump-func 'org-plot/gnuplot-to-data) 
>>> table num-cols params)
>>
>> Done in the attached patch.
>
> Thanks!
>
>>;; Dump table to datafile
>> -  (let ((dump-func (plist-get type :data-dump)))
>> +  (let* ((dump-func (plist-get type :data-dump)))
>
> Why let*?

Oops, leftover change from a previous iteration of the patch.  Now
fixed.

>> +(setq data-file (org-babel-temp-stable-file
>> + (list (or dump-func 'org-plot/gnuplot-to-data)
>> +   table num-cols params)
>> + "org-plot"))
>
> Please add in-code comment explaining why we need 
> `org-babel-temp-stable-file'.

Now done.

>From a922946b3965e117dc3bbbe5a4f3c67dcc832d68 Mon Sep 17 00:00:00 2001
From: Visuwesh 
Date: Sat, 15 Jun 2024 10:25:19 +0530
Subject: [PATCH] org-plot: Make data-file stable for replot-on-resize

* lisp/org-plot.el (org-plot/gnuplot): Use a stable data-file to make
replot-on-resize in GUI terminals work.

Reported-by: Visuwesh 
Link: https://orgmode.org/list/87mso7sl6g@gmail.com
---
 lisp/org-plot.el | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/lisp/org-plot.el b/lisp/org-plot.el
index 283d993..83483b2 100644
--- a/lisp/org-plot.el
+++ b/lisp/org-plot.el
@@ -662,8 +662,7 @@ (defun org-plot/gnuplot (&optional params)
   (looking-at "[[:space:]]*#\\+"))
 (setf params (org-plot/collect-options params
 ;; collect table and table information
-(let* ((data-file (make-temp-file "org-plot"))
-   (table (let ((tbl (save-excursion
+(let* ((table (let ((tbl (save-excursion
(org-plot/goto-nearest-table)
(org-table-to-lisp
 		(when (pcase (plist-get params :transpose)
@@ -681,12 +680,11 @@ (defun org-plot/gnuplot (&optional params)
 			   (nth 0 table
 	   (type (assoc (plist-get params :plot-type)
 			org-plot/preset-plot-types))
-   gnuplot-script)
+   gnuplot-script data-file)
 
   (unless type
 	(user-error "Org-plot type `%s' is undefined" (plist-get params :plot-type)))
 
-  (run-with-idle-timer 0.1 nil #'delete-file data-file)
   (when (eq (cadr table) 'hline)
 	(setf params
 	  (plist-put params :labels (car table))) ; headers to labels
@@ -697,6 +695,12 @@ (defun org-plot/gnuplot (&optional params)
 			(setf params (org-plot/collect-options params
   ;; Dump table to datafile
   (let ((dump-func (plist-get type :data-dump)))
+;; Use a stable temporary file to ensure that 'replot' upon
+;; resizing a GUI gnuplot terminal window works.
+(setq data-file (org-babel-temp-stable-file
+ (list (or dump-func 'org-plot/gnuplot-to-data)
+   table num-cols params)
+ "org-plot"))
 (if dump-func
 	(funcall dump-func table data-file num-cols params)
 	  (org-plot/gnuplot-to-data table data-file params)))
-- 
2.43.0

Re: [BUG] org-plot: Unable to use text xtics with type:2d (+ more) [9.7-pre (N/A @ /home/viz/lib/emacs/straight/build/org/)]

[Visuwesh]

[சனி ஜூன் 15, 2024] Ihor Radchenko wrote:

> Visuwesh  writes:
>
>> Sorry for the noise, I copied the wrong link in the commit message.
>> Please see attached instead.
>
> Thanks!
> I have some comments.
>
>> -   (type (assoc (plist-get params :plot-type)
>> -org-plot/preset-plot-types))
>> +   (type (cdr (assoc (plist-get params :plot-type)
>> + org-plot/preset-plot-types)))
>> gnuplot-script)
>
> This may break the existing customization.
> Later in the function, TYPE is used as an argument for
> `org-plot/gnuplot-term-extra' and `org-plot/gnuplot-script-preamble'.
> Some users may have these two custom options adjusted to the older
> calling convention.
>
> To not break things, we should pass the full `assoc' to these functions.

If you meant org-plot/gnuplot-script eventually calling these functions,
then I don't see how the above change will break things.  Even in
org-plot/gnuplot-script, TYPE passed to both these user options are

(let* ((type-name (plist-get params :plot-type))
   (type (cdr (assoc type-name org-plot/preset-plot-types

so there should be no harm done by the above change since TYPE is not an
argument taken by org-plot/gnuplot-script.

> Also, while you are at it, may your please clarify what TYPE means in
> the docstrings of `org-plot/gnuplot-term-extra' and
> `org-plot/gnuplot-script-preamble'?

OK.

>>  (setf params (org-plot/collect-options params
>> +  (setq params (org-combine-plists type params))
>
> May you also drop a short comment in the code that explains what this
> line does?

I will send an updated patch once my confusion is cleared.

Re: [BUG] orgalist send bug

[Rustom Mody]

Oops after sending the last mail, when I closed the debug window I see this
warning window below

⛔ Warning (emacs): Org version mismatch.
This warning usually appears when a built-in Org version is loaded
prior to the more recent Org version.

Version mismatch is commonly encountered in the following situations:

1. Emacs is loaded using literate Org config and more recent Org
   version is loaded inside the file loaded by ‘org-babel-load-file’.
   ‘org-babel-load-file’ triggers the built-in Org version clashing
   the newer Org version attempt to be loaded later.

   It is recommended to move the Org loading code before the
   ‘org-babel-load-file’ call.

2. New Org version is loaded manually by setting ‘load-path’, but some
   other package depending on Org is loaded before the ‘load-path’ is
   configured.
   This "other package" is triggering built-in Org version, again
   causing the version mismatch.

   It is recommended to set ‘load-path’ as early in the config as
   possible.

3. New Org version is loaded using straight.el package manager and
   other package depending on Org is loaded before straight triggers
   loading of the newer Org version.

   It is recommended to put

(straight-use-package 'org)

   early in the config.  Ideally, right after the straight.el
   bootstrap.  Moving ‘use-package’ :straight declaration may not be
   sufficient if the corresponding ‘use-package’ statement is
   deferring the loading.

4. A new Org version is synchronized with Emacs git repository and
   stale .elc files are still left from the previous build.

   It is recommended to remove .elc files from lisp/org directory and
   re-compile.
⛔ Warning (emacs): Org version mismatch.
This warning usually appears when a built-in Org version is loaded
prior to the more recent Org version.

Version mismatch is commonly encountered in the following situations:

1. Emacs is loaded using literate Org config and more recent Org
   version is loaded inside the file loaded by ‘org-babel-load-file’.
   ‘org-babel-load-file’ triggers the built-in Org version clashing
   the newer Org version attempt to be loaded later.

   It is recommended to move the Org loading code before the
   ‘org-babel-load-file’ call.

2. New Org version is loaded manually by setting ‘load-path’, but some
   other package depending on Org is loaded before the ‘load-path’ is
   configured.
   This "other package" is triggering built-in Org version, again
   causing the version mismatch.

   It is recommended to set ‘load-path’ as early in the config as
   possible.

3. New Org version is loaded using straight.el package manager and
   other package depending on Org is loaded before straight triggers
   loading of the newer Org version.

   It is recommended to put

(straight-use-package 'org)

   early in the config.  Ideally, right after the straight.el
   bootstrap.  Moving ‘use-package’ :straight declaration may not be
   sufficient if the corresponding ‘use-package’ statement is
   deferring the loading.

4. A new Org version is synchronized with Emacs git repository and
   stale .elc files are still left from the previous build.

   It is recommended to remove .elc files from lisp/org directory and
   re-compile.
⛔ Warning (emacs): Org version mismatch.
This warning usually appears when a built-in Org version is loaded
prior to the more recent Org version.

Version mismatch is commonly encountered in the following situations:

1. Emacs is loaded using literate Org config and more recent Org
   version is loaded inside the file loaded by ‘org-babel-load-file’.
   ‘org-babel-load-file’ triggers the built-in Org version clashing
   the newer Org version attempt to be loaded later.

   It is recommended to move the Org loading code before the
   ‘org-babel-load-file’ call.

2. New Org version is loaded manually by setting ‘load-path’, but some
   other package depending on Org is loaded before the ‘load-path’ is
   configured.
   This "other package" is triggering built-in Org version, again
   causing the version mismatch.

   It is recommended to set ‘load-path’ as early in the config as
   possible.

3. New Org version is loaded using straight.el package manager and
   other package depending on Org is loaded before straight triggers
   loading of the newer Org version.

   It is recommended to put

(straight-use-package 'org)

   early in the config.  Ideally, right after the straight.el
   bootstrap.  Moving ‘use-package’ :straight declaration may not be
   sufficient if the corresponding ‘use-package’ statement is
   deferring the loading.

4. A new Org version is synchronized with Emacs git repository and
   stale .elc files are still left from the previous build.

   It is recommended to remove .elc files from lisp/org directory and
   re-compile.
⛔ Warning (emacs): Org version mismatch.
This warning usually appears when a built-in Org version is loaded
prior to the more recent Org version.

Version mismatch is commonly encoun

Re: [BUG] orgalist send bug

[Rustom Mody]

On Sun, Jun 16, 2024 at 5:42 PM Ihor Radchenko  wrote:

>
> Please follow https://orgmode.org/manual/Feedback.html#Feedback
>
>
1. make repro
2. package-initialize (to get org-alist)
3. load file shown above
4. M-x orgalist-send-list



Backtrace

Debugger entered--Lisp error: (void-variable org-element-cache-version)
  (list 'version org-element-cache-version)
  (list '(elisp org-element--cache) (list 'version
org-element-cache-version))
  (org-persist-load (list '(elisp org-element--cache) (list 'version
org-element-cache-version)) (current-buffer) 'match-hash :read-related t)
  (progn (org-persist-load (list '(elisp org-element--cache) (list 'version
org-element-cache-version)) (current-buffer) 'match-hash :read-related t))
  (if (and org-element-cache-persistent org-element-use-cache) (progn
(org-persist-load (list '(elisp org-element--cache) (list 'version
org-element-cache-version)) (current-buffer) 'match-hash :read-related t)))
  (let ((delay-mode-hooks t)) (outline-mode) (setq major-mode 'org-mode)
(setq mode-name "Org") (progn (if (get 'outline-mode 'mode-class) (put
'org-mode 'mode-class (get 'outline-mode 'mode-class))) (if (keymap-parent
org-mode-map) nil (set-keymap-parent org-mode-map (current-local-map)))
(let ((parent (char-table-parent org-mode-syntax-table))) (if (and parent
(not (eq parent (standard-syntax-table nil (set-char-table-parent
org-mode-syntax-table (syntax-table (if (or (abbrev-table-get
org-mode-abbrev-table :parents) (eq org-mode-abbrev-table
local-abbrev-table)) nil (abbrev-table-put org-mode-abbrev-table :parents
(list local-abbrev-table (use-local-map org-mode-map) (set-syntax-table
org-mode-syntax-table) (setq local-abbrev-table org-mode-abbrev-table) (set
(make-local-variable 'org-mode-loading) t) (set (make-local-variable
'tab-width) 8) (org-load-modules-maybe) (if org-agenda-file-menu-enabled
(progn (org-install-agenda-files-menu))) (set (make-local-variable
'outline-regexp) org-outline-regexp) (set (make-local-variable
'outline-level) 'org-outline-level) (org-element-cache-reset) (if (and
org-element-cache-persistent org-element-use-cache) (progn
(org-persist-load (list '(elisp org-element--cache) (list 'version
org-element-cache-version)) (current-buffer) 'match-hash :read-related t)))
(org-set-regexps-and-options) (add-to-invisibility-spec '(org-link))
(org-fold-initialize (or (and (stringp org-ellipsis) (not (equal ""
org-ellipsis)) org-ellipsis) "...")) (make-local-variable
'org-link-descriptive) (if (eq org-fold-core-style 'overlays) (progn
(add-to-invisibility-spec '(org-hide-block . t (if (and (stringp
org-ellipsis) (not (equal "" org-ellipsis))) (progn (if org-display-table
nil (setq org-display-table (make-display-table))) (set-display-table-slot
org-display-table 4 (vconcat (mapcar #'(lambda ... ...) org-ellipsis)))
(setq buffer-display-table org-display-table)))
(org-set-font-lock-defaults) (if (and org-tag-faces (not
org-tags-special-faces-re)) (progn (org-set-tag-faces 'org-tag-faces
org-tag-faces))) (set (make-local-variable 'calc-embedded-open-mode) "# ")
(set-syntax-table (make-syntax-table org-mode-syntax-table)) (set
(make-local-variable 'font-lock-unfontify-region-function)
'org-unfontify-region) (set (make-local-variable
'org-table-may-need-update) t) (add-hook 'before-change-functions
'org-before-change-function nil 'local) (add-hook 'kill-buffer-hook
'org-check-running-clock nil 'local) (org-fold--advice-edit-commands)
(org-macro-initialize-templates) (org-update-radio-target-regexp) (set
(make-local-variable 'indent-line-function) 'org-indent-line) (set
(make-local-variable 'indent-region-function) 'org-indent-region)
(org-setup-filling) (org-setup-comments-handling) (set (make-local-variable
'beginning-of-defun-function) 'org-backward-element) (set
(make-local-variable 'end-of-defun-function) #'(lambda nil (if (not
(org-at-heading-p)) (org-forward-element) (org-forward-element)
(forward-char -1 (set (make-local-variable 'next-error-function)
'org-occur-next-match) (set (make-local-variable
'add-log-current-defun-function) #'org-add-log-current-headline) (if
org-enforce-todo-dependencies (add-hook 'org-blocker-hook
'org-block-todo-from-children-or-siblings-or-parent) (remove-hook
'org-blocker-hook 'org-block-todo-from-children-or-siblings-or-parent)) (if
org-enforce-todo-checkbox-dependencies (add-hook 'org-blocker-hook
'org-block-todo-from-checkboxes) (remove-hook 'org-blocker-hook
'org-block-todo-from-checkboxes)) (set (make-local-variable
'align-mode-rules-list) '((org-in-buffer-settings (regexp . "^[
\11]*#\\+[A-Z_]+:\\(\\s-*\\)\\S-+") (modes quote (org-mode) (set
(make-local-variable 'pcomplete-command-completion-function)
#'org-pcomplete-initial) (set (make-local-variable
'pcomplete-command-name-function) #'org-command-at-point) (set
(make-local-variable 'pcomplete-default-completion-function) #'ignore) (set
(make-local-variable 'pcomplete-parse-arguments-function)
#'org-parse-arguments) (set (make-local-variable

Re: ob-shell: possibly missing initiate-session functions?

[Suhail Singh]

Matt  writes:

> I agree, defining an alias within "org-babel-shell-initialize" seems
> reasonable.  However, it's important that "explicit-shell-file-name"
> is set to the appropriate shell name.  In your case, I suspect it's a
> coincidence that aliasing "org-babel-sh-initiate-session" works.  A
> "shell" block defaults to "explicit-shell-file-name".  This, I
> suspect, is "bash" in your case, yet may be something different on
> another system.

In my case, explicit-shell-file-name has the default value of nil.
However, shell-file-name (which is initialized from $SHELL) is
"/bin/bash".

> The alias would need to close over "explicit-shell-file-name" like is
> done with "org-babel-execute:name".

And what should be done in the case of org-babel-shell-initiate-session?
Should shell-file-name and explicit-shell-file-name remain unaltered
from their existing settings in that case?

I'm happy to send a patch, though it may be simpler for you to do so
yourself in case you have write-access.  Please let me know.

-- 
Suhail

Re: ob-shell: async support in "shell" vs "bash"

[Suhail Singh]

Matt  writes:

> :async should work for all shell types.

Thank you for confirming the expected behaviour.

> Are you finding that it's not?

No.  However, on language-specific shell code blocks (e.g. "bash")
org-lint reports warnings.

> Regarding org-babel-header-args, "org-babel-header-args:shell" is
> explicitly set to '((async . ((yes no, it seems, to quiet the
> linter

Indeed, it does.  For "shell" source blocks, but not for specific
language blocks such as "bash".

> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=40d1352b

I believe there's a bug in that commit.  Specifically, the commit
message states:

#+begin_quote
  Propagate base `org-babel-header-args:shell'
  to`org-babel-header-args:'.
#+end_quote

However, the code initializes org-babel-header-args: to nil,
instead.

> For other languages, such as "org-babel-header-args:bash", it's set to
> nil.  AFAICT, what you see are simply the default values.

Indeed.  It seems to me that the appropriate default for
org-babel-header-args: should be the same as that for
org-babel-header-args:shell , yet it's not.

> I'm curious, what caused you to notice this inconsistency?

Org-lint (in my specific case, I have it configured to run on save via
flycheck).

-- 
Suhail

Re: ob-shell: possibly missing initiate-session functions?

[Matt]

  On Mon, 10 Jun 2024 04:48:10 +0200  Suhail Singh  wrote --- 
 > Currently (Org 9.7.3), org-babel-switch-to-session invokes
 > org-babel-initiate-session which tries to invoke
 > org-babel--initiate-session.  On a related note, ob-shell defines
 > org-babel-sh-initiate-session (and uses it within
 > org-babel-execute:shell).
 > 
 > However, there are no definitions for org-babel-shell-initiate-session
 > nor org-babel-bash-initiate-session etc.  This means that trying to use
 > org-metadown (which invokes org-babel-switch-to-session) on shell and
 > bash language session blocks results in the following errors
 > respectively:
 > 
 > #+begin_comment
 >   org-babel-initiate-session: No org-babel-initiate-session function for 
 > shell!
 >   org-babel-initiate-session: No org-babel-initiate-session function for 
 > bash!
 > #+end_comment
 > 
 > Is this a bug?  If so, does org-babel-shell-initialize need to be
 > patched to set up function aliases pointing to
 > org-babel-sh-initiate-session ?
 > 
 > FWIW, I have been using such aliases in my personal config for bash and
 > shell language blocks and they seem to behave as expected.

Thank you for your message!  Yes, I would consider this a bug.

I agree, defining an alias within "org-babel-shell-initialize" seems 
reasonable.  However, it's important that "explicit-shell-file-name" is set to 
the appropriate shell name.  In your case, I suspect it's a coincidence that 
aliasing "org-babel-sh-initiate-session" works.  A "shell" block defaults to 
"explicit-shell-file-name".  This, I suspect, is "bash" in your case, yet may 
be something different on another system.  The alias would need to close over 
"explicit-shell-file-name" like is done with "org-babel-execute:name".

--
Matt Trzcinski
Emacs Org contributor (ob-shell)
Learn more about Org mode at https://orgmode.org
Support Org development at https://liberapay.com/org-mode

Re: ob-shell: async support in "shell" vs "bash"

[Matt]

  On Mon, 10 Jun 2024 04:33:26 +0200  Suhail Singh  wrote --- 
 > On my system both "shell" and "bash" language blocks use "bash"

 > However, the value of org-babel-header-args:lang differs:
 > 
 > #+begin_src emacs-lisp :results value replace verbatim
 >   org-babel-header-args:shell
 > #+end_src
 > 
 > #+RESULTS:
 > : ((async (yes no)))
 > 
 > #+begin_src emacs-lisp :results value replace verbatim
 >   org-babel-header-args:bash
 > #+end_src
 > 
 > #+RESULTS:
 > : nil
 > 
 > Is this a bug, or are async blocks only currently allowed in "shell"
 > language blocks and not "bash" language blocks?

Thank you for your message!

:async should work for all shell types.  Are you finding that it's not?

In practice, anything but a "no" or "none" for :async should work (see 
"org-babel-comint-use-async").  For example, the following should run the block 
asynchronously:

#+begin_src bash :session *my-bash* :async banana
echo "hi"
sleep 3
echo "bye"
#+end_src

All shell languages use the "explicit-shell-file-name" to define a comint 
buffer using the "shell" command.  For "shell" blocks, Babel uses whatever the 
user has set for "explicit-shell-file-name", most likely "shell-file-name" 
which defaults to /bin/sh.  Many systems symlink /bin/sh to bash for 
interactive shells.  I suspect this is what's happening on your system.  You 
can check this with something like 'ls -la' or 'readlink $(which sh)'.

For all other shell languages, "explicit-shell-file-name" gets set to the block 
language name, literally, when "org-babel-shell-initialize" is called, such as 
when ob-shell is first loaded.  That word, such as "bash", will resolve 
according to, I believe, "exec-path" (which is basically PATH).

Regarding org-babel-header-args, "org-babel-header-args:shell" is explicitly 
set to '((async . ((yes no, it seems, to quiet the linter 
(https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=40d1352b).  
For other languages, such as "org-babel-header-args:bash", it's set to nil.  
AFAICT, what you see are simply the default values.

I'm curious, what caused you to notice this inconsistency?

--
Matt Trzcinski
Emacs Org contributor (ob-shell)
Learn more about Org mode at https://orgmode.org
Support Org development at https://liberapay.com/org-mode

Re: [BUG] "Stack overflow in regexp matcher" when tangling or noweb referencing large blocks [9.7.4 (9.7.4-1387e3 @ /home/andrea/.emacs.d/elpa/org-9.7.4/)]

[Ihor Radchenko]

Andrea  writes:

> Thanks again for Org Mode!
> I recently moved to Org Mode 9.7 and I have a boring bug to share:
> on tangling or :noweb yes <> src blocks, I get "Stack
> overflow in regexp matcher".
>
> This was working with 9.6, maybe the regexp changed in a faulty way?

The change was very small:
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=eaf274909

>Debugger entered--Lisp error: (error "Stack overflow in regexp matcher")
>  org-babel-where-is-src-block-head()
>  org-babel-tangle((4))
>  funcall-interactively(org-babel-tangle (4))
>  command-execute(org-babel-tangle)
> ...
>Debugger entered--Lisp error: (error "Stack overflow in regexp matcher")
>  re-search-forward("^[ \11]*#\\+name:[ \11]*xx[ \11]*\\(?:\n[ 
> \11]*#\\+\\S-+:.*\\)*?..." nil t)

Confirmed.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] Trailing dash is not included in link [9.7.3 (9.7.3-2f1844 @ /home/mwillcock/.emacs.d/elpa/org-9.7.3/)]

[Ihor Radchenko]

Max Nikulin  writes:

>> +*** Trailing =-= is now allowed in plain links
>
> After a look into
>
> 7dcb1afb6 2021-03-24 21:27:24 +0800 Ihor Radchenko: Improve 
> org-link-plain-re
>
> I suspect, it worked prior to v9.5. Without a unit test it may be 
> accidentally broken again.

No, it did not work.
If you can, please do not make such assertions without testing.

>> +: https://domain/test-
>
> example.org, example.net, example.com are domains reserved for usage in 
> examples: 
> 

And so?

>> (or (regexp "[^[:punct:] \t\n]")
>
> I have realized that some Org regexps use [:punct:] *regexp class* and 
> others *syntax class*, see latex math regexp. I am in doubts if the 
> discrepancy is intentional.

It is not intentional, but using syntax classes can sometimes be
fragile.

> I have noticed that the following change
>
> 09ced6d2c 2024-02-03 15:15:46 +0100 Ihor Radchenko: org-link-plain-re: 
> Improve regexp heuristics
>
> that causes
>
>  (link http://example.org/a
> input is exported as
>
>  
>  (link  href="http://example.org/a%3Cb)">http://example.org/a%3Cb)
>
> I expect that ")" should not be parsed as a part of the link. Balanced 
> brackets are tricky with regexps (and it is not possible to match 
> arbitrary nested ones).

It is heuristics. We cannot be 100% right. So, it is what it is.

> Perhaps "[^[:punct:] \t\n]" is too strict in respect to spaces. It does 
> not allow the recommended workaround with zero width space:

You don't need zero width space for links.
Just use .

> As to the original bug report, while reading it, I noticed that 
> thunderbird includes dash into the recognized link for
>
>"https://domain/test-";
>
> I decided to look into its implementation and to my surprise I found: 
> ``punctation chars and "-" at the end are stipped off.'' I realized that 
> double quotes along with angle brackets are treated as a recommended way 
> to mark URLs in plain text. Thunderbird does not consider dash as a part 
> of links for e.g. http://example.org/t- It might be an attempt to 
> reserve possibility to assemble URLs wrapped into several lines with 
> added hyphenation marks, but it has not been implemented (RFC2396 
> appendix E warns about accidentally added hyphens).
>
> https://www.bucksch.org/1/projects/mozilla/16507/
> https://searchfox.org/mozilla-central/source/netwerk/streamconv/converters/mozTXTToHTMLConv.cpp#line-243
> mozTXTToHTMLConv::FindURLEnd
>
> Implementation is tricky, I have not noticed anything that may be reused 
> to improve heuristics for Org. Nowadays it is likely better to inspect 
> autolinking code for GitHub/GitLab or widely used python packages.

If you have concrete proposals, please share them.

> I would consider [:space:] or \s-.

Do you mean "[^[:punct:][:space:]\t\n]"?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] Trailing dash is not included in link [9.7.3 (9.7.3-2f1844 @ /home/mwillcock/.emacs.d/elpa/org-9.7.3/)]

[Max Nikulin]

On 14/06/2024 21:04, Ihor Radchenko wrote:

Morgan Willcock writes:


i.e. Inserting "https://domain/test-"; into the buffer will create a
clickable link for "https://domain/test";.


I improved the heuristics we use to detect plain links.
Fixed, on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=73da6beb5



+++ b/etc/ORG-NEWS

[...]

+*** Trailing =-= is now allowed in plain links


After a look into

7dcb1afb6 2021-03-24 21:27:24 +0800 Ihor Radchenko: Improve 
org-link-plain-re


I suspect, it worked prior to v9.5. Without a unit test it may be 
accidentally broken again.



+: https://domain/test-


example.org, example.net, example.com are domains reserved for usage in 
examples: 




(or (regexp "[^[:punct:] \t\n]")


I have realized that some Org regexps use [:punct:] *regexp class* and 
others *syntax class*, see latex math regexp. I am in doubts if the 
discrepancy is intentional.


I have noticed that the following change

09ced6d2c 2024-02-03 15:15:46 +0100 Ihor Radchenko: org-link-plain-re: 
Improve regexp heuristics


that causes

(link http://example.org/a
(link href="http://example.org/a%3Cb)">http://example.org/a%3Cb)


I expect that ")" should not be parsed as a part of the link. Balanced 
brackets are tricky with regexps (and it is not possible to match 
arbitrary nested ones).


Perhaps "[^[:punct:] \t\n]" is too strict in respect to spaces. It does 
not allow the recommended workaround with zero width space:


(org-export-string-as
 "http://example.org\N{ZERO WIDTH SPACE}[fn::footnote]" 'html 'body)
"
href=\"http://example.org​[fn::footnote]\";>http://example.org​[fn::footnote]

"

Actually some kind of non-breakable space should be better in such cases:

(org-export-string-as
 "http://example.org\N{NO-BREAK SPACE}[fn::footnote]" 'html 'body)
"
href=\"http://example.org [fn::footnote]\";>http://example.org [fn::footnote]

"

I would consider [:space:] or \s-.

As to the original bug report, while reading it, I noticed that 
thunderbird includes dash into the recognized link for


  "https://domain/test-";

I decided to look into its implementation and to my surprise I found: 
``punctation chars and "-" at the end are stipped off.'' I realized that 
double quotes along with angle brackets are treated as a recommended way 
to mark URLs in plain text. Thunderbird does not consider dash as a part 
of links for e.g. http://example.org/t- It might be an attempt to 
reserve possibility to assemble URLs wrapped into several lines with 
added hyphenation marks, but it has not been implemented (RFC2396 
appendix E warns about accidentally added hyphens).


https://www.bucksch.org/1/projects/mozilla/16507/
https://searchfox.org/mozilla-central/source/netwerk/streamconv/converters/mozTXTToHTMLConv.cpp#line-243
mozTXTToHTMLConv::FindURLEnd

Implementation is tricky, I have not noticed anything that may be reused 
to improve heuristics for Org. Nowadays it is likely better to inspect 
autolinking code for GitHub/GitLab or widely used python packages.

[BUG] Updating first subheading checkbox cookie resets parent cookie

[Bruno Cardoso]

Hello everyone,

Consider the following tree:

* Task [1/3]
** TODO test1 [/]
- [x] a
- [ ] b  
** DONE test2
** TODO test3 [/]
- [X] x
- [ ] y

With point at the "test1" heading cookie ("[/]"), after =C-c C-c=, it gets 
updated to "[1/2]", but it also resets its parent's "Task" cookie to "[0/0]".

This only happens with the first heading of a subtree (updating the "task3" 
cookie works as expected).

Org mode version 9.8-pre (release_9.7.3-73-g96d149)


Best,

Bruno.

Fw: place a header from an archive file back in its original location

[JARZz]

Is there any org-mode function (or even a package) that can place a header back 
in its original file from an archive based on the:ARCHIVE_FILE:?

I understand this can be done with refile, but this a manual process which also 
requires the ARCHIVE properties to be cleaned.

for context, I wrote about it on my blog: 
https://taonaw.com/2024/06/15/rethinking-and-organizing.html

- JTR

>

Re: org-babel-execute-src-block filters characters from :session *shell* output

[Max Nikulin]

On 15/06/2024 20:19, Ihor Radchenko wrote:

The underlying cause is limitation of Emacs API for interactive shells -
we cannot easily distinguish command output from prompt and other extra
staff your shell/other interactive command spits into the buffer.
So, we have to either filter output the prompts ourselves to get the
command output reliably or redirect output to files, where nothing
litters the actual output with prompts.


Some shells support "semantic shell" that allows terminal applications 
e.g. to copy whole command output. It is based on escape sequences.


- 
https://docs.kde.org/stable5/en/konsole/konsole/semantic-shell-integration.html
- 
https://gitlab.freedesktop.org/Per_Bothner/specifications/blob/master/proposals/semantic-prompts.md

- https://github.com/tmux/tmux/issues/3064

Re: Please document the caching and its user options

[Ihor Radchenko]

Daniel Clemente  writes:

> In particular, when setting (setq org-element-cache-persistent nil)
> org-mode *should not* create an org-persist directory anywhere. And I
> think it shouldn't activate org-persist timers (it does now) or hooks.
> The user's preference should be respected.

Nope. "org-persist" directory is not only used by org-element. If some
other parts of Org need to cache something, they can also store cache
there.

> That's a code change.
> If you just want to update documentation, a starting point can be
> org-element-cache-persistent's documentation, which is just "Non-nil
> when cache should persist between Emacs sessions.", and doesn't
> mention that some files will always be created even if it's nil. It
> also doesn't explicitly mention that it will create files (better be
> explicit about this), or where (or how to control where), or which
> content (i.e. just statistics, or parts of possible private org
> files).

May you suggest an alternative docstring?

> I suggest making an explicit difference between "caching in memory"
> and "caching by storing files on disk".
> For instance:
> (defvar org-element-use-cache t
>   "Non-nil when Org parser should cache its results.")
> From that description, it's not clear to a new user whether they're
> creating files on disk (as caches often do) or not.

Do you mean something like

"Non-nil when Org parser should cache its results.

The cache is stored in-memory and may also be stored on disk if
`org-element-cache-persistent' is non-nil (the default)."

?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [PATCH] function and symbol for headline and olp for org-capture-templates

[Ihor Radchenko]

Nafiz Islam  writes:

> I've updated my tests with `org-test-at-time' and updated the commit 
> message to fit within default column of 70.

That does not help, unfortunately. My "0" time is still different from
yours.

If will be more reliable to use a specific timestamp. See other uses of
`org-test-at-time' in the tests.

Also, please rebase the patch onto the latest main.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] orgalist send bug

[Ihor Radchenko]

Rustom Mody  writes:

>> I cannot reproduce using the latest Org mode.
>>
>>
> Maybe relevant to "cant reproduce"
>
> I tried to run it with make vanilla -- got the error of mismatching org
> versions
> Tried to remove the elc files as suggested in that error message -- the
> error became more involved.
>
> DIdn't think it very relevant to the bug bugging me ... but...
> Can supply more details if required

Please follow https://orgmode.org/manual/Feedback.html#Feedback

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Please document the caching and its user options

[Eli Zaretskii]

> From: Ihor Radchenko 
> Cc: emacs-orgmode@gnu.org, emacs-de...@gnu.org, michael.albi...@gmx.de
> Date: Sun, 16 Jun 2024 09:05:02 +
> 
> Eli Zaretskii  writes:
> 
> >> I was referring to some kind of global option that defines cache
> >> directory, data directory, etc. Something akin XDG.
> >
> > We already have xdg-cache-home (and a few others in xdg.el).  Is that
> > what you meant?
> 
> Yes, except that `xdg-cache-home' is limited:
> 
> 1. It cannot be customized by users

Of course it can: just make the default value of a defcustom be
derived by xdg-cache-home, and users can then customize the option to
a different value if they want.

> 2. It may sometimes return nil

The fallback is well-known.

> 3. It is limited to XDG - not all the Emacs platforms

No, it's supported on all platforms, even if XDG isn't.

> What I had in mind is a new custom option for cache dir (defaulting to
> OS-specific cache like XDG on Linux or something equivalent on Windows)
> + a new API function like `system-cache-home' that will be guaranteed to
> return some kind of meaningful dir.

Using xdg-cache-home and its fallbacks is a de-facto standard of
solving this in Emacs, and it supports all the platforms.  Even
startup.el uses it (albeit by customized code, to avoid interfering
with user customizations) when looking for init files and suchlikes.

So I think you raise a problem that is already solved in Emacs.

> >> Also, caching is not as simple, because caches may contain sensitive
> >> data. (see
> >> https://list.orgmode.org/orgmode/cam9alr8fusu0yws1sehrw7syxprjfx-r2juxd_dgvcyvkqc...@mail.gmail.com/)
> >> Some users may want to move caches to read-restricted location
> >> or even to location dependent on where the cache is originating from
> >> (separate caches depending on whether default-directory is from
> >> encrypted volume, remote mount, etc)
> >
> > AFAIK, Emacs has APIs for at least some of that, but whether to use
> > them is up to the application, I think.
> 
> What are those APIs?

Making files and directories readable only by the owner, for example:
set-file-modes and with-file-modes.  All the other Lisp programs in
Emacs use that, so why would Org need something special?

> >> Finally, we got several requests to have caches cleared up upon exiting
> >> Emacs, which is also something that should be better managed centrally,
> >> by Emacs, for all possible kinds of cache/history data.
> >
> > Deleting files in a directory, recursively if needed, is already
> > available.  is that what you meant?
> 
> No. I mean a new user option like `clear-caches-on-exit' that will work
> across all the packages.

Having a single option for all the caches makes little sense to me.
This must be a per-cache setting.

However, users on XDG platforms can have that via XDG system-wide
settings.

> Having to set this for each specific package (with some packages not
> documenting that they use cache, or users not expecting that cache may
> be used and not reading _all_ the docs carefully enough) is not ideal,
> IMHO.

I cannot disagree more.  Each cache has its own logic for when it is a
good time to empty the cache.

> > Can we first fix the problems for which I started this thread?  The
> > more general issues should be subjects of separate discussions, IMO.
> 
> If there is a global Emacs-wide customization how to handle caches,
> there will be no need to document it in Org mode manual.

I respectfully ask the Org developers to solve this particular issue
first, without waiting for some hypothetical general Emacs feature,
which may or may not materialize.

> like to see if introducing such global customization is feasible before
> making non-trivial changes to Org manual. (I am not even sure where to
> document these things in the manual yet; they seem way too generic wrt
> Org mode's scope)

A new chapter should be fine, if no existing chapter is relevant.

TIA

Re: Pending contents in org documents

[Ihor Radchenko]

Bruno Barbier  writes:

>>- After failure, the "!" fringe indicator is visible, but it is not
>>  obvious at all that user can click to get details
>>  I first tried to click on the fringe itself to no avail. Then, I
>>  randomly clicked on the text and got the description buffer; but
>>  that was unexpected - the text I clicked did not have any
>>  indication of its "clickability" - neither some kind of underline
>>  face, nor an overlay or a mouse hint.
>
> I couldn't find a way to answer a click on a fringe.  Is there a way?

Well. I do not know a way either :)

> About how much to decorate, it depends on the user, I guess.  For
> example, when org-pending is used for org babel, it should be obvious
> that one has to click on "#+RESULTS:".
>
> The current decoration is not the best for discoverability, indeed.
> But decorating the whole outcome region would be too much IMHO, and,
> it might interfer with other buffer fontifications.

We may do what flycheck/flyspell do. Maybe fontifying not the whole
region, but at least the region part where the indication was placed.

I really find the current behavior unintuitive even for experienced Emacs
users.

> I've refactored the code and added two variables so that it's now
> configurable, see 'org-pending-outcome-pre-display-function' and
> 'org-pending-outcome-post-display-function'.

Makes sense. Although, "display" part in the names is very confusing -
it sounds way too similar to `pre-redisplay-functions', which is
something entirely different.

What about removing org-pending-output-pre-display-function entirely (we
can add it later, if necessary) and renaming
org-pending-outcome-post-display-function to

`org-pending-on-outcome-functions' - an abnormal hook executed after
ON-OUTCOME action.

>> 2. I tried to do M-x undo while the reglock was active, and got an
>>error. I'd expect that undo would work, skipping the region.
>
> I'm not sure exactly what you did, nor which error you got.  I've
> noticed that the hacks (to handle indirect buffers) were flagging the
> buffer as "modified" (using text properties).  I've fixed that.
>
> Is the problem solved now ?

No.

What I did is:

1. make repro
2. Insert the example code from the top comment and uncomment it
3. M-x eval-buffer
4. *While regloc is active*, press C-x u
5. Observe

Debugger entered--Lisp error: (org-pending-error "Cannot modify a region 
containing pending content")
  signal(org-pending-error ("Cannot modify a region containing pending 
content"))
  (closure (t) (&rest _) (signal 'org-pending-error (list "Cannot modify a 
region containing pending content")))(1 81)
  primitive-undo(1 ((1 . 81) (t . 0)))
  undo-more(1)
  undo(nil)
  funcall-interactively(undo nil)
  command-execute(undo)

>> 3. I tried M-x undo _after_ reglock was unlocked, and I got "TO REPLACE"
>>word highlighted. I did not expect it to be highlighted.
>
> I couldn't get that behavior, but the undo wasn't correct either.
>
> org-pending is now directly instrumenting the buffer-undo-list, and
> manually adding an undo-boundary.
>
> Do you still see your problem ?

No, this problem is solved now.

>> 4. If I try to cancel the reglock, it does get canceled, but *Region
>>Lock* buffer is not updated - Live? still displays "yes Cancel".
>
> It's by design.
>
> The function `org-pending-describe-reglock' works like
> `describe-variable' and other similar functions.  You need to revert
> the buffer (g) to update it.

I still find it confusing.
Mostly because it is not clear if pressing "cancel" does anything.

> The reglock is live in its buffer.

What do you mean by that??
How can reglock be active in the buffer that does not contain the locked
text? How can even have different state in different buffers?

>> Does it mean that clicking "cancel" does not guarantee that the region
>> will not be updated by the running process/timer?
>
> Yes, org-pending does not enforce that; and it should not, else it
> would forbid valid use cases of org-pending. A given user of
> org-pending may decide to garantuee that though (using a suitable
> function for :on-outcome).

Imagine that the process that locked the region hangs. As a user, I'd
like to be able to edit text in buffer if I see that the lock is taking
too long. How can I do it without closing the buffer?

>> In my eyes, there is no difference between user request and "kill". If
>> users asks things to stop, no modifications should be allowed to
>> the region.
>
> There is no relation between "kill" and "cancel".
>
> For "kill", *Emacs* is killing the owner of the lock; there is nothing
> to update.  This is synchronous, immediate and definitive.
>
> For "cancel", the *user* is sending an asynchronous message to
> org-pending that it would be nice to release this particular lock
> sooner, if possible; that message implies that the user doesn't care
> about the outcome, but, if that outcome is available, then, just don't
> waste it: insert it in 

Re: Please document the caching and its user options

[Ihor Radchenko]

Eli Zaretskii  writes:

>> I was referring to some kind of global option that defines cache
>> directory, data directory, etc. Something akin XDG.
>
> We already have xdg-cache-home (and a few others in xdg.el).  Is that
> what you meant?

Yes, except that `xdg-cache-home' is limited:

1. It cannot be customized by users
2. It may sometimes return nil
3. It is limited to XDG - not all the Emacs platforms

What I had in mind is a new custom option for cache dir (defaulting to
OS-specific cache like XDG on Linux or something equivalent on Windows)
+ a new API function like `system-cache-home' that will be guaranteed to
return some kind of meaningful dir.

>> Also, caching is not as simple, because caches may contain sensitive
>> data. (see
>> https://list.orgmode.org/orgmode/cam9alr8fusu0yws1sehrw7syxprjfx-r2juxd_dgvcyvkqc...@mail.gmail.com/)
>> Some users may want to move caches to read-restricted location
>> or even to location dependent on where the cache is originating from
>> (separate caches depending on whether default-directory is from
>> encrypted volume, remote mount, etc)
>
> AFAIK, Emacs has APIs for at least some of that, but whether to use
> them is up to the application, I think.

What are those APIs?

>> Finally, we got several requests to have caches cleared up upon exiting
>> Emacs, which is also something that should be better managed centrally,
>> by Emacs, for all possible kinds of cache/history data.
>
> Deleting files in a directory, recursively if needed, is already
> available.  is that what you meant?

No. I mean a new user option like `clear-caches-on-exit' that will work
across all the packages. Then, concerned users may set it to non-nil to
delete *all* the caches upon exiting Emacs.

Having to set this for each specific package (with some packages not
documenting that they use cache, or users not expecting that cache may
be used and not reading _all_ the docs carefully enough) is not ideal,
IMHO.

> Can we first fix the problems for which I started this thread?  The
> more general issues should be subjects of separate discussions, IMO.

If there is a global Emacs-wide customization how to handle caches,
there will be no need to document it in Org mode manual. So, I would
like to see if introducing such global customization is feasible before
making non-trivial changes to Org manual. (I am not even sure where to
document these things in the manual yet; they seem way too generic wrt
Org mode's scope)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at