Re: Async Python src block behavior with :dir header property

[Jack Kamm]

Ihor Radchenko  writes:

> What we can do is to introduce a new backend template function
> org-babel-session-buffer: that will be passed a session name and
> src block params and return the session buffer name.
>
> If such function is not defined, we fall back to assumption that session
> buffer is named the same as the session.
>
> WDYT?

Sounds good -- I think this is the best solution.

Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Ihor Radchenko]

Jim Porter  writes:

> On 2/5/2024 7:07 AM, Ihor Radchenko wrote:
>> It would make sense to add a number of alists:
>> - bounds-of-thing-at-point-provider-alist
>> - same for 'forward-op, 'beginning-op, 'end-op.
>> 
>> After Emacs have those, we can add Org mode support.
>
> That sounds reasonable enough to me; does anyone else have opinions on 
> this? Otherwise, I'll get to work on a patch (though probably not for a 
> couple weeks).

CCing Stefan and Eli.
Please, let us know if the above is something not wanted upstream.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Jim Porter]

On 2/5/2024 7:07 AM, Ihor Radchenko wrote:

It would make sense to add a number of alists:
- bounds-of-thing-at-point-provider-alist
- same for 'forward-op, 'beginning-op, 'end-op.

After Emacs have those, we can add Org mode support.


That sounds reasonable enough to me; does anyone else have opinions on 
this? Otherwise, I'll get to work on a patch (though probably not for a 
couple weeks).

Re: [DISCUSSION] Add "Recent News" to orgmode.org

[Jean Louis]

* Ihor Radchenko  [2024-02-04 22:40]:
> What do you think about an idea to modify Org mode front page
> (https://orgmode.org/), adding the most recent blog posts and
> discussions about Org mode?
> 
> We might use Org-related records from Sacha's news and/or
> https://planet.emacslife.com/ as a source, scrape it regularly (once per
> day/week or on every export), and embed the relevant links into the
> orgweb/index.html
> 
> This way, visitors can easily see how active Org mode community is and
> discover Org-related blogs/forums.

Very good idea!


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: [BUG] Unsolicited download of remote resources

[Ihor Radchenko]

Leo Butler  writes:

> Q: if #+setupfile points to a real file available to download, does Org
> evaluate that file?

keywords and startup options are taken from there. No Elisp code present
in #+SETUPFILE is evaluated.

That said, if the file defines babel header arguments with elisp or
"eval" macros, they may be used later, during export or when src block
evaluation is requested by the user.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] Unsolicited download of remote resources

[Leo Butler]

On Sun, Feb 04 2024, Max Nikulin  wrote:

> On 03/02/2024 02:04, Leo Butler wrote:
>> When I opened your email in Gnus, I was greeted with the same
>> (bewildering) message. Given that Org still tried to download the
>> setupfile after being told not to, I think this is a majour security
>> hole.
>> This is also related to another thread concerning Org and email.
>> https://list.orgmode.org/orgmode/87cyteyhif.fsf@localhost/
>
> Sorry for sending a message with this kind of attachment, but from the
> discussion of that Emacs bug I expected that almost no Gnus users
> should be affected since their media type handler is set for
> text/x-org while Thunderbird uses "Content-Type: text/org".
>
> I would not classify this kind of issues as security ones. I am
> unaware of Org features that may make content of "#+setupfile:" more
> dangerous than the same snippet is included into attachment
> directly. (OK, antivirus might have a chance to detect something as
> dangerous code and "#+setupfile:" would bypass such protection.)
>
> I consider it as a privacy issue. It may allow spammers to track if
> their messages are delivered successfully.

There's no need to apologize--I was surprised at the whole episode.

Q: if #+setupfile points to a real file available to download, does Org
evaluate that file?

Leo

Re: [BUG] URI handling is overly complicated and nonstandard [9.6.7 (N/A @ /gnu/store/mg7223g8mw90lccp6mm5g6f3mpjk70si-emacs-org-9.6.7/share/emacs/site-lisp/org-9.6.7/)]

[Ihor Radchenko]

Max Nikulin  writes:

> On 05/09/2023 16:42, Ihor Radchenko wrote:
>> Max Nikulin writes:
>>>
>>>   From my point of view it will be more sane behavior. However it may
>>> require update of 3rd party ox backends.
>> 
>> Yes. The main problem is that I fail to understand the motivation behind
>> the current behaviour. git logs reveal that the code is there from the
>> initial version of the library.
>
> Just a guess, likely unrelated to actual decision. For links like 
> "lisp:" or "shell:" keeping link type does not have much sense (however 
> stripping it is questionable as well).
>
>  From my point of view, e.g.  should be exported 
> as plain text (identity "a") rather than an (invalid due to 
> not escaped quotes inside href) link (identity 
> "a").
>
> I still believe that fallback export should preserve link type. Code 
> links should define their export functions.

Let's get started on tackling this problem from not stripping the link
type.

I am attaching tentative patch to that effect, as a first step.

>From d0b6d3ff62749aaee40ca37964095bbaa3120128 Mon Sep 17 00:00:00 2001
Message-ID: 
From: Ihor Radchenko 
Date: Mon, 5 Feb 2024 16:39:05 +0100
Subject: [PATCH] org-export: Do not strip link type by default during export

* lisp/ox-html.el (org-html-link):
* lisp/ox-latex.el (org-latex-link):
* lisp/ox-man.el (org-man-link):
* lisp/ox-md.el (org-md-link):
* lisp/ox-odt.el (org-odt-link--inline-image):
* lisp/ox-texinfo.el (org-texinfo-link): Preserve link type during
export for all the links, not just for a hard-coded subset.
* etc/ORG-NEWS (Built-in HTML, LaTeX, Man, Markdown, ODT, and Texinfo
exporters preserve the link protocol during export): Document the
breaking change.

Link: https://list.orgmode.org/orgmode/878r9nofpw.fsf@localhost/
---
 etc/ORG-NEWS   | 18 ++
 lisp/ox-html.el|  4 +---
 lisp/ox-latex.el   |  4 +---
 lisp/ox-man.el |  4 +---
 lisp/ox-md.el  |  4 +---
 lisp/ox-odt.el |  6 ++
 lisp/ox-texinfo.el |  4 +---
 7 files changed, 25 insertions(+), 19 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 965872d23..c5e84b7c7 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -13,6 +13,24 @@ Please send Org bug reports to mailto:emacs-orgmode@gnu.org.
 
 * Version 9.7 (not released yet)
 ** Important announcements and breaking changes
+*** Built-in HTML, LaTeX, Man, Markdown, ODT, and Texinfo exporters preserve the link protocol during export
+
+Previously, some link types where not exported as =protocol:uri= but
+as bare =uri=. This is now changed.
+
+When a link is known by Org mode and does not have a custom ~:export~
+parameter (see A.3 Adding Hyperlink Types section of the manual), the
+link protocol is now not stripped.
+
+For example, if one adds a link type =tel=, but does not define
+~:export~ parameter
+: (org-link-set-parameters "tel")
+=[[tel:12345][John Doe]]= link will be correctly exported to LaTeX as
+=\href{tel:12345}{John Doe}=, not =\href{12345}{John Doe}=.
+
+However, links like =[[elisp:(+ 1 2)]]= will be exported as
+=\url{elisp:(+ 1 2)}=, which may be somewhat unexpected.
+
 *** Org mode now fontifies whole table lines (including newline) according to ~org-table~ face
 
 Previously, leading indentation and trailing newline in table rows
diff --git a/lisp/ox-html.el b/lisp/ox-html.el
index 976c24584..9812ac2b7 100644
--- a/lisp/ox-html.el
+++ b/lisp/ox-html.el
@@ -3231,8 +3231,6 @@ (defun org-html-link (link desc info)
 	 (desc (org-string-nw-p desc))
 	 (path
 	  (cond
-	   ((member type '("http" "https" "ftp" "mailto" "news"))
-	(url-encode-url (concat type ":" raw-path)))
 	   ((string= "file" type)
 	;; During publishing, turn absolute file names belonging
 	;; to base directory into relative file names.  Otherwise,
@@ -3259,7 +3257,7 @@ (defun org-html-link (link desc info)
 		  (concat raw-path
 			  "#"
 			  (org-publish-resolve-external-link option path t))
-	   (t raw-path)))
+	   (t (url-encode-url (concat type ":" raw-path)
 	 (attributes-plist
 	  (org-combine-plists
 	   ;; Extract attributes from parent's paragraph.  HACK: Only
diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index e3edef3bd..060a01b0e 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -2925,12 +2925,10 @@ (defun org-latex-link (link desc info)
 		  link (plist-get info :latex-inline-image-rules)))
 	 (path (org-latex--protect-text
 		(pcase type
-		  ((or "http" "https" "ftp" "mailto" "doi")
-		   (concat type ":" raw-path))
 		  ("file"
 		   (org-export-file-uri raw-path))
 		  (_
-		   raw-path)
+		   (concat type ":" raw-path))
 (cond
  ;; Link type is handled by a special function.
  ((org-export-custom-protocol-maybe link desc 'latex info))
diff --git a/lisp/ox-man.el b/lisp/ox-man.el
index 1f296baeb..16058fb9a 100644
--- a/lisp/ox-man.el
+++ b/lisp/ox-man.el
@@ -614,10 +614,8 @@ (defun org-man-link (link desc info)
  ;; Ensure DESC really exists, or set it to nil.
  

Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Ihor Radchenko]

Jim Porter  writes:

>> Looking into the source code of `bounds-of-thing-at-point', I see that
>> for standard "things" (like url),
>> `thing-at-point-bounds-of-url-at-point' is used unconditionally. In the
>> case of Org links, we may have something like [[https://orgmode.org]]
>> that will not match default URL regexp as is. AFAIU, there is no
>> documented way to customize the behaviour of `bounds-of-thing-at-point'
>> and `forward-thing'.
>
> I think it would make sense to add some sort of 
> 'bounds-of-thing-at-point-provider-alist' (that's a mouthful!) that 
> would let modes override the behavior of 'botap', but I don't think 
> that's necessary for the narrower purpose of asking, "I want the value 
> of THING at point, if any."

It would make sense to add a number of alists:
- bounds-of-thing-at-point-provider-alist
- same for 'forward-op, 'beginning-op, 'end-op.

After Emacs have those, we can add Org mode support.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Exporting multiple #+AUTHOR keywords

[Ihor Radchenko]

Max Nikulin  writes:

>>  Another option is to have a new set of keywords:
>>  #+LATEX_AUTHOR: 
>>  #+HTML_AUTHOR: ...
> ...
> Another idea:
>
> #+metadata:
> - author ::
>- John Doe
>- Luke Skywalker
> - title :: Some text
>
> With overrides for specific backends

This implies that #+AUTHOR may contain markup, while #+metadata should
not; as opposed to my idea with #+AUTHOR being reserved to "plain"
author and #+BACKEND_AUTHOR defining more specific markup.

I feel that my approach is slightly better because you can have
different author markup for different backends when using my suggestion.

> Perhaps backends may declare if they support footnotes, etc. by defining 
> some backend property.

The comment about footnotes belongs to parent thread, not to this
branch.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Another low-hanging fruit

[Ihor Radchenko]

Pedro Andres Aranda Gutierrez  writes:

> Fixed!
> ...
> From 0767bccaabde21241576eaac1103e917c7649f40 Mon Feb 05 00:00:00 2024
> From: "Pedro A. Aranda Gutiérrez" 
> Date: Sat, 5 Feb 2024 07:30:00 +0100
> Subject: [PATCH] Silent anonymous footnote creation

Applied, onto main, with amendments.
I changed the commit message to follow our ChangeLog entry conventions,
as described in
https://orgmode.org/worg/org-contribute.html#commit-messages
I reworded the commit message and the ORG-NEWS entry.
I also fixed the quoting in `org-startup-options'.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=10d2868c5

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] Unexpected result when evaluating python src block asynchronously [9.7-pre (release_9.6.17-1131-gc9ed03.dirty @ /home/yantar92/.emacs.d/straight/build/org/)]

[Ihor Radchenko]

Jack Kamm  writes:

>> So, we have to insert some kind of indicator for async result.
>
> I meant that we could return something like "async:uuid-abcd-1234" or
> "async:/path/to/tmpfile", so that `org-babel-comint-async-filter' could
> still find the result.

Of course, we could. But that would not solve all the possible problems.
In particular, when header arguments tell Org babel to write result to a
file, returning UUID will still not work.

>> Of course, the existing scheme of coordination between
>> `org-babel-insert-result' and `org-babel-comint-async-filter' is
>> erroneous:
>>
>> 1. We have the problem with :results file value discussed here
>> 2. We have a worse problem with :results file :file foo when the result
>>may not be unique
>> 3. We have :results append/prepend completely broken because
>>`org-babel-comint-async-filter' simply calls
>>`org-babel-insert-result' implicitly assuming that the existing
>>indicator is replaced.
>>
>> The whole thing should be re-designed.
> ...
> A bit of a tangent, but if you are thinking about re-designing this,
> then it may be worth considering ob-jupyter's implementation of async
> sessions [1]. In particular, I believe it leaves a marker [2] where it
> needs to insert the future result. I don't remember the details,
> e.g. how it keeps track of which marker is for which result. But it
> seems neat, and might work better for some cases such as
> appending/prepending results.

Markers are not as reliable as you think. If text around marker gets
deleted, the marker will still exist potentially causing the async
result to be inserted in the middle of unexpected place.

Having an actual text indicator is more reliable - if user removes it
before the async evaluation is completed, we will not write anything
unexpected.

Also,
https://github.com/emacs-jupyter/jupyter/blob/master/ob-jupyter.el#L540

;; KLUDGE: Remove the file result-parameter so that
;; `org-babel-insert-result' doesn't attempt to handle it while
;; async results are pending.  Do the same in the synchronous
;; case, but not if link or graphics are also result-parameters,
;; only in Org >= 9.2, since those in combination with file mean
;; to interpret the result as a file link, a useful meaning that
;; doesn't interfere with Jupyter style result insertion.

They also had to work around the same problem.

> I agree that it would be good to redesign it, but am not sure where to
> start.

For example,

1. Change `org-babel-comint-async-register' to return UUID and to store
   PARAMS as passed by the backend (current approach with PARAMS being
   derived from src blocks prevents backends to transform src block
   PARAMS dynamically).
2. Change `org-babel-insert-result' to handle :async t specially,
   inserting something reliable, like #+async:  in place of result
   without performing extra transformations.
3. Change `org-babel-insert-result' to accept an internal parameter
   that will make it replace #+async:  keyword rather than perform
   normal result insertion.
4. Change `org-babel-comint-async-filter' to use the previously passed
   PARAMS, remove :async t from them, and arrange the call to
   `org-babel-insert-result' to replace the #+async:  keyword.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Async Python src block behavior with :dir header property

[Ihor Radchenko]

Jack Kamm  writes:

> It's because ob-python starts the session in buffer "*pysession*" (it
> adds earmuffs around the session name when missing). So the following
> doesn't find the inferior Python:

Good point. Some backends indeed do not have comint buffer named the
same as :session name.

What we can do is to introduce a new backend template function
org-babel-session-buffer: that will be passed a session name and
src block params and return the session buffer name.

If such function is not defined, we fall back to assumption that session
buffer is named the same as the session.

WDYT?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] Prompt appears in async shell results

[Ihor Radchenko]

Matt  writes:

> #+begin_src emacs-lisp
> (org-babel-do-load-languages 'org-babel-load-languages '((shell . t)))
> #+end_src
>
> #+begin_src sh :results output :session *test* :async t
> cd /tmp
> echo "hello world"
> #+end_src
>
> #+RESULTS:
> : org_babel_sh_prompt> hello world

Confirmed.

> ** Thoughts
> A quick fix is:
>
> #+begin_src diff
> modified   lisp/ob-shell.el
> @@ -289,7 +289,7 @@ See `org-babel-comint-async-indicator'.")
>  (defun ob-shell-async-chunk-callback (string)
>"Filter applied to results before insertion.
>  See `org-babel-comint-async-chunk-callback'."
> -  (replace-regexp-in-string comint-prompt-regexp "" string))
> +  (replace-regexp-in-string org-babel-sh-prompt "" string))
> ...
> I'm not sure this is the best way.

It is not. It works by accident.
Other edge cases may appear with chunks containing parts of the prompt.

> It seems to me that we should extract the filter from 
> =org-babel-comint-with-output= and use it in both 
> =org-babel-comint-with-output= and =org-babel-comint-async-filter=.  

Yes. The right fix would be extracting the filter from
`org-babel-comint-with-output' and re-using it.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG]: contradictory requirements between resolving links and noweb result block

[Ihor Radchenko]

gerard.vermeu...@posteo.net writes:

> * mwe-noweb-ok
>
> ...
> Resolving the last link fails, because it has an org-id
> #+begin_src latex
> Listing \ref{noop}, \ref{make-noweb}, and \ref{orgd105392}.
> #+end_src
> while the caption of the linked listing has a user label.

Fixed, on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f4414f5db

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at