Re: Patch to allow adjusting latex fragment display scale factor

[Matt Huszagh]

Timothy  writes:

> Hi Matt and Ihor,
> I must admit that with the new system, I don’t see much value in accepting a
> function: we now scale the previews based on a combination of the :scale
> parameter and their actual LaTeX display size.

Yep, we can close/cancel this patch. I had been using that feature to
permit rescaling latex fragments for font size changes, but that's
already handled by the current implementation.

Matt

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Matt Huszagh]

Matt Huszagh  writes:

> Another minor issue I've encountered. Using different delimiters results
> in images of slightly different sizes.
>
> For example, in a file containing:
>
> ```
> Using first delimiters: \(F=ma\).
>
> Using other delimiters: $F=ma$.
> ```
>
> The snippet with dollar sign delimiters is slightly larger than the one
> using \(\).

The following code, where you compute the ascent value, seems relevant
to this observation:

;; The baseline seems to tend to sit slightly
;; lower than it should be, and a very mild
;; bias seems to improve the visual result.
;; From testing with a collecting of LaTeX
;; maths fonts (cm, cmbright, arev, pxfonts,
;; notomath, nextxsf, eulervm) decreacing the
;; depth measurement by 0.02pt in the baseline
;; calculation seems to work well.
;; I have yet to come across any situation
;; where this results in a negative depth,
;; however we may as well ensure that never
;; occurs.
(round (* 100 (- 1 (/ (max 0.0 (- depth 0.02))
  height

When using \(\) delimiters, using a depth adjustment of 0 (instead of
0.02) looks correct to me. I checked this by blowing up the fragment
with a very large scale factor (eg 10) and then baseline misalignments
become more obvious. This is how I ensured my baseline computation was
correct when I wrote that patch aligning the baseline several years
ago. I /think/ that's a valid method, and I've been using my code for
the last couple years and the baseline has always looked correct.

Anyway, can you explain more why you came to the conclusion of that
slight depth adjustment? Are you using $$ delimiters? That also appears
to produce other visual imperfections. For $F=ma$, I see the bottom of
the "m" and "a" cut off slightly. I wonder why different delimiters
produce different results. I used slightly different settings for
dvisvgm in my implementation (including --exact-bbox). I wonder if that
has any relevance... I also used a different document class - standalone
in preview mode. Now that I think about it, IIRC that was to address
another corner case I ran into, which is that for large images, article
will crop it before it gets to dvisvgm. It's been a while since I did
this and my memory is hazy, but I think that's why I used standalone. I
can try to investigate that with a minimal issue.

Matt

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Matt Huszagh]

Timothy,

Another minor issue I've encountered. Using different delimiters results
in images of slightly different sizes.

For example, in a file containing:

```
Using first delimiters: \(F=ma\).

Using other delimiters: $F=ma$.
```

The snippet with dollar sign delimiters is slightly larger than the one
using \(\).

Matt

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Matt Huszagh]

Hi Timothy,

Thanks for your work on this.

I've run into an issue related to the change you made in
209e5f5f4047a34db27c3b5dff4077bb1da9ceed that makes org-latex-compile
asynchronous.

This breaks the org-babel-latex-pdf-svg-process functionality I
introduced in commit ae35a345903c640397a8d29812112d72a9f8494a.

To reproduce the issue, you can create an org file with the following
single latex source block:

#+header: :file "out.svg"
#+begin_src latex :hidden
\begin{equation}\tag{1.6.3.1}
  \mathbf{F} = q \mathbf{E},
\end{equation}
#+end_src

If you execute this with C-c C-c, it complains about the PDF file not
being present. The problem can be found starting in line 182 of
ob-latex.el of your current (as of this writing) feature branch commit
8384289762b41b26e75d2e80c37ec84bcc552d32. The issue is that
org-babel-latex-tex-to-pdf (which delegates to org-latex-compile)
creates the PDF after org-compile-file (in line 185) tries to generate
an SVG file from it. In fact, I was initially confused because I
couldn't reproduce the issue with edebug (obviously, because that gave a
long enough delay for the file to be created before attempting the svg
creation).

org-latex-compile seems like it could be a dangerous function to change
to be asynchronous, as it's used in a lot of places and its very
possible that other people (like myself) were relying on it being
synchronous when they made some change that used it. I haven't looked at
its other invocations, but I wouldn't be surprised if this breaks other
things too. I hesitate to suggest alternatives because I don't know
enough about your use case. But, maybe you could achieve what you want
through a new org-latex-compile-async function, or add a nondefault
option to org-latex-compile?

Matt

Re: Patch to allow adjusting latex fragment display scale factor

[Matt Huszagh]

Hi,

Bringing this back up. Ihor, I've added you to the thread directly
(hope that's ok).

Any thoughts? I have not yet updated it for the most recent changes to
main, but I can do that.

Matt

On Sun, Oct 10, 2021 at 9:40 PM Matt Huszagh  wrote:
>
> Matt Huszagh  writes:
>
> > I've created a patch to allow adjusting the scale factor used for inline
> > latex image fragments. This involves a customizable variable that can
> > either be set to a scale factor (defaults to 1.0) or a function that
> > evaluates to a scale factor.
> >
> > This feature is in addition to the existing scale factor adjustment
> > capability provided by `org-preview-latex-process-alist' through
> > `:image-size-adjust'. Wherease image-size-adjust performs scaling at the
> > time of image generation, the new change performs it during
> > display. This can lead to significant time saving and suffers no loss of
> > quality for vector graphics.
> >
> > As an example of use, I have several functions for changing frame
> > scaling. I've added
> >
> > (if (eq major-mode 'org-mode)
> >   (progn
> > (org-clear-latex-preview)
> > ;; 16 corresponds to the C-u C-u arg prefix.
> > (org-latex-preview 16)))
> >
> > to these functions so that changing the frame scaling also
> > correspondingly changes the latex preview/fragment scaling to match the
> > new size of the surrounding text. Because of this new feature, this
> > change is effectively instantaneous for reasonably numbers of
> > overlays. Obviously, something similar could be done for
> > `text-scale-adjust' (e.g., through `advice-add').
> >
> > Feedback appreciated.
>
> Apologies, the patch I sent is slightly wrong. The line numbers also
> reflect an earlier patch I made. Here is a corrected version.
>

Re: [Patch] Align baseline of latex fragments and surrounding text

[Matt Huszagh]

Ihor Radchenko  writes:

> Matt Huszagh  writes:
>
>> These are nice ideas, and ones I hadn't considered. Thanks for the
>> suggestions! I'll think about it a bit and may modify the patch
>> accordingly.
>
> May I ask if you further considered changing the patch?

Hey Ihor,

I'm sorry for the late reply. I'm just now looking at several patches I
submitted a few years ago and left hanging...

The answer to this question is yes I did, and IIRC got most of the way
through implementing them but ran into a largely unrelated issue that I
didn't, at the time, find a solution for. It looks like I still have the
branch with those changes and I can take a look back at it and try to
resolve it.

However, I noticed that there appears to be another (large) patch in the
works that also does this (email subject line "Overhaul of the LaTeX
preview system"). That's a long thread and I haven't followed it. Is
there still scope for my change? Is it worth getting this in?

Matt

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Matt Huszagh]

Juan Manuel Macías  writes:

> TL;DR: A list of use cases where using LuaTeX is more advantageous than
> using pdfTeX
>
> 
>
> Many times Org users who frequently need to export their documents to
> LaTeX, but who do not have much LaTeX experience (or their knowledge of
> the TeX ecosystem is somewhat out of date), find themselves confused by
> the different versions of the TeX engine: pdfTeX, XeTeX, LuaTeX… In Org
> pdfTeX is the default engine, which in 2022 has a few limitations and is
> really old, but still perfectly meets the needs of a significant group
> of users. However, there may be a number of cases where it is more
> advantageous to compile with LuaTeX, so here I will leave a short list
> of those cases where LuaTeX may be a happy choice over pdfTeX.
>
> But first it is worth clarifying some things about LuaTeX along with
> some misunderstandings:
>
> • LuaTeX is the evolution of pdfTeX, therefore LuaTeX can be considered
>   as the current de facto TeX engine. It is intended to replace pdfTeX,
>   and in fact many of us already consider pdfTeX obsolete and
>   deprecated.
>
> • To use LuaTeX it is not necessary to learn anything new or to know how
>   to program in Lua. LuaTeX includes a Lua interpreter and the ability
>   to bypass TeX primitives through Lua scripting (hence called LuaTeX).
>   But all of that is more on the side of developers and packagers. For
>   example, I am currently writing two LaTeX packages (one in
>   collaboration with a colleague) where 80% of the code is Lua and 20%
>   is (La)TeX. Of course, any user who knows Lua can take advantage of
>   the \directlua primitive or the luacode environment in their
>   documents.
>
> • A standard LaTeX document is always a LaTeX document, regardless of
>   the flavor of TeX used to compile that document. There will be some
>   minor differences. For example, in LuaLaTeX it is unnecessary to add
>   fontenc and inputenc commands in the preamble.
>
> And now we go with the non-exhaustive list of cases where compiling with
> LuaTeX can be more advantageous for the user:
>
> 1. When you need to work in a *real* Unicode environment and not in
>pdfTeX’s 'fake Unicode'. And, especially, when it is required to work
>with languages that use a non-Latin writing system: Greek, Arabic,
>Hebrew, all the languages that use Cyrillic, oriental languages, etc.
>An extreme example you can see in this small code that I wrote for
>LuaTeX in order to be able to use the syllabograms of the ancient
>Mycenaean script:
>
>
>
> 2. When using truetype or opentype fonts is required. The pdfTeX user is
>limited to using only the included type1 fonts, the number of which
>is very limited. Besides, type1 is a deprecated and pre-unicode
>format. In fact, it almost always ends up leaving the default
>Computer Modern font. In LuaTeX we can use not only all the fonts
>installed on our system but also any font (just indicate the path),
>which is an important advantage over XeTeX. A basic command could be
>something like (loading the fontspec package):
>
>┌
>│ \setmainfont{Palatino Linotype}
>└
>
>And with the latest versions of Babel we can associate fonts for
>different writing systems, without the need to change languages
>explicitly with a \selectlanguage.
>
>We can define all the font families we want or redefine the default
> families. For example, with this command I define the default
> monospaced font and request that it be scaled according to the
> height of the lowercase of the main font:
>
>┌
>│ \setmonofont{Iosevka Fixed Curly}[Scale=MatchLowercase]
>└
>
> 3. When you need to take advantage, to a greater or lesser extent, of
>the opentype features of a font. For example, here we define the main
>font to use old style numbers:
>
>┌
>│ \setmainfont{Palatino Linotype}[Numbers=Lowercase]
>└
>
>We can also load the otf tags directly:
>
>┌
>│ \setmainfont{Palatino Linotype}[RawFeature=+onum]
>└
>
>The fontspec package for managing fonts in LuaTeX and XeTeX is very
> versatile and powerful. We can also associate a different font as
> italic for an already defined font family, use different optical
> resolutions of a font, etc. If what you are looking for is precise
> and absolute fine-tuning of the fonts used, of course LuaTeX is the
> ideal choice.
>
> 4. In general, when professional-level typographic fine-tuning is needed
>(and far superior to that offered by dtp programs like InDesign or
>QuarkXpress). For example, we can define on the fly new position
>opentype properties for a specific font, without having to edit the
>font. It is a non-destructive method that uses the
>fonts.handlers.otf.addfeature lua function. For example, we can
>define a new kerning value for the 

Re: [PATCH] Fix caption format for custom latex src block

[Matt Huszagh]

Timothy  writes:

> As mentioned by Ihor, you’ll want to take a look at the curent version of
> ox-latex, specifically `org-latex-src-block--custom' which now contains `(?c .
> ,caption)'. You may want to consider modifying 
> `org-latex--caption/label-string' to
> accept the form `(element info  content-only)' or reimplementing 
> just the
> bits needed for this purpose — which ever works out nicer. Looking at
> `org-latex--caption/label-string' a substring approach looks quite fragile, 
> though
> you might be able to get away with an application of 
> `replace-regexp-in-string',
> though I’d personally consider this a bit of a last resort.

Thanks for the directions Timothy.

I failed to reproduce the old issue with the latest commit to main, so
it looks like this patch is no longer needed. Nice! I'll post back if I
notice any issues in the future, but otherwise I'd say we can consider
this closed.

Matt

Re: [PATCH] Remove additional newline at end of results block

[Matt Huszagh]

Ihor Radchenko  writes:

> Thanks for the patch! And sorry for the late reply.

No worries for the delay - I know you have a lot of these. Thanks for
the review.

> Your explanation is quite confusing, but I think I managed to understand
> what you referred to:
>
> --
>  We can write the simplest equation as
>  #+begin_src latex :exports results
>  \begin{equation}
>1 + 1 = 2,
>  \end{equation}
>  #+end_src
> the paragraph should continue here.
> 
>
> will be exported as
>
> --
> We can write the simplest equation as
> \begin{equation}
>   1 + 1 = 2,
> \end{equation}
>
> the paragraph should continue here.
> --
>
> Note the extra empty line below equation environment.
>
> However, I am not confident if the proposed change is going to be safe
> for other uses of src blocks.
>
> I'd like to request other people who use export and source blocks
> extensively to try the patch and see if it breaks anything.
>
> Meanwhile, could you please reword the commit message and make it more
> concise and clear?

Can you clarify what you find to be unclear? Rereading my own commit
message, my only problem with it is how it starts: I'd add one sentence
to contextualize it a bit. For instance,

The previous behavior always ensured the presence of an empty line
between the results block of a source block and the subsequent
text. However, inserting this newline prevents a valid use-case and
protects against an edge-case that is completely avoidable without the
additional guarantee it provides. ... (the rest remains unchanged)

Oh and I also made a typo. A sentence further down should read
"...generate an SVG image...".

This commit message isn't short, but I think it's very clear. It
describes the previous behavior, explains the rationale for that
behavior, and then illustrates (with a complete example) how this
prevents a valid use case. It also explains why the new change does not
prohibit any behavior that was previously possible.

As someone who frequently uses git log, I'd much rather see a commit
message like this than the typical (usually) unhelpful one or two
sentences that fail to provide the motivation for a change. There's no
downside to a long commit message, and this one is structured such that
it proceeds from more general to more specific information - not
everyone has to read the entire thing.

I'm not trying to be difficult :) but I do care about the quality of
this codebase (as I know do you), so I'm reluctant to change something
in a way I feel is inferior. But, if you have specific parts etc you
feel are unclear I'm more than happy to address those.

Matt

Re: [PATCH] Fix behavior of lambda default header arg vars

[Matt Huszagh]

Ihor Radchenko  writes:

> This patch does not apply. It does not appear to be based on the
> original main branch:
>
> Applying: ob-core.el: Improve org-babel-default-header-args docstring
> error: sha1 information is lacking or useless (lisp/ob-core.el).
> error: could not build fake ancestor

My mistake. Try these.

Matt

>From 6894d7b441fe975bc90177d85102c031cffe3c46 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Mon, 27 Jun 2022 20:41:02 -0700
Subject: [PATCH 1/2] ob-core.el: Fix behavior of lambda default header arg
 vars

* lisp/ob-core.el: Permit multiple :var default header arguments when
using closures.
---
 lisp/ob-core.el | 5 +
 1 file changed, 5 insertions(+)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 75a6a167d..0d9ec5c84 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2749,6 +2749,11 @@ parameters when merging lists."
 	(pcase pair
 	  (`(:var . ,value)
 	   (let ((name (cond
+;; Default header arguments can accept lambda
+;; functions. We uniquely identify the var
+;; according to the full string contents of
+;; the lambda function.
+			((functionp value) value)
 			((listp value) (car value))
 			((string-match "^\\([^= \f\t\n\r\v]+\\)[ \t]*=" value)
 			 (intern (match-string 1 value)))
-- 
2.31.1

>From 12833bc6c089af9d94c9535835e0a68d189c8496 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Mon, 27 Jun 2022 20:42:27 -0700
Subject: [PATCH 2/2] ob-core.el: Improve org-babel-default-header-args
 docstring

* lisp/ob-core.el: Provide example illustrating one benefit of using
closures as default header arguments. Additionally, explain how to
provide the same type of header argument multiple times in the default
alist.
---
 lisp/ob-core.el | 25 ++---
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 0d9ec5c84..d85c66e76 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -482,12 +482,14 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'."
 This is a list in which each element is an alist.  Each key
 corresponds to a header argument, and each value to that header's
 value.  The value can either be a string or a closure that
-evaluates to a string.  The closure is evaluated when the source
-block is being evaluated (e.g. during execution or export), with
-point at the source block.  It is not possible to use an
-arbitrary function symbol (e.g. \\='some-func), since org uses
-lexical binding.  To achieve the same functionality, call the
-function within a closure (e.g. (lambda () (some-func))).
+evaluates to a string.
+
+A closure is evaluated when the source block is being
+evaluated (e.g. during execution or export), with point at the
+source block.  It is not possible to use an arbitrary function
+symbol (e.g. 'some-func), since org uses lexical binding.  To
+achieve the same functionality, call the function within a
+closure (e.g. (lambda () (some-func))).
 
 To understand how closures can be used as default header
 arguments, imagine you'd like to set the file name output of a
@@ -504,7 +506,16 @@ this with:
 
 Because the closure is evaluated with point at the source block,
 the call to `org-element-at-point' above will always retrieve
-information about the current source block.")
+information about the current source block.
+
+Some header arguments can be provided multiple times for a source
+block.  An example of such a header argument is :var.  This
+functionality is also supported for default header arguments by
+providing the header argument multiple times in the alist.  For
+example:
+
+'((:var . \"foo=\\\"bar\\\"\")
+  (:var . \"bar=\\\"foo\\\"\"))")
 
 (put 'org-babel-default-header-args 'safe-local-variable
  (org-babel-header-args-safe-fn org-babel-safe-header-args))
-- 
2.31.1

Re: [PATCH] Fix caption format for custom latex src block

[Matt Huszagh]

Ihor Radchenko  writes:

> I think that it will be better if you use
> org-latex--caption/label-string instead. It will take care about short
> captions as well.

Changing this to (?c . ,caption-str) yields for the original example

\caption{\caption{Perform inter-sample interpolation.}
}

I could use substring to remove the leading \caption{ and trailing
}. What do you think?

Matt

Re: [PATCH] Fix behavior of lambda default header arg vars

[Matt Huszagh]

Hi Ihor,

Modifications made and split into two patches. Let me know if any
remaining issues.

Thanks
Matt

>From fdc6e103932c68b450c72d44d062b4746ec2202c Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Mon, 27 Jun 2022 20:41:02 -0700
Subject: [PATCH 1/2] ob-core.el: Fix behavior of lambda default header arg
 vars

* lisp/ob-core.el: Permit multiple :var default header arguments when
using closures.
---
 lisp/ob-core.el | 5 +
 1 file changed, 5 insertions(+)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 7a9467b0e..80086f12c 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2718,6 +2718,11 @@ parameters when merging lists."
 	(pcase pair
 	  (`(:var . ,value)
 	   (let ((name (cond
+;; Default header arguments can accept lambda
+;; functions. We uniquely identify the var
+;; according to the full string contents of
+;; the lambda function.
+			((functionp value) value)
 			((listp value) (car value))
 			((string-match "^\\([^= \f\t\n\r\v]+\\)[ \t]*=" value)
 			 (intern (match-string 1 value)))
-- 
2.31.1

>From 7af7ad3990410ca388c841e045ec66ce694b404a Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Mon, 27 Jun 2022 20:42:27 -0700
Subject: [PATCH 2/2] ob-core.el: Improve org-babel-default-header-args
 docstring

* lisp/ob-core.el: Provide example illustrating one benefit of using
closures as default header arguments. Additionally, explain how to
provide the same type of header argument multiple times in the default
alist.
---
 lisp/ob-core.el | 25 ++---
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 80086f12c..6c3c79126 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -478,12 +478,14 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'."
 This is a list in which each element is an alist.  Each key
 corresponds to a header argument, and each value to that header's
 value.  The value can either be a string or a closure that
-evaluates to a string.  The closure is evaluated when the source
-block is being evaluated (e.g. during execution or export), with
-point at the source block.  It is not possible to use an
-arbitrary function symbol (e.g. 'some-func), since org uses
-lexical binding.  To achieve the same functionality, call the
-function within a closure (e.g. (lambda () (some-func))).
+evaluates to a string.
+
+A closure is evaluated when the source block is being
+evaluated (e.g. during execution or export), with point at the
+source block.  It is not possible to use an arbitrary function
+symbol (e.g. 'some-func), since org uses lexical binding.  To
+achieve the same functionality, call the function within a
+closure (e.g. (lambda () (some-func))).
 
 To understand how closures can be used as default header
 arguments, imagine you'd like to set the file name output of a
@@ -500,7 +502,16 @@ this with:
 
 Because the closure is evaluated with point at the source block,
 the call to `org-element-at-point' above will always retrieve
-information about the current source block.")
+information about the current source block.
+
+Some header arguments can be provided multiple times for a source
+block.  An example of such a header argument is :var.  This
+functionality is also supported for default header arguments by
+providing the header argument multiple times in the alist.  For
+example:
+
+'((:var . \"foo=\\\"bar\\\"\")
+  (:var . \"bar=\\\"foo\\\"\"))")
 
 (put 'org-babel-default-header-args 'safe-local-variable
  (org-babel-header-args-safe-fn org-babel-safe-header-args))
-- 
2.31.1

Re: [PATCH] Fix behavior of lambda default header arg vars

[Matt Huszagh]

> It would help if closure part and multi-variable part were split into
> separate paragraphs.

The closure part and muliple header argument part are already in
separate paragraphs. The multiple header argument part, however, is
incorporated into an introductory paragraph that very briefly describes
the value syntax of org-babel-default-header-args and the types of alist
values it supports. I did this because that introductory information is
short and simple and so I felt it acceptable to incorporate the multiple
header argument information. I don't feel this places a significant
intellectual burden on the reader to follow, but I'm happy to insert a
newline before "Some header arguments..." if you'd prefer.

> Are you saying that _only some_ backends support multiple vars? Are
> there backends that do not support?

I think you're confused about "(e.g., :var for some language backends)":

It's been a while since I created this patch and I don't remember
exactly why I wrote it this way. I think it was based on the belief that
not all language backends support variable passing in header arguments,
though I honestly couldn't tell you at the moment whether this is
true. In that vein, a semantically equivalent way to write this would be
"(e.g., :var for language backends that support it)". Maybe all language
backends support variable header arguments, in which case "(e.g., :var)"
could be used here instead. In any case, the "some language backends"
part of the phrase is not a qualification of "multiple", but of
"variables". Nor is it correct to read it this way, as the statement is
grammatically clear and correct. An equivalent statement would be:

"""
Some header arguments can be provided multiple times for a source
block. An example of such a header argument is :var.
"""

> Or are you talking about multiple assignments to the same variable?

No. The topic of multiple assignments to the same variable is not
discussed here.

> Also, the example is not helpful here.

The example *is* helpful. It provides explicit direction for how to
handle the non-obvious case of header arguments that can be passed
multiple times, which isn't described much in the documentation.

> The new docstring is confusing. The same paragraph is talking about
> closures, then multiple header args, and gives an example without
> closures.

I don't think there's anything particularly confusing about this
docstring. As already mentioned, that paragraph discusses acceptable
alist values and multiple header args. Closures are mentioned only as
one of the acceptable alist values, which puts them on the same level as
strings. The more in-depth discussion of closures happens later, in its
own paragraphs.

Taking a step back, the structure of this docstring is:

1. A single sentence briefly describing org-babel-default-header-args.
2. A description of the syntax and acceptable values that can be
   assigned to org-babel-default-header-args.
3. A discussion of how to handle header arguments that can be provided
   multiple times. This explicit treatment is justified because this
   case is not intuitively obvious.
4. A discussion of closures (including why functions need to be provided
   as closures) and an example illustrating their usefulness.

2 and 3 are part of the same paragraph and 4 is several paragraphs (the
precise number depends on whether you consider code blocks to be their
own paragraph and how they break up surrounding text). This is a
perfectly reasonable way to structure this information. There's no rule
requiring that there be one topic per paragraph. Instead, paragraphs
should be structured in a way that's clear, and in many cases (not all)
that does result in a paragraph discussing one topic. But again, a
reasonable alternative structure would be to separate 2 and 3 into their
own paragraphs and I'm happy to do that. The multiple header argument
discussion also doesn't need to be located where it is. It could
alternatively go after the discussion of closures if you prefer that.

Matt

[PATCH] Remove additional newline at end of results block

[Matt Huszagh]

Inserting this newline prevents a valid use-case and protects against
an edge-case that is completely avoidable without the additional
guarantee it provides. The original intention for inserting the
newline was to avoid the edge case in which a user does not insert a
newline between a source block and the subsequent text and the
subsequent text is merged with the results.

However, there are valid cases in which a user would not want a
newline between a results block and the subsequent buffer text. For
example, many display equations in LaTeX are considered as part of the
surrounding paragraph. Additionally, it is possible to setup a LaTeX
source block to be executable and insert results into the org
buffer. Consider the following example:

some org file (leading colons to prevent git ignoring lines starting
with #):
```
: We can write the simplest equation as
: #+begin_src latex
: \begin{equation}
:   1 + 1 = 2,
: \end{equation}
: #+end_src
: and hope that no one is confused by this.
```

We might then execute this source block to generate some output. For
example, this might generate and SVG image of the block and insert it
into the buffer. That should result in something like

```
: We can write the simplest equation as
: #+begin_src latex
: \begin{equation}
:   1 + 1 = 2,
: \end{equation}
: #+end_src

: #+RESULTS:
: #+begin_results
: [[file:some/file.svg]]
: #+end_results
: and hope that no one is confused by this.
```

When formatted this way, the resulting tex
file (org-latex-export-to-latex) will be:

```
We can write the simplest equation as
\begin{equation}
  1 + 1 = 2,
\end{equation}
and hope that no one is confused by this.
```

which will render correctly. Specifically, the display math
environment will not start a new paragraph after the leading "as" and
a new paragraph will not start between the end of the math display and
the trailing "and".

However, the current behavior results in

```
: We can write the simplest equation as
: #+begin_src latex
: \begin{equation}
:   1 + 1 = 2,
: \end{equation}
: #+end_src

: #+RESULTS:
: #+begin_results
: [[file:some/file.svg]]
: #+end_results

: and hope that no one is confused by this.
```

This blank line necessarily starts a new paragraph in TeX.

Finally, as previously stated, it is entirely possible to control
whether there is a newline between the results block and subsequent
text by leaving a newline between the source block and text. For
example,

```
: We can write the simplest equation as
: #+begin_src latex
: \begin{equation}
:   1 + 1 = 2,
: \end{equation}
: #+end_src

: and hope that no one is confused by this.
```

would still result in

```
: We can write the simplest equation as
: #+begin_src latex
: \begin{equation}
:   1 + 1 = 2,
: \end{equation}
: #+end_src

: #+RESULTS:
: #+begin_results
: [[file:some/file.svg]]
: #+end_results

: and hope that no one is confused by this.
```
From 667ba67570361bad878c74afdebe068b7916a1bb Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Thu, 30 Dec 2021 06:54:24 -0800
Subject: [PATCH] lisp/ob-core.el: Remove additional newline at end of results

* lisp/ob-core.el (org-babel--insert-results-keyword): Remove newline
at end of results block.

Inserting this newline prevents a valid use-case and protects against
an edge-case that is completely avoidable without the additional
guarantee it provides. The original intention for inserting the
newline was to avoid the edge case in which a user does not insert a
newline between a source block and the subsequent text and the
subsequent text is merged with the results.

However, there are valid cases in which a user would not want a
newline between a results block and the subsequent buffer text. For
example, many display equations in LaTeX are considered as part of the
surrounding paragraph. Additionally, it is possible to setup a LaTeX
source block to be executable and insert results into the org
buffer. Consider the following example:

some org file (leading colons to prevent git ignoring lines starting
with #):
```
: We can write the simplest equation as
: #+begin_src latex
: \begin{equation}
:   1 + 1 = 2,
: \end{equation}
: #+end_src
: and hope that no one is confused by this.
```

We might then execute this source block to generate some output. For
example, this might generate and SVG image of the block and insert it
into the buffer. That should result in something like

```
: We can write the simplest equation as
: #+begin_src latex
: \begin{equation}
:   1 + 1 = 2,
: \end{equation}
: #+end_src

: #+RESULTS:
: #+begin_results
: [[file:some/file.svg]]
: #+end_results
: and hope that no one is confused by this.
```

When formatted this way, the resulting tex
file (org-latex-export-to-latex) will be:

```
We can write the simplest equation as
\begin{equation}
  1 + 1 = 2,
\end{equation}
and hope that no one is confused by this.
```

which will render correctly. Specifically, the display math
environment will not start a new paragra

[PATCH] Fix caption format for custom latex src block

[Matt Huszagh]

This patch fixes an issue in which captions for custom listing
environments are not converted correctly. To illustrate the issue,
take the following MWE.

file.org:
```
#+caption: Perform inter-sample interpolation.
#+begin_src python
import numpy as np
#+end_src
```

file.el:
```
;; Loading straight isn't necessary if you don't use it, just need to
;; load org.
(load "~/.config/emacs/straight/repos/straight.el/bootstrap.el")
(straight-use-package 'org)

(find-file "file.org")
(setq org-latex-listings 'minted)
(setq org-latex-custom-lang-environments
  '((python "\\begin{listing}
\\begin{minted}[%o]{python}
%s\\end{minted}
\\caption{%c}
\\label{%l}
\\end{listing}")))
(org-latex-export-to-latex)
```

Run the example with:
emacs -Q --batch -l file.el

Before the patch, you get:
```
[...]
\begin{listing}
\begin{minted}[]{python}
import numpy as np
\end{minted}
\caption{(((Perform inter-sample interpolation.)))}
\label{nil}
\end{listing}
[...]
```

After the patch, you get:
```
[...]
\begin{listing}
\begin{minted}[]{python}
import numpy as np
\end{minted}
\caption{Perform inter-sample interpolation.}
\label{nil}
\end{listing}
[...]
```

I'm not actually 100% confident that the patch is the optimal way to
do this. The API wasn't totally clear to me in this case. Any advice
here is appreciated.

Matt
From 5b0601d6d3b8034202c4b9b820c64a719e3129b9 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Wed, 29 Dec 2021 19:35:42 -0800
Subject: [PATCH] ox-latex.el: Fix caption format for custom latex src block

2021-12-29  Matt Huszagh  

	* lisp/ox-latex.el (org-latex-src-block): Use
	`org-export-get-caption' to extract caption from
	element. Otherwise, the full caption contains a large number
	of unnecessary parentheses.
---
 lisp/ox-latex.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 8187119ec..a9c6a4b5c 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -3021,7 +3021,8 @@ contextual information."
 		  custom-env)
 	(format-spec custom-env
 			 `((?s . ,formatted-src)
-			   (?c . ,caption)
+			   (?c . ,(org-export-data (org-export-get-caption src-block)
+   info))
 			   (?f . ,float)
 			   (?l . ,(org-latex--label src-block info))
 			   (?o . ,(or (plist-get attributes :options) "")))
-- 
2.31.1

Re: [Patch] Align baseline of latex fragments and surrounding text

[Matt Huszagh]

Sébastien Miquel  writes:

> My mistake. For some reason, I had thought that
> =\documentclass[preview]{standalone}= was used by default for LaTeX
> previews. Setting it as described in your patch, it now works
> properly, even with the default value of =dvisvgm=.

No worries; glad you got it working.

> To make this behavior easier to toggle, you could
>   1. Change the default value of =org-format-latex-header=. The
>      =standalone= class makes sense, but I don't know if that might
>      break things.
>   2. Specify the =:latex-header= of the default =dvisvgm= option. Same
>      caveat applies.
>   3. Add a =dvisvgm-with-ascent= option to the default value of
>      =org-preview-latex-process-alist=.
>
> Instead of the new variable =org-latex-fragment-overlay-ascent=,
> perhaps the function used to compute the ascent could be provided as
> another property, such as =:ascent=, added to the relevant options in
> =org-preview-latex-process-alist=. It seems to make more sense since
> it only applies to svg output, and it makes it easier to have this
> behavior as default. It would require =org--make-preview-overlay= to
> take the ascent as an additional argument.

These are nice ideas, and ones I hadn't considered. Thanks for the
suggestions! I'll think about it a bit and may modify the patch
accordingly.

> Please note that I am not a maintainer, these are just a few thoughts.
> I do hope your work can be applied and that LaTeX fragments can be
> properly aligned by default.
>
> You should add [PATCH] to the subject of your mail, so that it gets
> listed at https://updates.orgmode.org/ and not forgotten. A maintainer
> will reply eventually, but it might take up to a few months.

Also a good suggestion. Fortunately, "Patch" seems to have been caught
by that tool. But [PATCH] seems more typical and I've updated the
subject for that.

Matt

Re: Patch to align baseline of latex fragments and surrounding text

[Matt Huszagh]

Sébastien Miquel  writes:

> This looks great indeed but I've failed to reproduce in my
> environment.

Thanks for testing this Sébastien.

> I couldn't get ~org--match-text-baseline-ascent~ to compute the
> ascent : the ~xml-get-attribute~ call returns
>   : ("-16.945024" "12.153473" "16.148855" "8.064997")
> which gives an ascent < -100, and the code then defaults to 'center.

I'd need to know more about your setup for generating latex
fragments. Did you follow all the directions in
org--match-text-baseline-ascent? How is your org-format-latex-header
set? In particular, are you using \documentclass[preview]{standalone}?
If you can provide me with the TeX file used to generate the fragment,
as well as the SVG file you get as a result, that would be helpful too.

> The options described in your =my-dvisvgm= seem outdated, you can
> check the latest default value of =dvisvgm= : =use-xcolor= is
> deprecated and a =:image-size-adjust= property is provided for the
> images to be sized properly. Are the arguments =--no-fonts= and
> =--exact-bbox= necessary ?

Hm, I'm not actually sure why I put use-xcolor in there, but it isn't
necessary. I've removed it in the updated patch (attached).

:image-size-adjust isn't necessary, I just mentioned it to point out
that baseline alignment works regardless of the size of a latex fragment
(I have another open PR that allows setting the size of fragments in a
context dependent way, which can be used for instance, to keep fragments
size-aligned to the surrounding text). I expect using the scale
parameter in org-format-latex-options will work similarly, but I'll
investigate.

--no-fonts is the same as -n. The --exact-bbox flag is necessary to
avoid cropping in certain cases (see
https://github.com/mgieseki/dvisvgm/issues/8). You're free to use
--bbox=min, but your glyphs may be cut off in places and this may affect
the baseline location too, though I haven't tested it.

> If there are no drawbacks, perhaps this behaviour should be the
> default. Otherwise, it should at least be easier to toggle.

I didn't attempt to make this the default because it requires a specific
setup, which is also different from the current default setup in other
respects. Most importantly, it requires using the standalone document
class, though I believe article is used at the moment.

> Can something similar be done with =dvipng= ?

Unfortunately I don't think so. This code isn't doing anything fancy; it
has no way of computing the text baseline from some arbitrary image file
displaying text. Instead, it relies on some other tool providing this
information inside the image file. In this case, standalone encodes this
information in the viewbox of the SVG file.

Thanks
Matt

>From 2640b24d2cce8e2b61101125c348db35800570ff Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Sun, 10 Oct 2021 15:46:05 -0700
Subject: [PATCH] org--make-preview-overlay: Add ability to set vertical
 alignment of latex fragments

* lisp/org.el (org--match-text-baseline-ascent): Add function that
computes the value of :ascent to match the baseline of the fragment to
the surrounding text.
(org-latex-fragment-overlay-ascent): Add custom variable that allows a
function to be used to compute the value of :ascent.
(org--make-preview-overlay): Incorporate the custom variable.
---
 lisp/org.el | 88 +++--
 1 file changed, 79 insertions(+), 9 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 405f0f0f9..91a4d0bf5 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -320,6 +320,73 @@ identifier."
   :version "24.1"
   :group 'org-id)
 
+(defun org--match-text-baseline-ascent (imagefile imagetype)
+  "Set :ascent to match the text baseline of an image to the surrounding text.
+IMAGEFILE is the path of the image file and IMAGETYPE is the
+image file type.
+
+This function currently only works for SVG images (defaulting to
+'center otherwise).  It also requires that the SVG's viewbox is
+correctly set according to the text baseline position.
+Specifically, it computes
+
+ascent = 100 * -min-y / height
+
+where min-y and height are taken from the viewbox.  If this
+computation returns a non-sensical value (below 0 or above 100),
+it will return 'center instead.
+
+It's possible to include text baseline information in the viewbox
+by using \\documentclass[preview]{standalone} in the input LaTeX
+file, compiling first to DVI and then converting this to an SVG
+image with dvisvgm.
+
+For example,
+
+(setq my-dvisvgm
+  '(my-dvisvgm :programs (\"latex\" \"dvisvgm\")
+   :description \"dvi > svg\"
+   :message \"you need to install latex and dvisvgm.\"
+   :image-input-type \"dvi\"
+   :image-output-type \"svg\"
+   :latex-compiler (\"latex -o

Re: [PATCH] Fix behavior of lambda default header arg vars

[Matt Huszagh]

Matt Huszagh  writes:

> Hey Timothy,
>
> Any thoughts on this bug fix?
>
> Thanks
> Matt

Apologies, meant to include the org-mode mailing list in this too.

Thanks
Matt

Re: Patch to align baseline of latex fragments and surrounding text

[Matt Huszagh]

Timothy, looping you in on this.

I feel that maybe it would be useful to attach screenshots to show the
improvement from this patch? Anyway, I've attached two images: one with
the correct baseline alignment to surrounding text and the other with
the current, incorrect, baseline alignment.

I think a lot of people would like this functionality. It looks much
better than the current behavior.

Anyway, feedback appreciated.

Re: [PATCH] Fix regex for determining image width from attribute

[Matt Huszagh]

Timothy  writes:

> Thanks for your thoughtful deliberation on this.

No worries, and thanks for continuing to engage with it.

>> The other consideration is if we take the first point as a given (that
>> org should use width directives for other backends), should it also
>> attempt to interpret directives that are ambiguous? In this case, do we
>> interpret 1.2 as 1.2? If  could only be ,
>> I’d be inclined to agree that this is logical. I also understand the
>> case for , though this is slightly less clear. But, what if
>> someone used 1.2? Seems a bit unusual I know, but maybe
>> someone would want this. Again, I don’t think we should guess if there’s
>> a chance we could be wrong.
>
> I feel this very much depends on how bad “guessing wrong” is, and as 
> previously
> discussed, since it’s rather easy to correct or set `org-image-actual-width' 
> to
> prevent this, I’m not sure it warrants being terribly concerned about.

I think the problem here is that `org-image-actual-width` is essentially
a global solution to a potentially local problem. I can't set
`org-image-actual-width` to nil for just one image.

> Thanks for continuing with this. Moving forward, I think it would be best to:
> ⁃ Make a patch just for prioritising `#+attr_org'
> ⁃ Make a patch just improving the regex (before or after the `#+attr_org' 
> patch)
> ⁃ Discuss changing the behaviour of image previews separately later / in 
> another
>   thread, linking to this thread when doing so.
>
> How does that sound?

Yep, sounds good.

> Lastly, a comment on your documentation patch from earlier. I like the changes
> to `org-image-actual-width', however I think you’ve been over-eager with 
> scrapping
> the current docstring for `org-display-inline-image--width'. Since the 
> behaviour
> is implemented there, I think it should at a minimum be documented there.
> The docstring for a function referring to a variable’s documentation for how 
> it’s
> handled by the function seems a bit weird.

I was torn on this as well. However, I ended up feeling that it would be
worse to duplicate the same information in two places. This requires
updating the information in two places instead of one, and, worse, the
documentation could easily diverge. Because the function isn't
public-facing, but the variable very much is, I felt it better just to
include the documentation for the variable.

I also don't think removing the documentation from the function is that
bad. A lot of what that function does is to follow what the variable
tells it to do. Again, the function won't be called directly by an
end-user, so requiring that the developer refer to other documentation
for understanding implementation details seems reasonable. Finally,
there are numerous functions in org that tell you to refer to
documentation elsewhere (especially defcustoms) for further information.

Anyway, I still feel that the earlier patch (before regex changes is the
right one). But, if you want me to revert the documentation removal from
the function I will.

Matt

Re: [PATCH] Fix regex for determining image width from attribute

[Matt Huszagh]

Timothy  writes:

> What would be a more sensible interpretation in your mind? The “true” value
> depends on the number of columns, and fetching that information seems a bit
> unreasonable. Since this isn’t just used if nothing else if given, I see a 
> 120%
> interpretation as fairly reasonable.

I think there are several different questions/considerations here, which
I'll address in a second. But first, I think the essential disagreement
is whether org should take an action if not explicitly told to do so. I
think org should only perform some action if given a clear directive. In
this context, I feel that org is guessing what the user wants and taking
an action based on that guess.

Ok, back to the fact that there are multiple considerations here. The
first issue is whether specifying a width for a backend reflects an
intention to have that same width in the org buffer. As I previously
stated, I don't agree that one implies the other. But, as also
previously discussed, this was a decision that was made almost 10 years
ago, so changing it would be a breaking change, etc. Because of that,
I'm not totally sure what org should do, and I expect a lot of people
won't want to change this.

The other consideration is if we take the first point as a given (that
org should use width directives for other backends), should it also
attempt to interpret directives that are ambiguous? In this case, do we
interpret 1.2\somemacro as 1.2? If \somemacro could only be \linewidth,
I'd be inclined to agree that this is logical. I also understand the
case for \columnwidth, though this is slightly less clear. But, what if
someone used 1.2\columnsep? Seems a bit unusual I know, but maybe
someone would want this. Again, I don't think we should guess if there's
a chance we could be wrong.

I totally agree with you that we don't want to implement a pseudo latex
parser here. But I feel like all this complexity is easily resolved by
just requiring that people be explicit about their intentions (i.e.,
specify #+attr_org: :width). That would avoid all the complex behavior
and surprises that could result from making intelligent guesses about
what the user wants.

Anyway, let me know what you want in terms of the patch. I still think
prioritizing attr_org should be its own patch and changing the regex and
all the other behavior should be a separate issue. But, if you'd like me
to perform the change I mentioned in my last email, I can take the time
to write that up and include it in the same patch.

Thanks
Matt

Re: [PATCH] Fix regex for determining image width from attribute

[Matt Huszagh]

Max Nikulin  writes:

> I am confused. I can not figure out how to create the following as HTML 
> export result:
>
> 
>
> Attempt to add quotes leads to  and does not prevent ":width" to 
> become another attribute.
>
>#+attr_html: :alt An image without :width 600 attribute
>[[file:img.png]]
>
>
> 
>
> My current variant:
>
> ":\\(?:[^\n]*?[[:blank:]]\\)?:width[[:blank:]]+\\(\\S-+\\)"
>
> The regexp should not match e.g.
>
> #+attr_html: :alt something
> :width 600
>
> P.S. I would prefer to use the same parser as ox does.

(cadr par) contains all the attr_ keywords with their values. I think it
would be better to use this instead of doing a regex search, which I
expect would be slower and more prone to errors. If we want to be strict
(and probably more correct), we could use
org-export-registered-backends. Otherwise, we could look for any key
that starts with attr_.

I can probably implement this if people want. But, let me know if I
should indeed use org-export-registered-backends. However, I'm starting
to feel like this should be separate patch (the goal for mine was just
to prioritize attr_org). I also still don't really like the behavior
here. I don't think it makes sense to interpret a width as 120% if we
have something like

#+attr_latex: :width 1.2\columnwidth

Matt

Re: [PATCH] Fix regex for determining image width from attribute

[Matt Huszagh]

Max Nikulin  writes:

> This is related solely to docscring.
>
>> +that is not found, use the first #+ATTR_.*:width specification.
>
> I am unsure how to make this phrase more clear, maybe something like
> "use :width value from the first #+ATTR_,*" or even "#+ATTR_xxx" to 
> avoid ".*".
>
>> + (re-search-forward (funcall attr-re 
>> "[a-z]*?")
>> +par-end t
>
> https://orgmode.org/worg/dev/org-syntax.html#Keywords
> has no requirement that it may be letter only. I expect it to be more 
> coherent with
> http://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/lisp/org-element.el#n2387
> that uses "\\S-" non-space character.

Better?

Matt

>From 73f6d6d0c16d9b3312737463361cefe08b01d35c Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Mon, 22 Nov 2021 23:28:48 -0800
Subject: [PATCH 1/2] org.el: Prioritize attr_org when computing image width

	* lisp/org.el (org-display-inline-image--width): First look
	for attr_org: :width and then look for another attr_.* :width
	when that isn't specified.
---
 lisp/org.el | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 308bb7d51..3718d3118 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16853,13 +16853,20 @@ buffer boundaries with possible narrowing."
((listp org-image-actual-width)
 (let* ((case-fold-search t)
(par (org-element-lineage link '(paragraph)))
-   (attr-re "^[ \t]*#\\+attr_.*?: +.*?:width +\\(\\S-+\\)")
+   (attr-re (lambda (backend)
+  (concat "^[ \t]*#\\+attr_"
+  backend
+  ":+.*? :width +\\(\\S-+\\)")))
(par-end (org-element-property :post-affiliated par))
-   ;; Try to find an attribute providing a :width.
+   ;; If an attr_org provides a :width, use that. Otherwise,
+   ;; use the first attribute that provides it, if any.
(attr-width
 (when (and par (org-with-point-at
(org-element-property :begin par)
- (re-search-forward attr-re par-end t)))
+ (or (re-search-forward (funcall attr-re "org")
+par-end t)
+ (re-search-forward (funcall attr-re "\\S-+?")
+par-end t))))
   (match-string 1)))
(width
 (cond
-- 
2.31.1


>From 76e92428716f2dcde0fbd281f71739c44a9be9d3 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Mon, 22 Nov 2021 23:30:11 -0800
Subject: [PATCH 2/2] org.el: Clarify documentation for computing image width

	* lisp/org.el (org-display-inline-image--width)
	(org-image-actual-width): Specify documentation for computing
	an inline image width in org-image-actual-width and remove the
	redundant documentation from org-display-inline-image--width.
---
 lisp/org.el | 38 --
 1 file changed, 24 insertions(+), 14 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 3718d3118..b050cb0dd 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -15529,10 +15529,29 @@ When set to a number in a list, try to get the width from any
 
 and fall back on that number if none is found.
 
-When set to nil, try to get the width from an #+ATTR.* keyword
-and fall back on the original width if none is found.
-
-When set to any other non-nil value, always use the image width.
+When set to nil, first try to get the width from #+ATTR_ORG.  If
+that is not found, use the first #+ATTR_xxx :width specification.
+If that is also not found, fall back on the original image width.
+
+Finally, org is quite flexible in the width specifications it
+supports and intelligently interprets width specifications for
+other backends when rendering an image in an org buffer.  This
+behavior is described presently.
+
+1. A floating point value is interpreted as the percentage of the text
+   area that should be taken up by the image.
+2. A number followed by a percent sign is divided by 100 and then
+   interpreted as a floating point value.
+3. If a number is followed by other text, extract the number and
+   discard the remaining text.  That number is then interpreted as a
+   floating-point value.  For example,
+
+   #+ATTR_LATEX: :width 0.7\\linewidth
+
+   would be interpreted as 70% of the text width.
+4. If t is provided the original image width is used.  This is useful
+   when you want to specify a width for a backend, but still want to
+   use the original image width in the org buffer.
 
 This requires Emacs >= 24.1, built with imagemagick support."
   :group 'org-appearance
@@ -16838

Re: [PATCH] Fix regex for determining image width from attribute

[Matt Huszagh]

Max Nikulin  writes:

> I may be wrong, but it seems both the old and the new regexps match
>
>  #+attr_html : :width 50%
>
> that is not a keyword due to a space before ":". The dot in the regexp 
> is too permissive.

I agree.

> Despite ".*" includes ": " before ":width", I would prefer explicit 
> space before ":width".

Currently we have a space before .*. Would you prefer it after? Anyway,
I've also implemented this change. Let me know what you think.

Matt

>From 76a0c05cec8e449efd5cbffd8123338912815f17 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Mon, 22 Nov 2021 23:28:48 -0800
Subject: [PATCH 1/2] org.el: Prioritize attr_org when computing image width

	* lisp/org.el (org-display-inline-image--width): First look
	for attr_org: :width and then look for another attr_.* :width
	when that isn't specified.
---
 lisp/org.el | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 308bb7d51..5f9d120a2 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16853,13 +16853,20 @@ buffer boundaries with possible narrowing."
((listp org-image-actual-width)
 (let* ((case-fold-search t)
(par (org-element-lineage link '(paragraph)))
-   (attr-re "^[ \t]*#\\+attr_.*?: +.*?:width +\\(\\S-+\\)")
+   (attr-re (lambda (backend)
+  (concat "^[ \t]*#\\+attr_"
+  backend
+  ":+.*? :width +\\(\\S-+\\)")))
(par-end (org-element-property :post-affiliated par))
-   ;; Try to find an attribute providing a :width.
+   ;; If an attr_org provides a :width, use that. Otherwise,
+   ;; use the first attribute that provides it, if any.
(attr-width
 (when (and par (org-with-point-at
(org-element-property :begin par)
- (re-search-forward attr-re par-end t)))
+ (or (re-search-forward (funcall attr-re "org")
+par-end t)
+ (re-search-forward (funcall attr-re "[a-z]*?")
+par-end t
   (match-string 1)))
        (width
 (cond
-- 
2.31.1


>From 0bc320b895ffb80a2a3ca8fb494e0aabe76180a3 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Mon, 22 Nov 2021 23:30:11 -0800
Subject: [PATCH 2/2] org.el: Clarify documentation for computing image width

	* lisp/org.el (org-display-inline-image--width)
	(org-image-actual-width): Specify documentation for computing
	an inline image width in org-image-actual-width and remove the
	redundant documentation from org-display-inline-image--width.
---
 lisp/org.el | 38 --
 1 file changed, 24 insertions(+), 14 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 5f9d120a2..37369cdb6 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -15529,10 +15529,29 @@ When set to a number in a list, try to get the width from any
 
 and fall back on that number if none is found.
 
-When set to nil, try to get the width from an #+ATTR.* keyword
-and fall back on the original width if none is found.
-
-When set to any other non-nil value, always use the image width.
+When set to nil, first try to get the width from #+ATTR_ORG.  If
+that is not found, use the first #+ATTR_.*:width specification.
+If that is also not found, fall back on the original image width.
+
+Finally, org is quite flexible in the width specifications it
+supports and intelligently interprets width specifications for
+other backends when rendering an image in an org buffer.  This
+behavior is described presently.
+
+1. A floating point value is interpreted as the percentage of the text
+   area that should be taken up by the image.
+2. A number followed by a percent sign is divided by 100 and then
+   interpreted as a floating point value.
+3. If a number is followed by other text, extract the number and
+   discard the remaining text.  That number is then interpreted as a
+   floating-point value.  For example,
+
+   #+ATTR_LATEX: :width 0.7\\linewidth
+
+   would be interpreted as 70% of the text width.
+4. If t is provided the original image width is used.  This is useful
+   when you want to specify a width for a backend, but still want to
+   use the original image width in the org buffer.
 
 This requires Emacs >= 24.1, built with imagemagick support."
   :group 'org-appearance
@@ -16838,16 +16857,7 @@ buffer boundaries with possible narrowing."
 (defvar visual-fill-column-width) ; Silence compiler warning
 (defun org-display-inline-image--width (link)
   "Determine the display width of the image LINK, in pixels.
-- When `org-image-actual-width' is t, the image's pixel width is u

Re: [PATCH] Fix regex for determining image width from attribute

[Matt Huszagh]

Here are two patches that prioritize attr_org over other attr_
keywords. I believe this is what you had in mind Timothy. But let me
know if not.

The second patch (intended to be applied after the first) improves the
documentation. It describes behavior that wasn't previously documented
and removes redundant documentation between org-image-actual-width and
org-display-inline-image--width (now only in
org-image-actual-width). Please double check I got everything correct,
as I haven't used all this behavior myself.

Thanks
Matt

>From 22bbe7d651e1f27398597297c7c35ae4f3253773 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Mon, 22 Nov 2021 23:28:48 -0800
Subject: [PATCH 1/2] org.el: Prioritize attr_org when computing image width

	* lisp/org.el (org-display-inline-image--width): First look
	for attr_org: :width and then look for another attr_.* :width
	when that isn't specified.
---
 lisp/org.el | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 308bb7d51..bf5d08e09 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16853,13 +16853,20 @@ buffer boundaries with possible narrowing."
((listp org-image-actual-width)
 (let* ((case-fold-search t)
(par (org-element-lineage link '(paragraph)))
-   (attr-re "^[ \t]*#\\+attr_.*?: +.*?:width +\\(\\S-+\\)")
+   (attr-re (lambda (backend)
+  (concat "^[ \t]*#\\+attr_"
+  backend
+  ": +.*?:width +\\(\\S-+\\)")))
(par-end (org-element-property :post-affiliated par))
-   ;; Try to find an attribute providing a :width.
+   ;; If an attr_org provides a :width, use that. Otherwise,
+   ;; use the first attribute that provides it, if any.
(attr-width
 (when (and par (org-with-point-at
(org-element-property :begin par)
- (re-search-forward attr-re par-end t)))
+ (or (re-search-forward (funcall attr-re "org")
+par-end t)
+ (re-search-forward (funcall attr-re ".*?")
+par-end t
   (match-string 1)))
(width
 (cond
-- 
2.31.1

>From aff581922e8d8bf10f813cdb9bc8adf9c8632683 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Mon, 22 Nov 2021 23:30:11 -0800
Subject: [PATCH] org.el: Clarify documentation for computing image width

	* lisp/org.el (org-display-inline-image--width)
	(org-image-actual-width): Specify documentation for computing
	an inline image width in org-image-actual-width and remove the
	redundant documentation from org-display-inline-image--width.
---
 lisp/org.el | 38 --
 1 file changed, 24 insertions(+), 14 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index bf5d08e09..c8dc1815f 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -15529,10 +15529,29 @@ When set to a number in a list, try to get the width from any
 
 and fall back on that number if none is found.
 
-When set to nil, try to get the width from an #+ATTR.* keyword
-and fall back on the original width if none is found.
-
-When set to any other non-nil value, always use the image width.
+When set to nil, first try to get the width from #+ATTR_ORG.  If
+that is not found, use the first #+ATTR_.*:width specification.
+If that is also not found, fall back on the original image width.
+
+Finally, org is quite flexible in the width specifications it
+supports and intelligently interprets width specifications for
+other backends when rendering an image in an org buffer.  This
+behavior is described presently.
+
+1. A floating point value is interpreted as the percentage of the text
+   area that should be taken up by the image.
+2. A number followed by a percent sign is divided by 100 and then
+   interpreted as a floating point value.
+3. If a number is followed by other text, extract the number and
+   discard the remaining text.  That number is then interpreted as a
+   floating-point value.  For example,
+
+   #+ATTR_LATEX: :width 0.7\\linewidth
+
+   would be interpreted as 70% of the text width.
+4. If t is provided the original image width is used.  This is useful
+   when you want to specify a width for a backend, but still want to
+   use the original image width in the org buffer.
 
 This requires Emacs >= 24.1, built with imagemagick support."
   :group 'org-appearance
@@ -16838,16 +16857,7 @@ buffer boundaries with possible narrowing."
 (defvar visual-fill-column-width) ; Silence compiler warning
 (defun org-display-inline-image--width (link)
   "Determine the display width of the image LINK, in pixels.
-- When `org-image-actual-width' is t, the image's pixel width is used.
-- When `org-image-actu

Re: Why is an image width restricted to being between 0 and 200% of the text area

[Matt Huszagh]

Timothy  writes:

> It occurred to me that large values would only really appear when occurring 
> in a
> narrower scope, e.g. in a multi-column document an image which is multiple
> columns wide. Say,
> ┌
> │ #+attr_latex: 3.0\columnwidth
> └
>
> However, I doubt that when previewing the image in Org one would want the
> preview to be 3x the buffer text area width!

Good point. I could see someone using this as an argument against
interpreting :width 3.0\columnwidth as :width 3.0, but I do see the
value in interpreting :width 0.8\linewidth as :width 0.8, so I don't
know.

> I suppose the exception would be say something like
> ┌
> │ #+attr_org: :width 300%
> └
> Though I can’t imagine why you’d want that…
>
> I think you don’t raise an unreasonable point, however I’m tempted to let this
> sit till we hear from someone who actually runs into this as an issue.

Yeah I think that makes sense. Thanks for considering this anyway!

Matt

Re: Why is an image width restricted to being between 0 and 200% of the text area

[Matt Huszagh]

Timothy  writes:

> Just to be clear, this isn’t preventing people from setting image widths to 
> say
> `5.0\linewidth', it just prevents them from being previewed at 5x the buffer 
> text
> width as that seems a bit ridiculous. Happy to reconsider the [0,2] preview
> restriction, but I don’t think ridiculously large in-buffer images make much
> sense.

Thanks for the response Timothy.

I agree that requesting an image to be >2x the buffer text width is a
strange request, and it's not one I've ever tried to give. But, I think
the salient point is that it's a very clear request, and I think org
should carry it out. I'm all in favor of org helping people not shoot
themselves in the foot, but I don't think it should prevent people from
doing so, especially when they're clear about their intentions. I also
think this qualifies as a case where someone /might/ have a valid reason
for doing this.

I guess we could make the upper limit customizable and default to
2.0. But, this is a bit confusing because it doesn't apply to the
original image width. I also think adding too many customizable
variables adds to complexity. I don't know. Thoughts? This also isn't a
feature I've ever needed... so I'm happy to concede and revisit it in
the future if I have a valid use case for it.

Matt

Why is an image width restricted to being between 0 and 200% of the text area

[Matt Huszagh]

Hi,

In org-display-inline-image--width we have the restriction that
percentages and floating point width specifications must be between 0
and 200% (or 0.0 and 2.0) of the text area. Reading the comments, it
sounds like this is done because you might specify something like
1.2\linewidth, but you wouldn't write something like 5.0\linewidth. I
guess that's fair, but I can't say with certainty that there is never
any valid reason to, e.g., specify that you want a width of 500% of the
text area. I'm also not sure that we should prevent people from using
5.0\linewidth, even if it is a bit odd. The intent is clear.

Maybe I'm missing a reason as to why this is done?

Best
Matt

Re: [PATCH] Fix regex for determining image width from attribute

[Matt Huszagh]

Timothy  writes:

> This issue and Kyle’s change were resolved in another thread, just FYI this is
> fixed now. Thanks for mentioning it.

There is just one small residual error I could find. This patch fixes
it.

Matt

>From 3724b5bcadab6900367848dadcf470494b5b0d79 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Mon, 22 Nov 2021 21:36:03 -0800
Subject: [PATCH] org.el: Fix string-match-p function arguments

	* lisp/org.el (org-display-inline-image--width):
	string-match-p requires 2 arguments, but only one was given.
---
 lisp/org.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index 736b743c7..308bb7d51 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16867,7 +16867,7 @@ buffer boundaries with possible narrowing."
  ((string= attr-width "t") nil)
  ;; Fallback to `org-image-actual-width' if no interprable width is given.
  ((or (null attr-width)
-  (string-match-p "\\`[^0-9]"))
+  (string-match-p "\\`[^0-9]" attr-width))
   (car org-image-actual-width))
  ;; Convert numeric widths to numbers, converting percentages.
  ((string-match-p "\\`[0-9.]+%" attr-width)
-- 
2.31.1

Re: [PATCH] Fix regex for determining image width from attribute

[Matt Huszagh]

Timothy  writes:

> I’ve just pushed the change I described in 4514a32. This improves the
> interpretation of :width attributes and makes a value of “t” work as 
> discussed.
> I have not prioritised #+attr_org for now, but that sounds like a change we
> could make in the future.

Thanks Timothy. However, I think this change may have some issues. It
seems that it unbalances parentheses. This also no longer sets width
(so, e.g., (floatp width) won't work). Maybe attr-width was intended to
be renamed to width? Are you seeing the same?

I've got an implementation for prioritizing #+attr_org, but I want to
make sure your commit goes in the right way before I submit that.

Matt

Re: [PATCH] Fix regex for determining image width from attribute

[Matt Huszagh]

Timothy  writes:

> Actually, it’s almost possible with the current implementation. Consider this
> example:
> ┌
> │ #+attr_org: :width t
> │ #+attr_html: :width 20%
> │ [[file:image.png]]
> └
>
> At the moment Org tries to interpret `t' as a number (and obviously fails),
> however with a small tweak that I think would be very reasonable to make, this
> would cause the behaviour you describe.
>
> What do you think?

Yeah, I think it's ok. To be perfectly honest, I still don't love it,
but I understand now that my disagreement was with a decision made a
while ago (from a quick look at the commit history, 2012 or before), not
with the one you made recently.

In my opinion the most logical solution would be for the width to fall
back on specifically attr_org, not just any attr_ when
`org-image-actual-width' is nil. To clarify further, my main
disagreement is with the idea that setting attr_html (for example)
implies that one wants the same setting in the org buffer. But, it seems
that ship sailed a while ago and now this would be a breaking change.

So, given that, I concede and I think attr_org: :width t is an
acceptable compromise. Thanks for coming up with that!

>> (IIRC this is how it previously worked, but I could be mistaken).
>
> You are mistaken. The previous implementation looked for `#+attr_*' too, but
> didn’t recognise proportional values.

Ok, thanks for clarifying.

>> Maybe a solution to accomplish all goals would be to add an #+attr_fallback
>> (or attr_default, attr_any, attr_all, etc.) that is used for any backend
>> unless a specific setting is made for that backend.
>
> Hmmm, I’m not sure this is called for.

Yeah, I agree this is wrong. I'd misunderstood the current behavior.

Matt

Re: [PATCH] Fix window width when line numbers present

[Matt Huszagh]

Bastien  writes:

> See https://orgmode.org/worg/org-maintenance.html#emacs-compatibility

Ah, my mistake. Thanks for pointing me to that. So does this stay on
some other branch until the Emacs official release is at 28?

Matt

Re: [PATCH] Fix window width when line numbers present

[Matt Huszagh]

Nicolas Goaziou  writes:

> `line-number-display-width' is Emacs 26+. So I guess it is unfortunately
> not acceptable on main branch.

Does org-mode not require 26+ even though the Emacs official release is
on 27?

Matt

Re: [PATCH] Fix regex for determining image width from attribute

[Matt Huszagh]

Timothy  writes:

> Once again, thank you for the patch. The fact that the current regexp matches
> `#+attr_latex' and `#+attr_html' is in fact by design though*. This is 
> because I
> consider it safe to assume that a `#+attr_*' which gives non-integer width 
> between
> 0 and 2 can be safely assumed to be that proportion of the page width. e.g.
> `#+attr_latex: :width 0.7\linewidth' or `#+attr_html: :width 70%'.
> This way, a good guess can be made without having do have both an
> `#+attr_latex/html' /and/ an `#+attr_org' line for the width. Should this 
> assumption
> be incorrect, a subsequent `#+attr_org' line will override the other 
> `#+attr_*'.
>
> Should there be edge-cases where this assumption doesn’t hold, I’d be 
> interested
> in patches that improves the logic here. As long as this holds more often than
> not though, this should be a net positive for user experience as I see it.
>
> Do please let me know if there are any good examples of unintended /
> counter-intuitive behaviour you’re aware of.

Thanks for explaining the logic behind the current implementation.

Unfortunately, I think this makes a valid use case
impossible. Specifically, I like to be able to set an image width
explicitly with #+attr_org (or some other attr_ for the corresponding
export) and fall back to the actual image width when this isn't
specified. This includes the ability to use the actual image width in an
org buffer, but an explicitly-set image width for export. For example,
for my blog I often have attr_html set, but I want the image to use its
actual width when displayed in org. I don't see how this is possible
with the current implementation. But, it works naturally with the
implementation I have in mind (IIRC this is how it previously worked,
but I could be mistaken).

I do understand the desire to avoid specifying a whole bunch of
redundant attr_ settings, but I don't think it should make fine-tuned
use cases like the one above impossible. I also find the current
implementation a bit counterintuitive, if less verbose. Maybe a solution
to accomplish all goals would be to add an #+attr_fallback (or
attr_default, attr_any, attr_all, etc.) that is used for any backend
unless a specific setting is made for that backend.

Matt

[PATCH] Fix regex for determining image width from attribute

[Matt Huszagh]

Hi,

A recent patch started computing the inline image width from any attr_
line. This is incorrect, as it matches settings like attr_latex, or
attr_html. We only want to look for settings specifically for the org
buffer. This patch fixes that.

Thanks
Matt

>From b9b8fb9b31dbb9145c2fe73af04eccd59f7a9973 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Sun, 21 Nov 2021 11:02:37 -0800
Subject: [PATCH] lisp/org.el: Fix regex for determining image width from
 attribute

	* lisp/org.el (org-display-inline-image--width): The regex
	should only match attr_org, not any attr_.  This would match
	attributes set for all export backends, such as latex and
	HTML, which is incorrect.
---
 lisp/org.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index eeefb4af3..92e765373 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16853,7 +16853,7 @@ buffer boundaries with possible narrowing."
((listp org-image-actual-width)
 (let* ((case-fold-search t)
(par (org-element-lineage link '(paragraph)))
-   (attr-re "^[ \t]*#\\+attr_.*?: +.*?:width +\\(\\S-+\\)")
+   (attr-re "^[ \t]*#\\+attr_org: +.*?:width +\\(\\S-+\\)")
(par-end (org-element-property :post-affiliated par))
;; Try to find an attribute providing a :width.
(attr-width
-- 
2.31.1

[PATCH] Fix window width when line numbers present

[Matt Huszagh]

Hello,

This is a very small patch that I believe fixes a previous, incorrect
fix for computing the window text area when line numbers are
present. The previous fix seemed to assume that
display-line-numbers-width being nil meant that this means line numbers
are not present. However, it can also mean (as described in the
variable documentation) that the line number width is computed
dynamically.

Thanks
Matt

>From 5faaf45bf54c0cd3135b9c5a747c22797fe1d290 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Sun, 21 Nov 2021 10:30:30 -0800
Subject: [PATCH] org.el: Fix inline image width calculation when line numbers
 present

* lisp/org.el (org-display-inline-image--width): When
display-line-numbers-width is nil, the width is computed dynamically.
This does not mean that the line number width is necessarily 0.
---
 lisp/org.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index eeefb4af3..331bd9f65 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16880,7 +16880,7 @@ buffer boundaries with possible narrowing."
 (/ (or (and (bound-and-true-p visual-fill-column-mode)
 (or visual-fill-column-width auto-fill-function))
(when auto-fill-function fill-column)
-   (- (window-text-width) (or display-line-numbers-width 0)))
+   (- (window-text-width) (line-number-display-width)))
(float (window-total-width)
 width)))
((numberp org-image-actual-width)
-- 
2.31.1

[PATCH] Fix behavior of lambda default header arg vars

[Matt Huszagh]

Hello,

:var header arguments can be provided multiple times. This is supported
directly at the source block and through the default header argument
facility. However, this was not handled correctly when the var was
evaluated from a closure in a default header argument (only the last var
was taken). This patch fixes that. I've also added documentation
explaining how to set multiple vars in the default header argument
facility.

Thanks,
Matt

>From 5f02fe4e896e6909fab18e8d90a7e14bdce9d5c4 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Wed, 27 Oct 2021 07:47:20 -0700
Subject: [PATCH] ob-core.el: Fix behavior of lambda default header arg vars

* lisp/ob-core.el: Permit multiple :var default header arguments when
using closures. Additionally, document how to use default header
arguments for arguments that can be provided multiple times.
---
 lisp/ob-core.el | 26 --
 1 file changed, 20 insertions(+), 6 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 7a9467b0e..21f461a08 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -478,12 +478,21 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'."
 This is a list in which each element is an alist.  Each key
 corresponds to a header argument, and each value to that header's
 value.  The value can either be a string or a closure that
-evaluates to a string.  The closure is evaluated when the source
-block is being evaluated (e.g. during execution or export), with
-point at the source block.  It is not possible to use an
-arbitrary function symbol (e.g. 'some-func), since org uses
-lexical binding.  To achieve the same functionality, call the
-function within a closure (e.g. (lambda () (some-func))).
+evaluates to a string.  Some header arguments (e.g., :var for
+some language backends) can be provided multiple times for a
+source block.  This functionality is also supported for default
+header arguments by providing the header argument multiple times
+in the alist.  For example:
+
+'((:var . \"foo=\\\"bar\\\"\")
+  (:var . \"bar=\\\"foo\\\"\"))
+
+A closure is evaluated when the source block is being
+evaluated (e.g. during execution or export), with point at the
+source block.  It is not possible to use an arbitrary function
+symbol (e.g. 'some-func), since org uses lexical binding.  To
+achieve the same functionality, call the function within a
+closure (e.g. (lambda () (some-func))).
 
 To understand how closures can be used as default header
 arguments, imagine you'd like to set the file name output of a
@@ -2718,6 +2727,11 @@ parameters when merging lists."
 	(pcase pair
 	  (`(:var . ,value)
 	   (let ((name (cond
+;; Default header arguments can accept lambda
+;; functions. We uniquely identify the var
+;; according to the full string contents of
+;; the lambda function.
+			((functionp value) value)
 			((listp value) (car value))
 			((string-match "^\\([^= \f\t\n\r\v]+\\)[ \t]*=" value)
 			 (intern (match-string 1 value)))
-- 
2.31.1

Re: Patch to allow adjusting latex fragment display scale factor

[Matt Huszagh]

Matt Huszagh  writes:

> I've created a patch to allow adjusting the scale factor used for inline
> latex image fragments. This involves a customizable variable that can
> either be set to a scale factor (defaults to 1.0) or a function that
> evaluates to a scale factor.
>
> This feature is in addition to the existing scale factor adjustment
> capability provided by `org-preview-latex-process-alist' through
> `:image-size-adjust'. Wherease image-size-adjust performs scaling at the
> time of image generation, the new change performs it during
> display. This can lead to significant time saving and suffers no loss of
> quality for vector graphics.
>
> As an example of use, I have several functions for changing frame
> scaling. I've added
>
> (if (eq major-mode 'org-mode)
>   (progn
> (org-clear-latex-preview)
> ;; 16 corresponds to the C-u C-u arg prefix.
> (org-latex-preview 16)))
>
> to these functions so that changing the frame scaling also
> correspondingly changes the latex preview/fragment scaling to match the
> new size of the surrounding text. Because of this new feature, this
> change is effectively instantaneous for reasonably numbers of
> overlays. Obviously, something similar could be done for
> `text-scale-adjust' (e.g., through `advice-add').
>
> Feedback appreciated.

Apologies, the patch I sent is slightly wrong. The line numbers also
reflect an earlier patch I made. Here is a corrected version.

>From 3c0e74a8659edb52c1200a02f8a20216b348c4ac Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Sun, 10 Oct 2021 21:20:31 -0700
Subject: [PATCH] org.el: Allow customizing overlay-put scale factor

* lisp/org.el (org-latex-fragment-scale): Add customizable variable
that is equal to or evaluates to a scale factor used to scale inline
latex fragments.
(org--make-preview-overlay): Adjust latex fragment overlay generation
to account for the new custom scale factor.
---
 lisp/org.el | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index 405f0f0f9..052212efb 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -320,6 +320,15 @@ identifier."
   :version "24.1"
   :group 'org-id)
 
+(defcustom org-latex-fragment-scale 1.0
+  "Scaling factor used for LaTeX image fragments.
+This can either be a number or a function that takes the image
+data and image type as parameters and evaluates to a number.  The
+function can be useful for context-aware scaling, such as setting
+the scale factor to be consistent with the surrounding text size."
+  :type '(number function)
+  :group 'org)
+
 ;;; Version
 (org-check-version)
 
@@ -15936,7 +15945,13 @@ as a string.  It defaults to \"png\"."
 			 (delete-overlay o
 (overlay-put ov
 		 'display
-		 (list 'image :type imagetype :file image :ascent 'center
+		 (list 'image
+   :type imagetype
+   :file image
+   :ascent 'center
+   :scale (if (functionp org-latex-fragment-scale)
+  (funcall org-latex-fragment-scale image imagetype)
+org-latex-fragment-scale)
 
 (defun org-clear-latex-preview ( beg end)
   "Remove all overlays with LaTeX fragment images in current buffer.
-- 
2.31.1

Patch to allow adjusting latex fragment display scale factor

[Matt Huszagh]

Hi,

I've created a patch to allow adjusting the scale factor used for inline
latex image fragments. This involves a customizable variable that can
either be set to a scale factor (defaults to 1.0) or a function that
evaluates to a scale factor.

This feature is in addition to the existing scale factor adjustment
capability provided by `org-preview-latex-process-alist' through
`:image-size-adjust'. Wherease image-size-adjust performs scaling at the
time of image generation, the new change performs it during
display. This can lead to significant time saving and suffers no loss of
quality for vector graphics.

As an example of use, I have several functions for changing frame
scaling. I've added

(if (eq major-mode 'org-mode)
  (progn
(org-clear-latex-preview)
;; 16 corresponds to the C-u C-u arg prefix.
(org-latex-preview 16)))

to these functions so that changing the frame scaling also
correspondingly changes the latex preview/fragment scaling to match the
new size of the surrounding text. Because of this new feature, this
change is effectively instantaneous for reasonably numbers of
overlays. Obviously, something similar could be done for
`text-scale-adjust' (e.g., through `advice-add').

Feedback appreciated.

Matt

>From 8247947aa6141cde9c58205e0266f0e674226f95 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Sun, 10 Oct 2021 21:20:31 -0700
Subject: [PATCH] org.el: Allow customizing overlay-put scale factor

* lisp/org.el (org-latex-fragment-scale): Add customizable variable
that is equal to or evaluates to a scale factor used to scale inline
latex fragments.
(org--make-preview-overlay): Adjust latex fragment overlay generation
to account for the new custom scale factor.
---
 lisp/org.el | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index dbc288a3c..f5e9ff8d1 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -388,6 +388,15 @@ text."
   :type '(symbol function integer)
   :group 'org)
 
+(defcustom org-latex-fragment-scale 1.0
+  "Scaling factor used for LaTeX image fragments.
+This can either be a number or a function that takes the image
+data and image type as parameters and evaluates to a number.  The
+function can be useful for context-aware scaling, such as setting
+the scale factor to be consistent with the surrounding text size."
+  :type '(number function)
+  :group 'org)
+
 ;;; Version
 (org-check-version)
 
@@ -16007,7 +16016,13 @@ as a string.  It defaults to \"png\"."
 			   (delete-overlay o
   (overlay-put ov
 		   'display
-		   (list 'image :type imagetype :file image :ascent ascent)
+		   (list 'image
+ :type imagetype
+ :file image
+ :ascent ascent
+ :scale (if (functionp org-latex-fragment-scale)
+(funcall org-latex-fragment-scale image imagetype)
+  org-latex-fragment-scale))
 
 (defun org-clear-latex-preview ( beg end)
   "Remove all overlays with LaTeX fragment images in current buffer.
-- 
2.31.1

Re: Patch to align baseline of latex fragments and surrounding text

[Matt Huszagh]

Matt Huszagh  writes:

> I've created a patch to align the baseline of latex image fragments to
> the surrounding text. The patch consists of several parts. First, it
> adds a customizable variable that can be set to a user supplied function
> to compute the value of :ascent passed to `overlay-put'. It can also be
> set to a symbol (e.g., 'center, which is the current setting and still
> the default), or an integer (:ascent can take an integer
> percentage). The patch also modifies `org--make-preview-overlay' to use
> this new variable.
>
> The other part of the patch is a new function that computes the correct
> value for :ascent from an SVG file.
>
> Unfortunately, this isn't a general-purpose solution to baseline text
> alignment. It currently only works with SVG images and requires that
> these SVG images encode the text baseline in the viewBox attribute (I've
> explained in the function documentation how to achieve this).
>
> I was not initially planning to include the function because it only
> works in certain special cases (though if you setup your SVG image
> generation as I've described in the documentation it should work
> correctly for all of your LaTeX fragments). However, I ultimately
> decided it was beneficial to include it for several reasons: (1) it
> would not be obvious for many users how to write this function on their
> own, (2) by including it, others can improve upon it, and (3) I tried to
> minimize corner cases by leaving the default value as 'center and having
> the function return 'center whenever it detects an incorrect :ascent
> value.

I should also mention that I've tested this for different image scaling
factors (`:image-size-adjust' in `org-preview-latex-process-alist') and
it works regardless of the scale factor. That is, the baseline will be
correctly aligned irrespective of the scale factor you set (though
obviously the text median
(https://en.wikipedia.org/wiki/Baseline_(typography))) will only match
the surrounding text for the correct scaling factor.

Matt

Patch to align baseline of latex fragments and surrounding text

[Matt Huszagh]

Hello,

I've created a patch to align the baseline of latex image fragments to
the surrounding text. The patch consists of several parts. First, it
adds a customizable variable that can be set to a user supplied function
to compute the value of :ascent passed to `overlay-put'. It can also be
set to a symbol (e.g., 'center, which is the current setting and still
the default), or an integer (:ascent can take an integer
percentage). The patch also modifies `org--make-preview-overlay' to use
this new variable.

The other part of the patch is a new function that computes the correct
value for :ascent from an SVG file.

Unfortunately, this isn't a general-purpose solution to baseline text
alignment. It currently only works with SVG images and requires that
these SVG images encode the text baseline in the viewBox attribute (I've
explained in the function documentation how to achieve this).

I was not initially planning to include the function because it only
works in certain special cases (though if you setup your SVG image
generation as I've described in the documentation it should work
correctly for all of your LaTeX fragments). However, I ultimately
decided it was beneficial to include it for several reasons: (1) it
would not be obvious for many users how to write this function on their
own, (2) by including it, others can improve upon it, and (3) I tried to
minimize corner cases by leaving the default value as 'center and having
the function return 'center whenever it detects an incorrect :ascent
value.

Let me know what you think!

Best
Matt

>From 6c33fa1875cc169616d878b59054d50980f741f3 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Sun, 10 Oct 2021 15:46:05 -0700
Subject: [PATCH] org--make-preview-overlay: Add ability to set vertical
 alignment of latex fragments

* lisp/org.el (org--match-text-baseline-ascent): Add function that
computes the value of :ascent to match the baseline of the fragment to
the surrounding text.
(org-latex-fragment-overlay-ascent): Add custom variable that allows a
function to be used to compute the value of :ascent.
(org--make-preview-overlay): Incorporate the custom variable.
---
 lisp/org.el | 89 +++--
 1 file changed, 80 insertions(+), 9 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 405f0f0f9..dbc288a3c 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -320,6 +320,74 @@ identifier."
   :version "24.1"
   :group 'org-id)
 
+(defun org--match-text-baseline-ascent (imagefile imagetype)
+  "Set :ascent to match the text baseline of an image to the surrounding text.
+IMAGEFILE is the path of the image file and IMAGETYPE is the
+image file type.
+
+This function currently only works for SVG images (defaulting to
+'center otherwise).  It also requires that the SVG's viewbox is
+correctly set according to the text baseline position.
+Specifically, it computes
+
+ascent = 100 * -min-y / height
+
+where min-y and height are taken from the viewbox.  If this
+computation returns a non-sensical value (below 0 or above 100),
+it will return 'center instead.
+
+It's possible to include text baseline information in the viewbox
+by using \\documentclass[preview]{standalone} in the input LaTeX
+file, compiling first to DVI and then converting this to an SVG
+image with dvisvgm.
+
+For example,
+
+(setq my-dvisvgm
+  '(my-dvisvgm :programs (\"latex\" \"dvisvgm\")
+   :description \"dvi > svg\"
+   :message \"you need to install latex and dvisvgm.\"
+   :use-xcolor t
+   :image-input-type \"dvi\"
+   :image-output-type \"svg\"
+   :latex-compiler (\"latex -output-directory=%o %f\")
+   :image-converter (\"dvisvgm --no-fonts --exact-bbox -c %S -o %O %f\")))
+
+(add-to-list 'org-preview-latex-process-alist my-dvisvgm)
+(setq org-preview-latex-default-process 'my-dvisvgm)
+
+(setq org-format-latex-header
+  \"\\documentclass[preview]{standalone}
+  [PACKAGES]
+  [DEFAULT-PACKAGES]\")"
+  (if (eq imagetype 'svg)
+  (let* ((viewbox (split-string
+   (xml-get-attribute (car (xml-parse-file imagefile)) 'viewBox)))
+ (min-y (string-to-number (nth 1 viewbox)))
+ (height (string-to-number (nth 3 viewbox)))
+ (ascent (round (* -100 (/ min-y height)
+(if (or (< ascent 0) (> ascent 100))
+'center
+  ascent))
+'center))
+
+(defcustom org-latex-fragment-overlay-ascent 'center
+  "Value of :ascent used for LaTeX image fragments.
+Org uses `overlay-put' to overlay LaTeX image fragments on inline
+math.  `overlay-put' takes an :ascent parameter that can specify
+the vertical offset of this image fragment.  See the 'Image
+Descriptors' section of the elisp manual for more information.
+
+If this variable

Re: babel default header args as functions

[Matt Huszagh]

Timothy  writes:

> I just read your docs patch, and that’s lead to a quick question: does this 
> patch
> support a header arg function in the form “(:file . 
> #’my-org-file-name-generator)”?

Unfortunately it doesn't. It's been about a year since I wrote this
patch, so I'm a bit hazy on some of the details, but IIRC it has to deal
with org-mode's use of lexical binding. If you're interested in digging
deeper (or maybe even getting it to work, which would be great) I'd look
at `org-babel-eval-headers'. The `functionp' line in there returns nil
and the function isn't evaluated unless it's a closure.

Matt

Re: babel default header args as functions

[Matt Huszagh]

Matt Huszagh  writes:

> I've tested it, and if you revert
> 78783f4e47901255695031dae0efcbb301a40878 and apply the new patch, it
> will apply with conflicts. Let me know if you run into any difficulties,
> have any concerns, etc.

That "with" should be "without"...

Matt

Re: babel default header args as functions

[Matt Huszagh]

Matt Huszagh  writes:

> Thanks Bastien, and no worries about the delay. However, I hate to say
> it but I think you may have applied an old patch. The most recent patch
> is
>
> https://lists.gnu.org/archive/html/emacs-orgmode/2020-09/txtzi_PffIaG1.txt
>
> Let me know what I can do.

I've tested it, and if you revert
78783f4e47901255695031dae0efcbb301a40878 and apply the new patch, it
will apply with conflicts. Let me know if you run into any difficulties,
have any concerns, etc.

>> It deserves an entry in etc/ORG-NEWS for Org 9.5, would you be willing
>> to submit a patch for this?
>
> Yep, I'll send one over.

Here's the patch for the news item. Bear in mind that the last part
about lazy evaluation is only true for the newest patch.

>From ae721d089854e3b6b2d71ab6829b0cace25f0968 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Tue, 28 Sep 2021 18:26:04 -0700
Subject: [PATCH] etc/ORG-NEWS: Add news item about
 org-babel-default-header-args accepting closures

---
 etc/ORG-NEWS | 29 -
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index b0a893946..db6df83f2 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -151,7 +151,7 @@ enable Calc units simplification mode.
 
 *** Support fontification of inline export snippets
 
-Inline 
+Inline
 
 See [[msg:87im57fh8j@gmail.com][this thread]].
 
@@ -443,6 +443,33 @@ See [[msg:875z8njaol@protesilaos.com][this thread]].
 attachment directory at calls of ~org-attach-sync~.  There is
 Never delete, Always delete and Query the user (default).
 
+*** ~org-babel-default-header-args~ can now be specified as closures or strings
+
+~org-babel-default-header-args~ now also accepts closures that
+evaluate to a string. Previously, only direct strings were
+supported. These closures are evaluated when point is at the source
+block, which allows them to make use of contextual information at the
+relevant source block. One example that illustrates the usefulness of
+this addition (also given in the documentation for
+~org-babel-default-header-args~) is:
+
+#+begin_src elisp
+(defun org-src-sha ()
+  (let ((elem (org-element-at-point)))
+(concat (sha1 (org-element-property :value elem)) \".svg\")))
+
+(setq org-babel-default-header-args:latex
+  `((:results . \"file link replace\")
+(:file . (lambda () (org-src-sha)
+#+end_src
+
+This will set the ~:file~ header argument to the sha1 checksum of the
+contents of the current latex source block.
+
+Finally, the closures are only evaluated if they're not overridden for
+a source block. This improves efficiency in cases where the result of
+a compute-expensive closure would otherwise be discarded.
+
 ** Miscellaneous
 *** =org-bibtex= includes =doi= and =url= entries when exporting to BiBTeX
 =doi= and =url= entries have been made optional for some publication
-- 
2.31.1


Matt

Re: babel default header args as functions

[Matt Huszagh]

Bastien  writes:

> Applied, thanks, and sorry again for the lng delay.

Thanks Bastien, and no worries about the delay. However, I hate to say
it but I think you may have applied an old patch. The most recent patch
is

https://lists.gnu.org/archive/html/emacs-orgmode/2020-09/txtzi_PffIaG1.txt

Let me know what I can do.

> It deserves an entry in etc/ORG-NEWS for Org 9.5, would you be willing
> to submit a patch for this?

Yep, I'll send one over.

Matt

Customizing the agenda item format

[Matt Huszagh]

I'm attempting to create a custom agenda, but having a bit of trouble
getting this to work in the way that I want. In particular, I'd like to
be able to specify the display format for each item.

Here's what I've got so far:

(setq org-agenda-custom-commands
  '(("c" "Custom agenda view"
 ((tags-todo "hardware|electronics|mechanical_engineering"
 ((org-agenda-skip-function '(org-agenda-skip-entry-if 
'todo '("HOLD")))
  (org-agenda-overriding-header "Hardware")
  (org-agenda-prefix-format "  ")
  (org-agenda-hide-tags-regexp ".*")))

This gives me something like

TODO some headline title

This is close to what I want. But, all my items are TODO so I don't want
to display this. In other words, I just want

some headline title

Then, I'd also like to display the title of the page, right-justified,
in parentheses.

So,

some headline title(some page title)

Another display I'm considering is to present the full outline path,
rather than just the title, so something like

some/headline/path (some page title)

I can provide functions to get this information, but I don't know how to
present it in the agenda.

Are displays like this possible? `org-agenda-prefix-format' gives a lot
of flexibility for the prefix format, but I wasn't able to find
something similar for the actual item. The "Presentation and Sorting"
part of the manual also didn't give me any leads.

Thanks
Matt

Re: Account for latex snippet width in fill paragraph

[Matt Huszagh]

Eric S Fraga  writes:

> I don't have this issue because I use visual-line-mode (together with
> org-indent-mode) and text reflows automatically.  Every paragraph is a
> single "line" in the org file but is (soft) wrapped automatically at the
> window edge.  But this may not suit you, of course.

This works very well. Thanks for the suggestion!

Matt

Account for latex snippet width in fill paragraph

[Matt Huszagh]

Hello,

I make extensive use of inline latex image snippets in my Org
buffers. One thing that has annoyed me for a while is that
`org-fill-paragraph' (unsurprisingly) uses the width of the underlying
text when determining the characters which make up each line. Because
the characters which make up the source of an image are typically wider
than the image itself, this creates short lines when many snippets are
used. For example, the following text

```
This gives some degree of temperature-stability because although
\(V_{\mathrm{BE}}\) changes somewhat with temperature, the change is
small. If the quiescent output voltage is designed for say
\(10V_{\mathrm{BE}}\), and \(V_{\mathrm{BE}}\) varies \(\SI{0.1}{V}\)
over temperature changes, then the quiescent point will vary
\(\SI{1}{V}\) over temperature changes. This isn't great, but may
sometimes be good enough.
```

appears something like

```
This gives some degree of temperature-stability because although
VBE changes somewhat with temperature, the change is
small. If the quiescent output voltage is designed for say
10VBE, and VBE varies 0.1V
over temperature changes, then the quiescent point will vary
1V over temperature changes. This isn't great, but may
sometimes be good enough.
```

Before I take a crack at this, has anyone else attempted to remedy this?
I wasn't able to find anything on the mailing list. Would anyone else be
interested in this functionality? I figure a proper solution would
appear to the user as a configurable variable that could enable filling
based on image width (this behavior might be undesirable in some
cases). My currently uninformed view is that it should be possible to
query the overlay image width and perform a fill based on that. Any
thoughts, concerns, etc. are appreciated.

Matt

Re: babel default header args as functions

[Matt Huszagh]

Bastien,

Any chance you've had the time to look at this? I know how much time you
put into org-mode, so no rush if you need more time! Let me know if
there's anything I can do on my end.

Thanks!
Matt

Re: [PATCH] babel latex headers and image generation commands

[Matt Huszagh]

Bastien  writes:

> sorry for the delayed answer.

No worries!

> Can you provide a patch against etc/ORG-NEWS announce this?

Attached. Let me know what you think.

Matt

>From 51fb3ef9843ae45884803142f150c5d2f4f4d4c9 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Sat, 24 Oct 2020 09:36:56 -0700
Subject: [PATCH] etc/ORG-NEWS: Describe new behavior of babel LaTeX SVG images

---
 etc/ORG-NEWS | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 7f935bf52..b9b0c1271 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -35,6 +35,28 @@ omit a file description was to omit the header argument entirely,
 which made it difficult/impossible to provide a default value for
 =file-desc=.
 
+*** New behavior for babel LaTeX SVG image files
+
+Org babel now uses a two-stage process for converting latex source
+blocks to SVG image files (when the extension of the output file is
+~.svg~). The first stage in the process converts the latex block into
+a PDF file, which is then converted into an SVG file in the second
+stage. The TeX->PDF part uses the existing infrastructure for
+~org-babel-latex-tex-to-pdf~. The PDF->SVG part uses a command
+specified in a new customization,
+~org-babel-latex-pdf-svg-process~. By default, this uses inkscape for
+conversion, but since it is fully customizable, any other command can
+be used in its place. For instance, dvisvgm might be used here. This
+two-part processing replaces the previous use of htlatex to process
+LaTeX directly to SVG (htlatex is still used for HTML conversion).
+
+Conversion to SVG exposes a number of additional customizations that
+give the user full control over the contents of the latex source
+block. ~org-babel-latex-preamble~, ~org-babel-latex-begin-env~ and
+~org-babel-latex-end-env~ are new customization options added to allow
+the user to specify the preamble and code that preceedes and proceeds
+the contents of the source block.
+
 ** New features
 *** =ob-python= improvements to =:return= header argument 
 
-- 
2.28.0

Re: babel default header args as functions

[Matt Huszagh]

rey-coyrehourcq  writes:

> I'm unfamilliar with patch by mail but i try to apply your patch to my melpa 
> local org 9.4 version used by doom emacs.
> Patch hang on Hunk #3, i attach the .rej file.

You might need to start from a different org commit. I believe
6a182b690f works. Otherwise, you could fix the merge conflicts (this is
described online, e.g., https://stackoverflow.com/a/16968982/5710525).

Finally, a very crude solution would be to manually apply the changes
yourself if you're not too conversant with git. The git solutions are
better, but since the changes in this patch are minimal it wouldn't take
too long.

I will at some point fix the conflicts myself and send a new patch, but
I'd prefer to wait until it gets the green light to merge so that I
don't have to do it multiple times.

Matt

Re: babel default header args as functions

[Matt Huszagh]

rey-coyrehourcq  writes:

> I'm interested by this functionality, do you know if it was merged or i need 
> to apply patch locally ?

Hi SR,

This hasn't been merged yet. I believe it's ready, but we're just
waiting on a maintainer to apply it upstream. If you're able to apply
the patch locally and provide feedback that always helps.

Matt

Re: [PATCH] Omit file description when :file-desc has nil value

[Matt Huszagh]

Kyle Meyer  writes:

> So, with the typo/spurious space change clean-ups, this looks good to
> me.  IIRC from a previous thread, you haven't yet completed the
> copyright paperwork.  Is that the case?

I've made those fixes and attached the updated patch.

I also sent you the paperwork separately (didn't post to the thread
since the PDF is too large).

Thanks
Matt

>From 7452f3e8315be63fa8ae160f6be00963bac898a7 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Tue, 29 Sep 2020 14:11:59 -0700
Subject: [PATCH] lisp/ob-core.el: Allow passing empty vector to :file-desc to
 omit description

* doc/org-manual.org (Type): Document empty vector argument for
file-desc.
* etc/ORG-NEWS (New argument for ~file-desc~ babel header): Add entry
to NEWS.
* lisp/ob-core.el (org-babel--file-desc): Add new function to evaluate
file description value.
(org-babel-execute-src-block): Correctly evaluate file description
when executing src block.
(org-babel-insert-result): Correctly evaluate file description value
when inserting the result of src block execution into the buffer.
* testing/lisp/test-ob.el (test-ob/file-desc-header-argument): Add
test case for new behavior.
---
 doc/org-manual.org  |  8 +---
 etc/ORG-NEWS| 11 +++
 lisp/ob-core.el | 16 +++-
 testing/lisp/test-ob.el | 20 +++-
 4 files changed, 46 insertions(+), 9 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index e7d25b90e..ef2dad9ef 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -17482,10 +17482,12 @@ default behavior is to automatically determine the result type.
   #+end_example
 
   #+cindex: @samp{file-desc}, header argument
-  The =file-desc= header argument defines the description (see
-  [[*Link Format]]) for the link.  If =file-desc= is present but has no value,
+  The =file-desc= header argument defines the description (see [[*Link
+  Format]]) for the link.  If =file-desc= is present but has no value,
   the =file= value is used as the link description.  When this
-  argument is not present, the description is omitted.
+  argument is not present, the description is omitted.  If you want to
+  provide the =file-desc= argument but omit the description, you can
+  provide it with an empty vector (i.e., :file-desc []).
 
   #+cindex: @samp{sep}, header argument
   By default, Org assumes that a table written to a file has
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 5dc68cba4..7f935bf52 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -24,6 +24,17 @@ Earlier, IDs generated using =ts= method had a hard-coded format (i.e. =20200923
 The new option allows user to customise the format.
 Defaults are unchanged.
 
+*** New argument for ~file-desc~ babel header
+
+It is now possible to provide the =file-desc= header argument for a
+babel source block but omit the description by passing an empty vector
+as an argument (i.e., :file-desc []).  This can be useful because
+providing =file-desc= without an argument results in the result of
+=file= being used in the description.  Previously, the only way to
+omit a file description was to omit the header argument entirely,
+which made it difficult/impossible to provide a default value for
+=file-desc=.
+
 ** New features
 *** =ob-python= improvements to =:return= header argument 
 
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 7300f239e..075e3f928 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -646,6 +646,14 @@ a list with the following pattern:
   (replace-regexp-in-string
(org-src-coderef-regexp coderef) "" expand nil nil 1
 
+(defun org-babel--file-desc (params result)
+  "Retrieve file description."
+  (pcase (assq :file-desc params)
+(`nil nil)
+(`(:file-desc) result)
+(`(:file-desc . ,(and (pred stringp) val)) val)
+(`(:file-desc . []) nil)))
+
 ;;;###autoload
 (defun org-babel-execute-src-block ( arg info params)
   "Execute the current source code block.
@@ -749,8 +757,7 @@ block."
 		(let ((*this* (if (not file) result
 (org-babel-result-to-file
  file
- (let ((desc (assq :file-desc params)))
-   (and desc (or (cdr desc) result)))
+ (org-babel--file-desc params result)
 		  (setq result (org-babel-ref-resolve post))
 		  (when file
 			(setq result-params (remove "file" result-params))
@@ -2257,9 +2264,8 @@ INFO may provide the values of these header arguments (in the
 	 (setq result (org-no-properties result))
 	 (when (member "file" result-params)
 	   (setq result (org-babel-result-to-file
-			 result (when (assq :file-desc (nth 2 info))
-  (or (cdr (assq :file-desc (nth 2 info)))
-  result))
+			 result
+			 (org-babel--file-desc (nth 2 info) result)
 	((listp result))
 	(t (setq result (format "%S" result
   (if (and result-params (member "silent" result-params))
diff --git a/testing/lisp/test-ob

Re: [PATCH] Omit file description when :file-desc has nil value

[Matt Huszagh]

Kyle Meyer  writes:

> I'd be happy for you to take what I sent and work it into a proper
> patch.  Here are some other loose ends in addition to the manual update
> you mentioned:
>
>   * a NEWS entry for 9.5
>
>   * decide whether (:file-desc . []) should be handled explicitly rather
> than the current "any value that org-babel-read doesn't map to nil
> or a string"
>
>   * check that there's a test case for each :file-desc scenario

Let me know if I missed anything, or any other issues. I've decided to
handle the empty vector case explicitly. I think this behavior is
clearer.

Thanks,
Matt

>From 749fd5ade6b65f9d07e87b4af44ebb1afef2bee6 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Tue, 29 Sep 2020 14:11:59 -0700
Subject: [PATCH] list/ob-core.el: Allow passing empty vector to :file-desc to
 omit description

* doc/org-manual.org (Type): Document empty vector argument for
file-desc.
* etc/ORG-NEWS (New argument for ~file-desc~ babel header): Add entry
to NEWS.
* lisp/ob-core.el (org-babel--file-desc): Add new function to evaluate
file description value.
(org-babel-execute-src-block): Correctly evaluate file description
when executing src block.
(org-babel-insert-result): Correctly evaluate file description value
when inserting the result of src block execution into the buffer.
* testing/lisp/test-ob.el (test-ob/file-desc-header-argument): Add
test case for new behavior.
---
 doc/org-manual.org  |  8 +---
 etc/ORG-NEWS| 13 -
 lisp/ob-core.el | 16 +++-
 testing/lisp/test-ob.el | 20 +++-
 4 files changed, 47 insertions(+), 10 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index e7d25b90e..a790f3225 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -17482,10 +17482,12 @@ default behavior is to automatically determine the result type.
   #+end_example
 
   #+cindex: @samp{file-desc}, header argument
-  The =file-desc= header argument defines the description (see
-  [[*Link Format]]) for the link.  If =file-desc= is present but has no value,
+  The =file-desc= header argument defines the description (see [[*Link
+  Format]]) for the link.  If =file-desc= is present but has no value,
   the =file= value is used as the link description.  When this
-  argument is not present, the description is omitted.
+  argument is not present, the description is omitted.  If you want to
+  provide the =file-disc= argument but omit the description, you can
+  provide it with an empty vector (i.e., :file-desc []).
 
   #+cindex: @samp{sep}, header argument
   By default, Org assumes that a table written to a file has
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 5dc68cba4..19f6af288 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -24,8 +24,19 @@ Earlier, IDs generated using =ts= method had a hard-coded format (i.e. =20200923
 The new option allows user to customise the format.
 Defaults are unchanged.
 
+*** New argument for ~file-desc~ babel header
+
+It is now possible to provide the =file-desc= header argument for a
+babel source block but omit the description by passing an empty vector
+as an argument (i.e., :file-desc []).  This can be useful because
+providing =file-desc= without an argument results in the result of
+=file= being used in the description.  Previously, the only way to
+omit a file description was to omit the header argument entirely,
+which made it difficult/impossible to provide a default value for
+=file-desc=.
+
 ** New features
-*** =ob-python= improvements to =:return= header argument 
+*** =ob-python= improvements to =:return= header argument
 
 The =:return= header argument in =ob-python= now works for session
 blocks as well as non-session blocks.  Also, it now works with the
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 7300f239e..075e3f928 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -646,6 +646,14 @@ a list with the following pattern:
   (replace-regexp-in-string
(org-src-coderef-regexp coderef) "" expand nil nil 1
 
+(defun org-babel--file-desc (params result)
+  "Retrieve file description."
+  (pcase (assq :file-desc params)
+(`nil nil)
+(`(:file-desc) result)
+(`(:file-desc . ,(and (pred stringp) val)) val)
+(`(:file-desc . []) nil)))
+
 ;;;###autoload
 (defun org-babel-execute-src-block ( arg info params)
   "Execute the current source code block.
@@ -749,8 +757,7 @@ block."
 		(let ((*this* (if (not file) result
 (org-babel-result-to-file
  file
- (let ((desc (assq :file-desc params)))
-   (and desc (or (cdr desc) result)))
+ (org-babel--file-desc params result)
 		  (setq result (org-babel-ref-resolve post))
 		  (when file
 			(setq result-params (remove "file" result-params))
@@ -2257,9 +2264,8 @@ INFO may provide the values of these header arguments (in the
 	 (setq result (org-no-p

Re: [PATCH] Omit file description when :file-desc has nil value

[Matt Huszagh]

Kyle Meyer  writes:

> But it's not a direct comparison against that use case and the use case
> you want to support.  The potential breakage of existing documents is a
> big factor to go against.

Yep, I agree. I think my phrasing could have just been better. I meant
to include the breakage as a factor against.

> Unfortunately, such a kludge is how I'd suggest to move forward.
> Perhaps an empty string, or perhaps any value (e.g., ":file-desc []")
> that org-babel-read won't treat as a string or nil (the two cases that
> mean something right now).  The rough patch below is an example of the
> latter.

I like this solution better than mine. I guess it's still a bit of a
hack, but it doesn't seem to be one that could break a use case, whereas
the empty string could conceivably be intended, though I'm still not
sure why.

> I'm not sure I get this.  What's next down the road in this scenario?
> With something like the above kludge, haven't we exhausted the cases for
> :file-desc?

Yes I think you're right. I was referring to my solution of an empty
string, which I didn't see a personal use for, but felt it might be a
valid use case for someone else. I really can't think of any reason the
empty vector would otherwise be valid.

> diff --git a/lisp/ob-core.el b/lisp/ob-core.el
> index 7300f239e..4483585a1 100644
> --- a/lisp/ob-core.el
> +++ b/lisp/ob-core.el
> @@ -646,6 +646,13 @@ (defun org-babel--expand-body (info)
>(replace-regexp-in-string
> (org-src-coderef-regexp coderef) "" expand nil nil 1
>  
> +(defun org-babel--file-desc (params result)
> +  (pcase (assq :file-desc params)
> +(`nil nil)
> +(`(:file-desc) result)
> +(`(:file-desc . ,(and (pred stringp) val)) val)
> +(_ nil)))
> +
>  ;;;###autoload
>  (defun org-babel-execute-src-block ( arg info params)
>"Execute the current source code block.
> @@ -749,8 +756,7 @@ (defun org-babel-execute-src-block ( arg info 
> params)
>   (let ((*this* (if (not file) result
>   (org-babel-result-to-file
>file
> -  (let ((desc (assq :file-desc params)))
> -(and desc (or (cdr desc) result)))
> +  (org-babel--file-desc params result)
> (setq result (org-babel-ref-resolve post))
> (when file
>   (setq result-params (remove "file" result-params))
> @@ -2257,9 +2263,8 @@ (defun org-babel-insert-result (result  
> result-params info hash lang)
>(setq result (org-no-properties result))
>(when (member "file" result-params)
>  (setq result (org-babel-result-to-file
> -  result (when (assq :file-desc (nth 2 info))
> -   (or (cdr (assq :file-desc (nth 2 info)))
> -   result))
> +  result
> +  (org-babel--file-desc (nth 2 info) result)
>   ((listp result))
>   (t (setq result (format "%S" result
>(if (and result-params (member "silent" result-params))
> diff --git a/testing/lisp/test-ob.el b/testing/lisp/test-ob.el
> index 648e9c115..e7a292de3 100644
> --- a/testing/lisp/test-ob.el
> +++ b/testing/lisp/test-ob.el
> @@ -1084,7 +1084,16 @@ (ert-deftest test-ob/file-desc-header-argument ()
>  (org-babel-execute-src-block)
>  (goto-char (point-min))
>  (should (search-forward "[[file:foo][bar]]" nil t))
> -(should (search-forward "[[file:foo][foo]]" nil t
> +(should (search-forward "[[file:foo][foo]]" nil t)))
> +  (should (string-match-p
> +(regexp-quote "[[file:foo]]")
> +(org-test-with-temp-text "
> +#+begin_src emacs-lisp :results file :file-desc []
> +  \"foo\"
> +#+end_src"
> +  (org-babel-next-src-block)
> +  (org-babel-execute-src-block)
> +  (buffer-substring-no-properties (point-min) (point-max))
>  
>  (ert-deftest test-ob/result-file-link-type-header-argument ()
>"Ensure that the result is a link to a file.

This patch looks good. I've tested it and it works well for me. Thanks
for coming up with a good solution! I think the one thing still missing
is some documentation in the info manual. Something along the lines of

 The ‘file-desc’ header argument defines the description (see *note
 Link Format::) for the link.  If ‘file-desc’ has no value, Org uses
 the generated file name for both the “link” and “description” parts
 of the link. If you want to omit the description (i.e., [[link]]),
 you can either omit the ‘file-desc’ header argument or provide it
 with an empty vector (i.e., :file-desc []).

Feel free to add this (or something else) to your patch. Or, if you'd
prefer that I created a patch for it, let me know.

Matt

Re: official orgmode parser

[Matt Huszagh]

"Gerry Agbobada"  writes:

> I'm currently toying with the idea of trying a tree-sitter parser for Org. 
> The very static nature of a shared object parser (knowing TODO keywords are 
> pretty dynamic for example) is a challenge I'm not sure to overcome ; to be 
> honest even without that I can't say I'll manage to do it.

A tree-sitter parser for org would be great! Please keep this list
posted on any developments you make on this front. I made some minimal
attempts at this a while back, but didn't get very far.

Matt

Re: basic org questions

[Matt Huszagh]

Emanuel Berg  writes:

> Yeah, but in LaTeX being left aligned is not some
> property of the table, everything is left-aligned,
> and if you want it otherwise, you put between
> \begin{center} and \end{center} ...

That page also describes how to set float parameters, which the table
can be within.

What is the latex your export is generating? What does your input org
file look like, and what sort of configuration parameters have you set?

Matt

Re: basic org questions

[Matt Huszagh]

Emanuel Berg via "General discussions about Org-mode."
 writes:

> Yes, after export to PDF, they are centered. they =
> the whole table items.

I think this link
(https://orgmode.org/manual/Tables-in-LaTeX-export.html) is the relevant
part of the documentation. I haven't used this though, so I can't help
you further.

In general, it would be much easier to help you if you specify these
things in the original question. The fact that you're asking about LaTeX
export processed to PDF is critical information. Also, please consult
the documentation before asking questions. There are gaps, but generally
it's well-written and a lot of people have put a lot of time into it.

Matt

Re: basic org questions

[Matt Huszagh]

Emanuel Berg via "General discussions about Org-mode."
 writes:

> Heh...? What's strange with that?
>
> Anyway that's what I mean.

I guess I'm just not sure what you mean. I was under the impression that
tables were always left-aligned and table alignment in the manual refers
to column alignment
(https://orgmode.org/manual/Column-Width-and-Alignment.html). But it's
certainly possible there's an org-mode feature I'm not aware of. Were
you referring to the tables being centered after export (I see from
another part of this chain that that's what you were referring to in the
3rd question) when you wanted them left-aligned? If that is the case, it
would really help if you specify that in your original
question. Appearance in an org-mode buffer and appearance in the PDF,
HTML, etc. after export are very different things with very different
configuration.

Matt

Re: basic org questions

[Matt Huszagh]

Emanuel Berg via "General discussions about Org-mode."
 writes:

> 2) How do I have tables not appear centered
> by default, but left-aligned?

Do you mean the alignment of columns, or the table itself? You can
control column alignment with ``, ``, ``, e.g.,

|  title   | description  |other desc |
|--+--+---|
|   |   ||
|some title| some description.|right desc |
| some other title | some longer description. | longer right desc |

If you mean the table itself, that would be very strange.

I'm not too sure about your other questions.

Matt

Re: [PATCH] Omit file description when :file-desc has nil value

[Matt Huszagh]

> Kyle Meyer  writes:
>
>> I also don't find the current behavior particularly intuitive.  (I'm
>> also not really a babel user, so my opinion probably shouldn't count for
>> much.)  If we were adding it today, I think what you describe would be
>> better, but, as you mention, breakage also now also weighs against
>> making a change here.
>>
>> In any case, I'd suggest raising the discussion on the list after the
>> 9.4 release.

Hello, just following up on this since 9.4 has been released. Thoughts?

Matt

Re: [PATCH] Omit file description when :file-desc has nil value

[Matt Huszagh]

Kyle Meyer  writes:

> I also don't find the current behavior particularly intuitive.  (I'm
> also not really a babel user, so my opinion probably shouldn't count for
> much.)  If we were adding it today, I think what you describe would be
> better, but, as you mention, breakage also now also weighs against
> making a change here.
>
> In any case, I'd suggest raising the discussion on the list after the
> 9.4 release.

Ok, I'll follow up on this then.

>>> Right, to reflect the current behavior established as a result of the
>>> above thread, I think that should be reworded to distinguish between an
>>> absent :file-desc header and one with no argument.  Sorry for not
>>> catching that when reviewing your initial patch.
>>
>> No worries, and I agree the documentation should be updated. I'm happy
>> to provide the patch myself, but I'd like to talk through whether the
>> current implementation is the correct one before I do.
>
> Thanks.  To avoid any confusion coming from this description making it
> into the 9.4 release, I've updated it in 4b2123fb7.

Thanks for fixing that Kyle.

Matt

Re: [PATCH] babel latex headers and image generation commands

[Matt Huszagh]

Bastien  writes:

> Prefer
>
>   * lisp/ob-latex.el (org-babel-latex-preamble): New option for LaTeX
>   preamble customization.
>
> "New option" is quite standard, an "option" being a customizable
> variable.  In this case, "New option" would probably be enough, given
> the name of the option is quite self-explanatory.  Also, some find it
> pedantic, but "LaTeX" is the correct spelling in a changelog I guess.

Fixed in new patch (attached).

> The first line of the docstring should contain a sentence, so you'd
> need to have a new paragraph after "runtime to the LaTeX preamble."

Also fixed. Making the first line a full sentence means that some lines
are a little longer than 80 characters. Is this acceptable?

> What for users who don't have inkscape?  

This is just a default, but I could use a dvisvgm command as the default
instead? Either way, converting a PDF to SVG will require an executable
outside Emacs, but I guess dvisvgm is more likely to be installed for
people using a texlive installation. My personal preference for inkscape
is because it should handle all PDF inputs, whereas there are some cases
where dvisvgm may fail (see
https://github.com/mgieseki/dvisvgm/issues/139) due to changes in
ghostscript. Still, dvisvgm generally does a very good job with PDF
inputs. Let me know your thoughts, I'd be happy to set the default to a
dvisvgm command instead.

Matt

>From 1ff86f2267b653dff225837ccf13ebf417f7ed03 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Fri, 28 Aug 2020 22:26:05 -0700
Subject: [PATCH] ob-latex.el: Make latex to svg compilation very customizable

* lisp/ob-latex.el (org-babel-latex-preamble): New option for LaTeX
preamble customization.
(org-babel-latex-begin-env): New option for LaTeX document environment
begin customization.
(org-babel-latex-end-env): New option for LaTeX document environment
end customization.
(org-babel-latex-pdf-svg-process): New option for converting a pdf to
svg.
(org-babel-execute:latex): Add specific case for svg generation from
LaTeX block.
---
 lisp/ob-latex.el | 59 +---
 1 file changed, 56 insertions(+), 3 deletions(-)

diff --git a/lisp/ob-latex.el b/lisp/ob-latex.el
index 4b343dd14..6a4f7a6ba 100644
--- a/lisp/ob-latex.el
+++ b/lisp/ob-latex.el
@@ -66,7 +66,46 @@
   "LaTeX-specific header arguments.")
 
 (defcustom org-babel-latex-htlatex "htlatex"
-  "The htlatex command to enable conversion of latex to SVG or HTML."
+  "The htlatex command to enable conversion of LaTeX to SVG or HTML."
+  :group 'org-babel
+  :type 'string)
+
+(defcustom org-babel-latex-preamble
+  (lambda (_)
+"\\documentclass[preview]{standalone}
+\\def\\pgfsysdriver{pgfsys-tex4ht.def}
+")
+  "Closure which evaluates at runtime to the LaTeX preamble.
+
+It takes 1 argument which is the parameters of the source block."
+  :group 'org-babel
+  :type 'function)
+
+(defcustom org-babel-latex-begin-env
+  (lambda (_)
+"\\begin{document}")
+  "Closure which evaluates at runtime to the begin part of the document environment.
+
+It takes 1 argument which is the parameters of the source block.
+This allows adding additional code that will be ignored when
+exporting the literal LaTeX source."
+  :group 'org-babel
+  :type 'function)
+
+(defcustom org-babel-latex-end-env
+  (lambda (_)
+"\\end{document}")
+  "Closure which evaluates at runtime to the end part of the document environment.
+
+It takes 1 argument which is the parameters of the source block.
+This allows adding additional code that will be ignored when
+exporting the literal LaTeX source."
+  :group 'org-babel
+  :type 'function)
+
+(defcustom org-babel-latex-pdf-svg-process
+  "inkscape --pdf-poppler %f -T -l -o %O"
+  "Command used to convert a PDF file to an SVG file when executing a latex source block."
   :group 'org-babel
   :type 'string)
 
@@ -114,12 +153,26 @@ This function is called by `org-babel-execute-src-block'."
 			 (mapconcat #'identity headers "\n"
 	   (org-create-formula-image
 body out-file org-format-latex-options in-buffer)))
+	 ((string= "svg" extension)
+	  (with-temp-file tex-file
+		 (insert (concat (funcall org-babel-latex-preamble params)
+			 (mapconcat #'identity headers "\n")
+			 (funcall org-babel-latex-begin-env params)
+			 body
+			 (funcall org-babel-latex-end-env params
+	  (let ((tmp-pdf (org-babel-latex-tex-to-pdf tex-file)))
+  (let* ((log-buf (get-buffer-create "*Org Babel LaTeX Output*"))
+ (err-msg "org babel latex failed")
+ (img-out (org-compile-file
+	   tmp-pdf
+   (list org-babel-latex-pdf-svg-process)
+  

Re: babel default header args as functions

[Matt Huszagh]

Bastien  writes:

> Also, if we integrate the change, `eval-default-headers' would be
> better named `org-babel-eval-default-headers'.

I've changed the function name to `org-babel-eval-headers'. The reason
for dropping "default" is that this function is now used as part of
`org-babel-merge-params' on all headers, not just defaults in order to
avoid evaluating closures until we know they are needed. See
https://lists.gnu.org/archive/html/emacs-orgmode/2020-09/msg00464.html
for the new patch and more info.

Matt

Re: babel default header args as functions

[Matt Huszagh]

Tom Gillespie  writes:

> [...] I have a number of use
> cases that I can imagine would benefit greatly from being able to
> define a :header-args: :header (lambda () "yay!") property as a
> closure (and actually I assumed that it would just work that way if I
> tried to do it, clearly not though). I can't tell for sure if the
> patch enables this behavior though or whether I would still get a
> Wrong type argument error.

This should work. Do you have reason for believing it might not?

For example, set:

(setq org-babel-default-header-args:bash
  `((:var . (lambda ()
  "a='yay'"

Then in file.org:
#+begin_src bash :results output
echo $a
#+end_src

Executing this will yield:

#+RESULTS:
: yay

> [...] Looking
> at the patch it seems that it preserves the behavior of performing the
> evaluation of the closures at the source block, but I'm not 100% sure.

I'm not sure I completely understand what you mean here. However, the
closures are evaluated when point is at the source block, during the
source block evaluation, not when the default headers are declared. This
allows the closures to use context-dependent functionality (e.g. you can
call `org-element-at-point' inside the closure and retrieve whatever
information you want). Does this address your concern? Please clarify if
I've missed your point.

> If the default header closures are being evaluated before checking
> whether they have been superseded by the headers on a block then that
> is incorrect and they should not be evaluated until it is clear that
> they are the value of the header for that block and have not been
> superseded.

I've fixed my patch (attached) so that now closures are only evaluated
when they are used as part of the final set of headers.

>From 4a461a90ec4f3c5f9634b687a6685ea3ba74f168 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Fri, 28 Aug 2020 11:05:59 -0700
Subject: [PATCH] ob-core.el: Add ability to use closures as default header
 arguments

* lisp/ob-core.el (org-babel-default-header-args): Document ability to
use closures.
(org-babel-eval-headers): New function to generate header arguments,
which adds the ability to evaluate closures during source block
execution or export.
(org-babel-merge-params): Only evaluate closures when we have our
final list of headers.
---
 lisp/ob-core.el | 60 -
 1 file changed, 55 insertions(+), 5 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 578622232..bef34d7c0 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -473,7 +473,35 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'."
 (defvar org-babel-default-header-args
   '((:session . "none") (:results . "replace") (:exports . "code")
 (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no"))
-  "Default arguments to use when evaluating a source block.")
+  "Default arguments to use when evaluating a source block.
+
+This is a list in which each element is an alist.  Each key
+corresponds to a header argument, and each value to that header's
+value.  The value can either be a string or a closure that
+evaluates to a string.  The closure is evaluated when the source
+block is being evaluated (e.g. during execution or export), with
+point at the source block.  It is not possible to use an
+arbitrary function symbol (e.g. 'some-func), since org uses
+lexical binding.  To achieve the same functionality, call the
+function within a closure (e.g. (lambda () (some-func))).
+
+To understand how closures can be used as default header
+arguments, imagine you'd like to set the file name output of a
+latex source block to a sha1 of its contents.  We could achieve
+this with:
+
+(defun org-src-sha ()
+  (let ((elem (org-element-at-point)))
+(concat (sha1 (org-element-property :value elem)) \".svg\")))
+
+(setq org-babel-default-header-args:latex
+  `((:results . \"file link replace\")
+(:file . (lambda () (org-src-sha)
+
+Because the closure is evaluated with point at the source block,
+the call to `org-element-at-point' above will always retrieve
+information about the current source block.")
+
 (put 'org-babel-default-header-args 'safe-local-variable
  (org-babel-header-args-safe-fn org-babel-safe-header-args))
 
@@ -584,6 +612,19 @@ the outer-most code block.")
 
 (defvar *this*)
 
+(defun org-babel-eval-headers (headers)
+  "Compute header list set with HEADERS.
+
+Evaluate all header arguments set to functions prior to returning
+the list of header arguments."
+  (let ((lst nil))
+(dolist (elem headers)
+  (if (and (cdr elem)
+	   (functionp (cdr elem)))
+  (push `(,(car elem) . ,(funcall (cdr elem))) lst)
+(push elem lst)))
+lst))
+
 (defun org-babel-get-src-block-info ( l

Re: [PATCH] Omit file description when :file-desc has nil value

[Matt Huszagh]

Matt Huszagh  writes:

> This patch omits a file description when :file-desc has a nil
> value.

I've modified the patch to yield the same effect when executing a source
block.

Matt

>From 24d156e421973b5a97f1c797d48f1daa95348898 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Wed, 2 Sep 2020 23:06:10 -0700
Subject: [PATCH] lisp/ob-core.el: Omit file description when :file-desc has
 nil value

* lisp/ob-core.el (org-babel-insert-result): Omit file description
when :file-desc value evaluates to nil.
(org-babel-execute-src-block): Perform the same functionality when
executing a src block.

The previous implementation makes it impossible to provide a default
:file-desc and in some cases override it to omit the description.
---
 lisp/ob-core.el | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 578622232..02c0a153c 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -750,7 +750,8 @@ block."
 (org-babel-result-to-file
  file
  (let ((desc (assq :file-desc params)))
-   (and desc (or (cdr desc) result)))
+   (and (and desc (cdr desc))
+	(cdr desc)))
 		  (setq result (org-babel-ref-resolve post))
 		  (when file
 			(setq result-params (remove "file" result-params))
@@ -2257,9 +2258,9 @@ INFO may provide the values of these header arguments (in the
 	 (setq result (org-no-properties result))
 	 (when (member "file" result-params)
 	   (setq result (org-babel-result-to-file
-			 result (when (assq :file-desc (nth 2 info))
-  (or (cdr (assq :file-desc (nth 2 info)))
-  result))
+			 result (when (and (assq :file-desc (nth 2 info))
+	   (cdr (assq :file-desc (nth 2 info
+  (cdr (assq :file-desc (nth 2 info
 	((listp result))
 	(t (setq result (format "%S" result
   (if (and result-params (member "silent" result-params))
-- 
2.28.0

[PATCH] Omit file description when :file-desc has nil value

[Matt Huszagh]

Hello,

This patch omits a file description when :file-desc has a nil
value. Previously, the following src block

#+BEGIN_SRC asymptote :results value file :file circle.pdf :file-desc 
:output-dir img/
  size(2cm);
  draw(unitcircle);
#+END_SRC

would yield

#+RESULTS:
[[file:img/circle.pdf][circle.pdf]]

This makes it impossible (I think) to provide :file-desc with a default
value and prevent the description in some cases. This patch would cause
the same code block to execute to

#+RESULTS:
[[file:img/circle.pdf]]

I feel I may be missing something in regard to why this previously had
the functionality it did. Is there a use case I've missed? To me, the
documentation seems to indicate that my patch is the desired behavior:

   The =file-desc= header argument defines the description (see
   [[*Link Format]]) for the link.  If =file-desc= has no value, the
   "description" part of the link will be omitted.

Full disclaimer: I wrote this section of the documentation as part of
this patch:

https://lists.gnu.org/archive/html/emacs-orgmode/2020-07/msg00320.html

Thanks
Matt

>From edcfa85add6ac71a1e13b7731779ccf4a8e12868 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Wed, 2 Sep 2020 23:06:10 -0700
Subject: [PATCH] lisp/ob-core.el: Omit file description when :file-desc has
 nil value

* lisp/ob-core.el (org-babel-insert-result): Omit file description
when :file-desc value evaluates to nil.

The previous implementation makes it impossible to provide a default
:file-desc and in some cases override it to omit the description.
---
 lisp/ob-core.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 578622232..55165ebc5 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2257,9 +2257,9 @@ INFO may provide the values of these header arguments (in the
 	 (setq result (org-no-properties result))
 	 (when (member "file" result-params)
 	   (setq result (org-babel-result-to-file
-			 result (when (assq :file-desc (nth 2 info))
-  (or (cdr (assq :file-desc (nth 2 info)))
-  result))
+			 result (when (and (assq :file-desc (nth 2 info))
+	   (cdr (assq :file-desc (nth 2 info
+  (cdr (assq :file-desc (nth 2 info
 	((listp result))
 	(t (setq result (format "%S" result
   (if (and result-params (member "silent" result-params))
-- 
2.28.0

Re: [PATCH] babel latex headers and image generation commands

[Matt Huszagh]

Matt Huszagh  writes:

> I've added a few changes to the patch that additionally allow custom the
> begin and end document environments. The purpose here is to allow latex
> code within the document environment that is ignored by the body
> export. For instance, I can set the page color with
> {\color{some-color}...} and this doesn't mess up latex exports.

I've fixed a minor bug with the previous patch.

>From 1bcd1d28dde6625d0c648c92243260b46433e1eb Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Fri, 28 Aug 2020 22:26:05 -0700
Subject: [PATCH] ob-latex.el: Make latex to svg compilation very customizable

* lisp/ob-latex.el (org-babel-latex-preamble): Add latex preamble
customization.
(org-babel-latex-begin-env): Add latex document environment begin
customization.
(org-babel-latex-end-env): Add latex document environment end
customization.
(org-babel-latex-pdf-svg-process): Add customization for converting a
pdf to svg.
(org-babel-execute:latex): Add specific case for svg generation from
latex block.
---
 lisp/ob-latex.el | 57 ++--
 1 file changed, 55 insertions(+), 2 deletions(-)

diff --git a/lisp/ob-latex.el b/lisp/ob-latex.el
index 4b343dd14..991873e2e 100644
--- a/lisp/ob-latex.el
+++ b/lisp/ob-latex.el
@@ -70,6 +70,45 @@
   :group 'org-babel
   :type 'string)
 
+(defcustom org-babel-latex-preamble
+  (lambda (_)
+"\\documentclass[preview]{standalone}
+\\def\\pgfsysdriver{pgfsys-tex4ht.def}
+")
+  "Closure which evaluates at runtime to the latex preamble.  It
+takes 1 argument which is the parameters of the source block."
+  :group 'org-babel
+  :type 'function)
+
+(defcustom org-babel-latex-begin-env
+  (lambda (_)
+"\\begin{document}")
+  "Closure which evaluates at runtime to the begin part of the
+document environment.  It takes 1 argument which is the
+parameters of the source block.  This allows adding additional
+code that will be ignored when exporting the literal latex
+source."
+  :group 'org-babel
+  :type 'function)
+
+(defcustom org-babel-latex-end-env
+  (lambda (_)
+"\\end{document}")
+  "Closure which evaluates at runtime to the end part of the
+document environment.  It takes 1 argument which is the
+parameters of the source block.  This allows adding additional
+code that will be ignored when exporting the literal latex
+source."
+  :group 'org-babel
+  :type 'function)
+
+(defcustom org-babel-latex-pdf-svg-process
+  "inkscape --pdf-poppler %f -T -l -o %O"
+  "Command used to convert a PDF file to an SVG file when
+executing a latex source block."
+  :group 'org-babel
+  :type 'string)
+
 (defcustom org-babel-latex-htlatex-packages
   '("[usenames]{color}" "{tikz}" "{color}" "{listings}" "{amsmath}")
   "Packages to use for htlatex export."
@@ -114,12 +153,26 @@ This function is called by `org-babel-execute-src-block'."
 			 (mapconcat #'identity headers "\n"
 	   (org-create-formula-image
 body out-file org-format-latex-options in-buffer)))
+	 ((string= "svg" extension)
+	  (with-temp-file tex-file
+		 (insert (concat (funcall org-babel-latex-preamble params)
+			 (mapconcat #'identity headers "\n")
+			 (funcall org-babel-latex-begin-env params)
+			 body
+			 (funcall org-babel-latex-end-env params
+	  (let ((tmp-pdf (org-babel-latex-tex-to-pdf tex-file)))
+  (let* ((log-buf (get-buffer-create "*Org Babel LaTeX Output*"))
+ (err-msg "org babel latex failed")
+ (img-out (org-compile-file
+	   tmp-pdf
+   (list org-babel-latex-pdf-svg-process)
+   extension err-msg log-buf)))
+(shell-command (format "mv %s %s" img-out out-file)
  ((string-suffix-p ".tikz" out-file)
 	  (when (file-exists-p out-file) (delete-file out-file))
 	  (with-temp-file out-file
 	(insert body)))
-	 ((and (or (string= "svg" extension)
-		   (string= "html" extension))
+	 ((and (string= "html" extension)
 	   (executable-find org-babel-latex-htlatex))
 	  ;; TODO: this is a very different way of generating the
 	  ;; frame latex document than in the pdf case.  Ideally, both
-- 
2.28.0

Re: [PATCH] babel latex headers and image generation commands

[Matt Huszagh]

Matt Huszagh  writes:

> Ok, I've finally gotten around to taking a crack at this. The patch is
> attached. Basically, it allows a lot more control when converting a
> latex source block into an svg image file.

I've added a few changes to the patch that additionally allow custom the
begin and end document environments. The purpose here is to allow latex
code within the document environment that is ignored by the body
export. For instance, I can set the page color with
{\color{some-color}...} and this doesn't mess up latex exports.

Please comment with any questions/concerns/thoughts.

Thanks!
>From 6dbd5ae840f02333f83d40a9c27be06968279563 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Fri, 28 Aug 2020 22:26:05 -0700
Subject: [PATCH] ob-latex.el: Make latex to svg compilation very customizable

* lisp/ob-latex.el (org-babel-latex-preamble): Add latex preamble
customization.
(org-babel-latex-begin-env): Add latex document environment begin
customization.
(org-babel-latex-end-env): Add latex document environment end
customization.
(org-babel-latex-pdf-svg-process): Add customization for converting a
pdf to svg.
(org-babel-execute:latex): Add specific case for svg generation from
latex block.
---
 lisp/ob-latex.el | 57 ++--
 1 file changed, 55 insertions(+), 2 deletions(-)

diff --git a/lisp/ob-latex.el b/lisp/ob-latex.el
index 4b343dd14..359179476 100644
--- a/lisp/ob-latex.el
+++ b/lisp/ob-latex.el
@@ -70,6 +70,45 @@
   :group 'org-babel
   :type 'string)
 
+(defcustom org-babel-latex-preamble
+  (lambda (_)
+"\\documentclass[preview]{standalone}
+\\def\\pgfsysdriver{pgfsys-tex4ht.def}
+")
+  "Closure which evaluates at runtime to the latex preamble.  It
+takes 1 argument which is the parameters of the source block."
+  :group 'org-babel
+  :type 'function)
+
+(defcustom org-babel-latex-begin-env
+  (lambda (_)
+"\\begin{document}")
+  "Closure which evaluates at runtime to the begin part of the
+document environment.  It takes 1 argument which is the
+parameters of the source block.  This allows adding additional
+code that will be ignored when exporting the literal latex
+source."
+  :group 'org-babel
+  :type 'function)
+
+(defcustom org-babel-latex-end-env
+  (lambda (_)
+"\\end{document}")
+  "Closure which evaluates at runtime to the end part of the
+document environment.  It takes 1 argument which is the
+parameters of the source block.  This allows adding additional
+code that will be ignored when exporting the literal latex
+source."
+  :group 'org-babel
+  :type 'function)
+
+(defcustom org-babel-latex-pdf-svg-process
+  "inkscape --pdf-poppler %f -T -l -o %O"
+  "Command used to convert a PDF file to an SVG file when
+executing a latex source block."
+  :group 'org-babel
+  :type 'string)
+
 (defcustom org-babel-latex-htlatex-packages
   '("[usenames]{color}" "{tikz}" "{color}" "{listings}" "{amsmath}")
   "Packages to use for htlatex export."
@@ -114,12 +153,26 @@ This function is called by `org-babel-execute-src-block'."
 			 (mapconcat #'identity headers "\n"
 	   (org-create-formula-image
 body out-file org-format-latex-options in-buffer)))
+	 ((string= "svg" extension)
+	  (with-temp-file tex-file
+		 (insert (concat (funcall org-babel-latex-preamble params)
+			 (mapconcat #'identity headers "\n")
+			 (funcall org-babel-latex-begin-env)
+			 body
+			 (funcall org-babel-latex-end-env
+	  (let ((tmp-pdf (org-babel-latex-tex-to-pdf tex-file)))
+  (let* ((log-buf (get-buffer-create "*Org Babel LaTeX Output*"))
+ (err-msg "org babel latex failed")
+ (img-out (org-compile-file
+	   tmp-pdf
+   (list org-babel-latex-pdf-svg-process)
+   extension err-msg log-buf)))
+(shell-command (format "mv %s %s" img-out out-file)
  ((string-suffix-p ".tikz" out-file)
 	  (when (file-exists-p out-file) (delete-file out-file))
 	  (with-temp-file out-file
 	(insert body)))
-	 ((and (or (string= "svg" extension)
-		   (string= "html" extension))
+	 ((and (string= "html" extension)
 	   (executable-find org-babel-latex-htlatex))
 	  ;; TODO: this is a very different way of generating the
 	  ;; frame latex document than in the pdf case.  Ideally, both
-- 
2.28.0

Re: babel default header args as functions

[Matt Huszagh]

Matt Huszagh  writes:

> I've generated a patch for this. Please let me know your thoughts. I
> believe this adds valuable flexibility to default header
> arguments.

I've added an additional fix that makes this work during export too.

>From aec4e905d5d72f9a124adfde877835a783bd637b Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Fri, 28 Aug 2020 11:05:59 -0700
Subject: [PATCH] ob-core.el: Add ability to use closures as default header
 arguments

* lisp/ob-core.el (org-babel-default-header-args): Document ability to
use functions.
(eval-default-headers): New function to generate default header
arguments, which adds the ability to evaluate function arguments at
runtime.
(org-babel-get-src-block-info): Use new header argument evaluate
function when retreiving src block info.

* lisp/ob-exp.el (org-babel-exp-src-block): Must use new
eval-default-headers when exporting as well.

The closures are evaluated at runtime.
---
 lisp/ob-core.el | 32 ++--
 lisp/ob-exp.el  |  2 +-
 2 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 578622232..4a22f17e7 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -473,7 +473,23 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'."
 (defvar org-babel-default-header-args
   '((:session . "none") (:results . "replace") (:exports . "code")
 (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no"))
-  "Default arguments to use when evaluating a source block.")
+  "Default arguments to use when evaluating a source block.
+
+This is a list in which each element is an alist.  Each key
+corresponds to a header argument, and each value to that header's
+value.  The value can either be a string or a closure that
+evaluates to a string at runtime.  For instance, imagine you'd
+like to set the file name output of a latex source block to a
+sha1 of its contents.  We could achieve this with:
+
+(defun org-src-sha ()
+  (let ((elem (org-element-at-point)))
+(concat (sha1 (org-element-property :value elem)) \".svg\")))
+
+(setq org-babel-default-header-args:latex
+  `((:results . \"file link replace\")
+(:file . (lambda () (org-src-sha)")
+
 (put 'org-babel-default-header-args 'safe-local-variable
  (org-babel-header-args-safe-fn org-babel-safe-header-args))
 
@@ -584,6 +600,18 @@ the outer-most code block.")
 
 (defvar *this*)
 
+(defun eval-default-headers (headers)
+  "Compute default header list set with HEADERS.
+
+  Evaluate all default header arguments set to functions prior to
+  returning the list of header arguments."
+  (let ((lst nil))
+(dolist (elem (eval headers t))
+  (if (listp (cdr elem))
+  (push `(,(car elem) . ,(funcall (cdr elem))) lst)
+(push elem lst)))
+lst))
+
 (defun org-babel-get-src-block-info ( light datum)
   "Extract information from a source block or inline source block.
 
@@ -615,7 +643,7 @@ a list with the following pattern:
 	   (apply #'org-babel-merge-params
 		  (if inline org-babel-default-inline-header-args
 			org-babel-default-header-args)
-		  (and (boundp lang-headers) (eval lang-headers t))
+		  (and (boundp lang-headers) (eval-default-headers lang-headers))
 		  (append
 		   ;; If DATUM is provided, make sure we get node
 		   ;; properties applicable to its location within
diff --git a/lisp/ob-exp.el b/lisp/ob-exp.el
index 34caf9546..13277f64f 100644
--- a/lisp/ob-exp.el
+++ b/lisp/ob-exp.el
@@ -103,7 +103,7 @@ Assume point is at block opening line."
 		   (apply #'org-babel-merge-params
 			  org-babel-default-header-args
 			  (and (boundp lang-headers)
-   (symbol-value lang-headers))
+   (eval-default-headers lang-headers))
 			  (append (org-babel-params-from-properties lang)
   (list raw-params)))
 	  (setf hash (org-babel-sha1-hash info :export)))
-- 
2.28.0

Re: babel latex headers and image generation commands

[Matt Huszagh]

Bastien  writes:

> If you find the time to share your change as a _patch_ (not the whole
> updated Elisp function), that will allow more readers on this list to
> quickly understand what is at stake.

Ok, I've finally gotten around to taking a crack at this. The patch is
attached. Basically, it allows a lot more control when converting a
latex source block into an svg image file.

Specifically, this new method does not place any restrictions on the
latex contents (the old svg generation required the use of
\documentclass[preview]{standalone}), or on the specific generation
commands (it does require first compiling to pdf then to svg, though I
guess this could be generalized if necessary). Additionally, it allows
you to use a setup independent of the setup used for inline latex
snippets. I think this is important because snippets and latex source
blocks have different use cases. For instance, I use snippets for simple
inline math (e.g. \(x=6\)) and source blocks for complicated full-blown
latex pictures and sets of equations, etc. that are not supported by
latex fragments.

I'll present my use case as an example (see the patch to understand the
customizations).

;; this is particular to my use case, see `latex-preamble-by-backend'
(defun by-backend (blist)
  (let ((ret nil))
(if org-export-current-backend
(let* ((backend-name org-export-current-backend)
   (elem (assoc backend-name blist)))
  (if elem
  (setq ret (cdr elem
  (let ((elem (assoc t blist)))
(setq ret (cdr elem
(eval ret)))

;; custom function for preamble generation
(defun latex-preamble-by-backend (params)
  (concat "\\documentclass{"
  (cdr (assoc :class params))
  "}"
  "\\definecolor{fg}{rgb}{"
  (by-backend '((html . "0,0,0")
(t . (org-latex-color :foreground
  "}\n"
  "\\definecolor{bg}{rgb}{"
  (by-backend '((html . "1,1,1")
(t . (org-latex-color :background
  "}\n"
  "\\def\\pc{"
  (by-backend '((html . "100")
(t . "20")))
  "}\n"))

;; actual set customization
(setq org-babel-latex-preamble
  (lambda (params)
(latex-preamble-by-backend params)))

Now if I define a source block:

#+header: :class math :results file link replace :file tmp/some-file.svg
#+begin_src latex :hidden
{\color{fg}
\begin{equation*}
|z| = \sqrt{x^2 + y^2}
\end{equation*}
}
#+end_src

Using my customization above, this will turn into the following latex

\documentclass{math}\definecolor{fg}{rgb}{0.819608,0.721569,0.592157}
\definecolor{bg}{rgb}{0.0235294,0.137255,0.160784}
\def\pc{20}
\begin{document}{\color{fg}
\begin{equation*}
|z| = \sqrt{x^2 + y^2}
\end{equation*}
}\end{document}

which is then turned into an svg with inkscape, though this could be
dvisvgm, or whatever you want.

Math is a custom document class I've defined. But my function also
allows the setting of other classes (which is useful because some
classes work better for tikz-like images and others work better for
math). In fact, since the preamble generation function can use any of
the source block parameters, I could make it do a bunch of other
conditional stuff. I've also configured it to set my color scheme in a
backend-dependent way. This allows me to get color schemes that match my
emacs theme when generating images for viewing within emacs and another
color scheme (in my case black foreground, white background) when
exporting to html. Note that none of this is enforced on other
people. My patch simply allows you to achieve this sort of flexibility.

I think this patch needs some discussing. I had some trouble deciding on
the best way to implement this functionality because its hard to get
anything to be thematically consistent with the current latex execute
command, which feels like a collection of loosely-related functionality
(for instance, the htlatex backend generates a completely different
latex source than the imagemagick backend, even though the outputs could
be quite similar -- svg and png). I think there's a lot of room for
improvement in the latex execute function that could make it more
general and flexible, but it's hard to do this in a way that doesn't
break existing workflows. My patch tries to be minimally invasive to
these workflows, though it will break workflows in an easily recoverable
way for anyone using htlatex for svg output.

Anyway, I'd be curious to hear thoughts and I'd be interested to discuss
options for further refactoring the latex execute function.

Matt

On Fri, Aug 28, 2020 at 11:10 PM Matt Huszagh  wrote:

> Bastien  writes:
>
> > If you find the time to share your change as a _patch_ (not the whole
> > updated Elisp function), that will allow more readers on thi

Re: babel default header args as functions

[Matt Huszagh]

Matt Huszagh  writes:

> I've added the ability in my own configuration to use lambda functions
> that evaluate to a string as babel default header arguments, instead of
> just the plain strings currently allowed. Would anyone else be
> interested in this feature? Shall I prepare a patch?
>
> There are a number of use cases for this, but to give you an idea,
> here's one I'm using myself.
>
>   (setq org-babel-default-header-args:latex
> `((:file . (lambda ()
>  (concat "img/"
>  (sha1 (org-element-property :value 
> (org-element-at-point)))
>  (by-backend '((html . "-html") (t . "-org")))
>  ".svg")
>
> This computes a filename based on the hash of the block contents.

I've generated a patch for this. Please let me know your thoughts. I
believe this adds valuable flexibility to default header
arguments.

Thanks!
Matt

>From 3dfb1066b211fdcc5e3ea1da8d36aa115dde9f9b Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Fri, 28 Aug 2020 11:05:59 -0700
Subject: [PATCH] ob-core.el: Add ability to use closures as default header
 arguments

* lisp/ob-core.el (org-babel-default-header-args): Document ability to
use functions.
(eval-default-headers): New function to generate default header
arguments, which adds the ability to evaluate function arguments at
runtime.
(org-babel-get-src-block-info): Use new header argument evaluate
function when retreiving src block info.

The closures are evaluated at runtime.
---
 lisp/ob-core.el | 32 ++--
 1 file changed, 30 insertions(+), 2 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 578622232..4a22f17e7 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -473,7 +473,23 @@ For the format of SAFE-LIST, see `org-babel-safe-header-args'."
 (defvar org-babel-default-header-args
   '((:session . "none") (:results . "replace") (:exports . "code")
 (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no"))
-  "Default arguments to use when evaluating a source block.")
+  "Default arguments to use when evaluating a source block.
+
+This is a list in which each element is an alist.  Each key
+corresponds to a header argument, and each value to that header's
+value.  The value can either be a string or a closure that
+evaluates to a string at runtime.  For instance, imagine you'd
+like to set the file name output of a latex source block to a
+sha1 of its contents.  We could achieve this with:
+
+(defun org-src-sha ()
+  (let ((elem (org-element-at-point)))
+(concat (sha1 (org-element-property :value elem)) \".svg\")))
+
+(setq org-babel-default-header-args:latex
+  `((:results . \"file link replace\")
+(:file . (lambda () (org-src-sha)")
+
 (put 'org-babel-default-header-args 'safe-local-variable
  (org-babel-header-args-safe-fn org-babel-safe-header-args))
 
@@ -584,6 +600,18 @@ the outer-most code block.")
 
 (defvar *this*)
 
+(defun eval-default-headers (headers)
+  "Compute default header list set with HEADERS.
+
+  Evaluate all default header arguments set to functions prior to
+  returning the list of header arguments."
+  (let ((lst nil))
+(dolist (elem (eval headers t))
+  (if (listp (cdr elem))
+  (push `(,(car elem) . ,(funcall (cdr elem))) lst)
+(push elem lst)))
+lst))
+
 (defun org-babel-get-src-block-info ( light datum)
   "Extract information from a source block or inline source block.
 
@@ -615,7 +643,7 @@ a list with the following pattern:
 	   (apply #'org-babel-merge-params
 		  (if inline org-babel-default-inline-header-args
 			org-babel-default-header-args)
-		  (and (boundp lang-headers) (eval lang-headers t))
+		  (and (boundp lang-headers) (eval-default-headers lang-headers))
 		  (append
 		   ;; If DATUM is provided, make sure we get node
 		   ;; properties applicable to its location within
-- 
2.28.0

Re: incorrect documentation for file-desc header argument?

[Matt Huszagh]

Kyle Meyer  writes:

> Yeah, that looks to be the intended result of that thread.  That
> thread's patch was applied with a58a4f0ad (new source block header
> argument :filelinkdescr, 2012-03-27).  However, shortly after, that
> treatment was intentionally changed (670c7f31c, 2012-03-31):
>
> simplified implementation of :file-desc header argument
>   
> This will no longer insert the value of the :file header argument as
> the description if the description is left blank (as this changes
> the meaning of the :file header argument).
>   
> The desc handling logic is moved to the `org-babel-result-to-file'
> function.
>
> So, I'd say this is a documentation bug.

Thanks for the clarification Kyle. I've attached a patch that I believe
clarifies the documentation to match the current behavior.

Matt
>From a9cd13f3d7b120a24b3416f8f8ba0892a8e47221 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Tue, 28 Jul 2020 22:10:12 -0700
Subject: [PATCH] org-manual.org: Modify file-desc header argument to match
 action

---
 doc/org-manual.org | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index b61644626..0f012d4df 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -17436,9 +17436,8 @@ default behavior is to automatically determine the result type.
 
   #+cindex: @samp{file-desc}, header argument
   The =file-desc= header argument defines the description (see
-  [[*Link Format]]) for the link.  If =file-desc= has no value, Org
-  uses the generated file name for both the "link" and
-  "description" parts of the link.
+  [[*Link Format]]) for the link.  If =file-desc= has no value, the
+  "description" part of the link will be omitted.
 
   #+cindex: @samp{sep}, header argument
   By default, Org assumes that a table written to a file has
-- 
2.27.0

Re: incorrect documentation for file-desc header argument?

[Matt Huszagh]

Matt Huszagh  writes:

> Hello,
>
> The org info documentation states
>
>  The ‘file-desc’ header argument defines the description (see *note
>  Link Format::) for the link.  If ‘file-desc’ has no value, Org uses
>  the generated file name for both the “link” and “description” parts
>  of the link.
>
> However, if I evaluate the
>
> #+BEGIN_SRC asymptote :results value file :file circle.pdf :output-dir img/
>   size(2cm);
>   draw(unitcircle);
> #+END_SRC
>
> I get
>
> #+RESULTS:
> [[file:img/circle.pdf]]
>
> I believe the documentation (and this is corroborated by the discussion
> of the original patch
> (https://lists.gnu.org/archive/html/emacs-orgmode/2012-04/msg00022.html)),
> that the intended result is
>
> #+RESULTS:
> [[file:img/circle.pdf][file:img/circle.pdf]]
>
> Is my assessment correct? Can anyone else confirm this bug?
>
> Thanks
> Matt

If this is indeed a bug, is there supposed to be a way to create the
current output?:

#+RESULTS:
[[file:img/circle.pdf]]

incorrect documentation for file-desc header argument?

[Matt Huszagh]

Hello,

The org info documentation states

 The ‘file-desc’ header argument defines the description (see *note
 Link Format::) for the link.  If ‘file-desc’ has no value, Org uses
 the generated file name for both the “link” and “description” parts
 of the link.

However, if I evaluate the

#+BEGIN_SRC asymptote :results value file :file circle.pdf :output-dir img/
  size(2cm);
  draw(unitcircle);
#+END_SRC

I get

#+RESULTS:
[[file:img/circle.pdf]]

I believe the documentation (and this is corroborated by the discussion
of the original patch
(https://lists.gnu.org/archive/html/emacs-orgmode/2012-04/msg00022.html)),
that the intended result is

#+RESULTS:
[[file:img/circle.pdf][file:img/circle.pdf]]

Is my assessment correct? Can anyone else confirm this bug?

Thanks
Matt

Re: patch: add custom latex->html conversion command

[Matt Huszagh]

Will do, thanks!

On Sun, Feb 16, 2020 at 5:10 PM Bastien  wrote:

> Hi Matt,
>
> Matt Huszagh  writes:
>
> > Thanks for the feedback. I've filled out the form you sent and sent it
> > to the email listed.
>
> Thanks!  It looks good.
>
> Let me know when you get the answer from the FSF.
>
> If you don't in four weeks, please ping them again cc'ing me.
>
> --
>  Bastien
>

Re: patch: add custom latex->html conversion command

[Matt Huszagh]

Thanks for the feedback. I've filled out the form you sent and sent it
to the email listed.

Adjusted patch below. I didn't add a value to the customization. I'm
honestly not very familiar with the customize interface since I never
use it. Is that a default value if the user selects string from the
customize interface? If it is I think it's probably best to leave it
blank since this is really meant to be open-ended. For instance, I doubt
the example I give would work on windows, so that might be more
confusing than helpful in that case. What do you think? Happy to set it
to whatever.

Matt

>From 6b2495c8aef0b67fd00ad27a0056e79f42c23c06 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Sun, 16 Feb 2020 16:52:02 -0800
Subject: [PATCH] add custom command option when converting latex fragments to
 html

* lisp/org.el (org-latex-to-html-convert-command): Add custom command
option to convert a latex fragment directly into HTML.
(org-format-latex): Add condition for this command to org-format-latex.
(org-format-latex-as-html): Command that ultimately does the
conversion work. It uses the user-specified command and applies to
the latex fragment. It then returns the resulting HTML.
* lisp/ox-html.el (org-html-with-latex): Document 'html symbol.
(org-html-format-latex): This custom HTML conversion, like mathjax,
doesn't require preprocessing.
(org-html-latex-fragment): Add condition to org-html-latex-fragment
for html symbol.

This allows you to set a custom command
`org-latex-to-html-convert-command' that will take as input a latex
fragment and use it to generate html for export. This is very
open-ended in the sense that you can use any shell-command you
want. I've added the ability in order to use latexml, but you could
use any other tool that generates HTML output text.
---
 lisp/org.el | 29 +
 lisp/ox-html.el |  9 +++--
 2 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 97ce7ec43..7cc9e8687 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3203,6 +3203,22 @@ When using LaTeXML set this option to
 	  (const :tag "None" nil)
 	  (string :tag "\nShell command")))
 
+(defcustom org-latex-to-html-convert-command nil
+  "Command to convert LaTeX fragments to HTML.
+This command is very open-ended: the output of the command will
+directly replace the latex fragment in the resulting HTML.
+Replace format-specifiers in the command as noted below and use
+`shell-command' to convert LaTeX to HTML.
+%i: The latex fragment to be converted.
+
+For example, this could be used with LaTeXML as
+\"latexmlc 'literal:%i' --profile=math --preload=siunitx.sty 2>/dev/null\"."
+  :group 'org-latex
+  :package-version '(Org . "9.5")
+  :type '(choice
+	  (const :tag "None" nil)
+	  (string :tag "\nShell command")))
+
 (defcustom org-preview-latex-default-process 'dvipng
   "The default process to convert LaTeX fragments to image files.
 All available processes and theirs documents can be found in
@@ -15613,6 +15629,10 @@ Some of the options can be changed using the variable
 		(if (string= (match-string 0 value) "$$")
 			(insert "\\[" (substring value 2 -2) "\\]")
 		  (insert "\\(" (substring value 1 -1) "\\)"
+		 ((eq processing-type 'html)
+		  (goto-char beg)
+		  (delete-region beg end)
+		  (insert (org-format-latex-as-html value)))
 		 ((assq processing-type org-preview-latex-process-alist)
 		  ;; Process to an image.
 		  (cl-incf cnt)
@@ -15778,6 +15798,15 @@ inspection."
   ;; Failed conversion.  Return the LaTeX fragment verbatim
   latex-frag)))
 
+(defun org-format-latex-as-html (latex-frag)
+  "Convert latex to html with a custom conversion command.
+`LATEX-FRAG' is the latex fragment
+Set the custom command with `org-latex-to-html-convert-command'."
+  (let ((cmd (format-spec org-latex-to-html-convert-command
+			  `((?i . ,latex-frag)
+(message "Running %s" cmd)
+(setq shell-command-output (shell-command-to-string cmd
+
 (defun org--get-display-dpi ()
   "Get the DPI of the display.
 The function assumes that the display has the same pixel width in
diff --git a/lisp/ox-html.el b/lisp/ox-html.el
index ea6aa63c3..b5cbf4cfc 100644
--- a/lisp/ox-html.el
+++ b/lisp/ox-html.el
@@ -776,6 +776,8 @@ e.g. \"tex:mathjax\".  Allowed values are:
   `verbatim'Keep everything in verbatim
   `mathjax', t  Do MathJax preprocessing and arrange for MathJax.js to
 be loaded.
+  `html'Use `org-latex-to-html-convert-command' to convert
+LaTeX fragments to HTML.
   SYMBOLAny symbol defined in `org-preview-latex-process-alist',
 e.g., `dvipng'."
   :group 'org-export-html
@@ -2776,12 +2778,13 @@ CONTENTS is nil.  INFO is a plist holding contextual information."
 (defun org

Re: patch: add custom latex->html conversion command

[Matt Huszagh]

Bastien,

Thanks for the reply. I’ll look into it. However, I’d initially ruled that
out because the final output is meant to be an image. So, to get just text
insertion into the html output I imagine would be a bit of a hack. For
instance what to set image-converter to etc.

Matt

On Sun, Feb 16, 2020 at 4:06 PM Bastien  wrote:

> Hi Matt,
>
> Matt Huszagh  writes:
>
> > The patch below allow's you to provide your own shell command the
> > generates HTML from a latex fragment and places the output directly in
> > the exported HTML file.
>
> Can you get the same output by configuring a new symbol in
> `org-preview-latex-process-alist' then setting `org-html-with-latex'
> to this new symbol?
>
> I've not tested it, so perhaps it does not work.
>
> But any patch about adding latexmlc support should surely look in this
> direction first.
>
> HTH,
>
> --
>  Bastien
>

patch: add custom latex->html conversion command

[Matt Huszagh]

Hi all,

The patch below allow's you to provide your own shell command the
generates HTML from a latex fragment and places the output directly in
the exported HTML file.

I've added this feature to be able to use latexml for processing latex
fragments to HTML. Unfortunately, mathjax can't process macros from the
siunitx package, but latexml can. Here's how I use it

  (setq org-html-with-latex 'html)
  (setq org-latex-to-html-convert-command "latexmlc 'literal:%i' --profile=math 
--preload=siunitx.sty 2>/dev/null")

Any/all feedback appreciated!

>From 056d23d9e5caa6fc22907014e0128519fcc84b6e Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Sat, 15 Feb 2020 18:42:11 -0800
Subject: [PATCH] add custom command option when converting latex fragments to
 html

This allows you to set a custom command
`org-latex-to-html-convert-command' that will take as input a latex
fragment and use it to generate html for export. This is very
open-ended in the sense that you can use any shell-command you want. I
envisioned this for use with latexml, but there's nothing preventing
you from using something else.
---
 lisp/org.el | 30 ++
 lisp/ox-html.el |  9 +++--
 2 files changed, 37 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 97ce7ec43..94557bf86 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3203,6 +3203,22 @@ When using LaTeXML set this option to
 	  (const :tag "None" nil)
 	  (string :tag "\nShell command")))
 
+(defcustom org-latex-to-html-convert-command nil
+  "Command to convert LaTeX fragments to HTML.
+This command is very open-ended: the output of the command will
+directly replace the latex fragment in the resulting HTML.
+Replace format-specifiers in the command as noted below and use
+`shell-command' to convert LaTeX to HTML.
+%i: The latex fragment to be converted.
+
+For example, this could be used with LaTeXML as
+\"latexmlc 'literal:%i' --profile=math --preload=siunitx.sty 2>/dev/null\"."
+  :group 'org-latex
+  :version "26.1"
+  :type '(choice
+	  (const :tag "None" nil)
+	  (string :tag "\nShell command")))
+
 (defcustom org-preview-latex-default-process 'dvipng
   "The default process to convert LaTeX fragments to image files.
 All available processes and theirs documents can be found in
@@ -15613,6 +15629,10 @@ Some of the options can be changed using the variable
 		(if (string= (match-string 0 value) "$$")
 			(insert "\\[" (substring value 2 -2) "\\]")
 		  (insert "\\(" (substring value 1 -1) "\\)"
+		 ((eq processing-type 'html)
+		  (goto-char beg)
+		  (delete-region beg end)
+		  (insert (org-format-latex-as-html value)))
 		 ((assq processing-type org-preview-latex-process-alist)
 		  ;; Process to an image.
 		  (cl-incf cnt)
@@ -15778,6 +15798,16 @@ inspection."
   ;; Failed conversion.  Return the LaTeX fragment verbatim
   latex-frag)))
 
+(defun org-format-latex-as-html (latex-frag)
+  "Convert latex to html with a custom conversion command.
+`LATEX-FRAG' is the latex fragment
+Set the custom command with `org-latex-to-html-convert-command'."
+  (let ((cmd (format-spec
+	  org-latex-to-html-convert-command
+	  `((?i . ,latex-frag)
+(message "Running %s" cmd)
+(setq shell-command-output (shell-command-to-string cmd
+
 (defun org--get-display-dpi ()
   "Get the DPI of the display.
 The function assumes that the display has the same pixel width in
diff --git a/lisp/ox-html.el b/lisp/ox-html.el
index ea6aa63c3..b5cbf4cfc 100644
--- a/lisp/ox-html.el
+++ b/lisp/ox-html.el
@@ -776,6 +776,8 @@ e.g. \"tex:mathjax\".  Allowed values are:
   `verbatim'Keep everything in verbatim
   `mathjax', t  Do MathJax preprocessing and arrange for MathJax.js to
 be loaded.
+  `html'Use `org-latex-to-html-convert-command' to convert
+LaTeX fragments to HTML.
   SYMBOLAny symbol defined in `org-preview-latex-process-alist',
 e.g., `dvipng'."
   :group 'org-export-html
@@ -2776,12 +2778,13 @@ CONTENTS is nil.  INFO is a plist holding contextual information."
 (defun org-html-format-latex (latex-frag processing-type info)
   "Format a LaTeX fragment LATEX-FRAG into HTML.
 PROCESSING-TYPE designates the tool used for conversion.  It can
-be `mathjax', `verbatim', nil, t or symbols in
+be `mathjax', `verbatim', `html', nil, t or symbols in
 `org-preview-latex-process-alist', e.g., `dvipng', `dvisvgm' or
 `imagemagick'.  See `org-html-with-latex' for more information.
 INFO is a plist containing export properties."
   (let ((cache-relpath "") (cache-dir ""))
-(unless (eq processing-type 'mathjax)
+(unless (or (eq processing-type 'mathjax)
+(eq processing-type 'html))
   (let ((bfn

babel default header args as functions

[Matt Huszagh]

I've added the ability in my own configuration to use lambda functions
that evaluate to a string as babel default header arguments, instead of
just the plain strings currently allowed. Would anyone else be
interested in this feature? Shall I prepare a patch?

There are a number of use cases for this, but to give you an idea,
here's one I'm using myself.

  (setq org-babel-default-header-args:latex
`((:file . (lambda ()
 (concat "img/"
 (sha1 (org-element-property :value 
(org-element-at-point)))
 (by-backend '((html . "-html") (t . "-org")))
 ".svg")

This computes a filename based on the hash of the block contents.

Matt

babel comma escape with :wrap

[Matt Huszagh]

There appears to be no way to disable the comma escape when using :wrap
for a babel source block.

I'm essentially trying to replicate this example from the manual

 #+NAME: attr_wrap
 #+BEGIN_SRC sh :var data="" :var width="\\textwidth" :results output
   echo "#+ATTR_LATEX: :width $width"
   echo "$data"
 #+END_SRC

 #+HEADER: :file /tmp/it.png
 #+BEGIN_SRC dot :post attr_wrap(width="5cm", data=*this*) :results drawer
   digraph{
   a -> b;
   b -> c;
   c -> a;
   }
 #+end_src

 #+RESULTS:
 :RESULTS:
 #+ATTR_LATEX :width 5cm
 [[file:/tmp/it.png]]
 :END:

But, my result type is a link not a drawer (which are mutually
exclusive). So, to get the same wrapping effect, I need to use the :wrap
argument. However, the comma escape negates attr_wrap's effect with this
code

 (let ((wrap
(lambda (start finish  no-escape no-newlines
  inline-start inline-finish)
  (when inline
(setq start inline-start)
(setq finish inline-finish)
(setq no-newlines t))
  (let ((before-finish (copy-marker end)))
(goto-char end)
(insert (concat finish (unless no-newlines "\n")))
(goto-char beg)
(insert (concat start (unless no-newlines "\n")))
(unless no-escape
  (org-escape-code-in-region
   (min (point) before-finish) before-finish))
(goto-char end
  [...]

But, since this file uses lexical binding (as it should) there appears
to be no way to set no-escape. I searched through the changelog a bit
and that seems to be a relic from an old version. At the very least,
that no-escape conditional should be made unconditional. However, I do
think there should be a way to avoid the comma escape, but I'm not sure
of the best way to do it. I'm more than happy to write the code, but I'm
curious what people think would be the best way to do it.

Maybe :wrap could take a special keyword argument (e.g. no-escape) that
would stop the effect. Thoughts?

Best,
Matt

babel link bug

[Matt Huszagh]

The patch below fixes a bug with the behavior of link without file for
babel source blocks. All explained in patch message, but let me know if
any concerns.

>From 25d363bbc3cd7122287364f25f9b5d653bcae232 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Fri, 7 Feb 2020 23:09:48 -0800
Subject: [PATCH] ob-core.el: fix silent ouput of babel link format
Cc: emacs 

The file type and link format are distinct according to the
manual. Previous code required file type in order for link to
work. This is distinct from the file header argument that is required
for link to work.

To see why this is a bug, try the code under link in the manual

 #+begin_src shell :results link :file "download.tar.gz"
 wget -c "http://example.com/download.tar.gz;
 #+end_src

This will download the file but will not generate any results.
---
 lisp/ob-core.el | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 1a0122192..53168edc8 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -709,8 +709,7 @@ block."
 			   (not (listp r)))
 			  (list (list r))
 			r)))
-	  (let ((file (and (member "file" result-params)
-			   (cdr (assq :file params)
+	  (let ((file (cdr (assq :file params
 		;; If non-empty result and :file then write to :file.
 		(when file
 		  ;; If `:results' are special types like `link' or
-- 
2.25.0

Re: latex fragments in html export

[Matt Huszagh]

Nvm, I found the issue.

latex fragments in html export

[Matt Huszagh]

Hello,

Has anyone had success getting the html export to process latex
fragments? It's leaving mine completely alone and I can't figure out
why. Latex babel source blocks work fine. I've set

(setq org-html-with-latex luasvgm)

luasvgm is a process-alist element I defined. I went through ox-html and
set up debug points in a bunch of functions meant to process latex
fragments, but none of them are being called.

Thanks
Matt

Re: babel latex headers and image generation commands

[Matt Huszagh]

I've thought about this more and the solution I presented above isn't
quite sufficient for me. I need something where I get complete control
over what goes in the latex source block on a block-by-block basis. In
other words, I don't want a user-configurable option like
org-format-latex-header plus a list of user-configurable packages in
every block.

I could add an option alongside the option I introduced above that tells
the execute function to only compose the tex file from the body (btw,
this doesn't have to be manual for the user, just define a class and use
snippets), but at some point I wonder how much it makes sense to keep
adding options to this function since it'll just make it harder to
maintain.

What do people think? If there's any interest, I'm more than happy to
put in the extra time and add this functionality to
latex-execute. Otherwise, I'll just advise the function for my own
specific needs.

I do think there's a real use-case here: namely, complete case-by-case
control over the source latex and its output, including export backend
dependent things like colors. But, maybe this is just me and the
existing functionality is fine with people.

Matt

Re: babel latex headers and image generation commands

[Matt Huszagh]

Ok, here's an implementation that seems to be working pretty well so
far.

`org-latex-img-process` is the new customization. Most of the execute
function is unaltered, but I've added the condition:
```
 ((and (not imagemagick)
   (assoc extension org-latex-img-process))
```

Here's the change in full.

```
(setq org-latex-pdf-process
  '("latexmk -f -interaction=nonstopmode -output-directory=%o %f"))

(setq org-latex-img-process
  '(("svg" . ("dvisvgm %f -P -n -b min -o %O"

(defun org-babel-execute:latex (body params)
  "Execute a block of Latex code with Babel.
This function is called by `org-babel-execute-src-block'."
  (setq body (org-babel-expand-body:latex body params))
  (if (cdr (assq :file params))
  (let* ((out-file (cdr (assq :file params)))
 (extension (file-name-extension out-file))
 (tex-file (org-babel-temp-file "latex-" ".tex"))
 (border (cdr (assq :border params)))
 (imagemagick (cdr (assq :imagemagick params)))
 (im-in-options (cdr (assq :iminoptions params)))
 (im-out-options (cdr (assq :imoutoptions params)))
 (fit (or (cdr (assq :fit params)) border))
 (height (and fit (cdr (assq :pdfheight params
 (width (and fit (cdr (assq :pdfwidth params
 (headers (cdr (assq :headers params)))
 (in-buffer (not (string= "no" (cdr (assq :buffer params)
 (org-latex-packages-alist
  (append (cdr (assq :packages params)) 
org-latex-packages-alist)))
(cond
 ((and (string-suffix-p ".png" out-file) (not imagemagick))
  (org-create-formula-image
   body out-file org-format-latex-options in-buffer))
 ((string-suffix-p ".tikz" out-file)
  (when (file-exists-p out-file) (delete-file out-file))
  (with-temp-file out-file
(insert body)))
 ((and (not imagemagick)
   (assoc extension org-latex-img-process))
  (with-temp-file tex-file
(insert (concat
 (org-latex-make-preamble
  (org-export-get-environment (org-export-get-backend 
'latex))
  org-format-latex-header
  'snippet)
 (if headers
 (concat "\n"
 (if (listp headers)
 (mapconcat #'identity headers "\n")
   headers) "\n")
   "")
 "\\begin{document}"
 body
 "\\end{document}")))
  (let ((tmp-pdf (org-babel-latex-tex-to-pdf tex-file)))
(when (file-exists-p out-file) (delete-file out-file))
(let* ((log-buf (get-buffer-create "*Org Babel LaTeX Output*"))
   (err-msg "fix")
   (img-out (org-compile-file
 tmp-pdf
 (cdr (assoc "svg" org-latex-img-process))
 extension err-msg log-buf)))
  (shell-command (format "mv %s %s" img-out out-file)
 ((and (or (string= "svg" extension)
   (string= "html" extension))
   (executable-find org-babel-latex-htlatex))
  ;; TODO: this is a very different way of generating the
  ;; frame latex document than in the pdf case.  Ideally, both
  ;; would be unified.  This would prevent bugs creeping in
  ;; such as the one fixed on Aug 16 2014 whereby :headers was
  ;; not included in the SVG/HTML case.
  (with-temp-file tex-file
(insert (concat
 "\\documentclass[preview]{standalone}
\\def\\pgfsysdriver{pgfsys-tex4ht.def}
"
 (mapconcat (lambda (pkg)
  (concat "\\usepackage" pkg))
org-babel-latex-htlatex-packages
"\n")
 (if headers
 (concat "\n"
 (if (listp headers)
 (mapconcat #'identity headers "\n")
   headers) "\n")
   "")
 "\\begin{document}"
 body
 "\\end{document}")))
  (when (file-exists-p out-file) (delete-file out-file))
  (let ((default-directory (file-name-directory tex-file)))
(shell-command (format "%s %s" org-babel-latex-htlatex 
tex-file)))
  

babel latex headers and image generation commands

[Matt Huszagh]

I spent some time today trying to get latex babel source blocks to work
for me and discovered that calling `org-babel-execute:latex` ignores the
:headers header if the output file is a png without setting imagemagick
to t. It's easy to see this in the source code: the conditions mentioned
above leads to calling org-create-formula-image without passing in the
headers. I think this is a bug, although maybe I missed it somewhere in
the documentation? Here's a MWE if you want it

#+header: :file "test.png"
#+header: :headers '("\\def\\hello{hello}")
#+begin_src latex :results output file link
\hello
#+end_src

doesn't work, but

#+header: :file "test.png"
#+header: :imagemagick t
#+header: :headers '("\\def\\hello{hello}")
#+begin_src latex :results output file link
\hello
#+end_src

does.

However, this got me thinking that I wish executing latex blocks behaved
a bit more like latex fragment previews. Particularly, the ability to
customize `image-converter`. So, I'm thinking about adding a
customization option that allows a user to use the image-converter
portion of an existing org-preview-latex-process-alist entry (I guess
the most obvious choice would be
org-preview-latex-default-process). Although I guess I could just add a
new variable like what we have with org-latex-pdf-process. Maybe that's
better since we'd only be using one part of the
latex-process-alist. What are people's thoughts on this?

preview src blocks that generate image files

[Matt Huszagh]

Hi,

I'm considering adding the ability to display an image preview overlay of
source blocks that generate an image file, much in the same way that
latex fragments can be previewed.

Is anyone else interested in this feature? Any general thoughts/feature
requests?

Matt

Re: Issues with nested begin..end blocks in inline math environments

[Matt Huszagh]

Nicolas Goaziou  writes:

> I don't think this patch is a good idea, as it mixes elements from
> different types: LaTeX fragment (inline), and LaTeX environment (block).
> For example, the regexp you modify is used as a paragraph separator,
> which doesn't make sense for inline LaTeX.

No worries. Thanks for taking a look at this anyway.

Matt

Re: Issues with nested begin..end blocks in inline math environments

[Matt Huszagh]

I'm submitting this as a patch. I've used it on hundreds of latex
fragments over the past week or so and haven't experienced any issues
(which is expected since the change is small).

>From a699b699ed4132839c39f1152868bb13364422c7 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Sat, 14 Dec 2019 19:54:41 -0800
Subject: [PATCH] org-element.el: allow environment blocks in math delimiters

* lisp/org-element.el (org-element--latex-begin-environment): Add a
non-capturing block for `\(' or `$' so that previously recognized
latex environments can also appear within an inline math environment.

* lisp/org-element.el (org-element--latex-end-environment): Match the
begin environment noncapturing block with `$' or `\)'.
---
 lisp/org-element.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/org-element.el b/lisp/org-element.el
index 110ff5624..6d7ec32c6 100644
--- a/lisp/org-element.el
+++ b/lisp/org-element.el
@@ -,14 +,14 @@ containing `:key', `:value', `:begin', `:end', `:post-blank' and
  Latex Environment
 
 (defconst org-element--latex-begin-environment
-  "^[ \t]*begin{\\([A-Za-z0-9*]+\\)}"
+  "^[ \t]*\\(?:(\\|\\$\\)?begin{\\([A-Za-z0-9*]+\\)}"
   "Regexp matching the beginning of a LaTeX environment.
 The environment is captured by the first group.
 
 See also `org-element--latex-end-environment'.")
 
 (defconst org-element--latex-end-environment
-  "end{%s}[ \t]*$"
+  "end{%s}[ \t]*\\(?:)\\|\\$\\)?$"
   "Format string matching the ending of a LaTeX environment.
 See also `org-element--latex-begin-environment'.")
 
-- 
2.24.0

Re: Removing horizontal space in latex fragments

[Matt Huszagh]

Nicolas Goaziou  writes:

> So, has anyone settled on which one to apply?

My vote goes for keeping the newlines to improve readability in the
generated tex file. But, again, I'm more than happy to be overuled.

> Minor nitpick:
>
>   (if (string-suffix-p string "\n") ...)
>
> is slightly less low-level.

Appreciate the nitpick; your version is better!

I've attached an updated patch.

Best,
Matt

>From bdb93a13a43d90ad6e66449797111e836a67a219 Mon Sep 17 00:00:00 2001
From: Matt Huszagh 
Date: Thu, 5 Dec 2019 23:25:32 -0800
Subject: [PATCH] org.el: Remove leading/trailing whitespace from latex
 fragment

* lisp/org.el (org-create-formula-image): Ensure user input ends
with a % character to remove trailing whitespace. Also, add %
characters between macros and newlines purely visual.
---
 lisp/org.el | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 9b84592ba..ae686e330 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16554,12 +16554,17 @@ a HTML file."
 	(setq bg (org-latex-color :background))
   (setq bg (org-latex-color-format
 		(if (string= bg "Transparent") "white" bg
+;; remove tex \par at end of snippet to avoid trailing
+;; whitespace
+(if (string-suffix-p string "\n")
+(aset string (- (length string) 1) ?%)
+  (setq string (concat string "%")))
 (with-temp-file texfile
   (insert latex-header)
   (insert "\n\\begin{document}\n"
-	  "\\definecolor{fg}{rgb}{" fg "}\n"
-	  "\\definecolor{bg}{rgb}{" bg "}\n"
-	  "\n\\pagecolor{bg}\n"
+	  "\\definecolor{fg}{rgb}{" fg "}%\n"
+	  "\\definecolor{bg}{rgb}{" bg "}%\n"
+	  "\n\\pagecolor{bg}%\n"
 	  "\n{\\color{fg}\n"
 	  string
 	  "\n}\n"
--
2.24.0

Re: Issues with nested begin..end blocks in inline math environments

[Matt Huszagh]

"Fraga, Eric"  writes:
> The identification of LaTeX fragments is somewhat fragile (in my
> experience).  I would suggest you enclose complex LaTeX code fragments
> within an #+begin_export latex ... #+end_export environment.
>
> Assuming your export target is LaTeX and/or PDF.

Changing the beginning and end regexes to the following seems to work
well (I've just added a non-capturing group for \(\) or $$ so that
begin-end blocks can be nested inside inline math environments).

(setq org-element--latex-begin-environment "^[ 
\t]*\\(?:(\\|\\$\\)?begin{\\([A-Za-z0-9*]+\\)}")
(setq org-element--latex-end-environment "end{%s}[ 
\t]*\\(?:)\\|\\$\\)?$")

I'm going to test this a bit before submitting it as a patch to make
sure it doesn't cause any issues. If you have any tricky fragments you
want to test it on please let me how it works!

Re: Issues with nested begin..end blocks in inline math environments

[Matt Huszagh]

"Fraga, Eric"  writes:
> The identification of LaTeX fragments is somewhat fragile (in my
> experience).  I would suggest you enclose complex LaTeX code fragments
> within an #+begin_export latex ... #+end_export environment.
>
> Assuming your export target is LaTeX and/or PDF.

Thanks for the suggestion. Unfortunately these are mostly just used as
latex previews in the org buffer. I'm going to try to improve support
for this when I get the time since I use this capability extensively. If
you have any thoughts/suggestions on best ways to improve this I'd be
happy to hear them.

Issues with nested begin..end blocks in inline math environments

[Matt Huszagh]

I'm experiencing incorrect and seemingly inconsistent behavior when nesting
`\begin` `\end` environments inside `\(\)` or `$$`. For example, the
following is valid latex code:
```
\(\begin{aligned}
b_n &= \frac{1}{\pi} \int_{0}^{\pi} x \sin{\left(nx\right)}dx \\
b_0 &= 0 \\
\int_{0}^{\pi} udv &= \left[uv\right]_{0}^{\pi}
  - \int_{0}^{\pi} vdu \\
dv &= \sin{\left(nx\right)}dx \\
v(x) &= \int \sin{\left(nx\right)}dx \\
&= -\frac{\cos{\left(nx\right)}}{n} \\
\left[uv\right]_{0}^{\pi} &=
  -\left[\frac{x\cos{\left(nx\right)}}{n}\right]_{0}^{\pi} \\
&= \left\{\begin{alignedat}{2}
  \frac{\pi}{n}&, && \quad \text{$n$ odd} \\
  -\frac{\pi}{n}&, && \quad \text{$n$ positive even} \\
\end{alignedat}\right. \\
\int_{0}^{\pi} vdu &=
  \int_{0}^{\pi} -\frac{\cos{\left(nx\right)}}{n} dx \\
&= -\frac{1}{n^2} \left.\sin{\left(nx\right)}\right|_{0}^{\pi} \\
&= 0 \\
b_{n\in \mathbb{Z}^+} &= \left\{\begin{alignedat}{2}
  \frac{1}{n}&, && \quad \text{$n$ odd} \\
  -\frac{1}{n}&, && \quad \text{$n$ positive even} \\
\end{alignedat}\right. \\
\end{aligned}\)
```
But this confuses org which is generating images for the `\begin{aligned}`
at the top and the `\text` macros, but nothing else. Iterations on this
sometimes work fine, however. For example,
```
\(\begin{aligned}
b_{n\in \mathbb{Z}^+} &= \left\{\begin{alignedat}{2}
  \frac{1}{n}&, && \quad \text{$n$ odd} \\
  -\frac{1}{n}&, && \quad \text{$n$ positive even} \\
\end{alignedat}\right. \\
\end{aligned}\)
```
typesets correctly. Has anyone else had trouble with the latex fragment
regex's?

Re: Removing horizontal space in latex fragments

[Matt Huszagh]

Thanks for the reply Eric. The thing I like about the newlines is that the
generated tex files are slightly easier to read. However, this is really
minor. I've created 2 separate patches: one keeping the newlines and the
other without. I'm happy to defer to you or anyone else in regard to which
is preferable.

On Thu, Dec 5, 2019 at 2:24 PM Fraga, Eric  wrote:

> On Thursday,  5 Dec 2019 at 11:03, Matt Huszagh wrote:
> > Is anyone else interested in this modification? Should I submit it as a
> > patch?
>
> I think so.  And I am not sure all those \n's are necessary.  Without
> them, you can probably also remove many of the %s.
> --
> Eric S Fraga via Emacs 27.0.50, Org release_9.2.6-544-gd215c3
>


remove-newlines.patch
Description: Binary data


keep-newlines.patch
Description: Binary data

Removing horizontal space in latex fragments

[Matt Huszagh]

I've modified the behavior of `org-create-formula-image' so that
`\definecolor' etc do not create unnecessary whitespace in the output PDF.
Here's the relevant change:

```
...
;; remove tex \par at end of line
(if (string= (substring string -1 nil) "\n")
(aset string (- (length string) 1) ?%)
  (setq string (concat string "%")))
(with-temp-file texfile
  (insert latex-header)
  (insert "\n\\begin{document}\n"
 "\\definecolor{fg}{rgb}{" fg "}%\n"
 "\\definecolor{bg}{rgb}{" bg "}%\n"
 "\n\\pagecolor{bg}%\n"
 "\n{\\color{fg}\n"
 string
 "\n}\n"
 "\n\\end{document}\n"))
...
```

This is useful if you (like me) have replaced the default document class
with standalone and are using dvisvgm. The change removes leading left and
right space, which is especially useful when using inline math mixed with
normal text. It shouldn't make a difference if using Imagemagick's convert
as a backend since that can get rid of whitespace boundaries.

Is anyone else interested in this modification? Should I submit it as a
patch?

Matt

latex preview parallel processes

[Matt Huszagh]

Hello. I have an org document with many latex snippets for which I use
the org latex preview facility. As the document has gotten bigger, any
changes requiring a rebuild of all images (e.g. changing
`org-format-latex-header') takes longer and longer to the point where
it now takes me over an hour to generate all images. Since all images
are mutually independent, I was wondering if it's possible to run
these tasks in parallel to make use of my multiple CPU cores. If this
is not currently possible, what would it take to get this working and
how could I help?

Thanks
Matt