Re: figures not exported properly by ox-latex

[Karthik Chikmagalur]

> it might've been fixed for figures as its been a while, but shouldnt
> latex environments with #+caption: and #+name: above them be exported
> as figures? because currently the user has to write \begin{figure} and
> \end{figure} explicitly which would work

This is not the case.  The LaTeX output I posted in my previous response
was the exported output from your example Org source.  I did not add the
\begin{figure} and \end{figure} tags manually.

Karthik

Re: figures not exported properly by ox-latex

[Karthik Chikmagalur]

> im on tecosaur's dev branch (version 9.7-pre)
> i have the following in an org file:
> ```org
> #+name: fig-switching-circuit-1
> #+caption: implementation of \(p\)
> [[attachment:circuit.svg]]
> ```
> it gets turned into this
> ```
> \includesvg[width=.9\linewidth]{/home/mahmooz/brain/notes/data/9e/5ba2ce-c383-4396-a2cb-891465f14d51/circuit}\\[0pt]
> ```

Cannot reproduce.  I get the following:

\begin{figure}[htbp]
\centering
\includesvg[width=.9\linewidth]{/tmp/circuit}
\caption{\label{fig-switching-circuit-1}implementation of \(p\)}
\end{figure}

Perhaps you have customized the LaTeX export process in some way?  Also
note that we don't touch any of the LaTeX generation code in ox-latex
except the parts handling LaTeX preview images.

Karthik

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Karthik Chikmagalur]

I don't see anything wrong in the diagnostic information you've
provided.  Here are some more questions for you.

1. What is your org-latex-compiler set to?

2. Could you try to use dvisvgm to generate previews, and when it fails,
check the contents of the buffer "*Org Preview Convert Output*"?  You
can include it with your reply.

3. Disable hl-line-mode if you use it, then move the cursor into one of
the LaTeX fragments and run the following using M-x eval-expression:

(overlay-get (car (overlays-at (point))) 'preview-image)

If it returns an svg file path, could you open that file and check if
the svg is being generated as expected?  If it returns nil, that will be
useful to know as well.

Karthik


Yaroslav Drachov  writes:

> Not yet.
>
> Best wishes,
> Yaroslav
>
>> 23 июня 2024 г., в 13:53, Ihor Radchenko  написал(а):
>> 
>> Yaroslav Drachov  writes:
>> 
>>> My system-configuration-features are
>>> 
>>> "ACL GIF GLIB GMP GNUTLS JPEG JSON LCMS2 LIBXML2 MODULES NOTIFY KQUEUE NS 
>>> PDUMPER PNG RSVG SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP 
>>> XIM XWIDGETS ZLIB".
>>> 
>>> RSVG is in the list.
>>> 
>>> org-latex-preview-check-health diagnostic info:
>>> ...
>> 
>> May I know if this problem has been resolved?
>> 
>> -- 
>> Ihor Radchenko // yantar92,
>> Org mode contributor,
>> Learn more about Org mode at .
>> Support Org development at ,
>> or support my work at 

Re: [PATCH] Fix regex for determining image width from attribute

[Karthik Chikmagalur]

> 1. #+attr_org is prioritised
> 2. Docstrings are updated
>
> Handled, on main.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=fede1c990

Just a note that we already prioritize #+attr_org when
aligning/centering images.  So this change makes sense.

Karthik

Re: [BUG] Attachments not resolved correctly from symlinked Org files

[Karthik Chikmagalur]

> I can see the problem.
> But what would you expect to happen if there was no attachment in the
> original directory? Should the attachment be created relative to the
> original file? To the symlink?

I don't know.  I would expect the attachment to always be created
relative to the original file, but other users might have the opposite
expectation.

Karthik

[BUG] Attachments not resolved correctly from symlinked Org files

[Karthik Chikmagalur]

Attachment paths are not resolved correctly when the Org file is
symlinked elsewhere. This may or may not be a bug, but I think it's
undesired behavior:

1. Create an Org file somewhere, say ~/Desktop/test.org.

2. In test.org, create an attachment: `C-c C-a m' as a file.  Either the
   "copy" or "move" attachment method can be used.

3. Symlink test.org from elsewhere, say

/tmp/test.org -> ~/Desktop/test.org

4. In a fresh Emacs session, open /tmp/test.org and try to open the
attachment directory (`C-c C-a f').

Two things happen:

- An empty directory is created, like
  /tmp/data/48/2697ee-913c-4eee-b1fb-636416d3fb6b

- The original attachment can't be found, since it's actually in
  ~/Desktop/data/48/2697ee-913c-4eee-b1fb-636416d3fb6b

Tested with `emacs -q' with Org 9.6.23, and with commit
1ae978f940c0f88473f2232177c63a0de7fb7a1c from two weeks ago.

Karthik

Re: Using org-latex-preview in other major modes

[Karthik Chikmagalur]

> Anyways, before I put this off for much longer, there is some more code
> attached. Live previews (and general environments) work now, and besides
> the above mentioned points there were no new surprises waiting—at least
> for getting the basic functionality to work.

Thank you for the update!  This looks promising.  I'll test it when I
have time.

> I did notice that precompilation being a bit flaky. In the end, this
> was the result of importing local packages; with the precompilation
> happening somewhere in /tmp/, access to these files wasn't given.
> haven't looked too closely into this—and it's probably a use-case
> that's specific to LaTeX-mode—but it seems worth considering

Sorry, I should have mentioned this in my original write-up.  There is a
reason for this, as explained below.  This issue should not be happening
in most cases though, so I'll need more details to help.

> (lots of people carry one big style file around that they include in
> all of their projects).

This is true of Org mode files that are intended for LaTeX export too,
but we avoid this problem in Org.

1. Why we precompile in /tmp:

   Precompilation is a blocking operation, so we want to do it as
   infrequently as possible. Imagine if LaTeX previews in every Org
   buffer/file you opened required a precompile step that blocked Emacs
   for 3 seconds.

   Precompiled dump files are identified by a hash that includes the
   preamble and a default-directory. If we precompile in /tmp (or some
   other fixed directory), we can reuse dump files for all Org buffers
   that have the same preamble, irrespective of where they're located.

   Precompiled files are stored in org-persist and don't expire for a
   long time.  So we'd like to avoid generating loads of them, and reuse
   them whenever possible.

   1a. Why is precompilation a blocking operation?

   It doesn't have to be, but including it in the async chain is a
   little annoying.  It can be made async in the future.

2. What happens if the user includes a directory-local .sty, .cls or
   other tex files in the header?

   When precompiling, we check the header to see if it has an \input{}
   or \include{} statement.  If it does, we assume that there are local
   dependencies and precompile it in the default-directory.  See
   org-latex-preview--create-tex-file for the implementation of this
   logic.

   Is there some other common way that a LaTeX file can have local
   dependencies?

3. How to force precompilation to occur in the default-directory instead
   of in /tmp:

   Based on 1 and 2, the easiest way would be to add an \input{} or
   \include{} statement to the preamble. The current test is very
   simple, you can even place it in a LaTeX comment and it should work.

Karthik

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Karthik Chikmagalur]

> I hope I understand correctly how to use this mail-thread. Thank you
> very much for such a great enhancement of Org-mode LaTeX preview
> capabilities. I am on macOS Big Sur and have some troubles with
> dvisvgm preview in your particular fork. Everything works just fine in
> stable release of Org-mode, and in your release dvipng method work
> almost as expected. dvisvgm preview method actually generate svg’s in
> /var/folders/3m…, but don’t make an overlay after all and equation
> environments remain being formatted as text.

Quick check: 

1. Is your Emacs compiled with svg support? (on Linux, this means
system-configuration-features will include RSVG)
2. Could you run the function at
https://gist.github.com/karthink/0ac48411a81459c0f3fd7557c4e817db
and supply us with the results?  It should pop up a buffer with some
diagnostics pertaining to LaTeX previews.

Karthik

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Karthik Chikmagalur]

> I've got problem when I use this version of org-latex preview:
>
> RUNNING: dvisvgm --page=1- --optimize --clipjoin --relative --no-fonts 
> --bbox=preview -o c\:/Users/artsi0m/AppData/Local/Temp/org-tex-DAJVTr-%9p.svg 
> c\:/Users/artsi0m/AppData/Local/Temp/org-tex-DAJVTr.dvi
>
> ERROR: can't open file 
> 'c\:/Users/artsi0m/AppData/Local/Temp/org-tex-DAJVTr.dvi' for reading
>
> As you can get from the traceback it is problem with windows. But I
> remember solving simillar problem with org-fc already[1][2]. I just removed
> escaping (quoting) from one of the elisp functions.

Thank you for testing the new LaTeX preview system, we haven't had any
reports yet from Windows.

As a temporary measure, could you try replacing calls to
shell-quote-argument with identity in the following two functions?

org-latex-preview--tex-compile-async
org-latex-preview--image-extract-async

I'd be interested to know if it works as expected otherwise.

> So, the main problem of the problem is that on Windows
> (shell-quote-argument) function will quote c:/ as c\:/ which breaks
> ability to find one or another file. It maybe other function with this
> quoting, but I suppose that it is it.

shell-quote-argument is used all over Emacs, so I'm quite surprised it
mangles file paths on Windows.

Karthik

org-persist-write slowing down kill-buffer

[Karthik Chikmagalur]

Hi,

I've been noticing that kill-buffer blocks Emacs for a noticeable
period when killing org buffers.  Here are elp results obtained by
instrumenting org-persist-* and kill-buffer:

| Function name | Call count | Elapsed time | Average time |
|---++--+--|
| kill-buffer   | 38 | 2.5647546329 | 0.0674935429 |
| org-persist-write-all-buffer  |  1 |  2.562779994 |  2.562779994 |
| org-persist-write-all |  1 |   2.56277329 |   2.56277329 |
| org-persist-write |  1 |  1.627834788 |  1.627834788 |
| org-persist--write-elisp-file |  1 |  0.312172392 |  0.312172392 |
| org-persist--find-index   |   4808 | 0.0480629129 | 9.996...e-06 |
| org-persist--normalize-associated |  4 |  0.007245059 | 0.0018112647 |
| org-persist--normalize-container  |  7 | 0.0001262719 | 1.803...e-05 |
| org-persist--get-collection   |  1 |  0.000113982 |  0.000113982 |
| org-persist-write:elisp   |  2 | 2.647...e-05 | 1.323...e-05 |
| org-persist--display-time |  1 |3.772e-06 |3.772e-06 |
| org-persist-write:version |  1 |1.467e-06 |1.467e-06 |

It blocks Emacs for about 3 seconds each time.  Any Org file with > 500
lines causes this behavior.

I've also attached the sampling profiler output from killing an Org
buffer.

Some facts that might be relevant:

- I'm using the WIP LaTeX preview system fork of Org, but there are no
  LaTeX previews in the Org buffers that I run these tests on.  (There
  aren't even any LaTeX fragments.)

- My org-persist-dir is 627MB in size.

This is Emacs 29.3 (pgtk branch), the Org commit that the LaTeX preview
patches are rebased on is 1ae978f940c0f88473f2232177c63a0de7fb7a1c.

Let me know if I can help with more information.

Karthik


kill-buffer-profiler-report.eld
Description: Binary data

Re: [Q] startup hook: How do I detect if the current buffer has been opened programmatically?

[Karthik Chikmagalur]

>> Org-mode occasionally opens files automatically, for instance, when
>> inserting or opening ID links, or during certain searches. I need to
>> determine if a buffer was opened programmatically or manually by the user
>> within the startup hooks. This distinction is important because, e.g., I
>> want to automatically preview all LaTeX fragments if the buffer was opened
>> by the user, but not if it was opened programmatically.
>
> AFAIK, there is no reliable way to do this.
> You may play around with `find-file-hook'. See `org-with-file-buffer'
> macro in org-macs.el (on main).

A heuristic I use is to check if the window is visible.  It's not
perfect but good enough for my use (which includes previewing LaTeX
fragments):

(defun my/latex-preview-maybe ()
  (when (window-live-p (get-buffer-window (current-buffer)))
(org-latex-preview 'buffer)
(org-latex-preview-auto-mode 1)))

(add-hook 'org-mode-hook #'my/latex-preview-maybe)

Karthik

Re: Using org-latex-preview in other major modes

[Karthik Chikmagalur]

> Thanks for taking a look! It indeed seems that caching will be the most
> work to properly implement. However, I'm quite happy to not care about
> numbers for now and make sure the other stuff works correctly first
> before tackling this.

Sure, this is a disproportionate amount of work for a minor feature.  I
suggest setting `org-latex-preview-numbered' to nil, FWIW.  The previews
look better this way.

> Besides `org-latex-preview-auto--maybe-track-element-here`, I was mostly
> talking about `org-latex-preview-live--src-buffer-setup` and
> `org-latex-preview-live--ensure-overlay`, which I think will be the main
> targets. The latter seems to be quite a straightforward translation
> (especially my current constraints of not caring about environments),

You can ignore `org-latex-preview-live--src-buffer-setup', this is for
previewing when using `org-edit-special' (C-c '), which does not apply
to any other major mode.  So `org-latex-preview-live--ensure-overlay' is
the only function you need to rewrite, which should be easy.

Keep us updated. Good luck!

Karthik

Re: Using org-latex-preview in other major modes

[Karthik Chikmagalur]

> Abstracting away previews is certainly welcome.
> RMS explicitly asked Org mode team to work towards this goal:
> https://list.orgmode.org/e1kikxv-0007iy...@fencepost.gnu.org/

I agree with RMS about this, and this was on our minds when we wrote
org-latex-preview.  The basic previewing process is explicitly written
to be Org-agnostic, but we weren't as rigorous when adding the fancier
features (like live previews).

> Ideally, we should have Org-independent library that does the previews,
> and an Org-specific code that re-uses this library. Eventually, we can
> move the generic library to Emacs core.

Here is the plan Timothy and I have discussed:

1. Merge org-latex-preview in its current state and continue to fix
bugs/edge cases.
2. Write an external package reimplementing in a more generic way the
parts of the API that are Org-specific.  This external package will
depend (heavily) on org-latex-preview.
3. Solicit from the community integrations of this with other
major-modes using this generic API.
4. Once this API is stable, replace the corresponding parts of
org-latex-preview.
5. Propose moving everything but the Org-specific parts to a
`latex-preview.el' package included with Emacs.

>From Tony's proof of concept, I think step 2 might be unnecessary and we
can incrementally modify org-latex-preview instead.

> We may go even further, and extend the previews to be not just for
> LaTeX. Might as well preview html/image links/pdf links/etc.

I agree in principle but I think this is difficult to do with
org-latex-preview because the async process chain and overlay handling are
highly tuned for low-latency LaTeX processing.[1] 

I do think Emacs could use a generic link-preview package, with an
org-link-preview adapter for Org mode.

Karthik

[1]: As low-latency as possible from Emacs without modifying
the LaTeX compiler or image renderer, as for example TeXpresso
does. https://github.com/let-def/texpresso

Re: Using org-latex-preview in other major modes

[Karthik Chikmagalur]

Hi Tony,

Just tried it out, it's very promising!  Thanks for taking the initiative
on this.

> I've prodded the code a little bit and, mostly just following [1],
> managed to implement basic previews in a relatively straightforward
> fashion. Attached is a proof-of-concept—aptly named
> latex-latex-preview.el :) The main user facing functions so far are
> 'latex-latex-preview' to preview the maths fragment at point, and
> 'latex-latex-preview-region' to preview all fragments in a region. Both
> functions currently ignore things like numbered equations and
> environments, though both do not seem exceptionally difficult to add
> back in.

There are two issues with numbering:
- providing an Org-agnostic API point to attach a numbering table to, and
- calculating the new numbering table in LaTeX (or other major-modes).

For The first issue, we need a way to provide an updated numbering table
during the auto-regeneration of edited fragments.  Currently this is
done implicitly by calling `org-latex-preview--place-from-elements' from
the `--regenerate-overlay` function.

The second requires fast numbering table updates.  We do it in Org by
mapping over the org-element cache (see
`org-latex-preview--get-numbered-environments').  Even this is too slow
sometimes, so we suspend numbering updates when live-previewing until
the cursor exits the fragment.  Parsing the LaTeX buffer from point to
the end when (re)generating each preview is going to be too slow, so
you'll have to create some kind of cache and update it incrementally.

> There's also a stub 'latex-latex-preview-auto-mode' minor mode. Stepping
> in and out of already rendered environments works fine, with the preview
> being regenerated if needed; only the implementation of
> 'org-latex-preview-auto--regenerate-overlay' had to change for that.

This was a pleasant surprise, I was expecting more trouble here.

> The mode does not currently feature live-previews. The innards of
> 'org-latex-preview-auto--detect-fragments-in-change' and
> 'org-latex-preview-live--setup' look a bit more org-specific, with
> queries to 'org-element-*' functions all over the place, but all of
> that looks fixable—at least from afar. I will continue prodding the
> code a little bit and will report back with any bumps that are hit
> along the way.

`org-latex-preview-auto--detect-fragments-in-change' is written for
speed. It only does quick text-matching and is thus mostly Org-agnostic,
except for a call to a numbering calculation near the end.  This should
be easy to adapt.  The problem is the function it calls,
`org-latex-preview-auto--maybe-track-element-here', which finds the
bounds of the inserted LaTeX fragment using org-element and
conditionally sets up a preview overlay.  You will need an equivalent of
this for LaTeX-mode.

Since you have auto-mode working already, live previews should be quite
easy to add.  From what I can see, you only need to provide your own
`org-latex-preview-live--ensure-overlay'.  This function creates the
preview overlay next to or under the LaTeX fragment.  All the other live
preview code only changes overlay properties or calls
`--regenerate-overlay`, which you've already implemented.

Karthik

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Karthik Chikmagalur]

>> Thanks for the patch!
>> May you please describe a use case when it is necessary to run 
>> latex
>> multiple times for previews?
>
> Sure: it's required whenever you have LaTeX that refers to
> other document elements whose positions are only known after
> they've been typeset. This often comes up with bibliographies; to
> quote the docstring for `org-latex-pdf-process': "The reason why
> this is a list is that it usually takes several runs of
> ‘pdflatex’, maybe mixed with a call to ‘bibtex’."

I assume this is a general example of a situation where you have to run
LaTeX multiple times, and not something applicable to Org's LaTeX
previews.

> I personally encountered it when using the tikzmark library from
> the tikzpicture package to draw  annotations on a table (LaTeX
> needed to be run once to typeset the table & only then did it
> "know" where the arrows needed to be drawn).

Was this for in-buffer previews or a LaTeX source block?  Could you
provide the LaTeX source for a minimal example?  I'd like to try it with
the new preview system.

Karthik

Re: [PATCH] org-manual: Describe export process flow

[Karthik Chikmagalur]

> The patch is attached.
> I'd appreciate feedback from people not familiar with ox.el.

- When exporting a sub-tree, at what stage of the export process is the
  buffer narrowed to the sub-tree?
- Are "inner" and "outer" templates described in the manual, and if they
  are could you add a link to those sections when mentioning them in
  this summary?  I only found references to the plist properties
  BEAMER_INNER_THEME etc.

Karthik

Re: [PATCH] Justify/align image previews in org-mode

[Karthik Chikmagalur]

>> I've incorporated the following suggestions:
>>
>> - Order of precedence:
>>   + #+attr_org overrides #+attr_html and #+attr_latex
>>   + `:center t' overrides `:align ...'
>
> Why only html and latex? I think that it will be more consistent to
> follow what we do in `org-display-inline-image--width' - any #+attr_*
> attribute. And apart from html/latex, we have
> beamer/hugo/koma-letter/what not is implemented or will be implemented
> deriving from html/latex backend or with support of :center attribute.

Changed implementation to support any #+attr_.* keyword.

>> - Add a checker for `:align nil' to org-lint.  `:align nil' is not
>>   supported.
>
> I meant :align  being not supported in #+attr_org. Not just nil.

org-lint now complains if anything other than `:align left|center|right'
or `:center t' are specified in #+attr_org.

>> +  #+vindex: org-image-align
>> +  Org mode can left-align, center or right-align the display of inline
>> +  images.  This setting is controlled (globally) by ~org-image-align~.
>> +  Only standalone links, /i.e/ links with no surrounding text in their
>
> Maybe "standalone images"?
> Also, please avoid i.e - this is a general Emacs documentation guideline
> to avoid specialized language like i.e., e.g., and iff.

Changed text, removed specialized language.

>> +  paragraphs (except whitespace) are affected.  Its value can be the
>> +  following:
>> +  - (default) nil, insert the image where the link appears in the
>> +buffer.
>
> Why not simply setting the default to 'left and not having nil at all?

The original idea was that `left' and `nil' do the same thing, so it
should be fine to leave it at `nil'.  I changed the user option to
default to `left', and `nil' has been removed as an available option.
(It is still not an error to set it to `nil'.)

>> +  - The symbol ~left~, which is the same as nil.
>> +  - The symbol ~center~, which will preview standalone links centered
>> +in the Emacs window.
>> +  - The symbol ~right~, which will preview standalone links
>> +right-aligned in the Emacs window.
>
> "standalone" links is redundant here - you already mentioned that  

Fixed.

>> +  Inline image alignment can be specified for each link using the
>> +  =#+ATTR.*= keyword if it matches an alignment specification like:
>> +  #+begin_example
>> +  ,#+ATTR_HTML: :align center
>> +  #+end_example
>> +  Supported values for =:align= are =left=, =center= and =right=.
>
> I think that we do not need to specify the supported values here - they
> are the same as for `org-image-align' just above.

Removed.

>> +  Inline image display can also be centered using =:center=, as in
>
> I'd explain that inline image adjustment is also taken from :center
> attribute supported by some export backends (like HTML, LaTeX, and
> Beamer), when #+attr_org is not set.

Tweaked the description of :center.

>> +  #+begin_example
>> +  ,#+ATTR_HTML: :center t
>> +  #+end_example
>> +  Org will use the alignment specification from any =#+ATTR.*=
>> +  keyword, such as =#+ATTR_HTML= or =#+ATTR_LATEX=, but =#+ATTR_ORG=
>> +  (if present) will override the others.  For instance, this link
>> +  #+begin_example
>> +  ,#+ATTR_HTML: :align right
>> +  ,#+ATTR_ORG: :align center
>> +  [[/path/to/image/file.png]]
>> +  #+end_example
>> +  will be displayed centered.
>
> ... "but exported right-aligned to HTML"

Added.

>> +(defun org-lint-invalid-image-alignment (ast)
>> +  (org-element-map ast 'paragraph
>> +(lambda (p)
>> +  (let ((bad-align-re ":align[[:space:]]+nil")
>> +(keyword-string (mapconcat
>> + (lambda (attr)
>> +   (or (car-safe (org-element-property attr p)) 
>> ""))
>> +     '(:attr_org :attr_latex :attr_html) " ")))
>
> :align nil is perfectly valid for HTML.
> I thought to warn only about using anything but :align left/right/center
> in #+attr_org. And about :center  in #+attr_org.

Yes, only the value of #+attr_org is checked now.

Patch v2 attached.

Karthik
>From 7176f43282749c06daf2756360633cc47d79b63c Mon Sep 17 00:00:00 2001
From: Karthik Chikmagalur 
Date: Mon, 18 Dec 2023 12:56:33 -0800
Subject: [PATCH] org: Add image alignment

* lisp/org.el (org-image--align, org-image-align,
org-toggle-inline-images): Add the ability to left-align, center
or right-align inline image previews in the Emacs window. This is
controlled globally using the new user option `org-image-align'.
Alignment can be specified per image using the `#+ATTR.*'
affi

Re: [PATCH] Justify/align image previews in org-mode

[Karthik Chikmagalur]

I've incorporated the following suggestions:

- Order of precedence:
  + #+attr_org overrides #+attr_html and #+attr_latex
  + `:center t' overrides `:align ...'

- Update doc/org-manual.org under the images section.

- Add a checker for `:align nil' to org-lint.  `:align nil' is not
  supported.

- Unspecified behaviors:
  + The behavior of `:center nil` is undefined
  + #+attr_html vs #+attr_latex

- Include a commit message in the patch.  The included patch is based on
  the main branch.

Karthik
>From eb1b287c009c2f7eb83e7e31d64980ba79f44527 Mon Sep 17 00:00:00 2001
From: Karthik Chikmagalur 
Date: Mon, 18 Dec 2023 12:56:33 -0800
Subject: [PATCH] org: Add image alignment

* lisp/org.el (org-image--align, org-image-align,
org-toggle-inline-images): Add the ability to left-align, center
or right-align inline image previews in the Emacs window. This is
controlled globally using the new user option `org-image-align'.
Alignment can be specified per image using the `#+ATTR.*'
affiliated keywords.  The function `org-image--align' determines
the kind of alignment for its argument link.

* lisp/org-lint.el (org-lint-invalid-image-alignment): Add an
org-lint checker for the pattern ":align nil" in `#+ATTR.*'
keywords.  This alignment specification is not supported.

* doc/org-manual.org: Document the new feature under the Images
section.
---
 doc/org-manual.org | 34 +
 lisp/org-lint.el   | 17 +
 lisp/org.el| 91 +++---
 3 files changed, 137 insertions(+), 5 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 5217e647d..0df730f2b 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -11501,6 +11501,40 @@ command:
 and fall back on the original width if none is found.
 
 
+  #+vindex: org-image-align
+  Org mode can left-align, center or right-align the display of inline
+  images.  This setting is controlled (globally) by ~org-image-align~.
+  Only standalone links, /i.e/ links with no surrounding text in their
+  paragraphs (except whitespace) are affected.  Its value can be the
+  following:
+  - (default) nil, insert the image where the link appears in the
+buffer.
+  - The symbol ~left~, which is the same as nil.
+  - The symbol ~center~, which will preview standalone links centered
+in the Emacs window.
+  - The symbol ~right~, which will preview standalone links
+right-aligned in the Emacs window.
+
+  Inline image alignment can be specified for each link using the
+  =#+ATTR.*= keyword if it matches an alignment specification like:
+  #+begin_example
+  ,#+ATTR_HTML: :align center
+  #+end_example
+  Supported values for =:align= are =left=, =center= and =right=.
+  Inline image display can also be centered using =:center=, as in
+  #+begin_example
+  ,#+ATTR_HTML: :center t
+  #+end_example
+  Org will use the alignment specification from any =#+ATTR.*=
+  keyword, such as =#+ATTR_HTML= or =#+ATTR_LATEX=, but =#+ATTR_ORG=
+  (if present) will override the others.  For instance, this link
+  #+begin_example
+  ,#+ATTR_HTML: :align right
+  ,#+ATTR_ORG: :align center
+  [[/path/to/image/file.png]]
+  #+end_example
+  will be displayed centered.
+
 #+vindex: org-cycle-inline-images-display
 Inline images can also be displayed when cycling the folding state.
 When custom option ~org-cycle-inline-images-display~ is set, the
diff --git a/lisp/org-lint.el b/lisp/org-lint.el
index 0e2967b6c..84bca9f48 100644
--- a/lisp/org-lint.el
+++ b/lisp/org-lint.el
@@ -964,6 +964,18 @@ Use \"export %s\" instead"
 		reports
 reports))
 
+(defun org-lint-invalid-image-alignment (ast)
+  (org-element-map ast 'paragraph
+(lambda (p)
+  (let ((bad-align-re ":align[[:space:]]+nil")
+(keyword-string (mapconcat
+ (lambda (attr)
+   (or (car-safe (org-element-property attr p)) ""))
+ '(:attr_org :attr_latex :attr_html) " ")))
+(when (string-match-p bad-align-re keyword-string)
+(list (org-element-begin p)
+  "nil is not a supported value for keyword attribute \":align\""))
+
 (defun org-lint-extraneous-element-in-footnote-section (ast)
   (org-element-map ast 'headline
 (lambda (h)
@@ -1390,6 +1402,11 @@ Use \"export %s\" instead"
   #'org-lint-invalid-keyword-syntax
   :trust 'low)
 
+(org-lint-add-checker 'invalid-image-alignment
+  "Report unsupported align attribute for keyword"
+  #'org-lint-invalid-image-alignment
+  :trust 'low)
+
 (org-lint-add-checker 'invalid-block
   "Report invalid blocks"
   #'org-lint-invalid-block
diff --git a/lisp/org.el b/lisp/org.el
index 59fe3d2d3..8a8bd977d 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16175,6 +16175,26 @@ cache   Display remote images, and open them in separate buffers
 	  (const :tag "Dis

Re: [PATCH] Justify/align image previews in org-mode

[Karthik Chikmagalur]

> I can only suggest something like
> 
> (equal (org-element-begin link)
> (save-excursion
>   (goto-char (org-element-contents-begin paragraph))
>   (skip-chars-forward "\t ")
>   (point)))
> 
> (equal (org-element-end link)
> (save-excursion
>   (goto-char (org-element-contents-end paragraph))
>   (skip-chars-backward "\t ")
>   (point)))

This should be fine, it's what we do in many places in org-latex-preview.

> > - Does org-element provide a property that can help here? I tried
> > :pre-blank but it was nil, I'm not sure what it does.
> 
> :pre-blank is for blank lines before contents. Leading whitespace is
> considered a part of the paragraph:
> (paragraph (...) "  " (link ...) "\n")

Good to know.

I've attached the latest version of the patch.

- 'justify is no longer an option

- 'left is now supported as a value of `org-image-align', although of
  course this is a noop.

- Leading and trailing whitespace are handled correctly.  The image
  overlay swallows up any trailing whitespace if there is nothing else
  on that line.

- Right alignment will still fail in some odd circumstances, such as if
  there's an overlay with a display property to the right of the image.
  To see an example of this, turn on whitespace-mode and try to
  right-align a standalone image.  I don't think it's worth handling
  cases like these at least for now.

Karthik
diff --git a/lisp/org.el b/lisp/org.el
index 30a4e7aef..1efd2f31b 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -15673,6 +15673,26 @@ cache   Display remote images, and open them in separate buffers
 	  (const :tag "Display and silently update remote images" cache))
   :safe #'symbolp)
 
+(defcustom org-image-align nil
+  "How to align images previewed using `org-display-inline-images'.
+
+Only stand-alone image links are affected by this setting.  These
+are links without surrounding text.
+
+Possible values of this option are:
+
+nil  Insert image at specified position.
+left Insert image at specified position (same as nil).
+center   Center image previews.
+rightRight-align image previews."
+  :group 'org-appearance
+  :package-version '(Org . "9.7")
+  :type '(choice
+  (const :tag "Left align (or don\\='t align) image previews" nil)
+	  (const :tag "Center image previews" center)
+	  (const :tag "Right align image previews" right))
+  :safe #'symbolp)
+
 (defun org--create-inline-image (file width)
   "Create image located at FILE, or return nil.
 WIDTH is the width of the image.  The image may not be created
@@ -15807,7 +15827,8 @@ buffer boundaries with possible narrowing."
   (when file (setq file (substitute-in-file-name file)))
 		  (when (and file (file-exists-p file))
 		(let ((width (org-display-inline-image--width link))
-			  (old (get-char-property-and-overlay
+			  (align (org-image--align link))
+  (old (get-char-property-and-overlay
 (org-element-begin link)
 'org-image-overlay)))
 		  (if (and (car-safe old) refresh)
@@ -15819,7 +15840,7 @@ buffer boundaries with possible narrowing."
    (progn
 	 (goto-char
 	  (org-element-end link))
-	 (skip-chars-backward " \t")
+	 (unless (eolp) (skip-chars-backward " \t"))
 	 (point)
   ;; FIXME: See bug#59902.  We cannot rely
   ;; on Emacs to update image if the file
@@ -15833,6 +15854,15 @@ buffer boundaries with possible narrowing."
 			   (list 'org-display-inline-remove-overlay))
 			  (when (boundp 'image-map)
 (overlay-put ov 'keymap image-map))
+  (when align
+(overlay-put
+ ov 'before-string
+ (propertize
+  " " 'face 'default
+  'display
+  (pcase align
+('center `(space :align-to (- center (0.5 . ,image
+('right  `(space :align-to (- right ,image)))
 			  (push ov org-inline-image-overlays
 
 (defvar visual-fill-column-width) ; Silence compiler warning
@@ -15894,6 +15924,43 @@ buffer boundaries with possible narrowing."
   org-image-actual-width)
  (t nil
 
+(defun org-image--align (link)
+  "Determine the alignment of the image link.
+
+This is controlled globally by the option `org-image-align', and
+per image by the value of `:align' in the affiliated keyword
+`#+attr_org'.
+
+The result is either nil or one of the symbols left, center or
+right.
+
+center will cause the image preview to be centered, right will
+cause it to be right-aligned.  A value of left or nil
+implies no special alignment."
+  (let ((par (org-element-lineage link 'paragraph)))
+;; Only apply when image is not surrounded by paragraph text:
+(when (and (= (org-element-begin link)
+  

Re: [PATCH] Justify/align image previews in org-mode

[Karthik Chikmagalur]

> It would make sense to allow 'left value as well (same as nil).

Done.

> I do not think that we need to consider 'justify value at this point.
> Maybe in future, when (or if) we add proper justification support to
> text. But not now.

Removed support for 'justify.

> This will not work when the image paragraph is indented:

* This is test
  #+attr_org: :align center
  [[file:~/Downloads/wallpaper.png]]

- Is there a better way to address this than checking if there is only
whitespace behind the link until the start of the paragraph?
- Does org-element provide a property that can help here? I tried
:pre-blank but it was nil, I'm not sure what it does.

Karthik

Re: [PATCH] Justify/align image previews in org-mode

[Karthik Chikmagalur]

Please ignore the previous patch and use this one instead.  I've fixed a
bug and a couple of formatting errors.

Karthik
diff --git a/lisp/org.el b/lisp/org.el
index 30a4e7aef..ad2ad2332 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -15673,6 +15673,25 @@ cache   Display remote images, and open them in separate buffers
 	  (const :tag "Display and silently update remote images" cache))
   :safe #'symbolp)
 
+(defcustom org-image-align nil
+  "How to align images previewed using `org-display-inline-images'.
+
+Only stand-alone image links are affected by this setting.  These
+are links without surrounding text.
+
+Possible values of this option are:
+
+nil  Insert image at specified position (same as left-alignment).
+center   Center image previews.
+rightRight-align image previews."
+  :group 'org-appearance
+  :package-version '(Org . "9.7")
+  :type '(choice
+  (const :tag "Don\\='t align image previews" nil)
+	  (const :tag "Center image previews" center)
+	  (const :tag "Right align image previews" right))
+  :safe #'symbolp)
+
 (defun org--create-inline-image (file width)
   "Create image located at FILE, or return nil.
 WIDTH is the width of the image.  The image may not be created
@@ -15807,7 +15826,8 @@ buffer boundaries with possible narrowing."
   (when file (setq file (substitute-in-file-name file)))
 		  (when (and file (file-exists-p file))
 		(let ((width (org-display-inline-image--width link))
-			  (old (get-char-property-and-overlay
+			  (align (org-image--align link))
+  (old (get-char-property-and-overlay
 (org-element-begin link)
 'org-image-overlay)))
 		  (if (and (car-safe old) refresh)
@@ -15833,6 +15853,16 @@ buffer boundaries with possible narrowing."
 			   (list 'org-display-inline-remove-overlay))
 			  (when (boundp 'image-map)
 (overlay-put ov 'keymap image-map))
+  (when align
+(overlay-put
+ ov 'before-string
+ (propertize
+  " " 'face 'default
+  'display
+  (pcase align
+((or 'center 'justify)
+ `(space :align-to (- center (0.5 . ,image
+('right  `(space :align-to (- right ,image)))
 			  (push ov org-inline-image-overlays
 
 (defvar visual-fill-column-width) ; Silence compiler warning
@@ -15894,6 +15924,37 @@ buffer boundaries with possible narrowing."
   org-image-actual-width)
  (t nil
 
+(defun org-image--align (link)
+  "Determine the alignment of the image link.
+
+This is controlled globally by the option `org-image-align', and
+per image by the value of `:align' in the affiliated keyword
+`#+attr_org'.
+
+The result is one of the symbols center, justify or right.  The
+first two will cause the image preview to be centered, the last
+will cause it to be right-aligned.  A return value of nil implies
+no special alignment -- the image preview is overlaid on the link
+exactly where it appears in the buffer."
+  (let ((par (org-element-lineage link 'paragraph)))
+;; Only apply when image is not surrounded by paragraph text:
+(when (and (= (org-element-property :begin link)
+  (org-element-property :contents-begin par))
+   (<= (- (org-element-property :contents-end par)
+  (org-element-property :end link))
+   1))  ;account for trailing newline
+  (save-match-data
+;; Look for a valid :align keyword (left, center, justify or right)
+(if-let* ((attr-org (car-safe (org-element-property :attr_org par)))
+  ((string-match ":align[[:space:]]+\\(\\w+\\)" attr-org))
+  (attr-align (car-safe
+   (memq (intern (match-string 1 attr-org))
+ '(left center justify right)
+(unless (eq attr-align 'left) attr-align)
+  ;; No image-specific keyword, check global alignment property
+  (when (memq org-image-align '(center justify right))
+org-image-align))
+
 (defun org-display-inline-remove-overlay (ov after _beg _end  _len)
   "Remove inline-display overlay if a corresponding region is modified."
   (when (and ov after)

[PATCH] Justify/align image previews in org-mode

[Karthik Chikmagalur]

Hi,

This patch allows image link previews in Org to be left-aligned, centered or 
right-aligned in the Emacs window.  "Inline" images that are surrounded by text 
are unaffected.  Here is an example of what this looks like:

Image: https://abode.karthinks.com/share/org-image-align.png

The alignment is persistent as the window (or frame) is resized:

Video demo: https://abode.karthinks.com/share/org-image-align-demo.mp4

Alignment can be set globally for all image previews using the (new) user 
option `org-image-align'.  It can be set locally for each image link using the 
`:align' parameter of the `#+attr_org' affiliated keyword, for example:

#+attr_org: :width 400px :align center

Allowed values for both settings are `center', `justify' and `right'.  Center 
and justify have the same effect, and a value of `left', while not an error, 
does nothing since this is the default behavior.

This patch does not update org-manual.org since it is a work in progress.  
Please let me know if you have suggestions.

Karthik
diff --git a/lisp/org.el b/lisp/org.el
index 30a4e7aef..2b82277a7 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -15673,6 +15673,25 @@ cache   Display remote images, and open them in separate buffers
 	  (const :tag "Display and silently update remote images" cache))
   :safe #'symbolp)
 
+(defcustom org-image-align nil
+  "How to align images previewed using `org-display-inline-images'.
+
+Only stand-alone image links are affected by this setting.  These
+are links without surrounding text.
+
+Possible values of this option are:
+
+nil  Insert image at specified position (same as left-alignment).
+center   Center image previews.
+rightRight-align image previews."
+  :group 'org-appearance
+  :package-version '(Org . "9.7")
+  :type '(choice
+  (const :tag "Don\\='t align image previews" nil)
+	  (const :tag "Center image previews" center)
+	  (const :tag "Right align image previews" right))
+  :safe #'symbolp)
+
 (defun org--create-inline-image (file width)
   "Create image located at FILE, or return nil.
 WIDTH is the width of the image.  The image may not be created
@@ -15807,7 +15826,8 @@ buffer boundaries with possible narrowing."
   (when file (setq file (substitute-in-file-name file)))
 		  (when (and file (file-exists-p file))
 		(let ((width (org-display-inline-image--width link))
-			  (old (get-char-property-and-overlay
+			  (align (org-image--align link))
+  (old (get-char-property-and-overlay
 (org-element-begin link)
 'org-image-overlay)))
 		  (if (and (car-safe old) refresh)
@@ -15833,6 +15853,16 @@ buffer boundaries with possible narrowing."
 			   (list 'org-display-inline-remove-overlay))
 			  (when (boundp 'image-map)
 (overlay-put ov 'keymap image-map))
+  (when align
+(overlay-put
+ ov 'before-string
+ (propertize
+  " " 'face 'default
+  'display
+  (pcase align
+((or 'center 'justify)
+ `(space :align-to (- center (0.5 . (,width)
+('right  `(space :align-to (- right (,width
 			  (push ov org-inline-image-overlays
 
 (defvar visual-fill-column-width) ; Silence compiler warning
@@ -15894,6 +15924,37 @@ buffer boundaries with possible narrowing."
   org-image-actual-width)
  (t nil
 
+(defun org-image--align (link)
+  "Determine the alignment of the image link.
+
+This is controlled globally by the option `org-image-align', and
+per image by the value of `:align' in the affiliated keyword
+`#+attr_org'.
+
+The result is one of the symbols center, justify or right.  The
+first two will cause the image preview to be centered, the last
+will cause it to be right-aligned.  A return value of nil implies
+no special alignment -- the image preview is overlaid on the link
+exactly where it appears in the buffer."
+  (let ((par (org-element-lineage link 'paragraph)))
+;; Only apply when image is not surrounded by paragraph text:
+(when (and (= (org-element-property :begin link)
+  (org-element-property :contents-begin par))
+   (<= (- (org-element-property :contents-end par)
+  (org-element-property :end link))
+   1))  ;account for trailing newline
+  (save-match-data
+;; Look for a valid :align keyword (left, center, justify or right)
+(if-let* ((attr-org (car-safe (org-element-property :attr_org par)))
+  (_ (string-match ":align[[:space:]]+\\(\\w+\\)" attr-org))
+  (attr-align (car-safe
+   (memq (intern (match-string 1 attr-org))
+   

Re: What is the purpose of \\ in org-add-note?

[Karthik Chikmagalur]

> It ensures a line break (say, when the document is exported). From the
> [[info:org#Paragraphs][org#Paragraphs]] section in the manual:
>
> #+begin_quote
> Paragraphs are separated by at least one empty line.  If you need to
> enforce a line break within a paragraph, use ‘\\’ at the end of a line.
> #+end_quote

Good to know, thank you.

Karthik

What is the purpose of \\ in org-add-note?

[Karthik Chikmagalur]

When I add a note to an entry (from Org agenda, say), it adds a \\ at
the end of the note line heading. Example:

- Note taken on [2021-03-01 Mon 20:44] \\
  Done. Check =ffmpeg-dispatch-load= and =ffmpeg-dispatch-save=.
  
The addition of this suffix is hardcoded in `org-store-log-note'. What
is its purpose?

Karthik

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Karthik Chikmagalur]

> I have lualatex installed as part of another texlive package but I am
> not sure if org uses it however.

If you haven't changed `org-latex-compiler' (globally or in the Org
buffer using a keyword), it's using pdflatex.

>> 3. If no, do you mind sharing this file (or a stripped down version
>> you're okay sharing that still has these issues) along with the LaTeX
>> preamble?  That would be `org-latex-preview-preamble' and
>> `org-latex-packages-alist'.  I would like to reproduce this bug.

Please let us know if you can reproduce the precompilation failure.
This shouldn't be happening.

> No, AFAICT I don't see the issue with png previews.  

Sizing issues with pdflatex+svg previews should be fixed soon.

> However, all latex previews, \( \) and \[ \], seem to be of the same
> size now.

Should they not be the same size?

>> Previewing unadorned LaTeX macros is officially unsupported but
>> partially supported in practice.  If you move the cursor over the \ch{}
>> and call `org-latex-preview' it should be previewed.  From that point it
>> should behave like a regular LaTeX fragment.  Editing it should cause
>> the preview to be auto-updated (If you are using
>> org-latex-preview-auto-mode).
>
> Is there no automagic way to do this?  Like changing a regexp variable
> and hoping it would work till the end of time™.  (:

You can change `org-latex-preview--tentative-math-re' to suit your
needs.  This variable is used to identify LaTeX fragments when calling
`org-latex-preview'.  All subsequent checks involve the org-element api,
but for reasons I can't recall at the moment the initial search is via a
regexp.

Karthik

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Karthik Chikmagalur]

> It worked well all around except for three things:
>
> 1. I had to set org-latex-preview-precompile to nil to produce the
>preview for ~2000 snippets.  I got errors in process filters such
>as arg-out-of-range, and Emacs completely blocked itself.  I can
>confirm that mylatexformat is installed

1. Are you using xelatex/lualatex?  
2. If yes, precompilation should have turned itself off with a warning.
3. If no, do you mind sharing this file (or a stripped down version
you're okay sharing that still has these issues) along with the LaTeX
preamble?  That would be `org-latex-preview-preamble' and
`org-latex-packages-alist'.  I would like to reproduce this bug.

Previewing 2000 snippets is exactly where the new async/speedy system is
expected to shine compared to the old one.

> 2. Apparently, org-latex-default-packages-alist no longer contain
>asmsymb and asmmath?  This change pulled the rug under me since
>it took me quite a while to figure out why some of my formulas
>were coloured red...

`org-latex-default-packages-alist' was modified to omit amsmath and
amssymb (among others) in preparation for a conditional export system
where required packages will be determined automatically from the buffer
text -- i.e. to avoid exactly the kind of headache you encountered.
Unfortunately this export system is not part of the patch set yet.

> 3. The image sizes of the same latex environment (e.g., \[ \]) is
>different for different formulas.  I can send a screenshot of
>this if required.

This is a recent regression.  Could you generate png previews by:

1. (setq org-latex-preview-default-process 'dvipng)
2. regenerating previews

and checking if this still happens?  If these look fine, this is a
dvisvgm issue and we will fix it soon.

>> If you do come across any issues, please let me know either in a reply here 
>> or
>> the org-mode matrix room. If you could also run
>>  and 
>> share the
>> diagnostic info, that would be quite helpful.
>
> If needed, I can do this after reproducing (1).
>
> BTW, would it possible to arrange other latex environments (\ch{} in my
> case) to be previewed as well?  Currently, I do \(\ch{}\) but omitting
> \( \) would be nice.

Previewing unadorned LaTeX macros is officially unsupported but
partially supported in practice.  If you move the cursor over the \ch{}
and call `org-latex-preview' it should be previewed.  From that point it
should behave like a regular LaTeX fragment.  Editing it should cause
the preview to be auto-updated (If you are using
org-latex-preview-auto-mode).

Karthik

Re: keep intermediary tex from latex preview

[Karthik Chikmagalur]

>> By the way, using Ihor's suggestion, I was able to debug the issue.
>> There is nothing wrong with my configuration. The issue is that the
>> font cannot deal with that symbol, and LaTeX just skips it. Still, it
>> would be convenient not to have to create a separate .tex to debug
>> the preview. Thanks!
>
> AFAIR, the new preview system will record possible issues with previews
> and mark them appropriately in a tooltip.

In this specific case, it was recognized as an error.  Here is an image
of the result and the tooltip message with the new LaTeX preview system.

I typed \(μp + 1\) because getting a screenshot of the tooltip with just
the rendered `p' was tricky.

Karthik

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Karthik Chikmagalur]

> Yeah, it kinda works ok now. It is kinda slow, but it probably takes
> the same time as the normal previews. May the fact that I use
> `lualatex' be the culprit?

This checks out.  I tested previews with lualatex for a bit, and it's
about 8-10x slower than pdflatex.  One big chunk of this gap is the
absence of precompilation, the rest appears to be inherent to lualatex.
I'm not familiar with lualatex, so if someone knows how it can be sped
up I'd be glad to be educated.  For now I'd recommend using pdflatex for
LaTeX previews.

As a result of this issue (plus other issues with xelatex), I'm
considering adding an option to specify different LaTeX compilers for
previews and exports, with the default option continuing to be
the value of `org-latex-compiler'.

> Both `buffer' and `eldoc' work as expected. Although, when I use `eldoc' and
> use the `eldoc-box' package, nothing is shown. But, this probably isn't
> in the scope of this conversation and it's really secondary.

I've had trouble getting LaTeX environments to show via eldoc in the
echo-area, whereas eldoc-box has worked fine.  I'm making a note to
investigate this later.

> I tried setting `org-latex-preview-default-process' to `dvipng' and
> found that previews as well appear blank. Don't know what might be the
> problem.

Is this in the context of previews or HTML exports?

-Karthik

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Karthik Chikmagalur]

> The preview now works as stated, although I find it kinda slow to
> regenerate, even if I set `org-latex-preview-throttle' to 0.2.

They shouldn't be any slower than when you manually call
`org-latex-preview' on a fragment.  To be sure, you could try

(setq org-latex-preview-debounce 0.3
  org-latex-preview-throttle 0.3)

and then turn on (or turn off + on) `org-latex-preview-auto-mode'.

> Also for some reason my previews appear to the right of the cursor,
> instead of under it like in the linked webm.

The video you linked to was a proof of concept.  Here is a demo of the
current design: https://tinyurl.com/ms2ksthc

If `org-latex-preview-live-display-type' is set to `eldoc':
- show live previews using Eldoc.  You may need to pop up the Eldoc doc
buffer for larger previews to be visible (`M-x eldoc-mode', followed by
`M-x eldoc-doc-buffer').

Else:
- Inline fragments are live-previewed inline
- LaTeX environments are previewed below the environment

> My last problem is that exports are kinda borked. For example, these
> days I started using `org-msg' and when my email exports to html and
> latex is included, one of these happen:
>
> - The fragment appears as a blank image when the options `tex:dvipg' or
>   `tex:dvisvg' are used.
> - The email doesn't even export to html with a message like
>   `org-html-latex-image: Expected LaTeX preview
> "424ed928d1eaca9e3c4c588469a4a0b87a7f4329" to exist in the cache'
>   with the option `tex:imagemagick'.

Tecosaur will have a better idea of what's happening here.  Note that
the export options for html are `tex:dvipng' or `tex:dvisvgm', not
`dvipg' and `dvisvg'.

>> If the live preview is not removed automatically after you move away
>> from a LaTeX fragment, you might have to manually eval
>> `(org-latex-preview-live--clearout)'.
>
> Yeah, it happens to me to. Even if I remove the fragment the preview
> persists.

It shouldn't be happening every time -- only when certain conditions are
met.  Deleting fragments might be one of these conditions.

-Karthik

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Karthik Chikmagalur]

> 1. Use TEC's org-mode =dev= branch, up to the =---NOPUSH PERSONAL
>DIVIDER---= commit.
> 2. Set some config options, which can be in the above link.
> 3. Test the new features.
> 4. When a latex fragment contained an error, its color would turn red,
>but no popup would appear on hover, or ever for that matter.
>
>My temporary solution is to directly check the =*Org Preview LaTeX
>Output*= buffer.

Is tooltip-mode turned on?

>> We don't think live previews will be ready or stable enough in time
>> for Org 9.7, so it is not currently part of the patch set.
>
> That's too bad, it looked really neat and useful.

We'd like to focus on improving the user experience of the core features
first.  If these issues are cleaned up before 9.7, I will work on
polishing the live previews feature.

>> It's available as a later WIP (work in progress) commit on
>> tecosaur's dev branch, if you are interested in trying it.
>
> I noticed that but as I said above, I already use that branch. Perhaps
> the bug with the error popups and this are related? After all, both have
> to do with popups. Also it is important to note that I have already set
> the configuration variables that are relevant to the /live/ preview (at
> least those that I could find).

There is no documentation for live-previews yet.  If you would like
to try it -- and I encourage you to -- you could set:

(setq org-latex-preview-auto-generate 'live)

and then turn on `org-latex-preview-auto-mode'.  The variables
`org-latex-preview-throttle' and `org-latex-preview-debounce' control
the responsiveness of the live previews.

If the live preview is not removed automatically after you move away
from a LaTeX fragment, you might have to manually eval
`(org-latex-preview-live--clearout)'.  This is one of the many edge
cases with the feature that need to be handled.

> All in all this is a huge improvement and a quality-of-life upgrade for
> all org-mode users. Thank you both for your work.

Thank you! The testing is very helpful as well.

-Karthik

Re: [Pre-PATCH] Overhaul of the LaTeX preview system

[Karthik Chikmagalur]

>From your diagnostic report, it looks like you're using lualatex +
precompilation.  This can cause some issues, but not the ones you report
above so this is interesting. 

> 1. When a snippet contains a mistake, it correctly turns red, but no
> error message is shown when I hover my mouse over it. I have to switch
> to the *Org Preview LaTeX Output* to see the error.

This is a bug, could you list the steps to reproduce it?

> 2. Live previews do not work (like shown here
> https://tinyurl.com/5fu7z27w), although I am using the related options
> and use your branch up to the ---PATCH SET DIVIDER--- commit.

We don't think live previews will be ready or stable enough in time for
Org 9.7, so it is not currently part of the patch set.  It's available
as a later WIP (work in progress) commit on tecosaur's dev branch, if
you are interested in trying it.

> 4. Some key-binds (like C-c C-e to publish or C-c C-x C-l for
> latex-preview) do not work when the cursor is at the top of the file,
> before the first header.

This is also a bug, thanks for the report.  I'll look into it.

-Karthik

Re: Recommended way to work on main without upgrading Org?

[Karthik Chikmagalur]

I can get it to work with emacs -Q as well, and this is good for testing, thank 
you.

But I'm looking for a way to retain the rest of my configuration and swap out 
Org as required.  Otherwise I can't do any interactive development.  Another 
alternative is to set up a completely different init directory and run Emacs 
with a chemacs2 profile or with the new --init-directory flag, but considering 
the complexity/size of my configuration this would be onerous.

Karthik

Recommended way to work on main without upgrading Org?

[Karthik Chikmagalur]

Hi,

I'm trying to work on the main branch of Org, with the intent of creating a 
patch.  However, I need to continue using Org 9.5 for everyday work in a 
separate Emacs session as I can't have things breaking.  Is there a recommended 
way to run two simultaneous instances of Emacs using two different Org versions?

I use Straight to install Org 9.5.  I've cloned the latest main branch from 
Savannah to work on, but I'm not able to test my changes cleanly since there 
are two org versions in the mix -- technically three including the built-in 
version that's been shadowed.

Karthik

Re: [PATCH] LSP support in org-src buffers

[Karthik Chikmagalur]

> This is not limited to Eglot support. M-x compile, eglot, project.el,
> xrefs, and similar tools all assume that current code buffer is
> associated with a real file in a real project folder, possibly
> containing all kinds of hints like .gitignore, .dir-locals.el, etc.

I hadn't considered this.  This makes context-aware org-src support even more 
important.

> This sounds a bit fragile and full of caveats.
>
> You already implemented a way to associate the org-edit-src buffer with
> the fully tangled code. Then, why not make it simple and do the real
> tangling first and then make org-edit-src work directly with a real
> file buffer associated with the tangled file?
>
> All the tools, including Eglot will then work naturally and as intended.

This will drastically simplify the patch, true.  I was working on the 
assumption that since tangling overwrites the file on disk, it should not be an 
implicit operation invoked as a side-effect of another action.  It causes other 
changes that the user might not have intended, like updating timestamps on the 
tangled file, etc.  What do you suggest?

Moreover, for Eglot to function correctly it is sufficient to (i) associate the 
buffer with a file -- any file, and (ii) Set the default-directory variable to 
the correct value.   "Tangling" to a file in /tmp (as I do in the patch) will 
not work with all the non-Eglot use-cases you describe above.

> The only tricky problem I am seeing with your approach is dealing with
> noweb references. Care should be taken about editing code blocks
> containing noweb.

If I reuse the actual tangling machinery in ob-tangle instead of writing my own 
version reusing only some of the primitives in this library, this should be 
handled automatically for me.  Is this correct?

> Our to-be-released main branch does tangling orders of magnitude faster.
> For example, my config.org with 660 src blocks tangles within 1.3 sec.
>
> This can be made even faster by extra caching if there is a real demand
> for super-fast tangling.

That's a giant improvement over the current implementation. I'll keep this in 
mind when working on the patch.

Also: org-src-context-mode works by advising some org-edit-src-* functions.  Is 
it preferable to edit these functions directly instead and add a check for 
whether org-src-context-mode is enabled?

Karthik

[PATCH] LSP support in org-src buffers

[Karthik Chikmagalur]

Hi folks,

I've added limited support for LSP via Eglot in org-src-mode buffers.  I was 
intending to publish it as a package but it was suggested to me that it could 
live as part of Org instead, especially now that Eglot is intended to be part 
of the upcoming Emacs release.  Here are some details:

I.   What it does.
II.  How to use it.
III. How it works.
IV.  Limitations and concerns.

I. What it does
===

It allows you to run Eglot in org-src buffers opened with org-edit-special (C-c 
'), giving you context-aware completion, linting, code actions etc.

II. How to use it


(i) Turn on org-src-context-mode.
(ii) Add :tangle header args to code blocks that you intend the Language Server 
(LS) to parse sequentially, as one file.  To parse all code blocks in the Org 
document using the LS as one file, you can set a global ":tangle yes" header 
argument. (You don't have to tangle anything -- this is to identify which code 
blocks constitute a "file".)
(iii) Use org-edit-special on a code block.
(iv) Run eglot or eglot-connect in the org-src-mode buffer.

>From this point on, you can use org-edit-special and org-edit-src-exit (both 
>bound by default to C-c ') as usual, and you should have LSP support via 
>Eglot.  This LSP connection is persistent across this Emacs session, you can 
>exit the org-src buffer or kill the Org buffer, and come back to it at will.

(v) You can shutdown the LSP connection with eglot-shutdown, as usual.

III. How it works


The main problem with reconciling org-src-mode and Language Server (LS) support 
is that the LSP requires and expects files, not buffers.  By default, org-src 
buffers are not associated with files.  Even if one were to associate an 
org-src-mode buffer with a file, set the correct default-directory for a 
project and start Eglot, it would only contain the small chunk of code from the 
current code block.  The LS cannot access enough code to form a useful picture 
of the project.

org-src-context-mode reuses the tangling machinery to populate an org-src 
buffer with code from all blocks associated with the current tangle file, and 
associates it with a temporary file.  This way, the LS has a better picture -- 
if still limited to a single file -- of the project.

org-src-context-mode then checks if there's an appropriate Eglot LSP connection 
active, and reconnects to it.

Only the contents of the current code block are editable.  The other blocks are 
marked read-only and (by default) only visible by widening the buffer.  No 
actual tangling is done -- the default-directory of the org file is not touched 
at all.

If there's no :tangle header arg, org-src-context-mode does nothing.

IV. Limitations and concerns
=

(i) This creates temporary files with (ostensibly) the contents of code blocks 
in the Org file.

(ii) org-src-context-mode is implemented by advising org-edit-src-code and 
org-edit-src-exit. This is because it was originally intended to be a 
third-party package. These functions will need to be modified a bit otherwise.

(iii) I'm assuming this design will go through revisions, so I haven't updated 
the Org manual yet. (I did update the changelog.)

(iv) It is quite straightforward to add lsp-mode support with a user-option. 
(The LSP-specific part of this package is tiny.) Since lsp-mode is not part of 
Emacs and Eglot will be soon, I decided to focus on Eglot support.

(v) org-src-context-mode does some pseudo-tangling -- this is required to 
specify what constitutes a "file" for the LS to parse. This adds a performance 
penalty to org-edit-src-code that can be noticeable if you have many (100+?) 
code blocks with the same tangle file as the current block.

(vi) Consider this scenario: The code for your entire project resides in one or 
more Org files, and is intended to be tangled to several files under a project 
root directory.  Then the nature of LSP support depends on the state of 
tangling.
- Before tangling anything: LSP support with org-src-context-mode remains 
limited since it can only "see" one file, the one being edited.
- Post-tangling the entire project: You have full and veracious LSP support in 
all org-src buffers.
- Post-tangling and after edits to multiple code blocks: LSP support is now 
*incorrect* since it "sees" a combination of the current state of the "file" 
being edited in the org-src buffer, and the past state of tangled versions of 
other code blocks.

Still, I've found this to be a big improvement over having no LSP support for 
Org code blocks.

Please let me know if you have any feedback.

Karthik
>From 2798a292d293f1d0aeed34bd0014c6bb97079491 Mon Sep 17 00:00:00 2001
From: Karthik Chikmagalur 
Date: Fri, 7 Oct 2022 21:14:42 -0700
Subject: [PATCH] Add org-src-context.el

---
 etc/ORG-NEWS|  27 ++
 lisp/org-src-context.el | 186 ++

Bug: Org-babel output malformed for MATLAB source blocks running in a session [9.3 (release_9.3 @ /usr/share/emacs/27.1/lisp/org/)]

[Karthik Chikmagalur]

Remember to cover the basics, that is, what you expected to happen and
what in fact did happen.  You don't know how to make a good report?  See

 https://orgmode.org/manual/Feedback.html#Feedback

Your bug report will be posted to the Org mailing list.

Org-babel's handling of MATLAB block output with the =:results session= 
argument is broken. Here is some sample code along with the expected result:

#+begin_src matlab :results output :session *MATLAB*
a = 3;
b = 4;
c = a + b
#+end_src

#+RESULTS:
: c =
: 
:  7

However here is the actual result:

#+begin_src matlab :results output :session *MATLAB*
a = 3;
b = 4;
c = a + b
#+end_src

#+RESULTS:
#+begin_example
a = 3;
b = 4;
c = a + b

c =

 7
'org_babel_eoe'

ans =

'org_babel_eoe'
#+end_example

There are two separate problems:

1. The =org-babel-octave-eoe-indicator= is not being stripped.
2. The comint input is being echoed in the output.

#1 is easy to fix. The problem is in the function 
=org-babel-octave-evaluate-session=, in the line:

 #+begin_src emacs-lisp
 (cdr (reverse (delq "" (mapcar #'org-strip-quotes
(mapcar #'org-trim raw)
 #+end_src

Where empty lines in the output are being removed with =delq=. =delq= compares 
with =eq= instead of =equal=, which fails on blank lines. Replacing this with 
=delete= works fine.

#2 is a much trickier problem to solve. Here is the body of the input as it 
appears in the MATLAB session:

#+begin_example
>> a = 3;
b = 4;
c = a + b
'org_babel_eoe'
a = 3;
>> b = 4;
>> c = a + b

c =

 7

>> 'org_babel_eoe'

ans =

'org_babel_eoe'

>> 
#+end_example

Here `>>' is the shell prompt, which is absent from the raw comint output as 
given to org by comint-mode. =matlab-shell= does not work like other comint 
shells in that it doesn't echo bulk (/i.e./ multi-line) input all at once. 
Instead, it echoes each line of input and follows it with its output. This 
interspersal of input and output is causing =org-babel-comint-with-output=, the 
function (actually macro) responsible for removing the comint input echo text 
from the raw comint output, to fail. This macro assumes that the raw comint 
output looks like 
#+begin_example
<>

<>
#+end_example

as you can see from this code from =org-babel-comint-with-output=:
#+begin_src emacs-lisp
 (when (and ,remove-echo ,full-body
(string-match
 (replace-regexp-in-string
  "\n" "[\r\n]+" (regexp-quote (or ,full-body "")))
 string-buffer))
   (setq string-buffer (substring string-buffer (match-end 0
#+end_src

To fix this, either =org-babel-octave-evaluate-session= or 
=org-babel-comint-with-output= needs to be modified. This problem is local to 
MATLAB, it does not happen with GNU Octave, which shares most of its org-babel 
code with MATLAB's. So I wrote a patch to the former that does additional 
line-by-line processing on the raw comint output to detect and remove the 
echoed input. (Patch is attached.) However while this fixes the problem it's 
not a robust solution.

-Karthik

--- ob-octave.el	2020-11-23 11:22:01.473682045 -0800
+++ ob-octave-new.el	2020-11-23 11:10:07.961900383 -0800
@@ -187,6 +187,7 @@
 			(org-babel-process-file-name tmp-file 'noquote)))
 	   (org-babel-octave-import-elisp-from-file tmp-file))
 
+
 (defun org-babel-octave-evaluate-session
 (session body result-type  matlabp)
   "Evaluate BODY in SESSION."
@@ -237,12 +238,31 @@
   (`output
(setq results
 	 (if matlabp
-		 (cdr (reverse (delq "" (mapcar #'org-strip-quotes
+		 (cdr (reverse (delete "" (mapcar #'org-strip-quotes
 		(mapcar #'org-trim raw)
 	   (cdr (member org-babel-octave-eoe-output
 			(reverse (mapcar #'org-strip-quotes
 	 (mapcar #'org-trim raw)))
-   (mapconcat #'identity (reverse results) "\n")
+   ;; This kludge is to remove the input lines from the output. Because of
+   ;; the special way that MATLAB processes bulk comint output (the output
+   ;; of each line follows that line) the macro
+   ;; `org-babel-comint-with-output' cannot remove the echoed commands. The
+   ;; following handles this manually, by splitting both the original input
+   ;; (`BODY') and full output (`RESULTS') on newlines, comparing them line
+   ;; by line and removing all lines in BODY from RESULTS. Note that RESULTS
+   ;; is already a list of strings so additional care is needed.
+   (let* ((body-lines (split-string body "\n+"))
+  (result-lines (flatten-list
+ (mapcar
+  (lambda (entry) (reverse (split-string entry "\n")))
+  results
+ (mapconcat
+  #'identity
+  (reverse (cl-remove-if
+(lambda (line) (member line 

Org-babel Matlab (session) output is malformed

[Karthik Chikmagalur]

I'm trying to evaluate some Matlab code using org-babel in a session:

#+BEGIN_SRC matlab :session :results output
a = 4;
b = a + 3
c = a - 3
#+END_SRC

And following the org-babel documentation, (something like) this is the output 
I expect:

#+RESULTS:
#+begin_example
b =

 7

c =

 1
#+end_example

But this is the output I get:

#+RESULTS:
#+begin_example
a = 4;
b = a + 3

b =

 7
c = a - 3

c =

 1
'org_babel_eoe'

ans =

'org_babel_eoe'
#+end_example

(Including the =#+begin_example= etc)

After going through the source of the function =org-babel= calls, here is 
what's supposed to happen:

1. =org-babel-octave-evaluate-session= copies the code into the Matlab REPL 
(calling =matlab-shell= if necessary) and the shell runs it.
2. It then copies the result back as a string up to the end-of-evaluation 
token, and then removes the original code from the output string.

it looks like =org-babel-octave-evaluate-session= is not reading the 
end-of-evaluation token (='org_babel_eoe'=) correctly, and not removing the 
input code from the resulting string either. This looks like it should be an 
easy fix, but I couldn't figure it out.

Any ideas how to fix this?

Notes:

1. The relevant function(s) are in 
=/usr/share/emacs/27.0.90/lisp/org/ob-octave.el= or equivalent location. I am 
using Emacs 27.0.90.

2. The same org-babel module pulls double duty for Octave and Matlab with a 
flag to distinguish between uses, hence the name. When I run the same test as 
an Octave source block it runs correctly:
   
#+BEGIN_SRC octave :session :results output
a = 4;
b = a + 3
c = a - 3
#+END_SRC

#+RESULTS:
: 
: b =  7
: c =  1

3. This issue regards evaluation of source blocks with the =:results output= 
option, where org-babel includes in the results block everything that the REPL 
prints to stdout. (This is different from the default =:results value= option 
where only the return value of the block is considered as a result.)


- Karthik