Re: suggestion to change default org-latex-pdf-process to latexmk

[Colin Baxter]

> Stefan Nobis  writes:

> Colin Baxter  writes:
>> I really don't understand why the changed is needed.

> The upcoming citation support needs a reliable way to run the
> required bibliography processor (e.g. biber or bibtex). Therefore
> the current default setting that only runs latex three times does
> not suffice.

> On the other hand running biber unconditionally will result in
> errors if no citation and bibliography has been used in the
> document.

> So we need a more sophisticated method to run the LaTeX engine and
> associated tools. One way would be to use something that tries to
> do exactly this: Depending on the LaTeX document and/or the log
> file decide what's left to do to get a properly set result. With
> latexmk it is also easier to handle generation of index or
> glossaries files etc.

> Or we try to re-build something like this in Elisp (which would be
> nice but quite a lot of work; even if we utilize the lessons
> learned from latexmk).

Thank you for this useful explanation. I should have read the whole
thread, but I'm lazy.

I'm old school and use bibtex, and, before I adopted latexmk as a
solution, I remember setting org-latex-pdf-process to run multiple
times (5+) to cope with my setup of multiple bib files and multiple
layers of cross-referencing and citing within the bib files. It didn't
always work so I understand fully the motive behind using latexmk. 

Thanks again - and to others for explaining the thinking.

Best wishes, Colin.

Re: A formal grammar for Org

[David Masterson]

Jakob Schöttl  writes:

> Am 02.06.21 um 06:00 schrieb David Masterson:
>> Jakob Schöttl  writes:
>>
>>> Am 01.06.21 um 11:53 schrieb Tom Gillespie:
> We have a pretty similar project, org-parser[1]. It's also written
> in a Lisp dialect, Clojure, but it uses instaparse instead of brag
> as parser library.
 https://github.com/tgbugs/laundry/tree/next#similar-projects I managed
 to get it into my README as a reminder to myself to have a thorough
 look at it, but have been occupied with other work since then.
>>> Thanks, I'll also set a link in our README to related work.
>> Have either (or both) of you looked at BeOrg (http://beorg.app)?  This
>> is an (iOS) app that implements task management from Org files by
>> reading and updating the Org file structure.  I would assume it uses a
>> parser to breakdown the Org file structure and rebuild it later.  That
>> is what I see your parsers becoming.
> I haven't tried BeOrg myself, but it's proprietary and we have an open
> source, platform-independent alternative with Organice. See also 
> https://github.com/200ok-ch/organice#beorg
>
> org-parser is also open source and will finally replace Organice's
> somewhat hacky Parser as a library.

Nice (no pun intended)

-- 
David Masterson

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

Greg Minshall  writes:

> Bruce,
>
>> > I really don't understand why the changed is needed.
>> 
>> I've stated it twice, I believe, but it's now a longer thread, so:
>> 
>> Org is set to get native citation support.
>> 
>> Once that's merged, people who use this feature will be disappointed
>> to know that when using the natbib or biblatex export processors,
>> their citations and bibliographies will not render.
>> 
>> As in, from their perspective, it will be broken upfront.
>
> thanks.
>
> one important thing will be the manual section describing the new
> capability -- it should highlight the desirability of using latexmk ("or
> some process with similar behavior"), and describe the results one will
> get without it.  presumably, people will initially need to read this
> section to figure out the syntax, etc.?
>
> i worry about breaking people's currently working setups.
>
> it seems there are N classes of users:
>
> 1.  not using citation engine.
> 2.  using citation engine
> 2a. ... and using default latex-to-pdf processing
> 2b. ... having customized that processing
> 2bi. ... in a way that works with new citation engine
> 2bii. .. in a way that does *not* work with new citation engine
>
> class 1 could likely benefit from latexmk, but it's optional.
>
> class 2a is probably the main class that would find it really helpful to
> have an automated latexmk.
>
> from *our* (the org-mode provider) point of view, i guess the 2b
> subclasses are indistinguishable.  so, if the user finds herself in
> 2bii, she will have to grep the manual or the web.  again, it would be
> good if information in the manual stuck out.  maybe a "TROUBLESHOOTING"
> sub-section for the new feature?
>
> i wonder if we could detect 2a, and offer them the customization dialog?
> (one would want the user to be able to click the "do not show me this
> again" button, or equivalent.)
>
> packaging.  :)
>
> cheers, Greg

My suggestion is to do something like


(defvar org-latex-pdf-process (if (executable-find "latexmk")
 '("latexmk -f -pdf -%latex 
-interaction=nonstopmode -output-directory=%o %f")
   '("%latex -interaction nonstopmode 
-output-directory %o %f"
 "%latex -interaction nonstopmode 
-output-directory %o %f"
 "%latex -interaction nonstopmode 
-output-directory %o %f")))

This is a simplistic example. It would need some refinement to work
correctly with custom, but the principal is the same. With the above,
the default would be latexmk if latexmk is installed and on the
exec-path, otherwise it will be the default latex.

We can make this as sophisticated as we want. For example, we could also
check to verify the 'default' latex is available and issue a warning to
the user if it isn't or issue a warning if none of the options are
available etc.

I believe something like the above would have the least impact as

1. It would have no impact on anyone who already sets a preference for
org-latex-pdf-process
2. Would provide the advantages of latexmk for those who already have it
installed or once they install it (assuming no preference has been set
by the user)
3. Maintain the status quo for existing users who don't have latexmk
installed, avoiding breakage of their current setup.

Note that this is assuming that latexmk has no other 'side effects' not
raised in this thread. I don't use latexmk, so don't know precisely what
impact it would have, but based on posts from others, it would appear to
be fine. 
-- 
Tim Cross

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

"Dr. Arne Babenhauserheide"  writes:

> [[PGP Signed Part:Undecided]]
>
> Scott Randby  writes:
>
>> On 6/2/21 8:07 AM, Bruce D'Arcus wrote:
>>> On Wed, Jun 2, 2021 at 4:28 AM Tim Cross  wrote:
>>> 
 The more I think about it, I think the best solution would be to update
 the code which sets the default and have it check for latexmk. If it is
 found, set it as the default and if it is not found, set the existing
 default. This would have minimum impact on users and allow those who
 have installed latexmk to get the benefits while not forcing those who
 don't need it to install it.
>>> If this were possible, I agree it would be the best approach.
>>
>> Perhaps this is a stupid question, but what about the case in which TeX is
>> installed after Emacs is installed? I do this sometimes.
>
> Is there a reason not to have the default check for availability when
> run? It should be fast and would progressively improve (one step) when
> the user installs latexmk.
>

Yes, because some people want to set specific programs. If you have code
which checks for things at the point of being called, this becomes more
complicated than necessary. If on the other hand you just check when
first initialising, users who want to can override with a simple setq.
It is also more efficient as the check is only done once at load time
rather than every time you export to pdf.

It should also be noted that the custom framework has good support for
doing it this way. You can even add code which will check values entered
by the user to verify they exist (for example and assuming they use
custom to set a new value and not just a setq in their init file). 

-- 
Tim Cross

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

Scott Randby  writes:

> On 6/2/21 8:07 AM, Bruce D'Arcus wrote:
>> On Wed, Jun 2, 2021 at 4:28 AM Tim Cross  wrote:
>> 
>>> The more I think about it, I think the best solution would be to update
>>> the code which sets the default and have it check for latexmk. If it is
>>> found, set it as the default and if it is not found, set the existing
>>> default. This would have minimum impact on users and allow those who
>>> have installed latexmk to get the benefits while not forcing those who
>>> don't need it to install it.
>> If this were possible, I agree it would be the best approach.
>
> Perhaps this is a stupid question, but what about the case in which TeX is
> installed after Emacs is installed? I do this sometimes.
>

No impact. The code is initialised each time you load org. If you start
Emacs and then install TeX/LaTeX (including latexmk), you would have the 
existing default
until the next time you start Emacs, at which time the default will be
set to latexmk. 

This 'pattern' of setting a value based on code which selects an
appropriate value based on what is on the system is already used
extensively in Emacs. 

-- 
Tim Cross

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

Colin Baxter  writes:

>> Bruce D'Arcus  writes:
>
> > On Wed, Jun 2, 2021 at 4:28 AM Tim Cross  wrote:
> >> The more I think about it, I think the best solution would be to
> >> update the code which sets the default and have it check for
> >> latexmk. If it is found, set it as the default and if it is not
> >> found, set the existing default. This would have minimum impact
> >> on users and allow those who have installed latexmk to get the
> >> benefits while not forcing those who don't need it to install it.
>
> > If this were possible, I agree it would be the best approach.
>
> > Bruce
>
> What effect is this likely to have on people who have specified latexmk?
> For example, I have
>
> #+begin_src elisp
> (setq org-latex-pdf-process '("latexmk -outdir=%o -f %f"))
> (setq org-latex-compiler "pdflatex")
> #+end_src
>
> I assume any change will have no effect, but is this correct?
>

Correct. Changing any default setting has no impact on anyone who sets
the value to a non-default value.

If the default is changed to be latexmk if latexmk is installed and set
it to the existing default if not will only impact users who have
latexmk installed, but have not configured org-latex-pdf-process to use
it. For those who do not have it installed, it would have no effect.

If on the other hand we just change the default to be latexmk, it will
break org for anyone who does not have latexmk installed and who have
not set org-latex-pdf-program until they install latexmk. For new users,
it means needing to install latexmk (an optional package on some systems
and already installed on others).

As you can call elisp to initialise a variable, it would not be
difficult to add initialising code for org-latex-pdf-program which first
checks to see if latexmk is in the exec patch and if it is, set
org-latex-pdf-program accordingly and if it is not, set the current
default. I think this would have the least impact on all users.



-- 
Tim Cross

Re: suggestion to change default org-latex-pdf-process to latexmk

[Greg Minshall]

Bruce,

> > I really don't understand why the changed is needed.
> 
> I've stated it twice, I believe, but it's now a longer thread, so:
> 
> Org is set to get native citation support.
> 
> Once that's merged, people who use this feature will be disappointed
> to know that when using the natbib or biblatex export processors,
> their citations and bibliographies will not render.
> 
> As in, from their perspective, it will be broken upfront.

thanks.

one important thing will be the manual section describing the new
capability -- it should highlight the desirability of using latexmk ("or
some process with similar behavior"), and describe the results one will
get without it.  presumably, people will initially need to read this
section to figure out the syntax, etc.?

i worry about breaking people's currently working setups.

it seems there are N classes of users:

1.  not using citation engine.
2.  using citation engine
2a. ... and using default latex-to-pdf processing
2b. ... having customized that processing
2bi. ... in a way that works with new citation engine
2bii. .. in a way that does *not* work with new citation engine

class 1 could likely benefit from latexmk, but it's optional.

class 2a is probably the main class that would find it really helpful to
have an automated latexmk.

from *our* (the org-mode provider) point of view, i guess the 2b
subclasses are indistinguishable.  so, if the user finds herself in
2bii, she will have to grep the manual or the web.  again, it would be
good if information in the manual stuck out.  maybe a "TROUBLESHOOTING"
sub-section for the new feature?

i wonder if we could detect 2a, and offer them the customization dialog?
(one would want the user to be able to click the "do not show me this
again" button, or equivalent.)

packaging.  :)

cheers, Greg

Re: LaTeX fragment preview weirdness

[Scott Randby]

On 6/2/21 1:28 PM, Nick Dokos wrote:

You probably want #+LATEX_HEADER_EXTRA, not #+LATEX_HEADER. Do

(info "(org) Latex header and sectioning")

where it says:

#+BEGIN_QUOTE
The LaTeX export back-end appends values from ‘LATEX_HEADER’ and
‘LATEX_HEADER_EXTRA’ keywords to the LaTeX header.  The docstring for
‘org-latex-classes’ explains in more detail.  Also note that LaTeX
export back-end does not append ‘LATEX_HEADER_EXTRA’ to the header when
previewing LaTeX snippets (see *note Previewing LaTeX fragments::).
#+END_QUOTE



That worked after I deleted the old images in the ltximg directory. I hardly 
ever preview LaTeX fragments, but I'm changing #+latex_header: to 
#+latex_header_extra: everywhere.

Thanks,
Scott

Re: suggestion to change default org-latex-pdf-process to latexmk

[Scott Randby]

On 6/2/21 12:07 PM, Stefan Nobis wrote> I have


--8<---cut here---start->8---
(setq org-latex-pdf-process '("latexmk -f -pdf -%latex -outdir=%o %f"))
--8<---cut here---end--->8---



I could not get that setting to work. The only setting that worked for me was 
this one:

--8<---cut here---start->8---
(customize-set-variable 'org-latex-pdf-process '("latexmk -f -pdf -%latex 
-interaction=nonstopmode -output-directory=%o %f"))
--8<---cut here---end--->8---

I've added the above code to my configuration file.

Scott

Re: LaTeX fragment preview weirdness

[Nick Dokos]

Scott Randby  writes:

> First, I'm using Emacs 27.2 and Org 9.4.6.
>
> I have the following code in an Org file:
>
> #+latex_header: 
> \hypersetup{pdfauthor={Author},pdftitle={Title},pdfsubject={Subject},pdfkeywords={keyword},pdfproducer={Emacs
>  Org},pdfcreator={pdflatex}}
> #+latex_header: \hypersetup{colorlinks=true,urlcolor=blue,linkcolor=black}
>
> The above lines are in a section at the end of my document.
>
> When I do C-c C-x C-l in a section (or with a fragment at point), the 
> following appears above every  fragment that is previewed:
>
> pdfauthor=Author,pdftitle=Title,pdfsubject=Subject,pdfkeywords=keyword,pdfproducer=Emacs
>  Org,pdfcreator=pdflatex colorlinks=true,urlcolor=blue,linkcolor=black
>
> Here is a sample section from my Org file:
>
> --BEGIN--
>
> * Distance Formula
>
> Let $d$ be the distance between $(x_{1}, y_{1})$ and $(x_{2}, y_{2})$.
> \[
> d=\sqrt{\left(x_{1}-x_{2}\right)^{2}+\left(y_{1}-y_{2}\right)^{2}}
> \]
>
> --END--
>
>

You probably want #+LATEX_HEADER_EXTRA, not #+LATEX_HEADER. Do

   (info "(org) Latex header and sectioning")

where it says:

#+BEGIN_QUOTE
   The LaTeX export back-end appends values from ‘LATEX_HEADER’ and
‘LATEX_HEADER_EXTRA’ keywords to the LaTeX header.  The docstring for
‘org-latex-classes’ explains in more detail.  Also note that LaTeX
export back-end does not append ‘LATEX_HEADER_EXTRA’ to the header when
previewing LaTeX snippets (see *note Previewing LaTeX fragments::).
#+END_QUOTE

-- 
Nick

"There are only two hard problems in computer science: cache
invalidation, naming things, and off-by-one errors." -Martin Fowler

Re: suggestion to change default org-latex-pdf-process to latexmk

[Stefan Nobis]

Colin Baxter  writes:

> I really don't understand why the changed is needed.

The upcoming citation support needs a reliable way to run the required
bibliography processor (e.g. biber or bibtex). Therefore the current
default setting that only runs latex three times does not suffice.

On the other hand running biber unconditionally will result in errors
if no citation and bibliography has been used in the document.

So we need a more sophisticated method to run the LaTeX engine and
associated tools. One way would be to use something that tries to
do exactly this: Depending on the LaTeX document and/or the log file
decide what's left to do to get a properly set result. With latexmk it
is also easier to handle generation of index or glossaries files etc.

Or we try to re-build something like this in Elisp (which would be
nice but quite a lot of work; even if we utilize the lessons learned
from latexmk).

-- 
Until the next mail...,
Stefan.

LaTeX fragment preview weirdness

[Scott Randby]

First, I'm using Emacs 27.2 and Org 9.4.6.

I have the following code in an Org file:

#+latex_header: 
\hypersetup{pdfauthor={Author},pdftitle={Title},pdfsubject={Subject},pdfkeywords={keyword},pdfproducer={Emacs
 Org},pdfcreator={pdflatex}}
#+latex_header: \hypersetup{colorlinks=true,urlcolor=blue,linkcolor=black}

The above lines are in a section at the end of my document.

When I do C-c C-x C-l in a section (or with a fragment at point), the following 
appears above every  fragment that is previewed:

pdfauthor=Author,pdftitle=Title,pdfsubject=Subject,pdfkeywords=keyword,pdfproducer=Emacs
 Org,pdfcreator=pdflatex colorlinks=true,urlcolor=blue,linkcolor=black

Here is a sample section from my Org file:

--BEGIN--

* Distance Formula

Let $d$ be the distance between $(x_{1}, y_{1})$ and $(x_{2}, y_{2})$.
\[
d=\sqrt{\left(x_{1}-x_{2}\right)^{2}+\left(y_{1}-y_{2}\right)^{2}}
\]

--END--

Scott Randby

Re: suggestion to change default org-latex-pdf-process to latexmk

[Bruce D'Arcus]

On Wed, Jun 2, 2021 at 12:30 PM Colin Baxter  wrote:
>
>
> I'm beginning to think this change of the default should not
> happen. From this thread it seems to be that the chances of it breaking
> existing work flow is high.

I don't see how that follows. Can you clarify?

> I really don't understand why the changed is needed.

I've stated it twice, I believe, but it's now a longer thread, so:

Org is set to get native citation support.

Once that's merged, people who use this feature will be disappointed
to know that when using the natbib or biblatex export processors,
their citations and bibliographies will not render.

As in, from their perspective, it will be broken upfront.

Bruce

Re: bug#47885: [PATCH] org-table-import: Make it more smarter for interactive use

[Maxim Nikulin]

On 02/06/2021 22:08, Utkarsh Singh wrote:

;;;###autoload
  (defun org-table-import (file separator)
@@ -955,12 +971,13 @@ lines.  It can have the following values:
  - integer When a number, use that many spaces, or a TAB, as field separator.
  - regexp  When a regular expression, use it to match the separator."
(interactive (list (read-file-name "Import file: ")
- (prefix-numeric-value current-prefix-arg)))
+ current-prefix-arg))


It seems, prefix argument works now. Let me remind that file name 
completion was working better before your change.


I have noticed a couple of error messages. Unsure what is going wrong, I 
hope, command to launch emacs is correct (-Q -L ~/src/org-mode/lisp 
test.org).


At startup:
Eager macro-expansion failure: (error "rx ‘not’ syntax error: (or 10 44)")

In response to M-x org-table-import:
rx-check-not: rx ‘not’ syntax error: (or 10 44)


Currently I am trying to refactor CSV parsing by applying techniques
used in pcsv library (MELPA package) which I think you will also enjoy
to play with!


I do not know opinion of Org maintainers. Personally I believe that org 
can take advantage of Emacs core features or of other packages if they 
are available and fallback to minimal implementation otherwise. Unsure 
whether borrowing code from pcsv can cause license issues. Current CSV 
parser is not perfect but it works reasonably well.

Re: suggestion to change default org-latex-pdf-process to latexmk

[Robert Pluim]

> On Wed, 02 Jun 2021 17:29:49 +0100, Colin Baxter  said:

Colin> I'm beginning to think this change of the default should not
Colin> happen. From this thread it seems to be that the chances of it 
breaking
Colin> existing work flow is high.

Colin> I really don't understand why the changed is needed.

It sounds like a better idea is to update the defcustom to offer
latexmk as an option, with copious documentation as to when it
could/should be used.

Robert
-- 

Re: suggestion to change default org-latex-pdf-process to latexmk

[Colin Baxter]

I'm beginning to think this change of the default should not
happen. From this thread it seems to be that the chances of it breaking
existing work flow is high.

I really don't understand why the changed is needed.

Best wishes,

Formatting content in a footnote for ox-tufte

[Galaxy Being]

I'm using the org tufte  ox-tufte and
started tweaking the underlying tufte.css
to my needs. It's really meant
for org html export, not like the more elaborate org-to-latex Tufte
formatting from org mode luminary Thomas Dye. As such, it gives you the
Tufte look-and-feel with fonts and the famed sidenote/margin note feature
-- all for just making a footnote (that appears beside its link in the text
rather than the usual at the bottom of the page). The margin note
capability is gnarlier in that it is just a link in the main text that then
appears to the right of the main text

[[mn:1][Here is some text that will appear as a side column block]][[mn:2][And
putting one right after the first one makes it look like a new paragraph]]

Obviously, this is a limiting factor on how elaborate and expressive these
side columns can be, especially the margin note which has your content
inside the link. However, the sidenote as a redirected footnote isn't too
bad. Still, I'd like to be able to fully format what goes into a footnote
without jumping through a lot of hoops. Here's an example of one footnote
as sidenote that comes out looking okay despite the kludginess

[fn:1] Star chart for \\
\\
file:./images//Celestial_map_Hyi.png \\
\\
Just text for a few lines of text that is just meant as filler, not
anything important. Just here to show how text looks in a
sidenote. Cool, I think this is enough.\\
\\
~foldr :: (a → b → b) → b → [a] → b~ \\
~foldr f e [ ] = e~\\
~foldr f e (x : xs) = f x (foldr f e xs)~ \\
\begin{align*}
v = \frac{x}{t}
\end{align*}
[[
https://learning.edx.org/course/course-v1:EPFLx+MatlabeOctaveBeginnersX+1T2017/home][MatLab
and Octave training]]

For one, I couldn't get any sort of normal org block formatting (e.g.,
#+BEGIN_example...) for the code snippet to be recognized. Is there any
sort of "normal" content formatting for org footnotes? Obviously, the worst
is the margin note [[mn:... situation. That simply needs to be redone. But
a margin note differs from a sidenote only because the margin note is not
numbered. I tried the org-footnote-auto-label set to confirm, but it always
reverts back to plain numbering.

Any ideas for improvement appreciated. This would be a really good-looking
org html export option if it was bit more user-friendly.

⨽
Lawrence Bottorff
Grand Marais, MN, USA
borg...@gmail.com

Re: suggestion to change default org-latex-pdf-process to latexmk

[Dr. Arne Babenhauserheide]

Scott Randby  writes:

> On 6/2/21 8:07 AM, Bruce D'Arcus wrote:
>> On Wed, Jun 2, 2021 at 4:28 AM Tim Cross  wrote:
>> 
>>> The more I think about it, I think the best solution would be to update
>>> the code which sets the default and have it check for latexmk. If it is
>>> found, set it as the default and if it is not found, set the existing
>>> default. This would have minimum impact on users and allow those who
>>> have installed latexmk to get the benefits while not forcing those who
>>> don't need it to install it.
>> If this were possible, I agree it would be the best approach.
>
> Perhaps this is a stupid question, but what about the case in which TeX is 
> installed after Emacs is installed? I do this sometimes.

Is there a reason not to have the default check for availability when
run? It should be fast and would progressively improve (one step) when
the user installs latexmk.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken


signature.asc
Description: PGP signature

Re: bug#12972: 24.3.50; Move `org-open-file' and associated code out of Org mode

[Maxim Nikulin]

On 01/06/2021 13:56, Lars Ingebrigtsen wrote:

So I've now added this to Emacs 28 under the name
`mailcap-view-file'.


I am sorry if it is a false alarm. Feel free to close the bug again if 
something changed recently in `start-process-shell-command' or if you 
prefer to discuss the issue as another bug.


It seems that implementation of `mailcap-view-file' is unreliable due to 
creation of unnecessary terminal session and it can cause obscure and

difficult to reproduce failures similar to
https://lists.gnu.org/archive/html/emacs-orgmode/2020-09/msg00195.html
https://lists.gnu.org/archive/html/emacs-orgmode/2020-06/msg00332.html
The thread is actually longer than it is shown in the archive interface.
Another lengthy discussion:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44824

In Org latest change was required for compatibility reason:
https://code.orgmode.org/bzg/org-mode/commit/869b7a21b94ed112f6640c8f2711c2a68b661dea

Let-bind (process-connection-type nil) is a minimal required change to 
avoid unnecessary terminal session. However it is not friendly to users 
in the case when troubleshooting is required. `make-process' with 
sentinel is hopefully better.


The following could be ignored since it likely requires significant 
amount of work with unclear benefits.


1. `org-open-file' besides Org-specific stuff allows to specify precise 
target inside the file. It can be quite useful, e.g.


okular --page 11 --find "some pattern" file.pdf

PDF files have internal anchors as well. I have no consistent vision how 
to express additional "locators" in general API.


2. There are at least two sources of truth for MIME-handlers on linux 
desktop that are not necessary synchronized. Info from extracted from 
.desktop files may be configurable from desktop UI unlike mailcap. 
Distros may have some instruments to mitigate discrepancies. Debian adds 
entries from .desktop handlers to system-wide mailcap DB. Another 
approach is to add to maicap greedy xdg-open handler that tries to guess 
currently running desktop and pass arguments to appropriate command.


Maybe mailcap should be secondary MIME database in Emacs, not the 
primary one.


3. Currently only file suffix is inspected to determine MIME type of a 
file. libmagic (or file command) usually provides more precise info, so 
it is possible to open an incorrectly named file.


4. Mailcap has more features that are not addressed in Emacs. They may 
be handy if Emacs is launched in terminal on remote server. It might 
allow e.g. to open PDF file using pdftotext handler.

- A buffer for command output should be created for "copiousoutput"  option.
- A buffer should be created and terminal session should be enabled if 
an entry "needsterminal".
- There are more substitutions than "%s". However I am unsure if it is 
possible to provide more info than application can obtain from the file.

I think, it is intended for mail multipart messages and additional headers.

On the other hand mailcap handlers might expect safe file names (minimal 
ASCII subset), users may have files with arbitrary names (national 
charset or some special characters). I hope, almost all handlers do not 
have such problem.


In summary, during launch of external command terminal session must be 
suppressed. There is enough room for MIME-related improvements in Emacs 
in general and in Org mode in particular.

Re: suggestion to change default org-latex-pdf-process to latexmk

[Stefan Nobis]

Scott Randby  writes:

> (1) Which one of the above suggestions should I use? I don't want
> interactions when I'm exporting. I don't want to have to mess with a
> configuration file either.

> (2) How many shell commands will org-latex-pdf-process need? Right
> now there are three. Replace those three with just one latexmk?

I have

--8<---cut here---start->8---
(setq org-latex-pdf-process '("latexmk -f -pdf -%latex -outdir=%o %f"))
--8<---cut here---end--->8---

in my Emacs configuration. The nice thing with latexmk is, that a
single call will suffice. It reads the generated log and checks
whether multiple calls to the LaTeX engine are necessary (after each
run it check whether another run is need, up-to the configurable
maximum of 5 runs). Therefore in some cases (depending on used LaTeX
packages and commands) latexmk may even produce the PDF faster, if
only one or two passes are necessary. But sometimes LaTeX thinks it
may need more passes (e.g. having undefined references, unstable
output due to dynamically generated things at page breaks for example
with cleveref and varioref where floats/references are put on one page
in one run and another page in the following run) - and in these cases
creating the PDF may take a bit longer due to the maximum of 5
(instead of only 3) runs.

Hope that helps a little bit.

-- 
Until the next mail...,
Stefan.

Re: suggestion to change default org-latex-pdf-process to latexmk

[Scott Randby]

On 6/2/21 8:07 AM, Bruce D'Arcus wrote:

On Wed, Jun 2, 2021 at 4:28 AM Tim Cross  wrote:


The more I think about it, I think the best solution would be to update
the code which sets the default and have it check for latexmk. If it is
found, set it as the default and if it is not found, set the existing
default. This would have minimum impact on users and allow those who
have installed latexmk to get the benefits while not forcing those who
don't need it to install it.


If this were possible, I agree it would be the best approach.


Perhaps this is a stupid question, but what about the case in which TeX is 
installed after Emacs is installed? I do this sometimes.

Scott Randby

Re: suggestion to change default org-latex-pdf-process to latexmk

[Scott Randby]

On 6/2/21 1:33 AM, Stefan Nobis wrote:

"Bruce D'Arcus"  writes:


Here's the current command for the "latexmk" option:



"latexmk -g -pdf -pdflatex=\"%latex\" -outdir=%o %f"



... and here's what you have, minus shell-escape.



"latexmk -f -pdf -%latex -interaction=nonstopmode -output-directory=%o %f"


The option "-interaction=nonstopmode" forbids user interaction (like
asking for a missing file). Without this explicit option latexmk uses
"-interaction=batchmode" by default - in this case some diagnotic
messages to the terminal (not log file) are suppressed. Thus the
default option from latexmk might suffice.

Regarding "-f" versus "-g" I personally would slightly prefer "-f" for
performance reasons. With "-g" the document will always be newly
generated, even if nothing changed. This is only useful if the latexmk
config (or some environment variable) has changed. Other changes
should be detectable by latexmk, so "-f" should suffice.

The options "-output-directory" and "-outdir" are synonyms.

Regarding the choise of engine: "-pdflatex=\"%latex\"" explicitly sets
the command (including options) to be called by latexmk, without
letting latexmk know that the engine changed. Using "-%latex" (e.g.
the option "-lualatex" or "-xelatex") lets latexmk know that a
different engine should be used.

This matters at least for xelatex, because latexmk can utilize some
optimizations if it knows that xetex is used. In this case the first
runs only produce .xdv files and only the last run will produce the
full pdf to save time (see also option "-pdfxe" which is triggered by
"-xelatex").

Therefore I would tend to use this call:

 "latexmk -f -pdf -%latex -outdir=%o %f"

All other aspects of latexmk should be configured via config file
(IMHO).


In any case, whatever the command is, clearly it should.




I'm no longer sure that a change would break my ability to export to PDF. I'm 
using Linux with TeX Live, and latexmk is available. I'm willing to try 
changing org-latex-pdf-process to one of the above suggestions, but I have some 
questions. I've never used latexmk, and I really depend on Org to set things up 
so that they just work out of the box. It took me a long time to figure out how 
to get Emacs to find TeX so that previewing LaTeX fragments works (the solution 
is not in the Org manual).

(1) Which one of the above suggestions should I use? I don't want interactions 
when I'm exporting. I don't want to have to mess with a configuration file 
either.

(2) How many shell commands will org-latex-pdf-process need? Right now there 
are three. Replace those three with just one latexmk?

Scott Randby

Re: Texinfo exporter fails to export M-x helm-documentation

[Nicolas Goaziou]

Hello,

Jonas Bernoulli  writes:

> You did that in f99f26306c57d2342069880eac4dca324d7579ec
> but I think you added a off-by-one error in the process.
> In
>
> (>= (org-export-get-relative-level h info)
> (length sections))
>
> the ">=" should be a ">", or it can match with 4 >= 4, which
> should result in valid menus, but is currently disallowed by
> this check.

Indeed! Fixed. Thank you!

Regards,
-- 
Nicolas Goaziou

Re: bug#47885: [PATCH] org-table-import: Make it more smarter for interactive use

[Utkarsh Singh]

On 2021-06-02, 19:06 +0700, Maxim Nikulin  wrote:

> Notice that for "C-u M-x org-table-import" heuristics is not necessary, 
> the separator is specified explicitly. I see that your intention was to 
> improve user interface of org-table-import, but actually you broke it by 
> the "org-table-import: add file prompt" patch. I have not tried it, but 
> my expectation is that user prompt can be customized while keeping all 
> other things working. Maybe the docstring should be updated to make it 
> more friendly to novices (like me while reviewing your patch).

Thanks for pointing out the error, locally you can fix it by (patches
for patches):

;;;###autoload
 (defun org-table-import (file separator)
@@ -955,12 +971,13 @@ lines.  It can have the following values:
 - integer When a number, use that many spaces, or a TAB, as field separator.
 - regexp  When a regular expression, use it to match the separator."
   (interactive (list (read-file-name "Import file: ")
- (prefix-numeric-value current-prefix-arg)))
+ current-prefix-arg))

Currently I am trying to refactor CSV parsing by applying techniques
used in pcsv library (MELPA package) which I think you will also enjoy
to play with!

-- 
Utkarsh Singh
http://utkarshsingh.xyz

Re: suggestion to change default org-latex-pdf-process to latexmk

[Colin Baxter]

> Bruce D'Arcus  writes:

> On Wed, Jun 2, 2021 at 4:28 AM Tim Cross  wrote:
>> The more I think about it, I think the best solution would be to
>> update the code which sets the default and have it check for
>> latexmk. If it is found, set it as the default and if it is not
>> found, set the existing default. This would have minimum impact
>> on users and allow those who have installed latexmk to get the
>> benefits while not forcing those who don't need it to install it.

> If this were possible, I agree it would be the best approach.

> Bruce

What effect is this likely to have on people who have specified latexmk?
For example, I have

#+begin_src elisp
(setq org-latex-pdf-process '("latexmk -outdir=%o -f %f"))
(setq org-latex-compiler "pdflatex")
#+end_src

I assume any change will have no effect, but is this correct?

Best wishes,

Colin.

Re: suggestion to change default org-latex-pdf-process to latexmk

[Bruce D'Arcus]

On Wed, Jun 2, 2021 at 4:28 AM Tim Cross  wrote:

> The more I think about it, I think the best solution would be to update
> the code which sets the default and have it check for latexmk. If it is
> found, set it as the default and if it is not found, set the existing
> default. This would have minimum impact on users and allow those who
> have installed latexmk to get the benefits while not forcing those who
> don't need it to install it.

If this were possible, I agree it would be the best approach.

Bruce

Re: bug#47885: [PATCH] org-table-import: Make it more smarter for interactive use

[Maxim Nikulin]

On 02/06/2021 00:46, Utkarsh Singh wrote:

Org 9.4.5+patches M-x org-table-import

| 1,Word,66.3e-35 ||  |   |
| 2,Unquoted  | cell,2.7   |  |   |
| 3,"Quoted   | cell",3.14 |  |   |
| 4,"Cell | ""with | quotes""",2021-06-01 |   |
| 5,"Next | cell   | is   | empty","" |
| 6,"Cell | with   | new  |   |
| Line",6.28  ||  |   |

So my personal conclusion is that CSV file is imported incorrectly in
both cases: with guessed separator and with explicitly requested through
prefix argument. Completion works a bit worse too.


Currently `org-table-guess-separator' returns "," (COMMA as string) and
`org-table-covert-region' uses '(4) to represent COMMA as separator
which is the main cause of breakdown in importing.

To make importing work well we have to:

+ Guess right separator (`org-table-guess-separator')
+ Parse CSV with this separator (`org-table-covert-region')

As far as I can tell "guessing part" works well and now we just have to
make parser work well with new separators.


Notice that for "C-u M-x org-table-import" heuristics is not necessary, 
the separator is specified explicitly. I see that your intention was to 
improve user interface of org-table-import, but actually you broke it by 
the "org-table-import: add file prompt" patch. I have not tried it, but 
my expectation is that user prompt can be customized while keeping all 
other things working. Maybe the docstring should be updated to make it 
more friendly to novices (like me while reviewing your patch).

Re: TMIO Pre-release, request for feedback

[Eric S Fraga]

On Wednesday,  2 Jun 2021 at 00:38, Timothy wrote:
> Eric S Fraga  writes:
>> I would find this quite useful as I hate typing : when entering diary
>> entries.  Where does this support exist?
>
> This is for date entry using `org-time-stamp' at the very least. I'm not
> sure if anything else is affected.

Ah, yes, it is there for inserting time stamps directly.  Thank you.  In
any case, it would be good if diary functions would also support this.

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-549-ga0a87d

Re: Format babel code block for export?

[Juan Manuel Macías]

Hi Lawrence,

You can use multiple blocks and noweb reference syntax
(https://orgmode.org/manual/Noweb-Reference-Syntax.html):

#+NAME:block1
#+begin_src haskell :results silent :exports none
:{
#+end_src

#+NAME:block2
#+begin_src haskell :results silent :exports none
maximum'' :: (Ord a) => [a] -> a
maximum'' = foldl1 (\x acc -> if x > acc then x else acc)
#+end_src

#+NAME:block3
#+begin_src haskell :results silent :exports none
:}
#+end_src

# for export

#+begin_src haskell :noweb yes :exports code
<>
#+end_src

# for evaluation

#+begin_src haskell :noweb yes :results silent :exports none
<>
<>
<>
#+end_src

Best regards,

Juan Manuel

Galaxy Being  writes:

> I have this
>
> #+begin_src haskell :results silent :exports code
> :{
> maximum'' :: (Ord a) => [a] -> a
> maximum'' = foldl1 (\x acc -> if x > acc then x else acc)
> :}
> #+end_src
>
> but on export to HTML (or LaTex) I'd like to suppress the :{ and :} to
> just show the code
>
> maximum'' :: (Ord a) => [a] -> a
> maximum'' = foldl1 (\x acc -> if x > acc then x else acc)
>
> Also, this
>
> #+begin_src haskell :results verbatim :exports both
> maximum'' [1,3,5,2,4]
> #+end_src
>
> I'd like to export the code part with a > REPL prompt included, e.g.,
>
>> maximum'' [1,3,5,2,4]
>
> I once saw some similar tricks, but I've searched and searched and
> can't find them again. There's probably other html export formatting
> I'd like too, but if anyone can point me to that mystery doc I'd
> appreciate it.
>
> ⨽
> Lawrence Bottorff
> Grand Marais, MN, USA
> borg...@gmail.com
>

Re: suggestion to change default org-latex-pdf-process to latexmk

[Juan Manuel Macías]

Stefan Nobis writes:

> An alternative may be to use latexmk only if citations are found (new
> feature, new dependencies). Or a wrapper that checks whether latexmk
> is available and works (e.g. trying to call "latexmk --version") and
> falls back to the old routine of manually running the engine and
> bibtex/biber if necessary.

Another possibility would be to add to the documentation all the
instructions needed on how to properly configure `org-latex-pdf-process'
with latexmk, for those users who wish to use bibtex or biblatex.
Although latexmk is also useful for many packages that need multiple
compilations, indexes, etc.

Best regards,

Juan Manuel 

Re: suggestion to change default org-latex-pdf-process to latexmk

[Tim Cross]

Stefan Nobis  writes:

> Tim Cross  writes:
>
>> One reason is that latexmk is not installed on some systems.
>
> Just after starting to write that this is false nowadays I realized
> you are right. :)
>
> See: https://mg.readthedocs.io/latexmk.html
>
> For macOS latexmk is distributed with the default TeX installation
> MacTeX for quite some years (IIRC at least since 2012). And it is
> working OOTB.
>
> As far as I know the default TeX installation for Windows is MikTeX
> (is this still true?), which also includes latexmk but lacks the Perl
> part (therefore Perl needs to be installed manually). But there is
> TeXLive for Windows and as far as I understand in this case Perl is
> also installed and latexmk works OOTB.
>
> A manual installation of TeXLive for Linux should also install latexmk
> (and Perl should also be available on next to every Linux box). Only
> some distributions bundle latexmk in a separate package - that should
> be easy to install (but breaks PDF creation if forgotten).
>
> Therefore: latexmk is available on all plattforms and in most cases it
> is already installed with the TeX system or easily installable. But
> not in all cases does it work OOTB and requires more work like
> installing Perl on Windows.
>
> In my opinion its worth to depend on this tool as it makes handling
> LaTeX documents much easier. On the other hand it may raise the bar
> for some users just to high. Hard to say.
>
> An alternative may be to use latexmk only if citations are found (new
> feature, new dependencies). Or a wrapper that checks whether latexmk
> is available and works (e.g. trying to call "latexmk --version") and
> falls back to the old routine of manually running the engine and
> bibtex/biber if necessary.

The more I think about it, I think the best solution would be to update
the code which sets the default and have it check for latexmk. If it is
found, set it as the default and if it is not found, set the existing
default. This would have minimum impact on users and allow those who
have installed latexmk to get the benefits while not forcing those who
don't need it to install it.


-- 
Tim Cross

Re: A formal grammar for Org

[Jakob Schöttl]

Am 02.06.21 um 06:00 schrieb David Masterson:

Jakob Schöttl  writes:


Am 01.06.21 um 11:53 schrieb Tom Gillespie:

We have a pretty similar project, org-parser[1]. It's also written
in a Lisp dialect, Clojure, but it uses instaparse instead of brag
as parser library.

https://github.com/tgbugs/laundry/tree/next#similar-projects I managed
to get it into my README as a reminder to myself to have a thorough
look at it, but have been occupied with other work since then.

Thanks, I'll also set a link in our README to related work.

Have either (or both) of you looked at BeOrg (http://beorg.app)?  This
is an (iOS) app that implements task management from Org files by
reading and updating the Org file structure.  I would assume it uses a
parser to breakdown the Org file structure and rebuild it later.  That
is what I see your parsers becoming.
I haven't tried BeOrg myself, but it's proprietary and we have an open 
source, platform-independent alternative with Organice. See also 
https://github.com/200ok-ch/organice#beorg


org-parser is also open source and will finally replace Organice's 
somewhat hacky Parser as a library.


Regards, Jakob

Re: [PATCH] org-mac-link.el: Change homepage.

[Bastien]

Hi Aimé,

Aimé Bertrand  writes:

> would like to move the homepage.

Applied, thanks.

-- 
 Bastien

Re: suggestion to change default org-latex-pdf-process to latexmk

[Loris Bennett]

Nick Dokos  writes:

> "Bruce D'Arcus"  writes:
>
>> While testing org-cite a few weeks ago, I noticed by default bibtex
>> won't run with natbib export, or biblatex with biblatex export.
>>
>> Since org has not had native citations, this is not surprising.
>>
>> But with org-cite coming soon, this will change.
>>
>> At the time I suggested changing default to use latexmk, which will
>> handle both bibtex and biblatex runs.
>>
>> https://lists.gnu.org/archive/html/emacs-orgmode/2021-05/msg00486.html
>>
>> So what do LaTeX users think about changing the default for
>> "org-latex-pdf-process" to "latexmk"?
>>
>> Is there any reason not to do this?
>>
>
> The only reason I can think of is that it might not be available by
> default, so it will be one more thing for a noob to install before Org mode
> starts "working".

Of course, LaTeX itself, with or without latexmk, is something the noob
will have to install.  So the prerequisite could be "LaTeX with
latexmk".  With regards to Windows, Cygwin has latexmk in the
'texlive-collection-binextra' package and MikTeX has a 'latexmf'
package.  On Debian latexmf is also its own package and thus has to be
installed explicitly.

Personally I use Debian and have "org-latex-pdf-process" set to use
latexmk, so the change in the default wouldn't affect me.

Cheers,

Loris

-- 
This signature is currently under construction.

Re: [ORG] Moving the org-mode.git repo to savannah.gnu.org

[Bastien]

Hi Kaushal,

Kaushal Modi  writes:

> I am seeing that the master branch on https://code.orgmode.org/bzg/
> org-mode/commits/master has newer commits than the main branch on 
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/log/ .

Fixed, thanks - I sync repositories regularily, but since we don't use
the savannah repo for now, it's not critical.

> I am assuming that the complete switch to Savannah is still pending?

Yes.  I plan to make the switch early July.

Thanks,

-- 
 Bastien

Re: suggestion to change default org-latex-pdf-process to latexmk

[Stefan Nobis]

Tim Cross  writes:

> One reason is that latexmk is not installed on some systems.

Just after starting to write that this is false nowadays I realized
you are right. :)

See: https://mg.readthedocs.io/latexmk.html

For macOS latexmk is distributed with the default TeX installation
MacTeX for quite some years (IIRC at least since 2012). And it is
working OOTB.

As far as I know the default TeX installation for Windows is MikTeX
(is this still true?), which also includes latexmk but lacks the Perl
part (therefore Perl needs to be installed manually). But there is
TeXLive for Windows and as far as I understand in this case Perl is
also installed and latexmk works OOTB.

A manual installation of TeXLive for Linux should also install latexmk
(and Perl should also be available on next to every Linux box). Only
some distributions bundle latexmk in a separate package - that should
be easy to install (but breaks PDF creation if forgotten).

Therefore: latexmk is available on all plattforms and in most cases it
is already installed with the TeX system or easily installable. But
not in all cases does it work OOTB and requires more work like
installing Perl on Windows.

In my opinion its worth to depend on this tool as it makes handling
LaTeX documents much easier. On the other hand it may raise the bar
for some users just to high. Hard to say.

An alternative may be to use latexmk only if citations are found (new
feature, new dependencies). Or a wrapper that checks whether latexmk
is available and works (e.g. trying to call "latexmk --version") and
falls back to the old routine of manually running the engine and
bibtex/biber if necessary.

-- 
Until the next mail...,
Stefan.