Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Keita

Le 25/05/2018 à 06:59, Ikumi Keita a écrit :

Hi Jean,


jfbu  writes:



(b.t.w. I also git cloned the savannah, and
https://git.savannah.gnu.org/cgit/auctex.git has a typo it should be
https://git.savannah.gnu.org/git/auctex.git
)


Ah, sorry.  The former URL is for usual web browsers, e.g. firefox.
That gives overview of the git repository, and more.  The git repository
address for the command "git" is listed on the page:
git://git.savannah.gnu.org/auctex.git
https://git.savannah.gnu.org/git/auctex.git
ssh://git.savannah.gnu.org:/srv/git/auctex.git

(the last one requires suitable account on savannah, I think)


my bad, of course that is what happened the link you gave brought
me to web page and there I spotted URL to use in git

(my volatile memory re-boots every 30 minutes)





My testing so far indicates no problem on Mac OS X 10.9.5



I was intrigued by these changes related to quotes, but it turns out
when I had tried manually on the command line that I was not quoting
correctly.



With your patch on top of 12.1.0 I successfully generate preview for a file
with both spaces and utf-8 characters.


Thank you for checking.  Now I can be more confident.


I did not check the ini file because it is volatile but I think you
indicated how to modify cache-preamble to make it permanent.


Though I didn't write the way explicitly, it's relatively easy.  Change
the last part of `preview-cache-preamble' in preview.el as:

 (preview-format-kill ',format-cons))
  ; (delete-file ',dump-file) ;; comment out
) ;; leave this close paren
   (error (preview-log-error err "Dumping" process)))
 (preview-reraise-error process)))




Thanks a lot, I will keep this for reference.


A proposito, the above in the pdflatex command (once the format
is done)means \input will be with extra spaces



\input{"test my látéx.tex"}



but fortunately that works fine.


These spaces are intentional so that w32 shell quoting style would not
interfere with other quote chars.  It seems that consequtive quote chars
are interpreted differently on w32.



Thanks for the precise info, I have no knowledge of Windows.




Anyway, all seems fine!


Marvelous!


TeX-region-create



and surely what happens is that Keita has a patch of the



\\message in there



but it was not included in the one I tested


Yes.  More precisely, the required fix is injected in
`TeX-quote-filename' rather than `TeX-region-create'.  On my side, the
given example runs without error.
If you want to try it out for yourself, extract the relavant part from
"tentative-patch2" attached in my previous message
https://lists.gnu.org/archive/html/auctex-devel/2018-05/msg00024.html
.  (But note that "tentative-patch2" as a whole has not been finished
yet and cannot typeset file name with space.)



Confirmed, it all works fine, no problem with generate preview at point, now.

Although I may have altered your work
unintentionally I am joining a patch which combines the two patches
of yesterday. It is set-up to be used directly in an 
.emacs.d/elpa/auctex-12.1.1/
repertory (and one byte-recompiles afterwards) for people who want
to mess their installs at their own risks.



By the way, I have various things to do for 2-3 days, including my hobby
:-), my next post will be after that.


I too feel like I deserve a nice vacation :-)

Best,

Jean-François



ikumipatchscombined.diff.gz
Description: GNU Zip compressed data
___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Le 24/05/2018 à 20:03, jfbu a écrit :

Le 24/05/2018 à 19:39, jfbu a écrit :

Hi Keita


I have applied your patch (with diff of preview.el.in
transferred to preview.el) to my elpa installed 12.1.0

(later I
saw I had 12.1.1 on another computer and that preview.el
had changed a bit. As the elpa comes with .gitignore
which by bad luck for me has /preview.el, my initial
commit did not have the original one which I then
manually modified with your patch ---
I recovered later from the other computer the preview.el
there and re-did my commits)

(b.t.w. I also git cloned the savannah, and
https://git.savannah.gnu.org/cgit/auctex.git has a typo it should be
https://git.savannah.gnu.org/git/auctex.git
)


Le 24/05/2018 à 17:26, Ikumi Keita a écrit :

Hi Jean and all,

The proposed method, with slight modification explained below, works for
all the cases I tested.  Thank you again!
For now, I attach the patch for preview-latex part only so that the
reader can concentrate on the relavant changes.


jfbu  writes:



\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\pdfoutput=\the\pdfoutput\relax\the\everyjob}}%
\everyjob\next\catcode`\ 10 %
\catcode`/ 0 %
\def\AUCTEXINPUT##1{\catcode`/ 12\relax\catcode`\ 
9\relax\input{\detokenize{"##1"}}}%
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax%




Notice that when ##1 (i.e. the filename) will be grabbed) the / is of
catcode 0, so ##1 is tokenized accordingly. Which in principle could
perhaps cause an issue \detokenize{aaa/b23} would give aaa/b123
but as / is not legal in filenames in contexts I know of, I did not even
test this anticipation.


AUCTeX always runs TeX commands after changing the current directory to
that of the master file, so forward slash would never appear as an
argument of the command.  The region file is created in the same
directory as the master, thus region compilation should work as well.


I have also added quotes inside the \detokenize.


I tried removing those quotes and let the quotes added around the file
name with space to be put inside /AUCTEXINPUT like this:
pdflatex -foo -bar _ppp_qqq /AUCTEXINPUT{"ppp qqq"} (without escape for 
shell)
, because it's easier to absorb the difference in the way of quoting the
shell arguments between w32 and non-w32 platforms.  Experiments showed
no problems even for files without space, so the attached patch takes
this way.  If this has any shortcomings, please tell me.



My testing so far indicates no problem on Mac OS X 10.9.5



I may have been a bit optimistic

I have problem when adding an equation and asking to generate preview at point



I was intrigued by these changes related to quotes, but it turns out
when I had tried manually on the command line that I was not quoting
correctly.

With your patch on top of 12.1.0 I successfully generate preview for a file
with both spaces and utf-8 characters.

The top of preview generation log displays starts like this;

Running `Preview-LaTeX' on `test my látéx' with ``pdflatex -interaction=nonstopmode \_test_my_la\́te\́x 
"/AUCTEXINPUT{" \"test\ my\ la\́te\́x.tex\" "}"''

(the Mac OS uses a special form of Unicode for filenames with combining
characters)

I did not check the ini file because it is volatile but I think you
indicated how to modify cache-preamble to make it permanent.

Anyway, above file with contents




with this file


\documentclass{article}
\def\foo{E=mc^2}
\def\bar{x^n + y^n = z^n}
\def\foofoo{E=h\nu}
\begin{document}
\(\foo\)
\(\bar\)
\(\foofoo\)
\end{document}



when I add

\[x^3+y^3\]

and then try to generate preview "at point" I get this

```
Running `Preview-LaTeX' on `_region_' with ``pdflatex -interaction=nonstopmode 
\_test_my_la\́te\́x "/AUCTEXINPUT{" _region_.tex "}"''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
  restricted \write18 enabled.
entering extended mode
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
     .
  "test my látéx.tex"
  article.cls    2014/09/29 v1.4h Standard LaTeX document class
   size10.clo    2014/09/29 v1.4h Standard LaTeX file (size option)
     .
(./_region_.tex
No file _region_.aux.
(/path/to/.emacs.d/elpa/auctex-12.1.0/latex/preview.sty 
(/usr/local/texlive/2018/texmf-dist/tex/generic/luatex85/luatex85.sty) 
(/path/to/.emacs.d/elpa/auctex-12.1.0/latex/prtightpage.def) 
(/path/to/.emacs.d/elpa/auctex-12.1.0/latex/prauctex.def
No auxiliary output files.


(/path/to/.emacs.d/elpa/auctex-12.1.0/latex/prauctex.cfg)) 
(/path/to/.emacs.d/elpa/auctex-12.1.0/latex/prfootnotes.def)
Preview: Fontsize 10pt
Preview: PDFoutput 1
)
! Undefined control sequence.
\GenericError  ...
     #4  \errhelp \@err@ ...
l.8 \message{ !name(test my lá
     téx.tex) !offset(0) }
! Undefined control sequence.
\GenericError  ...
   \let \@err@   ...
l.8 

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Le 24/05/2018 à 19:39, jfbu a écrit :

Hi Keita


I have applied your patch (with diff of preview.el.in
transferred to preview.el) to my elpa installed 12.1.0

(later I
saw I had 12.1.1 on another computer and that preview.el
had changed a bit. As the elpa comes with .gitignore
which by bad luck for me has /preview.el, my initial
commit did not have the original one which I then
manually modified with your patch ---
I recovered later from the other computer the preview.el
there and re-did my commits)

(b.t.w. I also git cloned the savannah, and
https://git.savannah.gnu.org/cgit/auctex.git has a typo it should be
https://git.savannah.gnu.org/git/auctex.git
)


Le 24/05/2018 à 17:26, Ikumi Keita a écrit :

Hi Jean and all,

The proposed method, with slight modification explained below, works for
all the cases I tested.  Thank you again!
For now, I attach the patch for preview-latex part only so that the
reader can concentrate on the relavant changes.


jfbu  writes:



\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\pdfoutput=\the\pdfoutput\relax\the\everyjob}}%
\everyjob\next\catcode`\ 10 %
\catcode`/ 0 %
\def\AUCTEXINPUT##1{\catcode`/ 12\relax\catcode`\ 
9\relax\input{\detokenize{"##1"}}}%
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax%




Notice that when ##1 (i.e. the filename) will be grabbed) the / is of
catcode 0, so ##1 is tokenized accordingly. Which in principle could
perhaps cause an issue \detokenize{aaa/b23} would give aaa/b123
but as / is not legal in filenames in contexts I know of, I did not even
test this anticipation.


AUCTeX always runs TeX commands after changing the current directory to
that of the master file, so forward slash would never appear as an
argument of the command.  The region file is created in the same
directory as the master, thus region compilation should work as well.


I have also added quotes inside the \detokenize.


I tried removing those quotes and let the quotes added around the file
name with space to be put inside /AUCTEXINPUT like this:
pdflatex -foo -bar _ppp_qqq /AUCTEXINPUT{"ppp qqq"} (without escape for 
shell)
, because it's easier to absorb the difference in the way of quoting the
shell arguments between w32 and non-w32 platforms.  Experiments showed
no problems even for files without space, so the attached patch takes
this way.  If this has any shortcomings, please tell me.



My testing so far indicates no problem on Mac OS X 10.9.5



I may have been a bit optimistic

I have problem when adding an equation and asking to generate preview at point



I was intrigued by these changes related to quotes, but it turns out
when I had tried manually on the command line that I was not quoting
correctly.

With your patch on top of 12.1.0 I successfully generate preview for a file
with both spaces and utf-8 characters.

The top of preview generation log displays starts like this;

Running `Preview-LaTeX' on `test my látéx' with ``pdflatex -interaction=nonstopmode \_test_my_la\́te\́x 
"/AUCTEXINPUT{" \"test\ my\ la\́te\́x.tex\" "}"''

(the Mac OS uses a special form of Unicode for filenames with combining
characters)

I did not check the ini file because it is volatile but I think you
indicated how to modify cache-preamble to make it permanent.

Anyway, above file with contents




with this file


\documentclass{article}
\def\foo{E=mc^2}
\def\bar{x^n + y^n = z^n}
\def\foofoo{E=h\nu}
\begin{document}
\(\foo\)
\(\bar\)
\(\foofoo\)
\end{document}



when I add

\[x^3+y^3\]

and then try to generate preview "at point" I get this

```
Running `Preview-LaTeX' on `_region_' with ``pdflatex -interaction=nonstopmode 
\_test_my_la\́te\́x "/AUCTEXINPUT{" _region_.tex "}"''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
 restricted \write18 enabled.
entering extended mode
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
.
 "test my látéx.tex"
 article.cls2014/09/29 v1.4h Standard LaTeX document class
  size10.clo2014/09/29 v1.4h Standard LaTeX file (size option)
.
(./_region_.tex
No file _region_.aux.
(/path/to/.emacs.d/elpa/auctex-12.1.0/latex/preview.sty 
(/usr/local/texlive/2018/texmf-dist/tex/generic/luatex85/luatex85.sty) 
(/path/to/.emacs.d/elpa/auctex-12.1.0/latex/prtightpage.def) 
(/path/to/.emacs.d/elpa/auctex-12.1.0/latex/prauctex.def
No auxiliary output files.


(/path/to/.emacs.d/elpa/auctex-12.1.0/latex/prauctex.cfg)) 
(/path/to/.emacs.d/elpa/auctex-12.1.0/latex/prfootnotes.def)
Preview: Fontsize 10pt
Preview: PDFoutput 1
)
! Undefined control sequence.
\GenericError  ...
#4  \errhelp \@err@ ...
l.8 \message{ !name(test my lá
téx.tex) !offset(0) }
! Undefined control sequence.
\GenericError  ...
  \let \@err@   ...
l.8 \message{ !name(test my lá

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Keita


I have applied your patch (with diff of preview.el.in
transferred to preview.el) to my elpa installed 12.1.0

(later I
saw I had 12.1.1 on another computer and that preview.el
had changed a bit. As the elpa comes with .gitignore
which by bad luck for me has /preview.el, my initial
commit did not have the original one which I then
manually modified with your patch ---
I recovered later from the other computer the preview.el
there and re-did my commits)

(b.t.w. I also git cloned the savannah, and
https://git.savannah.gnu.org/cgit/auctex.git has a typo it should be
https://git.savannah.gnu.org/git/auctex.git
)


Le 24/05/2018 à 17:26, Ikumi Keita a écrit :

Hi Jean and all,

The proposed method, with slight modification explained below, works for
all the cases I tested.  Thank you again!
For now, I attach the patch for preview-latex part only so that the
reader can concentrate on the relavant changes.


jfbu  writes:



\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\pdfoutput=\the\pdfoutput\relax\the\everyjob}}%
\everyjob\next\catcode`\ 10 %
\catcode`/ 0 %
\def\AUCTEXINPUT##1{\catcode`/ 12\relax\catcode`\ 
9\relax\input{\detokenize{"##1"}}}%
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax%




Notice that when ##1 (i.e. the filename) will be grabbed) the / is of
catcode 0, so ##1 is tokenized accordingly. Which in principle could
perhaps cause an issue \detokenize{aaa/b23} would give aaa/b123
but as / is not legal in filenames in contexts I know of, I did not even
test this anticipation.


AUCTeX always runs TeX commands after changing the current directory to
that of the master file, so forward slash would never appear as an
argument of the command.  The region file is created in the same
directory as the master, thus region compilation should work as well.


I have also added quotes inside the \detokenize.


I tried removing those quotes and let the quotes added around the file
name with space to be put inside /AUCTEXINPUT like this:
pdflatex -foo -bar _ppp_qqq /AUCTEXINPUT{"ppp qqq"} (without escape for 
shell)
, because it's easier to absorb the difference in the way of quoting the
shell arguments between w32 and non-w32 platforms.  Experiments showed
no problems even for files without space, so the attached patch takes
this way.  If this has any shortcomings, please tell me.



My testing so far indicates no problem on Mac OS X 10.9.5

I was intrigued by these changes related to quotes, but it turns out
when I had tried manually on the command line that I was not quoting
correctly.

With your patch on top of 12.1.0 I successfully generate preview for a file
with both spaces and utf-8 characters.

The top of preview generation log displays starts like this;

Running `Preview-LaTeX' on `test my látéx' with ``pdflatex -interaction=nonstopmode \_test_my_la\́te\́x 
"/AUCTEXINPUT{" \"test\ my\ la\́te\́x.tex\" "}"''

(the Mac OS uses a special form of Unicode for filenames with combining
characters)

I did not check the ini file because it is volatile but I think you
indicated how to modify cache-preamble to make it permanent.

Anyway, above file with contents

\documentclass{article}
\def\foo{E=mc^2}
\def\bar{x^n + y^n = z^n}
\def\foofoo{E=h\nu}
\begin{document}
\(\foo\)
\(\bar\)
\(\foofoo\)
\end{document}

generates correct previews.

It also compiles fine because my TeX-command-list does not have the
 %` and %'.
And the non-\input form correctly escapes the spaces for the shell
so it seems.

A proposito, the above in the pdflatex command (once the format
is done)means \input will be with extra spaces

\input{"test my látéx.tex"}

but fortunately that works fine.

(one can check that by using this

\\def\\AUCTEXINPUT##1{\\catcode`/ 12\\relax\\catcode`\\ 
9\\relax\\message{+++\\detokenize{##1}+++}\\input{\\detokenize{##1}}}%

in preview.el and then check generate-preview compilation log)

LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
.
 "test my látéx.tex"
 article.cls2014/09/29 v1.4h Standard LaTeX document class
  size10.clo2014/09/29 v1.4h Standard LaTeX file (size option)
.
+++ "test my látéx.tex" +++ (./test my látéx.tex
No file "test my látéx".aux.

Anyway, all seems fine!

Thanks!

Jean-François


___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Keita,



Le 24/05/2018 à 12:44, Ikumi Keita a écrit :

Hi Jean,

Thanks a lot again, the proposed method works on the command line!  I'll
implement it in the actual preview-latex code.

By the way, if I remember correctly, filenames with multiple contiguous
spaces are not supported by TeX, aren't they?  Standard pdflatex just
fails for such file names:



ah you are right it was never supported by AUCTeX anyhow.

But it is possible to make it work (I used this kind of tricks in my Preview 
adventures)

$ pdflatex  -file-line-error   -interaction=nonstopmode "\relax{\catcode32 12 
\edef\x{\noexpand\input{\detokenize{\"test  spaces.tex\"}}}\expandafter}\x"
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
 restricted \write18 enabled.
entering extended mode
LaTeX2e <2018-04-01> patch level 4
(./test  spaces.tex
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo))
(./test  spaces.aux) [1{/usr/local/texlive/2018/texmf-var/fonts/map/pdftex/
updmap/pdftex.map}] (./test  spaces.aux) )
Output written on "test  spaces.pdf" (1 page, 23299 bytes).
Transcript written on "test  spaces.log".

Thus it is only a problem of getting the filename correctly to pdflatex.

I admit this is somewhat extreme. I have never tried until today. It shows that
the only problem is to bypass TeX tokenization of contiguous spaces as only one,
but once this is done the pdflatex binary has no issues.

(one may fear problems with makeindex bibtex etc... never tested, in fact
I recall some time ago I was surprised to discover pdftex supported filename
with a single space to start with, but now I am sort in a frenzy ...)

Thus it is reasonable to not insist that Preview supports filenames with
multiple contiguous spaces, as already AUCTeX has no a priori support for
such filenames.

But in principle it is possible to add it as above.

Best,

Jean-François



--
[bash]$ cp ppp\ qqq.tex aaa\ \ bbb.tex
[bash]$ pdflatex aaa\ \ bbb.tex
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
  restricted \write18 enabled.
entering extended mode
! I can't find file `"aaa bbb.tex"'.
<*> "aaa  bbb.tex"
   
(Press Enter to retry, or Control-D to exit)

Please type another input file name:
--
So I suppose that it is not necessary for preview-latex to support such
file names.


jfbu  writes:

In short, in presence of the /AUCTEXINPUT hack, the original space hack
simply ceases functioning because the space catcode is reset to "ignore"
too soon (at \everyjob)



How to fix this ?



With this new version of ini file



\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\pdfoutput=\the\pdfoutput\relax\the\everyjob}}%
\everyjob\next\catcode`\ 10 %
\catcode`/ 0 %
\def\AUCTEXINPUT##1{\catcode`/ 12\relax\catcode`\ 
9\relax\input{\detokenize{"##1"}}}%
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax%




Notice that when ##1 (i.e. the filename) will be grabbed) the / is of
catcode 0, so ##1 is tokenized accordingly. Which in principle could
perhaps cause an issue \detokenize{aaa/b23} would give aaa/b123
but as / is not legal in filenames in contexts I know of, I did not even
test this anticipation.



I have also added quotes inside the \detokenize.



I tested with a file with filename "test mylátex.tex"



I. Generation of format



etex -ini "" prv_test_mylátex.ini 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input{\detokenize{\"test mylátex.tex\"}}"



where file prv_test_mylátex.ini contains the above contents (from 
\ifx\pdfoutput to \relax%)
and attention to the added quotes inside the \detokenize argument.



II. Usage of format



pdflatex -fmt="prv_test_mylátex.fmt" /AUCTEXINPUT{test mylátex.tex}



Could you try this out ?



Best



Jean-François


Best regards,
Ikumi Keita





___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

Hi Jean,

Thanks a lot again, the proposed method works on the command line!  I'll
implement it in the actual preview-latex code.

By the way, if I remember correctly, filenames with multiple contiguous
spaces are not supported by TeX, aren't they?  Standard pdflatex just
fails for such file names:
--
[bash]$ cp ppp\ qqq.tex aaa\ \ bbb.tex
[bash]$ pdflatex aaa\ \ bbb.tex 
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
 restricted \write18 enabled.
entering extended mode
! I can't find file `"aaa bbb.tex"'.
<*> "aaa  bbb.tex"
  
(Press Enter to retry, or Control-D to exit)
Please type another input file name: 
--
So I suppose that it is not necessary for preview-latex to support such
file names.

> jfbu  writes:
> In short, in presence of the /AUCTEXINPUT hack, the original space hack
> simply ceases functioning because the space catcode is reset to "ignore"
> too soon (at \everyjob)

> How to fix this ?

> With this new version of ini file

> \ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
> \edef\next{{\pdfoutput=\the\pdfoutput\relax\the\everyjob}}%
> \everyjob\next\catcode`\ 10 %
> \catcode`/ 0 %
> \def\AUCTEXINPUT##1{\catcode`/ 12\relax\catcode`\ 
> 9\relax\input{\detokenize{"##1"}}}%
> \let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax%


> Notice that when ##1 (i.e. the filename) will be grabbed) the / is of
> catcode 0, so ##1 is tokenized accordingly. Which in principle could
> perhaps cause an issue \detokenize{aaa/b23} would give aaa/b123
> but as / is not legal in filenames in contexts I know of, I did not even
> test this anticipation.

> I have also added quotes inside the \detokenize.

> I tested with a file with filename "test mylátex.tex"

> I. Generation of format

> etex -ini "" prv_test_mylátex.ini 
> "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
>  "\input{\detokenize{\"test mylátex.tex\"}}"

> where file prv_test_mylátex.ini contains the above contents (from 
> \ifx\pdfoutput to \relax%)
> and attention to the added quotes inside the \detokenize argument.

> II. Usage of format

> pdflatex -fmt="prv_test_mylátex.fmt" /AUCTEXINPUT{test mylátex.tex}

> Could you try this out ?

> Best

> Jean-François

Best regards,
Ikumi Keita

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi again (!!!)

better approach using the format with & syntax at bottom


Le 24/05/2018 à 11:52, jfbu a écrit :

Hi Keita

ARRGH filenames with multiple contiguous spaces are a challenge

Because TeX coalesces multiples spaces into a single one.

The ini file can be modified simply but it is more complicated
for the generation of the format itself.

See at bottom of post.

Le 24/05/2018 à 10:46, jfbu a écrit :

Hi Keita

Le 24/05/2018 à 09:24, Ikumi Keita a écrit :

Hi Jean,


Ikumi Keita  writes:

Thank you very much for your all valuable works!  I'll test this with
other cases (TL2017, file names with space, on w32 platform) and propose
to commit again.


Unfortunately, it seems that the proposed method using /AUCTEXINPUT does
not work with file name with space.  The sample I'm using is a file named
"ppp qqq.tex" with the content:




Yes, I had a bad night tormented about this. But I understand why
and will explain below.




--- ppp qqq.tex --
\documentclass{article}
\begin{document}
Hello, World
\(xyz\)
\end{document}

%%% Local Variables:
%%% TeX-master: t
%%% End:
--

The format file is generated successfully:
--
Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -ini -interaction=nonstopmode "" prv_ppp_qqq.ini    
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" "{\detokenize{" \"ppp\ qqq.tex\" "}}"''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (INITEX)
  restricted \write18 enabled.
entering extended mode
(./prv_ppp_qqq.ini
LaTeX2e <2018-04-01> patch level 4
(/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx) 
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex File ignored))
No auxiliary output files.

(./ppp qqq.tex (/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo)) )
Beginning to dump on file prv_ppp_qqq.fmt
  (preloaded format=prv_ppp_qqq 2018.5.24)
7585 strings of total length 123256
58367 memory locations dumped; current usage is 219&56905
4191 multiletter control sequences
\font\nullfont=nullfont
\font\OMX/cmex/m/n/5=cmex10
[...]
\font\OT1/cmss/m/n/10=cmss10
5716 words of font info for 21 preloaded fonts
1141 hyphenation exceptions
Hyphenation trie of length 370691 has 8997 ops out of 35111
   143 for language 83
   110 for language 82
[...]
   181 for language 0
0 words of pdfTeX memory
0 indirect objects
No pages of output.
Transcript written on prv_ppp_qqq.log.

TeX Output finished at Thu May 24 16:09:49
--

However, the next run with the dumped format doesn't make its way:



Because spaces are ignored in the filename, due to \input form (indirectly via 
/AUCTEXINPUT)

I will explain below.


--
Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -interaction=nonstopmode 
\_ppp_qqq /AUCTEXINPUT{\"ppp\ qqq.tex\"}''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
  restricted \write18 enabled.
entering extended mode
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
 .
   "ppp qqq.tex"
  article.cls    2014/09/29 v1.4h Standard LaTeX document class
   size10.clo    2014/09/29 v1.4h Standard LaTeX file (size option)
 .

! LaTeX Error: File `"pppqqq.tex"' not found.

Type X to quit or  to proceed,
or enter new name. (Default extension: tex")

Enter file name:
! Emergency stop.

<*> _ppp_qqq /AUCTEXINPUT{"ppp qqq.tex"}
 ^^M
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on texput.log.

TeX Output exited as expected with code 1 at Thu May 24 16:11:23
LaTeX: LaTeX found no preview images
--

When I replace \_ppp_qqq with -fmt="prv_ppp_qqq.fmt" from xterm, the
error changes slightly:


That's a bit unexpected (by me).


--
[bash]$ pdflatex -interaction=nonstopmode -fmt="prv_ppp_qqq.fmt" /AUCTEXINPUT{\"ppp\ 
qqq.tex\"}
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=prv_ppp_qqq.fmt)
  restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
 .
   "ppp qqq.tex"
  article.cls    2014/09/29 v1.4h Standard LaTeX document class
   size10.clo    

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Keita

ARRGH filenames with multiple contiguous spaces are a challenge

Because TeX coalesces multiples spaces into a single one.

The ini file can be modified simply but it is more complicated
for the generation of the format itself.

See at bottom of post.

Le 24/05/2018 à 10:46, jfbu a écrit :

Hi Keita

Le 24/05/2018 à 09:24, Ikumi Keita a écrit :

Hi Jean,


Ikumi Keita  writes:

Thank you very much for your all valuable works!  I'll test this with
other cases (TL2017, file names with space, on w32 platform) and propose
to commit again.


Unfortunately, it seems that the proposed method using /AUCTEXINPUT does
not work with file name with space.  The sample I'm using is a file named
"ppp qqq.tex" with the content:




Yes, I had a bad night tormented about this. But I understand why
and will explain below.




--- ppp qqq.tex --
\documentclass{article}
\begin{document}
Hello, World
\(xyz\)
\end{document}

%%% Local Variables:
%%% TeX-master: t
%%% End:
--

The format file is generated successfully:
--
Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -ini -interaction=nonstopmode "" prv_ppp_qqq.ini    
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" "{\detokenize{" \"ppp\ qqq.tex\" "}}"''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (INITEX)
  restricted \write18 enabled.
entering extended mode
(./prv_ppp_qqq.ini
LaTeX2e <2018-04-01> patch level 4
(/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx) 
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex File ignored))
No auxiliary output files.

(./ppp qqq.tex (/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo)) )
Beginning to dump on file prv_ppp_qqq.fmt
  (preloaded format=prv_ppp_qqq 2018.5.24)
7585 strings of total length 123256
58367 memory locations dumped; current usage is 219&56905
4191 multiletter control sequences
\font\nullfont=nullfont
\font\OMX/cmex/m/n/5=cmex10
[...]
\font\OT1/cmss/m/n/10=cmss10
5716 words of font info for 21 preloaded fonts
1141 hyphenation exceptions
Hyphenation trie of length 370691 has 8997 ops out of 35111
   143 for language 83
   110 for language 82
[...]
   181 for language 0
0 words of pdfTeX memory
0 indirect objects
No pages of output.
Transcript written on prv_ppp_qqq.log.

TeX Output finished at Thu May 24 16:09:49
--

However, the next run with the dumped format doesn't make its way:



Because spaces are ignored in the filename, due to \input form (indirectly via 
/AUCTEXINPUT)

I will explain below.


--
Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -interaction=nonstopmode 
\_ppp_qqq /AUCTEXINPUT{\"ppp\ qqq.tex\"}''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
  restricted \write18 enabled.
entering extended mode
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
 .
   "ppp qqq.tex"
  article.cls    2014/09/29 v1.4h Standard LaTeX document class
   size10.clo    2014/09/29 v1.4h Standard LaTeX file (size option)
 .

! LaTeX Error: File `"pppqqq.tex"' not found.

Type X to quit or  to proceed,
or enter new name. (Default extension: tex")

Enter file name:
! Emergency stop.

<*> _ppp_qqq /AUCTEXINPUT{"ppp qqq.tex"}
 ^^M
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on texput.log.

TeX Output exited as expected with code 1 at Thu May 24 16:11:23
LaTeX: LaTeX found no preview images
--

When I replace \_ppp_qqq with -fmt="prv_ppp_qqq.fmt" from xterm, the
error changes slightly:


That's a bit unexpected (by me).


--
[bash]$ pdflatex -interaction=nonstopmode -fmt="prv_ppp_qqq.fmt" /AUCTEXINPUT{\"ppp\ 
qqq.tex\"}
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=prv_ppp_qqq.fmt)
  restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
 .
   "ppp qqq.tex"
  article.cls    2014/09/29 v1.4h Standard LaTeX document class
   size10.clo    2014/09/29 v1.4h Standard LaTeX file (size option)
 .
)
Runaway argument?
! File ended while scanning use of ^^M.

 

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Le 24/05/2018 à 10:46, jfbu a écrit :

Notice that when ##1 (i.e. the filename) will be grabbed) the / is of
catcode 0, so ##1 is tokenized accordingly. Which in principle could
perhaps cause an issue \detokenize{aaa/b23} would give aaa/b123
but as / is not legal in filenames in contexts I know of, I did not even
test this anticipation.


actually the \detokenize will use escapechar setting so
this would give aaa\b123 with a backslash
but anyway I decided / will not possibly be part of a filename...

J.-F.


___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Keita

Le 24/05/2018 à 09:24, Ikumi Keita a écrit :

Hi Jean,


Ikumi Keita  writes:

Thank you very much for your all valuable works!  I'll test this with
other cases (TL2017, file names with space, on w32 platform) and propose
to commit again.


Unfortunately, it seems that the proposed method using /AUCTEXINPUT does
not work with file name with space.  The sample I'm using is a file named
"ppp qqq.tex" with the content:




Yes, I had a bad night tormented about this. But I understand why
and will explain below.




--- ppp qqq.tex --
\documentclass{article}
\begin{document}
Hello, World
\(xyz\)
\end{document}

%%% Local Variables:
%%% TeX-master: t
%%% End:
--

The format file is generated successfully:
--
Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -ini -interaction=nonstopmode "" prv_ppp_qqq.ini
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" "{\detokenize{" \"ppp\ qqq.tex\" "}}"''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (INITEX)
  restricted \write18 enabled.
entering extended mode
(./prv_ppp_qqq.ini
LaTeX2e <2018-04-01> patch level 4
(/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx) 
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex File ignored))
No auxiliary output files.

(./ppp qqq.tex (/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo)) )
Beginning to dump on file prv_ppp_qqq.fmt
  (preloaded format=prv_ppp_qqq 2018.5.24)
7585 strings of total length 123256
58367 memory locations dumped; current usage is 219&56905
4191 multiletter control sequences
\font\nullfont=nullfont
\font\OMX/cmex/m/n/5=cmex10
[...]
\font\OT1/cmss/m/n/10=cmss10
5716 words of font info for 21 preloaded fonts
1141 hyphenation exceptions
Hyphenation trie of length 370691 has 8997 ops out of 35111
   143 for language 83
   110 for language 82
[...]
   181 for language 0
0 words of pdfTeX memory
0 indirect objects
No pages of output.
Transcript written on prv_ppp_qqq.log.

TeX Output finished at Thu May 24 16:09:49
--

However, the next run with the dumped format doesn't make its way:



Because spaces are ignored in the filename, due to \input form (indirectly via 
/AUCTEXINPUT)

I will explain below.


--
Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -interaction=nonstopmode 
\_ppp_qqq /AUCTEXINPUT{\"ppp\ qqq.tex\"}''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
  restricted \write18 enabled.
entering extended mode
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
 .
   "ppp qqq.tex"
  article.cls2014/09/29 v1.4h Standard LaTeX document class
   size10.clo2014/09/29 v1.4h Standard LaTeX file (size option)
 .

! LaTeX Error: File `"pppqqq.tex"' not found.

Type X to quit or  to proceed,
or enter new name. (Default extension: tex")

Enter file name:
! Emergency stop.

  
<*> _ppp_qqq /AUCTEXINPUT{"ppp qqq.tex"}

 ^^M
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on texput.log.

TeX Output exited as expected with code 1 at Thu May 24 16:11:23
LaTeX: LaTeX found no preview images
--

When I replace \_ppp_qqq with -fmt="prv_ppp_qqq.fmt" from xterm, the
error changes slightly:


That's a bit unexpected (by me).


--
[bash]$ pdflatex -interaction=nonstopmode -fmt="prv_ppp_qqq.fmt" /AUCTEXINPUT{\"ppp\ 
qqq.tex\"}
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=prv_ppp_qqq.fmt)
  restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
 .
   "ppp qqq.tex"
  article.cls2014/09/29 v1.4h Standard LaTeX document class
   size10.clo2014/09/29 v1.4h Standard LaTeX file (size option)
 .
)
Runaway argument?
! File ended while scanning use of ^^M.

 \par
<*> "/AUCTEXINPUT{ppp qqq.tex}
   "



I do not understand that for time being
but let's forget it as we have the \_ppp_qqq.fmt way.

UPDATE: at bottom of this my new proposal seems ok with -fmt
at least at my locale.



! LaTeX Error: 

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

Hi Jean, thanks a lot, your advice led me to the success!

> jfbu  writes:

> Putting altogether :

> 1. We need file prv_testmylátex.ini to be

> \ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
> \edef\next{{\catcode`\ 9 \pdfoutput=\the\pdfoutput\relax\the\everyjob}}%
> \everyjob\next\catcode`\ 10 %
> \catcode`/ 0 %
> \def\AUCTEXINPUT##1{\catcode`/ 12\relax\input{\detokenize{##1}}}%
> \let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax%

> REMARK: the above hack of \dump is conditional on \pdfoutput being
> defined which I think is good anyhow. \pdfoutput is defined for
> pdflatex whether dvi or pdf output.

> 2. We create the format with this incantation

> etex -ini -jobname prv_testmylátex ""  
> "\input{\detokenize{prv_testmylátex.ini}}" 
> "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
>  "\input{\detokenize{testmylátex.tex}}"

> the following less paranoid version (I don't wrap the name of ini file in 
> \detokenize) seems to work too

> etex -ini -jobname prv_testmylátex "" prv_testmylátex.ini 
> "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
>  "\input{\detokenize{testmylátex.tex}}"

I took this version.  I also omitted "-jobname prv_testmylátex" since
the original AUCTeX code doesn't have that part.  Do you recommend to
restore them?

> The detokenize at the end is really needed. (Or we could play the other games 
> with \UseRawInputEncoding)

> 3. We then use the format with this incantation

> pdflatex -fmt="prv_testmylátex.fmt" /AUCTEXINPUT{testmylátex.tex}

I used the form "_testmylátex" instead of -fmt="prv_testmylátex.fmt"
following the original AUCTeX code.  I expect this is harmless.

> It works in my testing in Terminal on Mac OS X (bash shell)

> (I tested it with /AUCTEXINPUT{some other file with non-ascii filename}
> to check macros defined in the preamble of testmylátex.tex are actually
> used, proving the created format was indeed used).

Thank you very much for your all valuable works!  I'll test this with
other cases (TL2017, file names with space, on w32 platform) and propose
to commit again.

For those interested in, the current tentative patch is attached below.

Regards,
Ikumi Keita



tentative-patch2.gz
Description: new approach v2
___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Keita

(still top-posting)

I have an "in principle" solution, although it will look a bit complicated.

First I will explain proof of principle, then concrete implementation.
I hope you can incorporate that in AUCTeX.

STEP I: generation of format.

I will explain first the proof of principle and later how to make in practice.

Imagine we can make sure the preamble of our document is like
in this file :

  primárias.tex
\documentclass{article}

\catcode`/ 0
\def\AUCTEXINPUT#1{\catcode`/ 12\relax\input{\detokenize{#1}}}

\begin{document}
Hello
\end{document}


Generate format via this kind of invocation: (bash shell)

etex -ini \ mylatex.ltx {\\detokenize{testmylátex.tex}}

STEP II. Use of format.

This will go via this kind of invocation: (bash shell)

pdflatex \ /AUCTEXINPUT{testmylátex.tex}

It works in my testing.


Now this was proof of principle.

How to insert the special code

\catcode`/ 0
\def\AUCTEXINPUT#1{\catcode`/ 12\relax\input{\detokenize{#1}}}

?

We need it to make it into the format. This is where the kind of
hack you reported as already present in AUCTeX can be used.

It was

\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\catcode`\ 9 
\pdfoutput=\the\pdfoutput\relax\the\everyjob}}\everyjob\next\catcode`\ 10 
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax

I am not sure why this, but anyway the point is that \dump is the last thing
executed by \begin{document} when mylatex.ltx is in use to generate a format.

So the above hacks \dump. We can hack it too to insert what we want from above.


Putting altogether :

1. We need file prv_testmylátex.ini to be

\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\catcode`\ 9 \pdfoutput=\the\pdfoutput\relax\the\everyjob}}%
\everyjob\next\catcode`\ 10 %
\catcode`/ 0 %
\def\AUCTEXINPUT##1{\catcode`/ 12\relax\input{\detokenize{##1}}}%
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax%

REMARK: the above hack of \dump is conditional on \pdfoutput being
defined which I think is good anyhow. \pdfoutput is defined for
pdflatex whether dvi or pdf output.

2. We create the format with this incantation

etex -ini -jobname prv_testmylátex ""  "\input{\detokenize{prv_testmylátex.ini}}" 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input{\detokenize{testmylátex.tex}}"

the following less paranoid version (I don't wrap the name of ini file in 
\detokenize) seems to work too

etex -ini -jobname prv_testmylátex "" prv_testmylátex.ini 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input{\detokenize{testmylátex.tex}}"

The detokenize at the end is really needed. (Or we could play the other games 
with \UseRawInputEncoding)

3. We then use the format with this incantation

pdflatex -fmt="prv_testmylátex.fmt" /AUCTEXINPUT{testmylátex.tex}



It works in my testing in Terminal on Mac OS X (bash shell)

(I tested it with /AUCTEXINPUT{some other file with non-ascii filename}
to check macros defined in the preamble of testmylátex.tex are actually
used, proving the created format was indeed used).

And by the way, yes you were right about mylatex.ltx \relax, then it
uses .tex with no extra setting needed.

Best

Jean-François


Le 23/05/2018 à 21:38, jfbu a écrit :

Hi Keita

first-off let me apologize for my numerous posts,
originating from someone like myself who

- does not know preview,
- does not know elisp,
- does not know auctex,
- does not know mylatex.ltx...
- ... and is going to top-post

I appreciate a lot your detailed explanations,
which avoid me plunging into auctex sources.

I have now made some experiments which show there
is a problem to solve at "mylatex.ltx" level.

Consider this source


\documentclass{article}

\def\foo{foo}

\begin{document}
Checking format was used:

\ttfamily\meaning\foo
\end{document}


### Experiment One:

1. save above as testmylatex.tex
2. execute etex -ini \ mylatex.ltx testmylatex.tex
3. execute pdftex -fmt=mylatex testmylatex

everything should go fine, and then testmylatex.pdf will contain

Checking format was used:
macro:->foo

### Experiment Two

1. save above file as testmylátex.tex
2a. Try pdflatex testmylátex.tex : it works
2b. Try now: etex -ini \ mylatex.ltx testmylátex.tex

It fails with

! I can't find file `testmyl'.

    \global
<*>  mylatex.ltx testmylá
    tex.tex
(Press Enter to retry, or Control-D to exit)

(attention that á especially on Mac OS may be with a combining character
so your mileage my vary in the error report)

3a. The format file mylatex.fmt from Experiment one should still be there
so we try

pdflatex -fmt=mylatex 

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Le 23/05/2018 à 21:38, jfbu a écrit :

Hi Keita

first-off let me apologize for my numerous posts,
originating from someone like myself who

- does not know preview,
- does not know elisp,
- does not know auctex,
- does not know mylatex.ltx...
- ... and is going to top-post

I appreciate a lot your detailed explanations,
which avoid me plunging into auctex sources.

I have now made some experiments which show there
is a problem to solve at "mylatex.ltx" level.

Consider this source


\documentclass{article}

\def\foo{foo}

\begin{document}
Checking format was used:

\ttfamily\meaning\foo
\end{document}


### Experiment One:

1. save above as testmylatex.tex
2. execute etex -ini \ mylatex.ltx testmylatex.tex
3. execute pdftex -fmt=mylatex testmylatex

everything should go fine, and then testmylatex.pdf will contain

Checking format was used:
macro:->foo




Arrgh, ok this says nothing the idea would have been to execute

pdftex -fmt=mylatex testmylatex2


with file testmylatex2.tex with an empty preamble


anyway, that's not important for the other things I blatter about

Sorry...

J.-F.


### Experiment Two

1. save above file as testmylátex.tex
2a. Try pdflatex testmylátex.tex : it works
2b. Try now: etex -ini \ mylatex.ltx testmylátex.tex

It fails with

! I can't find file `testmyl'.

    \global
<*>  mylatex.ltx testmylá
    tex.tex
(Press Enter to retry, or Control-D to exit)

(attention that á especially on Mac OS may be with a combining character
so your mileage my vary in the error report)

3a. The format file mylatex.fmt from Experiment one should still be there
so we try

pdflatex -fmt=mylatex testmylátex.tex

(recall that pdflatex testmylátex.tex worked out of the box)

It fails with

! I can't find file `testmyl'.

    \unhbox
<*> testmylá
  tex.tex
(Press Enter to retry, or Control-D to exit)

Now try

pdflatex -fmt=mylatex "\input{\detokenize{testmylátex.tex}}"

It fails with the kind of error message you reported

$ pdflatex -fmt=mylatex "\input{\detokenize{testmylátex.tex}}"
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=mylatex)
  restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
     .
  article.cls    2014/09/29 v1.4h Standard LaTeX document class
   size10.clo    2014/09/29 v1.4h Standard LaTeX file (size option)
     .
)
Runaway argument?
! File ended while scanning use of ^^M.

     \par
<*> \
  input{\detokenize{testmylátex.tex}}
?


All of the above problems are with "mylatex.ltx".

I think creating a file and using `\input` form might help

The doc says

%%% If you are on a Mac or using some shell that makes it inconvenient
%%% to use a command line such as the above examples then you may
%%% make a file `mylatex.tex' with the single line
%%% \input mylatex.ltx abc
%%% and then pass the file mylatex.tex to your (ini)tex shell to produce
%%% the format, ie something equivalent to initex  mylatex.tex.

and perhaps this can be used to solve our problems.

I can't work immediately on this but will later,

(or David Carlisle will show up and explain what to do)

Best,

Jean-François

Le 23/05/2018 à 20:12, Ikumi Keita a écrit :

Hi Jean,


jfbu  writes:



no bothering at all!



Perhaps I should give up this particular case (non-ascii file name +
preview-latex preamble cache) and commit the patch which fixes other
issues.



surely not!


Thank you!


What would helped me is to see the exact content of an ini file.


It is designed to be deleted automatically after the format file dump,
so I modified `preview-cache-preamble' temporarily not to delete it.
The content thus obtained is:
\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\catcode`\ 9 
\pdfoutput=\the\pdfoutput\relax\the\everyjob}}\everyjob\next\catcode`\ 10 
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax


Then it is intriguing that the comment says



"mylatex.ltx expects a file name to follow.  Bad."



and refers to empty file ".tex" but I see mylatex.ltx \relax not mylatex.ltx 
.tex ??


I don't know well, but the log at the dumping says
(/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx) 
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex File ignored))
so I think it really reads ".tex".



about this ".tex" it was missing from TeXLive for a while due to a LaTeX 
packaging
error but is since fixed
https://github.com/latex3/latex2e/issues/30



Yes, I had updated with tlmgr and restored ".tex" already.


It might take some time until I can say something about preview's
mechanism, as I am not even that familiar to its usage. I hope
David can kick in, that would be so much faster.


I hope so as well.


I anticipate the role of the format is to catch up some common 

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Keita

first-off let me apologize for my numerous posts,
originating from someone like myself who

- does not know preview,
- does not know elisp,
- does not know auctex,
- does not know mylatex.ltx...
- ... and is going to top-post

I appreciate a lot your detailed explanations,
which avoid me plunging into auctex sources.

I have now made some experiments which show there
is a problem to solve at "mylatex.ltx" level.

Consider this source


\documentclass{article}

\def\foo{foo}

\begin{document}
Checking format was used:

\ttfamily\meaning\foo
\end{document}


### Experiment One:

1. save above as testmylatex.tex
2. execute etex -ini \ mylatex.ltx testmylatex.tex
3. execute pdftex -fmt=mylatex testmylatex

everything should go fine, and then testmylatex.pdf will contain

Checking format was used:
macro:->foo

### Experiment Two

1. save above file as testmylátex.tex
2a. Try pdflatex testmylátex.tex : it works
2b. Try now: etex -ini \ mylatex.ltx testmylátex.tex

It fails with

! I can't find file `testmyl'.

   \global
<*>  mylatex.ltx testmylá
   tex.tex
(Press Enter to retry, or Control-D to exit)

(attention that á especially on Mac OS may be with a combining character
so your mileage my vary in the error report)

3a. The format file mylatex.fmt from Experiment one should still be there
so we try

pdflatex -fmt=mylatex testmylátex.tex

(recall that pdflatex testmylátex.tex worked out of the box)

It fails with

! I can't find file `testmyl'.

   \unhbox
<*> testmylá
 tex.tex
(Press Enter to retry, or Control-D to exit)

Now try

pdflatex -fmt=mylatex "\input{\detokenize{testmylátex.tex}}"

It fails with the kind of error message you reported

$ pdflatex -fmt=mylatex "\input{\detokenize{testmylátex.tex}}"
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=mylatex)
 restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
.
 article.cls2014/09/29 v1.4h Standard LaTeX document class
  size10.clo2014/09/29 v1.4h Standard LaTeX file (size option)
.
)
Runaway argument?
! File ended while scanning use of ^^M.

\par
<*> \
 input{\detokenize{testmylátex.tex}}
?


All of the above problems are with "mylatex.ltx".

I think creating a file and using `\input` form might help

The doc says

%%% If you are on a Mac or using some shell that makes it inconvenient
%%% to use a command line such as the above examples then you may
%%% make a file `mylatex.tex' with the single line
%%% \input mylatex.ltx abc
%%% and then pass the file mylatex.tex to your (ini)tex shell to produce
%%% the format, ie something equivalent to initex  mylatex.tex.

and perhaps this can be used to solve our problems.

I can't work immediately on this but will later,

(or David Carlisle will show up and explain what to do)

Best,

Jean-François

Le 23/05/2018 à 20:12, Ikumi Keita a écrit :

Hi Jean,


jfbu  writes:



no bothering at all!



Perhaps I should give up this particular case (non-ascii file name +
preview-latex preamble cache) and commit the patch which fixes other
issues.



surely not!


Thank you!


What would helped me is to see the exact content of an ini file.


It is designed to be deleted automatically after the format file dump,
so I modified `preview-cache-preamble' temporarily not to delete it.
The content thus obtained is:
\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\catcode`\ 9 
\pdfoutput=\the\pdfoutput\relax\the\everyjob}}\everyjob\next\catcode`\ 10 
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax


Then it is intriguing that the comment says



"mylatex.ltx expects a file name to follow.  Bad."



and refers to empty file ".tex" but I see mylatex.ltx \relax not mylatex.ltx 
.tex ??


I don't know well, but the log at the dumping says
(/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx) 
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex File ignored))
so I think it really reads ".tex".



about this ".tex" it was missing from TeXLive for a while due to a LaTeX 
packaging
error but is since fixed
https://github.com/latex3/latex2e/issues/30



Yes, I had updated with tlmgr and restored ".tex" already.


It might take some time until I can say something about preview's
mechanism, as I am not even that familiar to its usage. I hope
David can kick in, that would be so much faster.


I hope so as well.


I anticipate the role of the format is to catch up some common preamble,


These two paragraphs are quotes from preview-latex.info:
--
'C-c C-p C-f'
'preview-cache-preamble'
Preview/Turn preamble cache on
  Dump a pregenerated format file.  For the rest of the session, this
  file is 

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

Hi Jean,

> jfbu  writes:

> no bothering at all!

>> Perhaps I should give up this particular case (non-ascii file name +
>> preview-latex preamble cache) and commit the patch which fixes other
>> issues.

> surely not!

Thank you!

> What would helped me is to see the exact content of an ini file.

It is designed to be deleted automatically after the format file dump,
so I modified `preview-cache-preamble' temporarily not to delete it.
The content thus obtained is:
\ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{%
\edef\next{{\catcode`\ 9 
\pdfoutput=\the\pdfoutput\relax\the\everyjob}}\everyjob\next\catcode`\ 10 
\let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax

> Then it is intriguing that the comment says

> "mylatex.ltx expects a file name to follow.  Bad."

> and refers to empty file ".tex" but I see mylatex.ltx \relax not mylatex.ltx 
> .tex ??

I don't know well, but the log at the dumping says
(/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx) 
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex File ignored))
so I think it really reads ".tex".

> 
> about this ".tex" it was missing from TeXLive for a while due to a LaTeX 
> packaging
> error but is since fixed
> https://github.com/latex3/latex2e/issues/30
> 

Yes, I had updated with tlmgr and restored ".tex" already.

> It might take some time until I can say something about preview's
> mechanism, as I am not even that familiar to its usage. I hope
> David can kick in, that would be so much faster.

I hope so as well.

> I anticipate the role of the format is to catch up some common preamble,

These two paragraphs are quotes from preview-latex.info:
--
'C-c C-p C-f'
'preview-cache-preamble'
Preview/Turn preamble cache on
 Dump a pregenerated format file.  For the rest of the session, this
 file is used when running on the same master file.  Use this if you
 know your LaTeX takes a long time to start up, the speedup will be
 most noticeable when generating single or few previews.  If you
 change your preamble, do this again.  preview-latex will try to
 detect the necessity of that automatically when editing changes to
 the preamble are done from within Emacs, but it will not notice if
 the preamble effectively changes because some included file or
 style file is tampered with.
--
   * Automatically cache preambles

 Currently preview-latex asks you whether you want to cache the
 document preamble (everything before '\begin{document}') before it
 generates previews for a buffer the first time.  Caching the
 preamble will significantly speed up regeneration of previews.  The
 larger your preamble is, the more this will be apparent.  Once a
 preamble is cached, preview-latex will try to keep track of when it
 is changed, and dump a fresh format in that case.  If you
 experience problems with this, or if you want it to happen without
 asking you the first time, you can customize the variable
 'preview-auto-cache-preamble'.
--
I hope these help.

> and the file primárias.tex should not be used here.

I think the basic idea is to "embed" the particular document's preamble
in the format file for efficiency, so I suppose that it is essential to
input the document file (primárias.tex here).

> And "\input" "\detokenize{primárias.tex}" should be either

> "\expandafter\input" "\detokenize{primárias.tex}"

> or

> "\input" "{\detokenize{primárias.tex}}"

OK, I'll change the relavant code accordingly.

> PS: I am CCing to auctex-dev but it seems my messages get bounced

Sorry, I misspelled the address in my previous message.  The right
address is "auctex-de...@gnu.org", not "auctex-dev@...".

Best regards,
Ikumi Keita

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Ah, I understand better some errors reported by Keita :


Le 23/05/2018 à 15:39, Ikumi Keita a écrit :

(1) Add prv_primárias.ini argument to creat .fmt file.
etex -ini -interaction=nonstopmode -jobname prv_prim\árias "" prv_primárias.ini 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" "\detokenize{primárias.tex}"
-> No error
pdflatex -interaction=nonstopmode \_prim\árias "\input{" "\detokenize{" prim\árias.tex 
"}" "}"
-> error
--
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
  restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
 .
  article.cls2014/09/29 v1.4h Standard LaTeX document class
   size10.clo2014/09/29 v1.4h Standard LaTeX file (size option)
 .
)
Runaway argument?
! File ended while scanning use of ^^M.

 \par
<*> _primárias \
  input{ \detokenize{ primárias.tex } }
! Emergency stop.
<*> ...ias \input{ \detokenize{ primárias.tex } }
   
!  ==> Fatal error occurred, no output PDF file produced!

Transcript written on .log.
--



It appears that mylatex.ltx is at work here.

Here is a copy of its comments

% While the preamble is being skipped, the EOL is active
% and defined to grab each line and inspect it looking
% for \begin{document} or mylatex lines.

Now I see also that it makes the backslash \ active.

From extra look in latex.ltx
EOL (^^M) is looking for an argument delimite by itself ^^M
but it never finds here the second ^^M.

Also latex.ltx contains various incantations and the
\catcode`\ 9 thing is one of them.

I need to get a hold of what prv_primárias.ini looks like!
So I guess I have to go into auctex el files...

Best,

Jean-François



___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi again Keita

Le 23/05/2018 à 15:39, Ikumi Keita a écrit :

Hi Jean,
[...]
The literally same commands succeed here as well on xterm.  However,
after modifying seemingly to coordinate the actual preview-latex
commands, the latter fails.

(1) Add prv_primárias.ini argument to creat .fmt file.
etex -ini -interaction=nonstopmode -jobname prv_prim\árias "" prv_primárias.ini 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" "\detokenize{primárias.tex}"
-> No error


Any etex -ini  must be ended by \dump command. It it is in prv_primárias.ini 
then what follows will be ignored.

I anticipate the role of the format is to catch up some common preamble,
andthe file primárias.tex should not be used here.

And "\input" "\detokenize{primárias.tex}" should be either

"\expandafter\input" "\detokenize{primárias.tex}"

or

"\input" "{\detokenize{primárias.tex}}"

But this is probably not at all relevant to the format thing.

PS: I am CCing to auctex-dev but it seems my messages get bounced
So this time I tried with replacing the mailing list with the
gmane newsgroup

Best,

Jean-François

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Keita,

Le 23/05/2018 à 15:39, Ikumi Keita a écrit :

Hi Jean,


jfbu  writes:



It is not easy for me to try out any patch to official AUCTeX
(does it have a github repo?)


It has savannah git repository:
https://git.savannah.gnu.org/cgit/auctex.git
Unfortunately, it doesn't on github.


I gave it a try with a creating a format like this



etex -ini -interaction=nonstopmode -jobname prv_prim\árias ""  
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}\dump"



(CAVEAT: for reasons explained above I do not look into AUCTeX ELisp files,
I just started from things in Keita's message and malaxed until I could
execute on commande line)



Then I executed manually this line (which I copied from the continuation of 
your message)



pdflatex -interaction=nonstopmode \_prim\árias "\input{" "\detokenize{" prim\árias.tex 
"}" "}"



and it compiled seemingly successfully



$ pdflatex -interaction=nonstopmode \_prim\árias "\input{" "\detokenize{" prim\árias.tex 
"}" "}"
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)


The literally same commands succeed here as well on xterm.  However,
after modifying seemingly to coordinate the actual preview-latex
commands, the latter fails.

(1) Add prv_primárias.ini argument to creat .fmt file.
etex -ini -interaction=nonstopmode -jobname prv_prim\árias "" prv_primárias.ini 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" "\detokenize{primárias.tex}"
-> No error
pdflatex -interaction=nonstopmode \_prim\árias "\input{" "\detokenize{" prim\árias.tex 
"}" "}"
-> error
--
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
  restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
 .
  article.cls2014/09/29 v1.4h Standard LaTeX document class
   size10.clo2014/09/29 v1.4h Standard LaTeX file (size option)
 .
)
Runaway argument?
! File ended while scanning use of ^^M.

 \par
<*> _primárias \
  input{ \detokenize{ primárias.tex } }
! Emergency stop.
<*> ...ias \input{ \detokenize{ primárias.tex } }
   
!  ==> Fatal error occurred, no output PDF file produced!

Transcript written on .log.
--
(2) Add \dump furthermore.
etex -ini -interaction=nonstopmode -jobname prv_prim\árias "" prv_primárias.ini 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}\dump"
 "\input" "\detokenize{primárias.tex}"
-> No error
pdflatex -interaction=nonstopmode \_prim\árias "\input{" "\detokenize{" prim\árias.tex 
"}" "}"
-> No error, but doesn't do its expected job
--
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
  restricted \write18 enabled.
entering extended mode
LaTeX2e <2018-04-01> patch level 4
(./primárias.tex
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29v1.4h StandardLaTeXdocumentclass
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo)) )
(\dump is performed only by INITEX)
No pages of output.
Transcript written on primárias.log.
--


I will be busy for today,


I'm sorry to have bothered you many times.  I deeply appreciate your
kindness so far!



no bothering at all!



Perhaps I should give up this particular case (non-ascii file name +
preview-latex preamble cache) and commit the patch which fixes other
issues.



surely not!

What would helped me is to see the exact content of an ini file.

From another message of yours I have this

--
, which can be obtained by C-c C-p C-f.  The init file prv_primárias.ini
is prepared by preview-latex by the following elisp code in
`preview-cache-preamble':
--
  ;; mylatex.ltx expects a file name to follow.  Bad. `.tex'
  ;; in the tools bundle is an empty file.
  (write-region "\\ifx\\pdfoutput\\undefined\\else\
\\let\\PREVIEWdump\\dump\\def\\dump{%
\\edef\\next{{\\catcode`\\ 9 

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

Hi Jean,

> jfbu  writes:

> It is not easy for me to try out any patch to official AUCTeX
> (does it have a github repo?)

It has savannah git repository:
https://git.savannah.gnu.org/cgit/auctex.git
Unfortunately, it doesn't on github.

> I gave it a try with a creating a format like this

> etex -ini -interaction=nonstopmode -jobname prv_prim\árias ""  
> "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}\dump"

> (CAVEAT: for reasons explained above I do not look into AUCTeX ELisp files,
> I just started from things in Keita's message and malaxed until I could
> execute on commande line)

> Then I executed manually this line (which I copied from the continuation of 
> your message)

> pdflatex -interaction=nonstopmode \_prim\árias "\input{" "\detokenize{" 
> prim\árias.tex "}" "}"

> and it compiled seemingly successfully

> $ pdflatex -interaction=nonstopmode \_prim\árias "\input{" "\detokenize{" 
> prim\árias.tex "}" "}"
> This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
> format=pdflatex)

The literally same commands succeed here as well on xterm.  However,
after modifying seemingly to coordinate the actual preview-latex
commands, the latter fails.

(1) Add prv_primárias.ini argument to creat .fmt file.
etex -ini -interaction=nonstopmode -jobname prv_prim\árias "" 
prv_primárias.ini 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" "\detokenize{primárias.tex}"
-> No error
pdflatex -interaction=nonstopmode \_prim\árias "\input{" "\detokenize{" 
prim\árias.tex "}" "}"
-> error
--
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
.
 article.cls2014/09/29 v1.4h Standard LaTeX document class
  size10.clo2014/09/29 v1.4h Standard LaTeX file (size option)
.
)
Runaway argument?
! File ended while scanning use of ^^M.
 
\par 
<*> _primárias \
 input{ \detokenize{ primárias.tex } }
! Emergency stop.
<*> ...ias \input{ \detokenize{ primárias.tex } }
  
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on .log.
--
(2) Add \dump furthermore.
etex -ini -interaction=nonstopmode -jobname prv_prim\árias "" 
prv_primárias.ini 
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}\dump"
 "\input" "\detokenize{primárias.tex}"
-> No error
pdflatex -interaction=nonstopmode \_prim\árias "\input{" "\detokenize{" 
prim\árias.tex "}" "}"
-> No error, but doesn't do its expected job
--
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
 restricted \write18 enabled.
entering extended mode
LaTeX2e <2018-04-01> patch level 4
(./primárias.tex
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29v1.4h StandardLaTeXdocumentclass
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo)) )
(\dump is performed only by INITEX)
No pages of output.
Transcript written on primárias.log.
--

> I will be busy for today,

I'm sorry to have bothered you many times.  I deeply appreciate your
kindness so far!
Perhaps I should give up this particular case (non-ascii file name +
preview-latex preamble cache) and commit the patch which fixes other
issues.

Best regards,
Ikumi Keita

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Ikumi

I had not read your initial message closely enough.

Le 23/05/2018 à 13:34, Ikumi Keita a écrit :
> Thank you very much, Jean,
> 
>> jfbu  writes:
>>>
>>> On emacs invoked on UTF-8 locale, when I open the test file and type C-c
>>> C-p C-d, answering with y to "Cache preamble?", pdflatex succeeds to
>>> dump prv_primárias.fmt.  However, the run of pdflatex to generate actual
>>> image which follows that dump fails with TeXLive 2018 like this:
>>> --
>>> Running `Preview-LaTeX' on `primárias' with ``pdflatex 
>>> -interaction=nonstopmode \_prim\árias  prim\árias.tex ''
>>> This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
>>> format=pdflatex)
>>> restricted \write18 enabled.
>>> entering extended mode
>>> ! I can't find file `prim'.
>>> 
>>> \unhbox
>>> <*> _primárias primá
>>> rias.tex
>>> (Press Enter to retry, or Control-D to exit)
>>> Please type another input file name
>>> ! Emergency stop.
>>> 
>>> \unhbox
>>> <*> _primárias primá
>>> rias.tex
>>> !  ==> Fatal error occurred, no output PDF file produced!
>>> Transcript written on texput.log.
>>>
>>> TeX Output exited as expected with code 1 at Wed May 23 18:36:15
>>> LaTeX: LaTeX found no preview images
>>> --
> 
>> Can you please configure AUCTeX to issue this instead (I hope my mailer does 
>> not add linebreaks)
> 
>> pdflatex  -file-line-error   
>> "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
>>  "\input" {\\detokenize{prim\árias.tex}}
> 
>> indeed LaTeX's \input checks if there is an opening brace, if not it uses 
>> TeX's form of \input and the \detokenize is no good. Alternative
> 
>> pdflatex  -file-line-error   
>> "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
>>  "\expandafter\input" \\detokenize{prim\árias.tex}
> 
>> Hope it helps,
> 
> Unfortunately, it didn't work.


What happened is that I did something like C-cC-pC-d at my locale,
there was an error, I recovered (no idea how) the command line invocation
which had failed and ended up with the slight modification above
(which later I modified again in my numerous follow-ups) which works
at my locale on commande line.

But it has nothing to do with using a custom format, so here we
see I had not read your message at all but only copied pasted
the MWE. Besides I am not using dev repo of AUCTeX but stock
installation (updated some weeks ago if I recall well).

My TeX-command-list diverged long time ago from official one
and this may interfere too.

It is not easy for me to try out any patch to official AUCTeX
(does it have a github repo?)

I gave it a try with a creating a format like this

etex -ini -interaction=nonstopmode -jobname prv_prim\árias ""  
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}\dump"

(CAVEAT: for reasons explained above I do not look into AUCTeX ELisp files,
I just started from things in Keita's message and malaxed until I could
execute on commande line)


Then I executed manually this line (which I copied from the continuation of 
your message)

pdflatex -interaction=nonstopmode \_prim\árias "\input{" "\detokenize{" 
prim\árias.tex "}" "}"

and it compiled seemingly successfully

$ pdflatex -interaction=nonstopmode \_prim\árias "\input{" "\detokenize{" 
prim\árias.tex "}" "}"
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
 restricted \write18 enabled.
entering extended mode
LaTeX2e <2018-04-01> patch level 4
(./primárias.tex
(/path/to/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/path/to/texlive/2018/texmf-dist/tex/latex/base/size10.clo))
No file primárias.aux.
(/path/to/texlive/2018/texmf-dist/tex/latex/preview/preview.sty
(/path/to/texlive/2018/texmf-dist/tex/generic/luatex85/luatex85.sty
)
(/path/to/texlive/2018/texmf-dist/tex/latex/preview/prtightpage.def
) (/path/to/texlive/2018/texmf-dist/tex/latex/preview/prauctex.def
No auxiliary output files.


(/path/to/texlive/2018/texmf-dist/tex/latex/preview/prauctex.cfg))
(/path/to/texlive/2018/texmf-dist/tex/latex/preview/prfootnotes.def
)
Preview: Fontsize 10pt
Preview: PDFoutput 1
)
! Preview: Snippet 1 started.
<-><->
  
l.5 \(
  abc\)
Preview: Tightpage -32891 -32891 32891 32891
! Preview: Snippet 1 ended.(455111+0x911285).
<-><->
  
l.5 \(abc\)
   

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

> jfbu  writes:

> If filename is containing # or ~ it must have them like \string# and \string~ 
> in the above

> (I made a brief test with file primárias~.tex and the above with 
> \input{primárias\string~.tex} worked on command line)

Thanks, I think we don't have to care about "#" and "~" in the file
name on the command line.  The code in AUCTeX has not taken care of
those characters on the command line long since, and such file names has
not been processed right even before TL 2018.
I don't know why `TeX-quote-filename' pays special attention to "#" and
"~" whereas no special treatment is done on "#" and "~" in the file
name on the command line and doubt the necessity of such treatment, but
I tried to keep compatibility with the current code since I was not 100%
sure.

Regards,
Ikumi Keita

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

Thank you very much, Jean,

> jfbu  writes:
>> 
>> On emacs invoked on UTF-8 locale, when I open the test file and type C-c
>> C-p C-d, answering with y to "Cache preamble?", pdflatex succeeds to
>> dump prv_primárias.fmt.  However, the run of pdflatex to generate actual
>> image which follows that dump fails with TeXLive 2018 like this:
>> --
>> Running `Preview-LaTeX' on `primárias' with ``pdflatex 
>> -interaction=nonstopmode \_prim\árias  prim\árias.tex ''
>> This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
>> format=pdflatex)
>> restricted \write18 enabled.
>> entering extended mode
>> ! I can't find file `prim'.
>> 
>> \unhbox
>> <*> _primárias primá
>> rias.tex
>> (Press Enter to retry, or Control-D to exit)
>> Please type another input file name
>> ! Emergency stop.
>> 
>> \unhbox
>> <*> _primárias primá
>> rias.tex
>> !  ==> Fatal error occurred, no output PDF file produced!
>> Transcript written on texput.log.
>> 
>> TeX Output exited as expected with code 1 at Wed May 23 18:36:15
>> LaTeX: LaTeX found no preview images
>> --

> Can you please configure AUCTeX to issue this instead (I hope my mailer does 
> not add linebreaks)

> pdflatex  -file-line-error   
> "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
>  "\input" {\\detokenize{prim\árias.tex}}

> indeed LaTeX's \input checks if there is an opening brace, if not it uses 
> TeX's form of \input and the \detokenize is no good. Alternative

> pdflatex  -file-line-error   
> "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
>  "\expandafter\input" \\detokenize{prim\árias.tex}

> Hope it helps,

Unfortunately, it didn't work.
--
Running `Preview-LaTeX' on `primárias' with ``pdflatex -interaction=nonstopmode 
\_prim\árias "\input{" "\detokenize{" prim\árias.tex "}" "}"''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
.
 article.cls2014/09/29 v1.4h Standard LaTeX document class
  size10.clo2014/09/29 v1.4h Standard LaTeX file (size option)
.
)
Runaway argument?
! File ended while scanning use of ^^M.
 
\par 
<*> _primárias \
 input{ \detokenize{ primárias.tex } }
! Emergency stop.
<*> ...ias \input{ \detokenize{ primárias.tex } }
  
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on .log.

TeX Output exited as expected with code 1 at Wed May 23 20:06:31
LaTeX: LaTeX found no preview images
--

Unlike the case preamble cache is disabled, the current case does not
use the complex stuff of 
"\nonstopmode\nofiles\PassOptionsToPackage{...}{preview}\AtBeginDocument{...}"
at the pdflatex run generating the actual image.  It runs in a simple
form:

pdflatex -interaction=nonstopmode _primárias FILENAME

The complex stuff has been used already when dumping the customized
format prv_primárias.fmt and, I guess, embedded in that format file.
The output when preview-latex dumps the customized format is:
--
Running `Preview-LaTeX' on `primárias' with ``pdflatex -ini 
-interaction=nonstopmode "" prv_prim\árias.ini
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" "\detokenize{" prim\árias.tex "}"''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (INITEX)
 restricted \write18 enabled.
entering extended mode
(./prv_primárias.ini
LaTeX2e <2018-04-01> patch level 4
(/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx) 
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex File ignored))
No auxiliary output files.

(./primárias.tex (/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo)) )
Beginning to dump on file prv_primárias.fmt
 (preloaded format=prv_primárias 2018.5.23)
7584 strings of total length 123267
58322 memory locations dumped; current usage is 

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

You can replace \begingroup by { and \endgroup by }

pdflatex  -file-line-error   
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 
"{\ifdefined\UseRawInputEncoding\UseRawInputEncoding\fi\edef\x{\noexpand\input{primárias.tex}}\expandafter}\x"




and to be compatible with non-etex
as I see an \ifx\undefined test is used also before, so let's do same

pdflatex  -file-line-error   
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 
"{\ifx\UseRawInputEncoding\undefined\else\UseRawInputEncoding\fi\edef\x{\noexpand\input{primárias.tex}}\expandafter}\x"


If filename is containing # or ~ it must have them like \string# and \string~ 
in the above

(I made a brief test with file primárias~.tex and the above with 
\input{primárias\string~.tex} worked on command line)


Sorry for not having posted this initially

Jean-François

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Le 23/05/2018 à 12:46, jfbu a écrit :

Le 23/05/2018 à 12:33, jfbu a écrit :

Hi Keita

Can you please configure AUCTeX to issue this instead (I hope my mailer does 
not add linebreaks)

pdflatex  -file-line-error   
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" {\\detokenize{prim\árias.tex}}

indeed LaTeX's \input checks if there is an opening brace, if not it uses TeX's 
form of \input and the \detokenize is no good. Alternative

pdflatex  -file-line-error   
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\expandafter\input" \\detokenize{prim\árias.tex}



The alternative (as you mentioned earlier \detokenize doesn't play well with 
\string# or with naked #, in the style of my previous post





jfbu  writes:



Perhaps you can use this mouthful



pdflatex  -file-line-error   -interaction=nonstopmode 
"\begingroup\ifdefined\UseRawInputEncoding\UseRawInputEncoding\fi\edef\x{\noexpand\input{\string#éàè\string~\string#.tex}}\expandafter\endgroup\x"




would be

pdflatex  -file-line-error   
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 
"\begingroup\ifdefined\UseRawInputEncoding\UseRawInputEncoding\fi\edef\x{\noexpand\input{primárias.tex}}\expandafter\endgroup\x"




You can replace \begingroup by { and \endgroup by }

pdflatex  -file-line-error   
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 
"{\ifdefined\UseRawInputEncoding\UseRawInputEncoding\fi\edef\x{\noexpand\input{primárias.tex}}\expandafter}\x"


Jean-François


where the filename if containing # or ~ must have them escaped as \string# and 
\string~

I don't know what is proper quoting on Windows, but here on bash the á withing 
".." must not be shell-escaped.

The above works at my locale. It is the most powerful method because it can 
handle filenames with # and ~.

Best

Jean-François



___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Le 23/05/2018 à 12:33, jfbu a écrit :

Hi Keita

Can you please configure AUCTeX to issue this instead (I hope my mailer does 
not add linebreaks)

pdflatex  -file-line-error   
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" {\\detokenize{prim\árias.tex}}

indeed LaTeX's \input checks if there is an opening brace, if not it uses TeX's 
form of \input and the \detokenize is no good. Alternative

pdflatex  -file-line-error   
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\expandafter\input" \\detokenize{prim\árias.tex}



The alternative (as you mentioned earlier \detokenize doesn't play well with 
\string# or with naked #, in the style of my previous post





jfbu  writes:



Perhaps you can use this mouthful



pdflatex  -file-line-error   -interaction=nonstopmode 
"\begingroup\ifdefined\UseRawInputEncoding\UseRawInputEncoding\fi\edef\x{\noexpand\input{\string#éàè\string~\string#.tex}}\expandafter\endgroup\x"




would be

pdflatex  -file-line-error   
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 
"\begingroup\ifdefined\UseRawInputEncoding\UseRawInputEncoding\fi\edef\x{\noexpand\input{primárias.tex}}\expandafter\endgroup\x"

where the filename if containing # or ~ must have them escaped as \string# and 
\string~

I don't know what is proper quoting on Windows, but here on bash the á withing 
".." must not be shell-escaped.

The above works at my locale. It is the most powerful method because it can 
handle filenames with # and ~.

Best

Jean-François

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Keita


Le 23/05/2018 à 12:03, Ikumi Keita a écrit :

Hi Jean and all,

I could sort out most of the problem by a new approach attached with this
message, but am having difficulty with enabling preamble cache on
preview-latex when the document has non-ascii file name.  Could you help
me get out of this trouble if possible?

I'm using a test file primárias.tex (mimicking example of
https://tex.stackexchange.com/questions/429869/emacs-auctex-tl-2018-problems-with-non-ascii-characters-in-the-file-name)
with a math formula to test preview-latex functionality:
\documentclass{article}

\begin{document}

\(abc\)

\end{document}
%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:

On emacs invoked on UTF-8 locale, when I open the test file and type C-c
C-p C-d, answering with y to "Cache preamble?", pdflatex succeeds to
dump prv_primárias.fmt.  However, the run of pdflatex to generate actual
image which follows that dump fails with TeXLive 2018 like this:
--
Running `Preview-LaTeX' on `primárias' with ``pdflatex -interaction=nonstopmode 
\_prim\árias  prim\árias.tex ''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
  restricted \write18 enabled.
entering extended mode
! I can't find file `prim'.

\unhbox
<*> _primárias primá
   rias.tex
(Press Enter to retry, or Control-D to exit)
Please type another input file name
! Emergency stop.

\unhbox
<*> _primárias primá
   rias.tex
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on texput.log.

TeX Output exited as expected with code 1 at Wed May 23 18:36:15
LaTeX: LaTeX found no preview images
--

It doesn't help to change the command to
pdflatex \_prim\árias \\input \\detokenize\{prim\árias.tex}
on xterm.  Pdflatex says:
--
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
  restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
 .
  article.cls2014/09/29 v1.4h Standard LaTeX document class
   size10.clo2014/09/29 v1.4h Standard LaTeX file (size option)
 .
)
Runaway argument?
! File ended while scanning use of ^^M.

 \par
<*> _primárias \
  input \detokenize{primárias.tex}
! Emergency stop.
<*> ...imárias \input \detokenize{primárias.tex}
   
!  ==> Fatal error occurred, no output PDF file produced!

Transcript written on .log.
--

With TL2017, preview-latex works as expected even with my new patch.

Jean, do you have any idea, such as "magic spell" in the message below
you wrote previously?  I have vague impression that \UseRawInputEncoding
can solve this problem, but I have no experience of TeX programming and
could not arrage the given TeX codes so that prv_primárias.fmt accepts
the file name primárias.tex.



Can you please configure AUCTeX to issue this instead (I hope my mailer does 
not add linebreaks)

pdflatex  -file-line-error   
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\input" {\\detokenize{prim\árias.tex}}

indeed LaTeX's \input checks if there is an opening brace, if not it uses TeX's 
form of \input and the \detokenize is no good. Alternative

pdflatex  -file-line-error   
"\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}"
 "\expandafter\input" \\detokenize{prim\árias.tex}

Hope it helps,

Best

Jean-François




jfbu  writes:



Perhaps you can use this mouthful



pdflatex  -file-line-error   -interaction=nonstopmode 
"\begingroup\ifdefined\UseRawInputEncoding\UseRawInputEncoding\fi\edef\x{\noexpand\input{\string#éàè\string~\string#.tex}}\expandafter\endgroup\x"



It works to compile file with name #éàè~#.tex, the file containing itself
UTF-8 characters in its contents and is not doing \usepackage[utf8]{inputenc}
to check it does work nevertheless in PDF output (it would not if action of
\UseRawInputEncoding wasn't scoped)


Sincerly,
Ikumi Keita



___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex




___
auctex mailing list
auctex@gnu.org

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

Hi Jean and all,

I could sort out most of the problem by a new approach attached with this
message, but am having difficulty with enabling preamble cache on
preview-latex when the document has non-ascii file name.  Could you help
me get out of this trouble if possible?

I'm using a test file primárias.tex (mimicking example of
https://tex.stackexchange.com/questions/429869/emacs-auctex-tl-2018-problems-with-non-ascii-characters-in-the-file-name)
with a math formula to test preview-latex functionality:
\documentclass{article}

\begin{document}

\(abc\)

\end{document}
%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:

On emacs invoked on UTF-8 locale, when I open the test file and type C-c
C-p C-d, answering with y to "Cache preamble?", pdflatex succeeds to
dump prv_primárias.fmt.  However, the run of pdflatex to generate actual
image which follows that dump fails with TeXLive 2018 like this:
--
Running `Preview-LaTeX' on `primárias' with ``pdflatex -interaction=nonstopmode 
\_prim\árias  prim\árias.tex ''
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
 restricted \write18 enabled.
entering extended mode
! I can't find file `prim'.
 
   \unhbox 
<*> _primárias primá
  rias.tex
(Press Enter to retry, or Control-D to exit)
Please type another input file name
! Emergency stop.
 
   \unhbox 
<*> _primárias primá
  rias.tex
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on texput.log.

TeX Output exited as expected with code 1 at Wed May 23 18:36:15
LaTeX: LaTeX found no preview images
--

It doesn't help to change the command to
pdflatex \_prim\árias \\input \\detokenize\{prim\árias.tex}
on xterm.  Pdflatex says:
--
This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded 
format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex
LaTeX2e <2018-04-01> patch level 4
CUSTOMISED FORMAT. Preloaded files:
.
 article.cls2014/09/29 v1.4h Standard LaTeX document class
  size10.clo2014/09/29 v1.4h Standard LaTeX file (size option)
.
)
Runaway argument?
! File ended while scanning use of ^^M.
 
\par 
<*> _primárias \
 input \detokenize{primárias.tex}
! Emergency stop.
<*> ...imárias \input \detokenize{primárias.tex}
  
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on .log.
--

With TL2017, preview-latex works as expected even with my new patch.

Jean, do you have any idea, such as "magic spell" in the message below
you wrote previously?  I have vague impression that \UseRawInputEncoding
can solve this problem, but I have no experience of TeX programming and
could not arrage the given TeX codes so that prv_primárias.fmt accepts
the file name primárias.tex.

> jfbu  writes:

> Perhaps you can use this mouthful

> pdflatex  -file-line-error   -interaction=nonstopmode 
> "\begingroup\ifdefined\UseRawInputEncoding\UseRawInputEncoding\fi\edef\x{\noexpand\input{\string#éàè\string~\string#.tex}}\expandafter\endgroup\x"

> It works to compile file with name #éàè~#.tex, the file containing itself
> UTF-8 characters in its contents and is not doing \usepackage[utf8]{inputenc}
> to check it does work nevertheless in PDF output (it would not if action of
> \UseRawInputEncoding wasn't scoped)

Sincerly,
Ikumi Keita



tentative-patch.gz
Description: new approach
___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Le 21/05/2018 à 11:34, jfbu a écrit :

Le 21/05/2018 à 11:04, jfbu a écrit :

If AUCTeX provides a user-interface for extra TeX commands on
command-line, via some variable, couldn't it check for that
variable being or not nil. If nil, do not use \input, if not
nil, use the \input with whatever extra measures are needed
from the fact that any command will be executed after the format
is loaded, hence the non-ascii characters assumed to be utf-8
and their first byte is active.


Ah sorry for the confusion in this paragraph.

In fact, the utf-8 first bytes are active in both cases,
but in the case of

pdflatex filename

they have a temporary definition based on \string
and the thing works

For reference on this, these links

http://tug.org/pipermail/tex-live/2018-May/041669.html
https://github.com/latex3/latex2e/commit/93b375e40e3dbfd57093e5b527be1d357a942a99

indirectly explain it, because there was a problem on Windows
that utf-8 filenames are presented to the binary in another encoding.

The commit above contains modified definitions for the
LaTeX \UTFviii@ macros, but indirectly it does illustrate
that the mechanism is in place already in

pdflatex filename

situation.

The difference with

pdflatex "\input" filename

is that \input is processed at a later time and the LaTeX
\UFTviii@... macros have by then another meaning.

I think.
 
Best


Jean-François



Hi Keita, (sorry for having mixed-up firstname and lastname)

A.

In TeX-expand-list-builtin, there is in PDFout a possibility
if variable TeX-DVI-via-PDFTeX is t that the command line will
do \pdfoutput=0. This is a situation which would then necessitate
usage of \input, because it is either \commands or filename.

However that possibility looks very obsolete because it is
long since that "latex" command is aliased to pdfetex in a
way which sets \pdfoutput without having to do it manually.

B.

About this in the patch under discussion

+  ;; Since TeXLive 2018, the default encoding for LaTeX files has been
+  ;; changed to UTF-8 if used with classic TeX or pdfTeX.  I.e.,
+  ;; \usepackage[utf8]{inputenc} is enabled by default in (pdf)latex.
+  ;; c.f. LaTeX News issue 28
+  ;; Due to this change, \detokenize is required to recognize non
+  ;; ascii characters in the file name.

The exact explanation would be that this is required in case the
\input method of executing the file is used. It is not required
in the case of "pdflatex filename" invocation, not because the non-ascii
letters are not handled in a special way
but because they are handled in a self-innocenting way
as long as \everyjob has not been executed after which they acquire
their proper \usepackage[utf8]{inputenc} form.

Best

Jean-François




___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Le 21/05/2018 à 11:04, jfbu a écrit :

If AUCTeX provides a user-interface for extra TeX commands on
command-line, via some variable, couldn't it check for that
variable being or not nil. If nil, do not use \input, if not
nil, use the \input with whatever extra measures are needed
from the fact that any command will be executed after the format
is loaded, hence the non-ascii characters assumed to be utf-8
and their first byte is active.


Ah sorry for the confusion in this paragraph.

In fact, the utf-8 first bytes are active in both cases,
but in the case of

pdflatex filename

they have a temporary definition based on \string
and the thing works

For reference on this, these links

http://tug.org/pipermail/tex-live/2018-May/041669.html
https://github.com/latex3/latex2e/commit/93b375e40e3dbfd57093e5b527be1d357a942a99

indirectly explain it, because there was a problem on Windows
that utf-8 filenames are presented to the binary in another encoding.

The commit above contains modified definitions for the
LaTeX \UTFviii@ macros, but indirectly it does illustrate
that the mechanism is in place already in

pdflatex filename

situation.

The difference with

pdflatex "\input" filename

is that \input is processed at a later time and the LaTeX
\UFTviii@... macros have by then another meaning.

I think.

Best

Jean-François


___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Ikumi

Le 18/05/2018 à 08:36, Ikumi Keita a écrit :

Regarding your problem with \message{...}. Perhaps
\unexpanded will help you, as this shows (with April 2018 LaTeX)
\documentclass{article}
\begin{document}
\message{éàù}% gives about 72 macros 
\message{\unexpanded{éàù}}% just displays éàù
\message{\unexpanded{#}}% does not help the doubling... \string# helps
\end{document}

Thanks for suggestion.  It seems that \unexpanded and \detokenize have
basically the same effect within \message, but I suppose \unexpanded is
better for some reason, isn't it?



I don't see indeed a difference between \unexpanded and \detokenize
when used within \message (but I have not studied the e-TeX sources).

I mentioned \unexpanded because it was more natural to me in the
context of writing to a file external to TeX itself, whereas I
use \detokenize when I am still inside TeX. In effect \detokenize
changes catcodes which is a notion not having sense once the
data is outside of TeX. On the other hand \unexpanded inhibits
TeX expansion (macros, active characters) which is something else.
Traditional LaTeX code base was done before e-TeX (the latter
release date is I think 1999, and TeXLive makes it default for
pdflatex since 2004), and has a \protect mechanism when
writing to files, which uses TeX primitive \noexpand. Things
are much easier with \unexpanded.

It might be that \unexpanded which only prevents expansion has
less overhead than \detokenize which creates tokens with normalized
catcodes, but even if true that is not really important here.

Thus I _think_ \unexpanded and \detokenize achieve exact same
when used within \message, (they obey \escapechar the same,
they add a space after macro names except if \,
they double the #'s,...), but you may need to ping some authority
for confirmation. Perhaps ask for advice on the LaTeX-L mailing
list ? (https://github.com/latex3/latex2e#code-fixes)

Notice that both will require balanced braces inside their
arguments.

About the e-TeX extensions in the context of LaTeX, (not Plain TeX),
I am wondering if there exist realistically people using pdflatex
binary from 15 years ago together with up-to-date AUCTeX ?
(for an example although TL2004's pdflatex was with e-TeX,
I was myself stuck at a place with TL2004 for quite some time,
when I connected over via ssh, but Emacs itself was also very old).

About adding TeX commands on command line the current man page
of pdftex on TL2018 is like this

pdftex [options] [] [file|\commands]

which indeed says that it is either filename or commands, the latter
case being recognized by a starting backslash (once reaching pdftex
command line parsing).

If AUCTeX provides a user-interface for extra TeX commands on
command-line, via some variable, couldn't it check for that
variable being or not nil. If nil, do not use \input, if not
nil, use the \input with whatever extra measures are needed
from the fact that any command will be executed after the format
is loaded, hence the non-ascii characters assumed to be utf-8
and their first byte is active.

Best,

Jean-François





___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

Hi Jean, thank you very much for clarifying!

First of all, my thought about the behavior of \message was wrong.  I
wrote
> The output of \message{...} also changed in TeXLive 2018.  If
> non-ascii characters are in \message{...}, they are output in
> tokenized form by default in TL 2018.
before, but it didn't change in fact.  I'm sorry about that.  The dummy
document I used for test lacked \usepackage[utf8]{inputenc}, so it
looked to me as if the behavior had changed between TL 2017 and 2018.

And after reading carefully texdoc ltnews again, I noticed that it
states that "the default encoding for LaTeX files has been changed to
UTF-8 if used with classic TeX or pdfTeX" and realized latex variants
includeing xelatex, lualatex (and lambda, lamed?) were not affected by
the TL 2018 change.

> jfbu  writes:

> Perhaps you can use this mouthful

> pdflatex  -file-line-error   -interaction=nonstopmode 
> "\begingroup\ifdefined\UseRawInputEncoding\UseRawInputEncoding\fi\edef\x{\noexpand\input{\string#éàè\string~\string#.tex}}\expandafter\endgroup\x"
...
> I am indicating this if there exists context where using "\input" is needed,
> because preferred form is not use "\input" at all.

In fact `TeX-quote-filename' is used only within \message{}, so we don't
have to consider the command line argument as long as my patches are
concerned.  But thanks anyway.  (And your code above would be useful for
anyone else who prefers other approach than my patches.)

> Regarding your problem with \message{...}. Perhaps
> \unexpanded will help you, as this shows (with April 2018 LaTeX)

> \documentclass{article}

> \begin{document}
> \message{éàù}% gives about 72 macros 

> \message{\unexpanded{éàù}}% just displays éàù

> \message{\unexpanded{#}}% does not help the doubling... \string# helps
> \end{document}

Thanks for suggestion.  It seems that \unexpanded and \detokenize have
basically the same effect within \message, but I suppose \unexpanded is
better for some reason, isn't it?

Anyway, `TeX-quote-filename' has been long since incompatible with
\usepackage[utf8]{inputenc} if file name contains non ascii characters,
even before TL 2018.

> On my AUCTeX i have long since modified the "tex" invocation
> to be alias for "etex" because I need e-TeX daily even with Plain.

> But the "tex" can't be modified officially because it is Knuth tex.

I see.  Thanks.

> - I have absolutely no idea about Japanese platex/uplatex

It is known that pLaTeX and upLaTeX are also subject to the change about
default UTF-8 in TL 2018.  (discussed in
https://github.com/texjporg/platex/issues/67 .)

>> (a) Should we treat `doctex-mode' in the same way as `latex-mode' in
>> this case?  I'm not sure since I don't know docTeX.

> doctex mode is only an auctex mode related to the ltxdoc document
> class (or scrdoc) which incorporates doc package

> as such it is but slight variation of latex so yes it should be
> handled like latex-mode

Thanks to this instruction, I managed to make out a revised patch, which
is attached with this message.  I hope it satisfies those who are
interested in this topic.

> (I don't think doctex-mode has special facilities to emulate
> docstrip behaviour for extracting files, if it did some other problems
> may arise, for example I use doctex-mode sometimes for packages
> working with etex, not needing latex)

I'm not sure, but I suppose that would be a minor issue hopefully and
could be ignored at least for the moment.

Best regards,
Ikumi Keita



non-ascii-file-name-v2.gz
Description: revised patch
___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Le 17/05/2018 à 10:16, jfbu a écrit :


Perhaps you can use this mouthful

pdflatex  -file-line-error   -interaction=nonstopmode 
"\begingroup\ifdefined\UseRawInputEncoding\UseRawInputEncoding\fi\edef\x{\noexpand\input{\string#éàè\string~\string#.tex}}\expandafter\endgroup\x"



The above uses \ifdefined, which is from e-TeX. But LaTeX requires e-TeX since

quote from ltnews:

As of the January 2017
release of LATEX2e, e-TEX is required to build the format,
and attempting to build a format without the extensions
will fail.

For much longer major TeX distributions incorporated e-TeX extensions in 
pdflatex binary anyhow

Extract from TeXLive doc, entry for year 2004:

pdfetex is now the default engine for all formats except (plain) tex itself. 
(Of course it generates
DVI when run as latex, etc.) This means, among other things, that the 
microtypographic features
of pdftex are available in LATEX, ConTEXt, etc., as well as the e-TEX features

Best,

Jean-François

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Hi Ikumi

thanks a lot for tackling this,

Le 15 mai 2018 à 08:35, Ikumi Keita  a écrit :

> Hi all,
> 
>> (1) The output of \message{...} also changed in TeXLive 2018.  If
>> non-ascii characters are in \message{...}, they are output in tokenized
>> form by default in TL 2018.  This makes partial typset, such as C-c C-r,
>> C-c C-s and so on, to fail to recognize the right file name when source
>> correlate mode is enabeld, so I inserted another \detokenize{} in
>> `TeX-quote-filename'.
> 
> It turned out that my patch doesn't work as expected in some cases.
> When `TeX-quote-filename' inserts "\string" to escape "#" and "~" before
> \detokenize{} takes place, wrapping the file name inside \detokenize{}
> neutralizes the "\string" so that it appears as literal 7-character
> "\string", not escaping "#" and "~".  In addition, \detokenize{} doubles
> the pound sign "#" for some reason unknown to me.



Perhaps you can use this mouthful

pdflatex  -file-line-error   -interaction=nonstopmode 
"\begingroup\ifdefined\UseRawInputEncoding\UseRawInputEncoding\fi\edef\x{\noexpand\input{\string#éàè\string~\string#.tex}}\expandafter\endgroup\x"

It works to compile file with name #éàè~#.tex, the file containing itself
UTF-8 characters in its contents and is not doing \usepackage[utf8]{inputenc}
to check it does work nevertheless in PDF output (it would not if action of
\UseRawInputEncoding wasn't scoped)

The point is that the filename itself needs be quoted only old-fashioned way
with \string


I am indicating this if there exists context where using "\input" is needed,
because preferred form is not use "\input" at all.

pdflatex \#éàè\\string\~\#.tex

works fine where # only needs to be quoted from shell, not from LaTeX
but ~ needs to be quoted from LaTeX because it is active

I am ignorant in AUCTeX code base so it is a short in the dark)

Regarding your problem with \message{...}. Perhaps
\unexpanded will help you, as this shows (with April 2018 LaTeX)

\documentclass{article}

\begin{document}
\message{éàù}% gives about 72 macros 

\message{\unexpanded{éàù}}% just displays éàù

\message{\unexpanded{#}}% does not help the doubling... \string# helps
\end{document}



> 
>> (1) According to texdoc etex, \detokenize was added in e-TeX extension.
>> So if the user has quite old TeX distribution where e-TeX extension was
>> not incorporated in the engine (command binary) yet, \detokenize raises
>> error.  I hope this is a permissible incompatibility.
> 
> I realized that the default command binary "tex" for plain-TeX does not
> incorporate e-TeX extension, while the binaries of latex families
> ("latex", "pdflatex", "lualatex", ...) do.  (The e-TeX enabled binary
> producing dvi file for plain-TeX is "etex".)  We only need \detokenize
> for latex family because it is LaTeX kernel that the internal UTF-8-nize
> was performed on and other families including plain-TeX are not
> affected.

On my AUCTeX i have long since modified the "tex" invocation
to be alias for "etex" because I need e-TeX daily even with Plain.

But the "tex" can't be modified officially because it is Knuth tex.

As you said, the problem with active characters for support of UTF-8
is anyway a specific LaTeX one, and only for pdflatex, not lualatex/xelatex.


> 
> In addition, I heard that Omega does not have e-TeX extension, either.
> If this is the case, "lambda" is an exception in latex families and does
> not accept \detokenize.  (I cannot confirm because Omega binaries no
> longer exist in TeX Live these days.)


I can't help either

> 
> Summarizing these considerations, it seems that we have to do something
> like this:
> (defun TeX-quote-filename (file)
>  ... (snip) ...
>  (if (and (eq major-mode 'latex-mode)
>  (not (and (eq TeX-engine 'omega)
>;; lamed, Aleph version of lambda, has e-TeX extension.
>(equal LaTeX-Omega-command "lambda"
>  (replace-regexp-in-string "[[:multibyte:]]+"
>   "detokenize{\\&}" file t)
>file))
> 


Although \detokenize should not hurt (*) it is not needed
with lualatex/xelatex

(*) except for # problem

- if no \input, then no need for \detokenize

- if \input, possible to use \UseRawInputEncoding as in mouthful above,
but one must test for its existence
to remain compatible with LaTeX earlier than April 2018 (as is done in mouthful)
(I tested above "solution" both with TL2016 and TL2018)

- the above problems are only with LaTeX + 8bit engine 

- I have absolutely no idea about Japanese platex/uplatex


> Some notes.
> (a) Should we treat `doctex-mode' in the same way as `latex-mode' in
> this case?  I'm not sure since I don't know docTeX.

doctex mode is only an auctex mode related to the ltxdoc document class (or 
scrdoc)
which incorporates doc package

as such it is but slight variation of latex so yes it should be handled like
latex-mode (I don't think doctex-mode has special facilities to emulate

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

Hi all,

> (1) The output of \message{...} also changed in TeXLive 2018.  If
> non-ascii characters are in \message{...}, they are output in tokenized
> form by default in TL 2018.  This makes partial typset, such as C-c C-r,
> C-c C-s and so on, to fail to recognize the right file name when source
> correlate mode is enabeld, so I inserted another \detokenize{} in
> `TeX-quote-filename'.

It turned out that my patch doesn't work as expected in some cases.
When `TeX-quote-filename' inserts "\string" to escape "#" and "~" before
\detokenize{} takes place, wrapping the file name inside \detokenize{}
neutralizes the "\string" so that it appears as literal 7-character
"\string", not escaping "#" and "~".  In addition, \detokenize{} doubles
the pound sign "#" for some reason unknown to me.

> (1) According to texdoc etex, \detokenize was added in e-TeX extension.
> So if the user has quite old TeX distribution where e-TeX extension was
> not incorporated in the engine (command binary) yet, \detokenize raises
> error.  I hope this is a permissible incompatibility.

I realized that the default command binary "tex" for plain-TeX does not
incorporate e-TeX extension, while the binaries of latex families
("latex", "pdflatex", "lualatex", ...) do.  (The e-TeX enabled binary
producing dvi file for plain-TeX is "etex".)  We only need \detokenize
for latex family because it is LaTeX kernel that the internal UTF-8-nize
was performed on and other families including plain-TeX are not
affected.

In addition, I heard that Omega does not have e-TeX extension, either.
If this is the case, "lambda" is an exception in latex families and does
not accept \detokenize.  (I cannot confirm because Omega binaries no
longer exist in TeX Live these days.)

Summarizing these considerations, it seems that we have to do something
like this:
(defun TeX-quote-filename (file)
  ... (snip) ...
  (if (and (eq major-mode 'latex-mode)
   (not (and (eq TeX-engine 'omega)
 ;; lamed, Aleph version of lambda, has e-TeX extension.
 (equal LaTeX-Omega-command "lambda"
  (replace-regexp-in-string "[[:multibyte:]]+"
"detokenize{\\&}" file t)
file))

Some notes.
(a) Should we treat `doctex-mode' in the same way as `latex-mode' in
this case?  I'm not sure since I don't know docTeX.
(b) Note that preview-latex does not work with "lambda" even if
`TeX-quote-filename' is modified as above.  I hope this is not a major
drawback because very few users still use Omega now, IIUC.

> N.B.  The initial release of TeXLive 2018 lacks the file ".tex" [1],
> which makes preview-latex to fail to dump format file when preamble
> caching is enabled.  Running `tlmgr update --all' doesn't recover
> ".tex" at the time of writing this paragraph, thus it seems that the
> latest TL repositry still lacks ".tex".

Now TL repository has ".tex".  Running `tlmgr update --all' will restore
".tex" in the texmf tree of TL 2018.

Regards,
Ikumi Keita

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

Hi Mosè and David,

> David Kastrup  writes:
> Mosè Giordano  writes:

>> I seem to remember David explained that "\input" was used to make
>> something work (region-compilation?).  I looked for this explanation
>> in my inbox but I can't find it, hopefully David can tell us more.  If
>> "\input" is really not needed anymore then removing "\input" when
>> calling TeX is a good solution.

> I think it was for setting interaction mode in a reasonably portable
> way.  At the current point of time, we likely just have to deal with
> MikTeX and TeXlive.

Thank you for responses.  I performed search on the ML archive and seem
to find the David's message which Mosè mentioned:
https://lists.gnu.org/archive/html/auctex-devel/2008-05/msg00055.html

First, David worried about the case some TeX codes are put on the
command line like this:
latex -interaction=nonstopmode \SOME_CODE FILENAME
My impression is that such case is extremely rare, at least when the
default `TeX-command-list' is used.  (In theory, a user can put TeX
codes by either setting `TeX-command-extra-options' or customizing
`TeX-expand-list' to override the default expansion of the directives
such as "%(mode)", "%(file-line-error)" and so on, but...)  So I expect
we can live without \input as long as we take care of preview-latex to
work with \input.
David, what do you think about this?

Next, David pointed out that removing \input does not help %&-parsing of
the first line in the case of region compilation.  (This is not a reason
that \input is necessary, I think.)
This is a problem I looked over, and I expect that the attached
additional patch covers the point.

Regards,
Ikumi Keita

P.S.  Two notes about my previous patch
(1) According to texdoc etex, \detokenize was added in e-TeX extension.
So if the user has quite old TeX distribution where e-TeX extension was
not incorporated in the engine (command binary) yet, \detokenize raises
error.  I hope this is a permissible incompatibility.

(2) This patch incidentally fixes Bug#25276:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25276
The problem is present only in xetex engine on w32 platform.  Both the
form without \input and with \input \detokenize{} works well even when
the file name contains space, while the form with \input only (original
AUCTeX form) stops with an error.

diff --git a/tex-buf.el b/tex-buf.el
--- a/tex-buf.el
+++ b/tex-buf.el
@@ -2092,8 +2092,10 @@
 			   (if (not (re-search-forward TeX-header-end nil t))
 			   ""
 			 (re-search-forward "[\r\n]" nil t)
-			 (buffer-substring (point-min) (point)
+			 (buffer-substring-no-properties
+			  (point-min) (point)
 	 (header-offset 0)
+	 first-line
 	 ;; We search for the trailer from the master file, if it is
 	 ;; not present in the region.
 	 (trailer-offset 0)
@@ -2113,11 +2115,24 @@
 			  ;;(beginning-of-line 1)
 			  (re-search-backward "[\r\n]" nil t)
 			  (setq trailer-offset (TeX-current-offset))
-			  (buffer-substring (point) (point-max))
+			  (buffer-substring-no-properties
+			   (point) (point-max))
 ;; file name should be relative to master
 (setq original (TeX-quote-filename (file-relative-name
 	original (TeX-master-directory)))
 	  master-name (TeX-quote-filename master-name))
+
+;; Cater for % construct in the first line.
+(setq first-line (if (and (> (length header) 1)
+			  (string= (substring header 0 2) "%&"))
+			 ;; This would work even if header has no newline.
+			 (substring header 0 (string-match "\n" header))
+		   ""))
+(unless (string= first-line "")
+  ;; Remove first-line from header.
+  (setq header (substring header (length first-line)))
+  (setq first-line (concat first-line "\n")))
+
 (with-current-buffer file-buffer
   (setq buffer-read-only t
 	buffer-undo-list t)
@@ -2127,7 +2142,8 @@
 	(when (boundp 'buffer-file-coding-system)
 	  (setq buffer-file-coding-system
 		(with-current-buffer master-buffer buffer-file-coding-system)))
-	(insert "\\message{ !name(" master-name ")}"
+	(insert first-line
+		"\\message{ !name(" master-name ")}"
 		header
 		TeX-region-extra
 		"\n\\message{ !name(" original ") !offset(")
___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[David Kastrup]

Mosè Giordano  writes:

> Hi Keita,
>
> 2018-05-06 8:52 GMT+02:00 Ikumi Keita :
>> Thus I think it is basically unnecessary to adhere using \input and now
>> we can do just
>> latex -interaction=nonstopmode FILENAME
>> instead.
>
> I seem to remember David explained that "\input" was used to make
> something work (region-compilation?).  I looked for this explanation
> in my inbox but I can't find it, hopefully David can tell us more.  If
> "\input" is really not needed anymore then removing "\input" when
> calling TeX is a good solution.

I think it was for setting interaction mode in a reasonably portable
way.  At the current point of time, we likely just have to deal with
MikTeX and TeXlive.

-- 
David Kastrup

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Mosè Giordano]

Hi Keita,

2018-05-06 8:52 GMT+02:00 Ikumi Keita :
> Thus I think it is basically unnecessary to adhere using \input and now
> we can do just
> latex -interaction=nonstopmode FILENAME
> instead.

I seem to remember David explained that "\input" was used to make
something work (region-compilation?).  I looked for this explanation
in my inbox but I can't find it, hopefully David can tell us more.  If
"\input" is really not needed anymore then removing "\input" when
calling TeX is a good solution.

Thanks for tackling this issue.

Cheers,
Mosè

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

Hi all,

> My current impression is that it would be fine to change
> TeX-command-list so that tex command issued explicitly by C-c C-c, C-c
> C-r and so becomes like
> latex -interaction=nonstopmode FILENAME
> while internal call by preview-latex continues to use \input and
> incorporate \detokenize like
>   latex 
> \nonstopmode\nofiles\PassOptionsToPackage{...}\AtBeginDocument{...} \input 
> \detokenize{FILENAME}
> .
> How do others think about this idea?

I implemented the above idea in the attached patch.  It turned out that
we have to deal with the following two additional issues:
(1) The output of \message{...} also changed in TeXLive 2018.  If
non-ascii characters are in \message{...}, they are output in tokenized
form by default in TL 2018.  This makes partial typset, such as C-c C-r,
C-c C-s and so on, to fail to recognize the right file name when source
correlate mode is enabeld, so I inserted another \detokenize{} in
`TeX-quote-filename'.
(2) When preamble caching is enabled in preview-latex, the later runs of
latex to produce images must be in the form without \input.  In this
form, latex refuses to recognize the file name if \detokenize is
present.  Thus I had to modify `preview-undump-replacements' to strip
off not only \input but also \detokenize.

I confirmed that the attached patch works with both file name with a
non-ascii character and one with a space, with TL 2018 on freebsd and
w32.
I'd like all of you to test the patch extensively, please.

N.B.  The initial release of TeXLive 2018 lacks the file ".tex" [1],
which makes preview-latex to fail to dump format file when preamble
caching is enabled.  Running `tlmgr update --all' doesn't recover ".tex"
at the time of writing this paragraph, thus it seems that the latest TL
repositry still lacks ".tex".  You have to obtain ".tex", e.g. from TL
2017, and put it manually in order to use preamble caching in
preview-latex with TL 2018.

[1] https://github.com/latex3/latex2e/issues/30

Regards,
Ikumi Keita



non-ascii-file-name.gz
Description: proposal
___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[Ikumi Keita]

Hi Jean and all,

> jfbu  writes:
>> this is to draw attention to this 
>> https://tex.stackexchange.com/questions/429869/emacs-auctex-tl-2018-problems-with-non-ascii-characters-in-the-file-name

Thank you for report.  It seems that AUCTeX has to be fixed with the
change in texlive.

>> By the way, why does AucTeX use `\input` ?

According to my brief survey, AUCTeX used to call tex binaries in a form
like
latex \nonstopmode\input FILENAME
in its early days.  I suppose that the initial reason to use \input is
to run tex in nonstop mode by prefixing \nonstopmode.
At some point in its history, AUCTeX changed to use "-interaction"
option to achieve nonstop mode like this:
latex -interaction=nonstopmode \input FILENAME
Thus I think it is basically unnecessary to adhere using \input and now
we can do just
latex -interaction=nonstopmode FILENAME
instead.

However, there is one exception.  Preview-latex calls latex in a form
like
latex 
\nonstopmode\nofiles\PassOptionsToPackage{...}\AtBeginDocument{...} \input 
FILENAME
, so it seems mandatory to me to use \input in order to insert these
complex stuffs before the document file is actually read by latex.

>> It creates another problem, which is that the LaTeX version banner
>> (not the engine banner, I mean the LaTeX format version) is not
>> inserted in the log.
>> 
>> cf https://github.com/latex3/latex2e/issues/35

> a third related problem is the one of a format line at top of file. Afaict
> the `\input` causes it to be ignored.

> cf 
> https://tex.stackexchange.com/questions/427001/using-a-custom-fmt-with-emacs-auctex

My current impression is that it would be fine to change
TeX-command-list so that tex command issued explicitly by C-c C-c, C-c
C-r and so becomes like
latex -interaction=nonstopmode FILENAME
while internal call by preview-latex continues to use \input and
incorporate \detokenize like
latex 
\nonstopmode\nofiles\PassOptionsToPackage{...}\AtBeginDocument{...} \input 
\detokenize{FILENAME}
.
How do others think about this idea?

Regards,
Ikumi Keita

___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex

Re: [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018

[jfbu]

Le 03/05/2018 à 17:56, jfbu a écrit :

Hi,

this is to draw attention to this 
https://tex.stackexchange.com/questions/429869/emacs-auctex-tl-2018-problems-with-non-ascii-characters-in-the-file-name

By the way, why does AucTeX use `\input` ? It creates another problem,
which is that the LaTeX version banner (not the engine banner, I mean
the LaTeX format version) is not inserted in the log.

cf https://github.com/latex3/latex2e/issues/35

Best,

Jean-François



a third related problem is the one of a format line at top of file. Afaict
the `\input` causes it to be ignored.

cf 
https://tex.stackexchange.com/questions/427001/using-a-custom-fmt-with-emacs-auctex

Best



___
auctex mailing list
auctex@gnu.org
https://lists.gnu.org/mailman/listinfo/auctex