Re: [AUCTeX-devel] [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018
Hi Jean, > jfbuwrites: > (b.t.w. I also git cloned the savannah, and > https://git.savannah.gnu.org/cgit/auctex.git has a typo it should be > https://git.savannah.gnu.org/git/auctex.git > ) Ah, sorry. The former URL is for usual web browsers, e.g. firefox. That gives overview of the git repository, and more. The git repository address for the command "git" is listed on the page: git://git.savannah.gnu.org/auctex.git https://git.savannah.gnu.org/git/auctex.git ssh://git.savannah.gnu.org:/srv/git/auctex.git (the last one requires suitable account on savannah, I think) > My testing so far indicates no problem on Mac OS X 10.9.5 > I was intrigued by these changes related to quotes, but it turns out > when I had tried manually on the command line that I was not quoting > correctly. > With your patch on top of 12.1.0 I successfully generate preview for a file > with both spaces and utf-8 characters. Thank you for checking. Now I can be more confident. > I did not check the ini file because it is volatile but I think you > indicated how to modify cache-preamble to make it permanent. Though I didn't write the way explicitly, it's relatively easy. Change the last part of `preview-cache-preamble' in preview.el as: (preview-format-kill ',format-cons)) ; (delete-file ',dump-file) ;; comment out ) ;; leave this close paren (error (preview-log-error err "Dumping" process))) (preview-reraise-error process))) > A proposito, the above in the pdflatex command (once the format > is done)means \input will be with extra spaces > \input{"test my látéx.tex"} > but fortunately that works fine. These spaces are intentional so that w32 shell quoting style would not interfere with other quote chars. It seems that consequtive quote chars are interpreted differently on w32. > Anyway, all seems fine! Marvelous! > TeX-region-create > and surely what happens is that Keita has a patch of the > \\message in there > but it was not included in the one I tested Yes. More precisely, the required fix is injected in `TeX-quote-filename' rather than `TeX-region-create'. On my side, the given example runs without error. If you want to try it out for yourself, extract the relavant part from "tentative-patch2" attached in my previous message https://lists.gnu.org/archive/html/auctex-devel/2018-05/msg00024.html . (But note that "tentative-patch2" as a whole has not been finished yet and cannot typeset file name with space.) By the way, I have various things to do for 2-3 days, including my hobby :-), my next post will be after that. Regards, Ikumi Keita ___ auctex-devel mailing list auctex-devel@gnu.org https://lists.gnu.org/mailman/listinfo/auctex-devel
Re: [AUCTeX-devel] [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018
Hi Jean and all, The proposed method, with slight modification explained below, works for all the cases I tested. Thank you again! For now, I attach the patch for preview-latex part only so that the reader can concentrate on the relavant changes. > jfbuwrites: >> \ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{% >> \edef\next{{\pdfoutput=\the\pdfoutput\relax\the\everyjob}}% >> \everyjob\next\catcode`\ 10 % >> \catcode`/ 0 % >> \def\AUCTEXINPUT##1{\catcode`/ 12\relax\catcode`\ >> 9\relax\input{\detokenize{"##1"}}}% >> \let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax% >> Notice that when ##1 (i.e. the filename) will be grabbed) the / is of >> catcode 0, so ##1 is tokenized accordingly. Which in principle could >> perhaps cause an issue \detokenize{aaa/b23} would give aaa/b123 >> but as / is not legal in filenames in contexts I know of, I did not even >> test this anticipation. AUCTeX always runs TeX commands after changing the current directory to that of the master file, so forward slash would never appear as an argument of the command. The region file is created in the same directory as the master, thus region compilation should work as well. >> I have also added quotes inside the \detokenize. I tried removing those quotes and let the quotes added around the file name with space to be put inside /AUCTEXINPUT like this: pdflatex -foo -bar _ppp_qqq /AUCTEXINPUT{"ppp qqq"} (without escape for shell) , because it's easier to absorb the difference in the way of quoting the shell arguments between w32 and non-w32 platforms. Experiments showed no problems even for files without space, so the attached patch takes this way. If this has any shortcomings, please tell me. Regards, Ikumi Keita preview-diff.gz Description: diff for preview-latex only ___ auctex-devel mailing list auctex-devel@gnu.org https://lists.gnu.org/mailman/listinfo/auctex-devel
Re: [AUCTeX-devel] [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018
Hi Jean, Thanks a lot again, the proposed method works on the command line! I'll implement it in the actual preview-latex code. By the way, if I remember correctly, filenames with multiple contiguous spaces are not supported by TeX, aren't they? Standard pdflatex just fails for such file names: -- [bash]$ cp ppp\ qqq.tex aaa\ \ bbb.tex [bash]$ pdflatex aaa\ \ bbb.tex This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded format=pdflatex) restricted \write18 enabled. entering extended mode ! I can't find file `"aaa bbb.tex"'. <*> "aaa bbb.tex" (Press Enter to retry, or Control-D to exit) Please type another input file name: -- So I suppose that it is not necessary for preview-latex to support such file names. > jfbuwrites: > In short, in presence of the /AUCTEXINPUT hack, the original space hack > simply ceases functioning because the space catcode is reset to "ignore" > too soon (at \everyjob) > How to fix this ? > With this new version of ini file > \ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{% > \edef\next{{\pdfoutput=\the\pdfoutput\relax\the\everyjob}}% > \everyjob\next\catcode`\ 10 % > \catcode`/ 0 % > \def\AUCTEXINPUT##1{\catcode`/ 12\relax\catcode`\ > 9\relax\input{\detokenize{"##1"}}}% > \let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax% > Notice that when ##1 (i.e. the filename) will be grabbed) the / is of > catcode 0, so ##1 is tokenized accordingly. Which in principle could > perhaps cause an issue \detokenize{aaa/b23} would give aaa/b123 > but as / is not legal in filenames in contexts I know of, I did not even > test this anticipation. > I have also added quotes inside the \detokenize. > I tested with a file with filename "test mylátex.tex" > I. Generation of format > etex -ini "" prv_test_mylátex.ini > "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}" > "\input{\detokenize{\"test mylátex.tex\"}}" > where file prv_test_mylátex.ini contains the above contents (from > \ifx\pdfoutput to \relax%) > and attention to the added quotes inside the \detokenize argument. > II. Usage of format > pdflatex -fmt="prv_test_mylátex.fmt" /AUCTEXINPUT{test mylátex.tex} > Could you try this out ? > Best > Jean-François Best regards, Ikumi Keita ___ auctex-devel mailing list auctex-devel@gnu.org https://lists.gnu.org/mailman/listinfo/auctex-devel
Re: [AUCTeX-devel] [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018
Hi Keita Le 24/05/2018 à 09:24, Ikumi Keita a écrit : Hi Jean, Ikumi Keitawrites: Thank you very much for your all valuable works! I'll test this with other cases (TL2017, file names with space, on w32 platform) and propose to commit again. Unfortunately, it seems that the proposed method using /AUCTEXINPUT does not work with file name with space. The sample I'm using is a file named "ppp qqq.tex" with the content: Yes, I had a bad night tormented about this. But I understand why and will explain below. --- ppp qqq.tex -- \documentclass{article} \begin{document} Hello, World \(xyz\) \end{document} %%% Local Variables: %%% TeX-master: t %%% End: -- The format file is generated successfully: -- Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -ini -interaction=nonstopmode "" prv_ppp_qqq.ini "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}" "\input" "{\detokenize{" \"ppp\ qqq.tex\" "}}"'' This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (INITEX) restricted \write18 enabled. entering extended mode (./prv_ppp_qqq.ini LaTeX2e <2018-04-01> patch level 4 (/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx) (/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex File ignored)) No auxiliary output files. (./ppp qqq.tex (/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo)) ) Beginning to dump on file prv_ppp_qqq.fmt (preloaded format=prv_ppp_qqq 2018.5.24) 7585 strings of total length 123256 58367 memory locations dumped; current usage is 219&56905 4191 multiletter control sequences \font\nullfont=nullfont \font\OMX/cmex/m/n/5=cmex10 [...] \font\OT1/cmss/m/n/10=cmss10 5716 words of font info for 21 preloaded fonts 1141 hyphenation exceptions Hyphenation trie of length 370691 has 8997 ops out of 35111 143 for language 83 110 for language 82 [...] 181 for language 0 0 words of pdfTeX memory 0 indirect objects No pages of output. Transcript written on prv_ppp_qqq.log. TeX Output finished at Thu May 24 16:09:49 -- However, the next run with the dumped format doesn't make its way: Because spaces are ignored in the filename, due to \input form (indirectly via /AUCTEXINPUT) I will explain below. -- Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -interaction=nonstopmode \_ppp_qqq /AUCTEXINPUT{\"ppp\ qqq.tex\"}'' This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded format=pdflatex) restricted \write18 enabled. entering extended mode LaTeX2e <2018-04-01> patch level 4 CUSTOMISED FORMAT. Preloaded files: . "ppp qqq.tex" article.cls2014/09/29 v1.4h Standard LaTeX document class size10.clo2014/09/29 v1.4h Standard LaTeX file (size option) . ! LaTeX Error: File `"pppqqq.tex"' not found. Type X to quit or to proceed, or enter new name. (Default extension: tex") Enter file name: ! Emergency stop. <*> _ppp_qqq /AUCTEXINPUT{"ppp qqq.tex"} ^^M ! ==> Fatal error occurred, no output PDF file produced! Transcript written on texput.log. TeX Output exited as expected with code 1 at Thu May 24 16:11:23 LaTeX: LaTeX found no preview images -- When I replace \_ppp_qqq with -fmt="prv_ppp_qqq.fmt" from xterm, the error changes slightly: That's a bit unexpected (by me). -- [bash]$ pdflatex -interaction=nonstopmode -fmt="prv_ppp_qqq.fmt" /AUCTEXINPUT{\"ppp\ qqq.tex\"} This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded format=prv_ppp_qqq.fmt) restricted \write18 enabled. entering extended mode (/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex LaTeX2e <2018-04-01> patch level 4 CUSTOMISED FORMAT. Preloaded files: . "ppp qqq.tex" article.cls2014/09/29 v1.4h Standard LaTeX document class size10.clo2014/09/29 v1.4h Standard LaTeX file (size option) . ) Runaway argument? ! File ended while scanning use of ^^M. \par <*> "/AUCTEXINPUT{ppp qqq.tex} " I do not understand that for time being but let's forget it as we have the \_ppp_qqq.fmt way. UPDATE: at bottom of this my new proposal seems ok with -fmt at least at my locale. ! LaTeX Error:
Re: [AUCTeX-devel] [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018
Hi Jean, > Ikumi Keitawrites: > Thank you very much for your all valuable works! I'll test this with > other cases (TL2017, file names with space, on w32 platform) and propose > to commit again. Unfortunately, it seems that the proposed method using /AUCTEXINPUT does not work with file name with space. The sample I'm using is a file named "ppp qqq.tex" with the content: --- ppp qqq.tex -- \documentclass{article} \begin{document} Hello, World \(xyz\) \end{document} %%% Local Variables: %%% TeX-master: t %%% End: -- The format file is generated successfully: -- Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -ini -interaction=nonstopmode "" prv_ppp_qqq.ini "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}" "\input" "{\detokenize{" \"ppp\ qqq.tex\" "}}"'' This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (INITEX) restricted \write18 enabled. entering extended mode (./prv_ppp_qqq.ini LaTeX2e <2018-04-01> patch level 4 (/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx) (/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex File ignored)) No auxiliary output files. (./ppp qqq.tex (/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls Document Class: article 2014/09/29 v1.4h Standard LaTeX document class (/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo)) ) Beginning to dump on file prv_ppp_qqq.fmt (preloaded format=prv_ppp_qqq 2018.5.24) 7585 strings of total length 123256 58367 memory locations dumped; current usage is 219&56905 4191 multiletter control sequences \font\nullfont=nullfont \font\OMX/cmex/m/n/5=cmex10 [...] \font\OT1/cmss/m/n/10=cmss10 5716 words of font info for 21 preloaded fonts 1141 hyphenation exceptions Hyphenation trie of length 370691 has 8997 ops out of 35111 143 for language 83 110 for language 82 [...] 181 for language 0 0 words of pdfTeX memory 0 indirect objects No pages of output. Transcript written on prv_ppp_qqq.log. TeX Output finished at Thu May 24 16:09:49 -- However, the next run with the dumped format doesn't make its way: -- Running `Preview-LaTeX' on `ppp qqq' with ``pdflatex -interaction=nonstopmode \_ppp_qqq /AUCTEXINPUT{\"ppp\ qqq.tex\"}'' This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded format=pdflatex) restricted \write18 enabled. entering extended mode LaTeX2e <2018-04-01> patch level 4 CUSTOMISED FORMAT. Preloaded files: . "ppp qqq.tex" article.cls2014/09/29 v1.4h Standard LaTeX document class size10.clo2014/09/29 v1.4h Standard LaTeX file (size option) . ! LaTeX Error: File `"pppqqq.tex"' not found. Type X to quit or to proceed, or enter new name. (Default extension: tex") Enter file name: ! Emergency stop. <*> _ppp_qqq /AUCTEXINPUT{"ppp qqq.tex"} ^^M ! ==> Fatal error occurred, no output PDF file produced! Transcript written on texput.log. TeX Output exited as expected with code 1 at Thu May 24 16:11:23 LaTeX: LaTeX found no preview images -- When I replace \_ppp_qqq with -fmt="prv_ppp_qqq.fmt" from xterm, the error changes slightly: -- [bash]$ pdflatex -interaction=nonstopmode -fmt="prv_ppp_qqq.fmt" /AUCTEXINPUT{\"ppp\ qqq.tex\"} This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2018) (preloaded format=prv_ppp_qqq.fmt) restricted \write18 enabled. entering extended mode (/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex LaTeX2e <2018-04-01> patch level 4 CUSTOMISED FORMAT. Preloaded files: . "ppp qqq.tex" article.cls2014/09/29 v1.4h Standard LaTeX document class size10.clo2014/09/29 v1.4h Standard LaTeX file (size option) . ) Runaway argument? ! File ended while scanning use of ^^M. \par <*> "/AUCTEXINPUT{ppp qqq.tex} " ! LaTeX Error: Missing \begin{document}. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... <*> "/AUCTEXINPUT{ppp qqq.tex} " ! LaTeX Error: File `ppp qqq.tex' not found. Type X to quit or to proceed, or enter new name. (Default extension: tex) Enter file name: ! Emergency stop. <*> "/AUCTEXINPUT{ppp qqq.tex}
Re: [AUCTeX-devel] [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018
Hi Jean, thanks a lot, your advice led me to the success! > jfbuwrites: > Putting altogether : > 1. We need file prv_testmylátex.ini to be > \ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{% > \edef\next{{\catcode`\ 9 \pdfoutput=\the\pdfoutput\relax\the\everyjob}}% > \everyjob\next\catcode`\ 10 % > \catcode`/ 0 % > \def\AUCTEXINPUT##1{\catcode`/ 12\relax\input{\detokenize{##1}}}% > \let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax% > REMARK: the above hack of \dump is conditional on \pdfoutput being > defined which I think is good anyhow. \pdfoutput is defined for > pdflatex whether dvi or pdf output. > 2. We create the format with this incantation > etex -ini -jobname prv_testmylátex "" > "\input{\detokenize{prv_testmylátex.ini}}" > "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}" > "\input{\detokenize{testmylátex.tex}}" > the following less paranoid version (I don't wrap the name of ini file in > \detokenize) seems to work too > etex -ini -jobname prv_testmylátex "" prv_testmylátex.ini > "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}" > "\input{\detokenize{testmylátex.tex}}" I took this version. I also omitted "-jobname prv_testmylátex" since the original AUCTeX code doesn't have that part. Do you recommend to restore them? > The detokenize at the end is really needed. (Or we could play the other games > with \UseRawInputEncoding) > 3. We then use the format with this incantation > pdflatex -fmt="prv_testmylátex.fmt" /AUCTEXINPUT{testmylátex.tex} I used the form "_testmylátex" instead of -fmt="prv_testmylátex.fmt" following the original AUCTeX code. I expect this is harmless. > It works in my testing in Terminal on Mac OS X (bash shell) > (I tested it with /AUCTEXINPUT{some other file with non-ascii filename} > to check macros defined in the preamble of testmylátex.tex are actually > used, proving the created format was indeed used). Thank you very much for your all valuable works! I'll test this with other cases (TL2017, file names with space, on w32 platform) and propose to commit again. For those interested in, the current tentative patch is attached below. Regards, Ikumi Keita tentative-patch2.gz Description: new approach v2 ___ auctex-devel mailing list auctex-devel@gnu.org https://lists.gnu.org/mailman/listinfo/auctex-devel
Re: [AUCTeX-devel] [AUCTeX] files with non-ascii characters and LaTeX as in TeXLive 2018
Hi Jean, > jfbuwrites: > no bothering at all! >> Perhaps I should give up this particular case (non-ascii file name + >> preview-latex preamble cache) and commit the patch which fixes other >> issues. > surely not! Thank you! > What would helped me is to see the exact content of an ini file. It is designed to be deleted automatically after the format file dump, so I modified `preview-cache-preamble' temporarily not to delete it. The content thus obtained is: \ifx\pdfoutput\undefined\else\let\PREVIEWdump\dump\def\dump{% \edef\next{{\catcode`\ 9 \pdfoutput=\the\pdfoutput\relax\the\everyjob}}\everyjob\next\catcode`\ 10 \let\dump\PREVIEWdump\dump}\fi\input mylatex.ltx \relax > Then it is intriguing that the comment says > "mylatex.ltx expects a file name to follow. Bad." > and refers to empty file ".tex" but I see mylatex.ltx \relax not mylatex.ltx > .tex ?? I don't know well, but the log at the dumping says (/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx) (/usr/local/texlive/2018/texmf-dist/tex/latex/tools/.tex File ignored)) so I think it really reads ".tex". > > about this ".tex" it was missing from TeXLive for a while due to a LaTeX > packaging > error but is since fixed > https://github.com/latex3/latex2e/issues/30 > Yes, I had updated with tlmgr and restored ".tex" already. > It might take some time until I can say something about preview's > mechanism, as I am not even that familiar to its usage. I hope > David can kick in, that would be so much faster. I hope so as well. > I anticipate the role of the format is to catch up some common preamble, These two paragraphs are quotes from preview-latex.info: -- 'C-c C-p C-f' 'preview-cache-preamble' Preview/Turn preamble cache on Dump a pregenerated format file. For the rest of the session, this file is used when running on the same master file. Use this if you know your LaTeX takes a long time to start up, the speedup will be most noticeable when generating single or few previews. If you change your preamble, do this again. preview-latex will try to detect the necessity of that automatically when editing changes to the preamble are done from within Emacs, but it will not notice if the preamble effectively changes because some included file or style file is tampered with. -- * Automatically cache preambles Currently preview-latex asks you whether you want to cache the document preamble (everything before '\begin{document}') before it generates previews for a buffer the first time. Caching the preamble will significantly speed up regeneration of previews. The larger your preamble is, the more this will be apparent. Once a preamble is cached, preview-latex will try to keep track of when it is changed, and dump a fresh format in that case. If you experience problems with this, or if you want it to happen without asking you the first time, you can customize the variable 'preview-auto-cache-preamble'. -- I hope these help. > and the file primárias.tex should not be used here. I think the basic idea is to "embed" the particular document's preamble in the format file for efficiency, so I suppose that it is essential to input the document file (primárias.tex here). > And "\input" "\detokenize{primárias.tex}" should be either > "\expandafter\input" "\detokenize{primárias.tex}" > or > "\input" "{\detokenize{primárias.tex}}" OK, I'll change the relavant code accordingly. > PS: I am CCing to auctex-dev but it seems my messages get bounced Sorry, I misspelled the address in my previous message. The right address is "auctex-devel@gnu.org", not "auctex-dev@...". Best regards, Ikumi Keita ___ auctex-devel mailing list auctex-devel@gnu.org https://lists.gnu.org/mailman/listinfo/auctex-devel