Bug#1070150: texlive-latex-extra: performance regression under emulation with 2023.20240207-1
Control: reassign -1 texlive-binaries Control: severity -1 important On 01.05.2024 00:19, Steev Klimaszewski wrote: Hi all, Since the 2023.20240207-1 release, there is a major performance regression when running the tex-common postinst hook in an arm64 chroot on amd64 host. I assume this is important, further the package needs to be corrected. H. -- sigfault OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1070150: texlive-latex-extra: performance regression under emulation with 2023.20240207-1
Hi Hilmar, On Wed, May 1, 2024 at 4:17 PM Preuße, Hilmar wrote: > > On 01.05.2024 22:46, Steev Klimaszewski wrote: > > Hi, > > > May 01 14:28:46 catcodes, registers, parameters, > > May 01 14:28:46 LaTeX2e <2024-06-01> pre-release-0 (develop 2024-5-1 > > branch) > > May 01 14:28:46 (/usr/share/texlive/texmf-dist/tex/latex/l3kernel/expl3.ltx > > May 01 14:51:15 > > (/usr/share/texlive/texmf-dist/tex/latex/l3kernel/expl3-code.tex)) hacks, > > > May 01 14:51:19 document commands, control, par, spacing, files, font > > > > > > Noticed that too, that loading this file takes quite long. In Debian > stable (on a Rasbpi4) it are just a few seconds. In a chroot (Debian > unstable / experimental) it takes longer, but still not the 30 minutes I > see in your log. There is no difference between pdflatex and lualatex. > > H, > -- > sigfault Correct, when run on the same host architecture, it's fine. The issue comes in when running on an amd64 host, and in an arm64 chroot. So, e.g. running an arm64 docker container on amd64, or running an arm64 systemd-nspawn container on an amd64 host. While, yes, qemu-user emulation is slow (single threaded iirc?) stable takes 13 minutes to do this, testing/unstable/experimental take significantly longer as mentioned.
Bug#1070150: texlive-latex-extra: performance regression under emulation with 2023.20240207-1
On 01.05.2024 22:46, Steev Klimaszewski wrote: Hi, May 01 14:28:46 catcodes, registers, parameters, May 01 14:28:46 LaTeX2e <2024-06-01> pre-release-0 (develop 2024-5-1 branch) May 01 14:28:46 (/usr/share/texlive/texmf-dist/tex/latex/l3kernel/expl3.ltx May 01 14:51:15 (/usr/share/texlive/texmf-dist/tex/latex/l3kernel/expl3-code.tex)) hacks, > May 01 14:51:19 document commands, control, par, spacing, files, font Noticed that too, that loading this file takes quite long. In Debian stable (on a Rasbpi4) it are just a few seconds. In a chroot (Debian unstable / experimental) it takes longer, but still not the 30 minutes I see in your log. There is no difference between pdflatex and lualatex. H, -- sigfault OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1070150: texlive-latex-extra: performance regression under emulation with 2023.20240207-1
On 4/30/24 11:22 PM, Norbert Preining wrote: When running the hook with the 2023.20240207, this step takes around 3 hours. Ouch, that hurts. I'm guessing that this is related to enabling lua on aarch64 finally, but I'm not familiar at all with latex and how to get the smallest possible test case to show the issue. If you suspect lua, then could you please run (as root or via sudo) time fmtutil-sys --byengine luatex time fmtutil-sys --byengine luajittex and let us see the times there? I've additionally piped the first command to ts as well, but forgot to do that while I was still logging, so here is the fmtutil-sys --byengine luatex | ts output: root@sid:~# time fmtutil-sys --byengine luajittex | ts May 01 14:26:13 fmtutil: fmtutil is using the following fmtutil.cnf files (in precedence order): May 01 14:26:13 fmtutil: /usr/share/texmf/web2c/fmtutil.cnf May 01 14:26:13 fmtutil: /usr/share/texlive/texmf-dist/web2c/fmtutil.cnf May 01 14:26:13 fmtutil: fmtutil is using the following fmtutil.cnf file for writing changes: May 01 14:26:13 fmtutil: /etc/texmf/web2c/fmtutil.cnf May 01 14:26:13 fmtutil [INFO]: writing formats under /var/lib/texmf/web2c May 01 14:26:13 fmtutil [INFO]: Did not find entry for byengine=luajittex skipped May 01 14:26:13 fmtutil [INFO]: not selected formats: 17 May 01 14:26:13 fmtutil [INFO]: total formats: 17 May 01 14:26:13 fmtutil [INFO]: exiting with status 0 real 0m9.009s user 0m10.184s sys 0m0.156s root@sid:~# time fmtutil-sys --byengine luatex | ts May 01 14:26:46 fmtutil: fmtutil is using the following fmtutil.cnf files (in precedence order): May 01 14:26:46 fmtutil: /usr/share/texmf/web2c/fmtutil.cnf May 01 14:26:46 fmtutil: /usr/share/texlive/texmf-dist/web2c/fmtutil.cnf May 01 14:26:46 fmtutil: fmtutil is using the following fmtutil.cnf file for writing changes: May 01 14:26:46 fmtutil: /etc/texmf/web2c/fmtutil.cnf May 01 14:26:46 fmtutil [INFO]: writing formats under /var/lib/texmf/web2c May 01 14:26:46 fmtutil [INFO]: --- remaking luatex with luatex May 01 14:26:46 fmtutil: running `luatex -ini -jobname=luatex -progname=luatex luatex.ini' ... May 01 14:26:47 This is LuaTeX, Version 1.17.0 (TeX Live 2023/Debian) (INITEX) May 01 14:26:47 restricted system commands enabled. May 01 14:26:47 (/usr/share/texlive/texmf-dist/tex/generic/tex-ini-files/luatex.ini May 01 14:26:47 (/usr/share/texlive/texmf-dist/tex/generic/tex-ini-files/luatexconfig.tex May 01 14:26:47 (/var/lib/texmf/tex/generic/tex-ini-files/pdftexconfig.tex)) May 01 14:26:47 (/usr/share/texlive/texmf-dist/tex/generic/tex-ini-files/luatexiniconfig.tex) May 01 14:26:47 (/usr/share/texlive/texmf-dist/tex/generic/unicode-data/load-unicode-data.tex May 01 14:26:47 May 01 14:26:47 load-unicode-data.tex v1.17 (2023-09-18) May 01 14:26:47 Reading Unicode data May 01 14:26:47 # UnicodeData-15.1.0.txt May 01 14:26:47 # Modified 2023-09-18 08:45:00 GMT [JAW] May 01 14:28:20 ) (/usr/share/texlive/texmf-dist/tex/luatex/hyph-utf8/etex.src May 01 14:28:20 (/usr/share/texlive/texmf-dist/tex/plain/base/plain.tex May 01 14:28:21 Preloading the plain format: codes, registers, parameters, fonts, more fonts, May 01 14:28:21 macros, math definitions, output routines, hyphenation May 01 14:28:21 (/usr/share/texlive/texmf-dist/tex/generic/hyphen/hyphen.tex May 01 14:28:21 [skipping from \patterns to end-of-file...])) May 01 14:28:21 (/usr/share/texlive/texmf-dist/tex/plain/etex/etexdefs.lib May 01 14:28:21 Skipping module "grouptypes"; Loading module "interactionmodes"; May 01 14:28:21 Skipping module "nodetypes"; Skipping module "iftypes";) May 01 14:28:21 (/var/lib/texmf/tex/generic/config/language.def May 01 14:28:21 (/usr/share/texlive/texmf-dist/tex/generic/hyphen/hyphen.tex)) May 01 14:28:21 Augmenting the Plain TeX definitions: \tracingall; May 01 14:28:21 Adding new e-TeX definitions: \eTeX, \loggingall, \tracingnone, May 01 14:28:21 register allocation; extended register allocation; May 01 14:28:21 Recycling: \addlanguage, \@nswer (not defined), \@sk, \b@dresponsetrue, May 01 14:28:21 \b@dresponsefalse, \ch@ckforyn, \mayber@cycle, \et@xabort, \et@xbuf, May 01 14:28:21 \et@xfmtsrc, \et@xfilehdr, \et@xinf, \et@xpatterns, \l@ngdefnfile, \n@xt, May 01 14:28:21 \p@rse (not defined), \pr@mpt (not defined), \pr@mptloop (not defined), May 01 14:28:21 \forcer@cycle, \usef@llback, \usef@llbacktrue, \usef@llbackfalse, May 01 14:28:21 Retaining: \et@xerr, \et@xinput, \et@xlibhdr, \et@xmsg, \et@xtoks, \et@xwarn, May 01 14:28:21 \et@xl@@d, \et@xl@ad, \et@xload, \et@xlang, \et@xhash, \eTeX, \etexhdrchk, May 01 14:28:29 \etexstatus, \module, \uselanguage, \r@tain, \r@cycle,)) May 01 14:28:29 Beginning to dump on file luatex.fmt May 01 14:28:29 (format=luatex 2024.5.1) May 01 14:28:29 3389 strings using 10306 bytes May 01 14:28:29 68809 memory locations dumped; current usage is 149&7701 May 01 14:28:29 1883 multiletter control sequences May 01 14:28:29
Bug#1070150: texlive-latex-extra: performance regression under emulation with 2023.20240207-1
> When running the hook with the 2023.20240207, this step takes around 3 hours. Ouch, that hurts. > I'm guessing that this is related to enabling lua on aarch64 finally, but I'm > not familiar at all with latex and how to get the smallest possible test case > to > show the issue. If you suspect lua, then could you please run (as root or via sudo) time fmtutil-sys --byengine luatex time fmtutil-sys --byengine luajittex and let us see the times there? > A way to reproduce the issue would be to install debootstrap, qemu-user-static > and systemd-nspawn and then run apt-get install texlive-latex-extra Sorry, no Debian system around anymore. > On an arm64 host (VM on an original M1 Macbook Air), this is fine Nor that. > sudo debootstrap --arch arm64 sid sid > sudo systemd-nspawn -D sid > time apt-get install texlive-latex-extra -y > > real 181m11.687s > user 181m43.534s > sys 1m5.422s > > texlive-latex-recommended: 2023.20240207-1 Can you show the *actual* output? And, maybe install moreutils and run it through ts apt-get install texlive-latex-extra -y | ts That would give a hint which part is taking so much time. Best regards Norbert -- PREINING Norbert https://www.preining.info arXiv / Cornell University + IFMGA Guide + TU Wien + TeX Live GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13
Bug#1070150: texlive-latex-extra: performance regression under emulation with 2023.20240207-1
Package: texlive-latex-extra Version: 2023.20240207-1 Severity: wishlist User: de...@kali.org Usertags: origin-kali Dear Maintainer, Since the 2023.20240207-1 release, there is a major performance regression when running the tex-common postinst hook in an arm64 chroot on amd64 host. This was not the case with 2022.20230122-3. When running the hook with the 2023.20240207, this step takes around 3 hours. I'm guessing that this is related to enabling lua on aarch64 finally, but I'm not familiar at all with latex and how to get the smallest possible test case to show the issue. A way to reproduce the issue would be to install debootstrap, qemu-user-static and systemd-nspawn and then run apt-get install texlive-latex-extra On an arm64 host (VM on an original M1 Macbook Air), this is fine sudo debootstrap --arch arm64 stable stable sudo systemd-nspawn -D stable time apt-get install texlive-latex-extra -y real 1m4.121s user 0m52.893s system 0m6.116s (due to the t64 transition, testing is currentl unable to debootstrap) sudo debootstrap --arch arm64 sid sid sudo systemd-nspawn -D sid time apt-get install texlive-latex-extra -y real 1m24.647s user 1m14.897s sys 0m6.558s In the qemu emulated build however, sudo debootstrap --arch arm64 stable stable sudo systemd-nspawn -D stable time apt-get install texlive-latex-extra -y real 13m14.171s user 13m26.363s sys 0m53.424s texlive-latex-recommended: 2022.20230122-3 sudo debootstrap --arch arm64 sid sid sudo systemd-nspawn -D sid time apt-get install texlive-latex-extra -y real 181m11.687s user 181m43.534s sys 1m5.422s texlive-latex-recommended: 2023.20240207-1 So the version of latex in stable took less than 15 minutes to install, but the version in testing and unstable both take 2 1/2 hours! I did also test 2024.20240313.70630+ds-1 from experimental and it also takes the 2 1/2 hours. Currently, we are working around this by forcing our package list to not install any of the latex packages, but it would be nice to be able to have the packages installed instead of telling end users to install them manually. If you need me to do more testing or any sort of additional information, please let me know! -- steev -- System Information: Debian Release: kali-rolling APT prefers experimental APT policy: (1, 'experimental'), (1, 'unstable') Architecture: arm64 (aarch64) Kernel: Linux 6.8.8 (SMP w/8 CPU threads; PREEMPT) Kernel taint flags: TAINT_UNSIGNED_MODULE Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages texlive-latex-extra depends on: pn libcommons-logging-java pn libpdfbox-java pn preview-latex-style ii python33.11.6-1 pn tex-common pn texlive-base pn texlive-binaries pn texlive-latex-recommended pn texlive-pictures Versions of packages texlive-latex-extra recommends: pn default-jre pn libspreadsheet-parseexcel-perl pn texlive-fonts-recommended pn texlive-plain-generic Versions of packages texlive-latex-extra suggests: pn icc-profiles ii libfile-which-perl 1.27-2 ii python3-pygments 2.17.2+dfsg-1 pn texlive-latex-extra-doc pn texlive-science
