Your message dated Sat, 17 Jun 2023 09:20:45 +0000 with message-id <e1qas6v-006i9t...@fasolo.debian.org> and subject line Bug#1037198: fixed in glibc 2.37-2 has caused the Debian Bug report #1037198, regarding locales: please parallelise locale-gen to be marked as done.
This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact ow...@bugs.debian.org immediately.) -- 1037198: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1037198 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
--- Begin Message ---Package: locales Version: 2.36-9 Severity: wishlist Tags: patch Dear Maintainer, Posting as a bug per comment from Andrej; originally posted 2022-05-06 as https://salsa.debian.org/glibc-team/glibc/-/merge_requests/7 Patch based on current Salsa HEAD attached, incl. analysis. Best, наб -- System Information: Debian Release: 12.0 APT prefers unstable APT policy: (500, 'unstable') Architecture: x32 (x86_64) Foreign Architectures: amd64, i386 Kernel: Linux 6.1.0-2-amd64 (SMP w/2 CPU threads; PREEMPT) Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages locales depends on: ii debconf [debconf-2.0] 1.5.82 ii libc-bin 2.36-9 ii libc-l10n 2.36-9 locales recommends no packages. locales suggests no packages. -- debconf information: * locales/locales_to_be_generated: en_GB.UTF-8 UTF-8 * locales/default_environment_locale: en_GB.UTF-8From b6af0ad83f5517fd1987f9c7ac0493565bc0976d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=BD=D0=B0=D0=B1?= <nabijaczlew...@nabijaczleweli.xyz> Date: Fri, 6 May 2022 01:22:10 +0200 Subject: [PATCH] Parallelise locale-gen if possible MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Mutt-PGP: OS Assuming a very generous 200M free/localedef (because I saw a max RSS of 147M w/time(1)), this will attempt to keep all jobs saturated, and usually succeed. There's little starvation, since the vast majority of time is spent in gzip(1) ‒ 1:14 user vs 27:55 sys At 2.2ish seconds per locale, even on a low-end system of today with 4 CPUs (and 800 free MB), we can generate up to 4 locales at once for 6.6s' speed-up. Assuming no super-pathological cases, this globally scales in roughly ceil(locales/ncpus)*2.2s chunks, which is a massive win The only user-visible change is that, with nproc>1, the output is en_GB.UTF-8... <cursor here> instead of en_GB.UTF-8... <cursor here, will print "done\n" when it's done> MemFree: in /proc/meminfo is available on all supported Debian kernels, and, indeed, exactly what procps free(1) uses --- debian/local/usr_sbin/locale-gen | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/debian/local/usr_sbin/locale-gen b/debian/local/usr_sbin/locale-gen index 7fa3d772..f1632f4e 100755 --- a/debian/local/usr_sbin/locale-gen +++ b/debian/local/usr_sbin/locale-gen @@ -23,6 +23,18 @@ is_entry_ok() { fi } +nproc="$(nproc 2>/dev/null)" || nproc=1 +if [ "$nproc" -gt 1 ]; then + mem_free=0 + while read -r k v _; do + [ "$k" = "MemFree:" ] && mem_free="$v" && break || : + done < /proc/meminfo || : + mem_free=$(( mem_free / 1024 / 200 )) + [ "$mem_free" -lt 1 ] && mem_free=1 || : + [ "$mem_free" -lt "$nproc" ] && nproc="$mem_free" || : + jobs=0; pids= +fi 2>/dev/null + echo "Generating locales (this might take a while)..." while read -r locale charset; do if [ -z "$locale" ] || [ "${locale#\#}" != "$locale" ]; then continue; fi @@ -35,6 +47,7 @@ while read -r locale charset; do locale_at="${locale#*@}" [ "$locale_at" = "$locale" ] && locale_at= || locale_at="@$locale_at" printf " %s.%s%s..." "$locale_base" "$charset" "$locale_at" + [ "$nproc" -gt 1 ] && echo || : if [ -e "$USER_LOCALES/$locale" ]; then input="$USER_LOCALES/$locale" @@ -46,7 +59,20 @@ while read -r locale charset; do input="$USER_LOCALES/$input" fi fi - localedef -i "$input" -c -f "$charset" -A /usr/share/locale/locale.alias "$locale" || : - echo " done" + localedef -i "$input" -c -f "$charset" -A /usr/share/locale/locale.alias "$locale" & + if [ "$nproc" -gt 1 ]; then + pids="$pids$! " + jobs=$(( jobs + 1 )) + + if [ "$jobs" -ge "$nproc" ]; then + wait "${pids%% *}" || : + jobs=$(( jobs - 1 )) + pids="${pids#* }" + fi + else + wait + echo " done" + fi done < "$LOCALEGEN" +wait echo "Generation complete." -- 2.30.2signature.asc
Description: PGP signature
--- End Message ---
--- Begin Message ---Source: glibc Source-Version: 2.37-2 Done: Aurelien Jarno <aure...@debian.org> We believe that the bug you reported is fixed in the latest version of glibc, which is due to be installed in the Debian FTP archive. A summary of the changes between this version and the previous one is attached. Thank you for reporting the bug, which will now be closed. If you have further comments please address them to 1037...@bugs.debian.org, and the maintainer will reopen the bug report if appropriate. Debian distribution maintenance software pp. Aurelien Jarno <aure...@debian.org> (supplier of updated glibc package) (This message was generated automatically at their request; if you believe that there is a problem with it please contact the archive administrators by mailing ftpmas...@ftp-master.debian.org) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Format: 1.8 Date: Sat, 17 Jun 2023 11:10:20 +0200 Source: glibc Architecture: source Version: 2.37-2 Distribution: experimental Urgency: medium Maintainer: GNU Libc Maintainers <debian-glibc@lists.debian.org> Changed-By: Aurelien Jarno <aure...@debian.org> Closes: 1037198 Changes: glibc (2.37-2) experimental; urgency=medium . [ Samuel Thibault ] * debian/patches/hurd-i386/git-task_info_count.diff: Fix passing size to task_info. . [ Aurelien Jarno ] * debian/patches/git-updates.diff: update from upstream stable branch: - Fix FTBFS on hppa. * debian/sysdeps/alpha.mk, debian/control.in/main: build-depends on g++-12 (>= 12.3.0-2) on alpha instead of forcing -mcpu=ev56. . [ Andrej Shadura ] * debian/po/sk.po: Fix typos in the Slovak translation. . [ Helmut Grohne ] * debian/rules.d/build.mk: disable -Werror when dpkg-builflags contains -Wno-error. . [ наб ] * debian/local/usr_sbin/locale-gen: parallelise locales generation if possible. Closes: #1037198. Checksums-Sha1: 6e445e22051bf0f00e405d28bf0e2580766d1a00 9685 glibc_2.37-2.dsc 68d7fdaf8b88521a7bb62855573269aa3423656b 786416 glibc_2.37-2.debian.tar.xz 75749b933d4f11817d57b10ddb8078726b7e3bf0 9587 glibc_2.37-2_source.buildinfo Checksums-Sha256: 7734737fb59c29630a21b0a6e603f7fc73612d17122c77f94fc5c8e8b4a919f9 9685 glibc_2.37-2.dsc 0c9a943ffff1c246f032d5058e28c7fb8871abe569094c08726e38c9e0ebf4aa 786416 glibc_2.37-2.debian.tar.xz ce1b6fbc1d3582e00bb38a46b9fd57b01105279a6cccad76370c62e9e5043f93 9587 glibc_2.37-2_source.buildinfo Files: 4149eafdec39ba67839484c2d921206b 9685 libs required glibc_2.37-2.dsc db3087bebb96abee19f80b8c27825f74 786416 libs required glibc_2.37-2.debian.tar.xz 9d05ad59b80f6e2696ca8d32b0bc9eb8 9587 libs required glibc_2.37-2_source.buildinfo -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEUryGlb40+QrX1Ay4E4jA+JnoM2sFAmSNeQYACgkQE4jA+Jno M2s4+hAAiPx2iASWGPrM9HftZCr4wVqygVhjEr8sdKXNpm8IFJB/Iu76Ic1wn4sX K1Ovzl9mJeYhQJarIuqFBmUvPLcPDsGlr7Rw9aTnhFe8p7XS01gEUZlsID7dH5ex gNc0DdLbUOzytpu75NDxFmALB3wjSWXzOnkVwZggryYulIO3kanhd9+75LiKHm74 /CHw0+NXY4cx8JJP/Ixwn2fxHyzcjlGgaUYu3fetvkq7fADnJSrkEpYhkn2PNB3A Wikp53wV0FISiivkkCLX7rDjcSojvY3Wd5dX27z0RdbX+YIDkZu7yuX55zSP40Js 5L+twb+I/sM7WhmVLWsWEIF5LMSVbx0IvSYbQcCkP17+x+zECYVX/UaY0cFmbkxS Rw1+TOFB3i+jJTxg2xgmtyXG77bB+6Ibmpg8DYS1TwFM8JoBfsID1TTkHyfmtAR+ U2GFog672z1bRZQEWrhBGCF7pONo2FDh4FXTi2SL8tLJjZ+wC8UOgrKtHFOPgTPO EUm/UXZmbDaTJFI1AVf4gfNG2De4l5TZqPtMMNtv6heL5byUYr2JcvQr1j9l/DIq i53KpGvOULHLiS9r0LkYejUxxk+y0Csi4LI3XYW1q97Okj1sYCpncejB1s2rU40M jyW4uo5y4+/2DpApMCdmxn45J9up/3MmPFe+4HTClZIuHrRsEnU= =LpsO -----END PGP SIGNATURE-----
--- End Message ---