Bug#632798: libc6: broken LANGUAGE design
Vincent Lefevre wrote: My settings come from the installation. /etc/default/locale was: # File generated by update-locale #LANG=en_US.UTF-8 LANGUAGE=en_US:en Ah. localechooser sets LANGUAGE up, and then update-locale from the locales package preserves it. The weird part is that debconf (dpkg-reconfigure locales) controls the LANG setting but LANGUAGE has to be changed more directly (update-locale LANGUAGE=foo). See (*), below. Now, if LANGUAGE is set in /etc/default/locale, this change may not solve the problem due to: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=313317 Wow. The upstream discussion went nowhere fast. Have you tried the patch, and does it work well? However, even if this ssh bug is fixed, in case LANGUAGE is not defined on the ssh client's side, the system must not set it by default in the user's back. So, in short, LANGUAGE should not be set in /etc/default/locale. I am not sure I agree, even though I have my system set up locally not to define LANGUAGE. If ssh were to transmit it and let it override /etc/default/locale, wouldn't sending LANGUAGE= work? $ LANGUAGE= LC_ALL=de_DE.UTF-8 cp cp: Fehlendes Dateioperand „cp --help“ gibt weitere Informationen. LANGUAGE=fr:de LC_MESSAGES=de on an installation where most programs are translated to German. (It means: programs using catgets should use German, while programs using gettext should prefer French if possible. Intent: I'd prefer French, but barring that, please give me German instead of English). Hmm... OK, I thought that all programs were using gettext. The authors of update-locale must have thought the same thing. # update-locale LANG=de_DE.UTF-8 LANGUAGE=fr:de *** update-locale: Warning: LANGUAGE (fr:de) is not compatible with LANG (de_DE.UTF-8). Disabling it. See Bug#596695 for the strange history that led to that. This seems mildly broken. Consider the following sequence of events: (*) 1. Install, telling localechooser to use language A. 2. Reconfigure using dpkg-reconfigure to switch to language B. 3. Reconfigure using dpkg-reconfigure to switch back to language A. At step 1, LANG and LANGUAGE are set. At step 2, locales.postinst decides what LANG should be. LANG and LANGUAGE are mismatched, so the LANGUAGE line is automatically commented. At step 3, locales.postinst sets LANG back, but LANGUAGE is still commented. At last, the old mystery (1) is solved. The above must be what allowed LANGUAGE to be set on your system and not set on mine. Summary of proposed changes --- openssh-server Bug#313317: * Let user envvars from the whitelist override pam. There's a patch already; we should just make sure it still works, ping upstream, and consider applying it in the meantime as a Debian-specific change. Additional bugs to be cloned from this one: * openssh-server: expand the default envvar whitelist to include LANGUAGE * libc6: let C.UTF-8 locale suppress LANGUAGE, just like C does * libc6: let LANGUAGE take effect even if LC_MESSAGES refers to a non-existent locale * libc6: implicitly add LC_MESSAGES to the beginning of the LANGUAGE list, except when it is already in the list * locales: when changing the default locale, update LANGUAGE at the same time, so the resulting configuration can be more similar to what localechooser would write. Thanks again for all your help. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#632798: libc6: broken LANGUAGE design
Hi again, One more quick comment. Vincent Lefevre wrote: My settings come from the installation. /etc/default/locale was: # File generated by update-locale #LANG=en_US.UTF-8 LANGUAGE=en_US:en (I only added a LC_TIME=en_DK since, hoping it would be taken into account for the time displayed by gdm). FYI, I installed the machine on 2010-01-04. Now I'm confused again. That particular LANGUAGE setting is (I hope) redundant next to LANG, so it should not have been set, right? That is supposed to have been fixed in localechooser 1.07 (2006-03-26): * Only use LANGUAGELIST list of languages when really needed, ie when a language needs and benefits from something else than English. Should avoid reports like #356997. The only choices I see that should be setting LANGUAGE based on the current localechooser data file are Northern Sami, Norwegian Bokmaal, Norwegian Nynorsk, Portuguese, Malagasy, and Wolof. (Plus Chinese (Traditional), Chinese (Simplified), and Portuguese (Brazil), but I suspect those are bugs.) Am I misunderstanding how LANGUAGE is supposed to be used? Did localechooser regress? -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#632798: libc6: broken LANGUAGE design
On 2011-07-07 18:46:38 -0500, Jonathan Nieder wrote: Vincent Lefevre wrote: Now, if LANGUAGE is set in /etc/default/locale, this change may not solve the problem due to: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=313317 Wow. The upstream discussion went nowhere fast. Have you tried the patch, and does it work well? No, I haven't tried it. My current workaround is to more or less force the locales settings in my .zshenv (this is a bit more complex because I share my config among various systems). However, even if this ssh bug is fixed, in case LANGUAGE is not defined on the ssh client's side, the system must not set it by default in the user's back. So, in short, LANGUAGE should not be set in /etc/default/locale. I am not sure I agree, even though I have my system set up locally not to define LANGUAGE. If ssh were to transmit it and let it override /etc/default/locale, wouldn't sending LANGUAGE= work? Only if LANGUAGE is set. But what about systems not based on glibc? They could have a LANGUAGE environment variable with a different meaning and setting LANGUAGE= may have a bad effect. I wish software used some form of namespace, e.g. GLIBC_LANGUAGE. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon) -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#632798: libc6: broken LANGUAGE design
On 2011-07-07 19:30:38 -0500, Jonathan Nieder wrote: My settings come from the installation. /etc/default/locale was: # File generated by update-locale #LANG=en_US.UTF-8 LANGUAGE=en_US:en (I only added a LC_TIME=en_DK since, hoping it would be taken into account for the time displayed by gdm). FYI, I installed the machine on 2010-01-04. Now I'm confused again. That particular LANGUAGE setting is (I hope) redundant next to LANG, so it should not have been set, right? That is supposed to have been fixed in localechooser 1.07 (2006-03-26): * Only use LANGUAGELIST list of languages when really needed, ie when a language needs and benefits from something else than English. Should avoid reports like #356997. Could it be due to the following? * Prefer official language variants by adding the combination of selected language and country to the front on the language list (closes: #560045). If a preferred locale was selected, instead add the language and country combination from that to the languagelist as it will most likely also indicate the preferred language variant. (on 23 Dec 2009). -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon) -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#632798: libc6: broken LANGUAGE design
Vincent Lefevre wrote: On 2011-07-07 18:46:38 -0500, Jonathan Nieder wrote: If ssh were to transmit it and let it override /etc/default/locale, wouldn't sending LANGUAGE= work? Only if LANGUAGE is set. True. In particular, if the user doesn't know to defend against a remote LANGUAGE setting, this won't help him. Unconditionally prepending LC_MESSAGES to the LANGUAGE list is starting to sound more appealing. But what about systems not based on glibc? They could have a LANGUAGE environment variable with a different meaning and setting LANGUAGE= may have a bad effect. The only library I've heard of that pays attention to LANGUAGE is gettext's libintl, even on systems not based on glibc. I think the empty-LANGUAGE trick should be safe and portable to all systems where the ssh server allows it to override local defaults. Next step is to write some patches (and after that to pass them to upstreams and learn what they think), so I will probably go quiet for a while. Still, I'm very grateful for your help in thinking the design through. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#632798: libc6: broken LANGUAGE design
Jonathan Nieder wrote: 4. Despite what the gettext manual[*] says, setting LANG does not cause LANGUAGE to take effect. This is another bug, as far as I can tell. $ LANG=en_US LANGUAGE=de_DE cp cp: missing file operand Try `cp --help' for more information. The above explanation is nonsense --- the actual cause of the above behavior is that there is no plain en_US locale installed here. $ locale -a | grep en en_US.utf8 $ LC_ALL=gobbledegook LANGUAGE=de cp cp: missing file operand Try `cp --help' for more information. $ LANG=en_US.UTF-8 LANGUAGE=de cp cp: Fehlendes Dateioperand „cp --help“ gibt weitere Informationen. It _might_ be more intuitive for non-installed locales to enable LANGUAGE, too; if so, you can pretend I was complaining about that and call it #4. Otherwise, please feel free to ignore the example. Sorry for the confusion. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#632798: libc6: broken LANGUAGE design
Hi Jonathan, On 2011-07-05 22:22:51 -0500, Jonathan Nieder wrote: 1. On my local machine, I could not reproduce the same effect. That's probably because no default locale is configured here. After making the default locale de_DE.UTF-8 using dpkg-reconfigure -plow locales, /etc/environment is empty and /etc/default/locale looks like this # File generated by update-locale LANG=de_DE.UTF-8 #LANGUAGE=en_US and ssh localhost locale still shows LANGUAGE=. Any idea what's different between your setup and mine? My settings come from the installation. /etc/default/locale was: # File generated by update-locale #LANG=en_US.UTF-8 LANGUAGE=en_US:en (I only added a LC_TIME=en_DK since, hoping it would be taken into account for the time displayed by gdm). FYI, I installed the machine on 2010-01-04. I did: $ wget http://cdimage.debian.org/cdimage/daily-builds/daily/arch-latest/amd64/jigdo-cd/debian-testing-amd64-netinst.jigdo $ wget wget http://cdimage.debian.org/cdimage/daily-builds/daily/arch-latest/amd64/jigdo-cd/debian-testing-amd64-netinst.template $ jigdo-lite debian-testing-amd64-netinst.jigdo then growisofs -Z /dev/scd0=debian-testing-amd64-netinst.iso to burn a DVD. The installer asked me which language to use, and I chose English (could be American English). Note: I saw that on a machine reinstalled with Debian squeeze, /etc/default/locale only sets LANG, but this may have been manually set by the admin. 2. openssh-server has the following in its default sshd/config: AcceptEnv LANG LC_* That is unfortunate. Independently of everything else, this list ought to be expanded to include LANGUAGE. I agree. This would be a nice addition if /etc/default/locale does not set LANGUAGE. Now, if LANGUAGE is set in /etc/default/locale, this change may not solve the problem due to: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=313317 However, even if this ssh bug is fixed, in case LANGUAGE is not defined on the ssh client's side, the system must not set it by default in the user's back. So, in short, LANGUAGE should not be set in /etc/default/locale. I agree concerning 3 and 4 (with corrected explanations). 5. As you mentioned, when LANGUAGE has an effect, it takes precedence over LC_MESSAGES. The principle of least surprise might indicate that the locale in LC_MESSAGES should be used in preference to LANGUAGE, but that would make it impossible to express something like what is currently meant by LANGUAGE=fr:de LC_MESSAGES=de on an installation where most programs are translated to German. (It means: programs using catgets should use German, while programs using gettext should prefer French if possible. Intent: I'd prefer French, but barring that, please give me German instead of English). Hmm... OK, I thought that all programs were using gettext. If one is willing to break with 10 years of gettext behavior, it should be possible to change this without that downside by prepending $LC_MESSAGES to LANGUAGE when and only when it is not already an item in $LANGUAGE. A nice side-effect would be the ability to partially work around (2), as you mentioned. What do you think? What pieces does the above description miss? It would also be necessary that /etc/default/locale does not set LANGUAGE (but perhaps this is already fixed for the new installers). Said otherwise, as LANGUAGE is not standard (e.g. for portable Unix scripts and other settings) and has the precedence, it should never be set by the system, only by the user (who doesn't have necessarily root access) if he wishes too. And of course, the specification of LANGUAGE in the glibc manual should be correct (bug 632795). -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon) -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#632798: libc6: broken LANGUAGE design
Package: libc6 Version: 2.13-10 Severity: normal The current LANGUAGE design is broken. Here an example: ypig% locale LANG=POSIX LANGUAGE= LC_CTYPE=en_US.ISO8859-1 LC_NUMERIC=POSIX LC_TIME=en_DK LC_COLLATE=POSIX LC_MONETARY=POSIX LC_MESSAGES=fr_FR LC_PAPER=POSIX LC_NAME=POSIX LC_ADDRESS=POSIX LC_TELEPHONE=POSIX LC_MEASUREMENT=POSIX LC_IDENTIFICATION=POSIX LC_ALL= ypig% cp cp: opérande fichier manquant Saisissez « cp --help » pour plus d'informations. ypig% ssh localhost locale Connected to ypig (from ::1) LANG=POSIX LANGUAGE=en_US:en LC_CTYPE=en_US.ISO8859-1 LC_NUMERIC=POSIX LC_TIME=en_DK LC_COLLATE=POSIX LC_MONETARY=POSIX LC_MESSAGES=fr_FR LC_PAPER=POSIX LC_NAME=POSIX LC_ADDRESS=POSIX LC_TELEPHONE=POSIX LC_MEASUREMENT=POSIX LC_IDENTIFICATION=POSIX LC_ALL= ypig% ssh localhost cp Connected to ypig (from ::1) cp: missing file operand Try `cp --help' for more information. The problem is that the system sets LANGUAGE in the user's back, so that the standard user's choice (LC_MESSAGES=fr_FR) is no longer taken into account. The best solution would be to make the standard locale system override LANGUAGE, which could just be used as a fallback. Said otherwise, it is equivalent to internally prepend the current LC_MESSAGES setting (possibly coming from LANG or LC_ALL) to the LANGUAGE value. Alternatively the system that sets LANGUAGE could be smarter (but it's difficult to know what the user expects, in particular if SSH doesn't transmit LANGUAGE). -- System Information: Debian Release: wheezy/sid APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental') Architecture: amd64 (x86_64) Kernel: Linux 2.6.39-2-amd64 (SMP w/8 CPU cores) Locale: LANG=POSIX, LC_CTYPE=en_US.ISO8859-1 (charmap=ISO-8859-1) Shell: /bin/sh linked to /bin/dash Versions of packages libc6 depends on: ii libc-bin 2.13-10Embedded GNU C Library: Binaries ii libgcc1 1:4.6.1-2 GCC support library libc6 recommends no packages. Versions of packages libc6 suggests: ii debconf [debconf-2.0] 1.5.40 Debian configuration management sy ii glibc-doc 2.13-10Embedded GNU C Library: Documentat ii locales 2.13-10Embedded GNU C Library: National L -- debconf information: glibc/upgrade: true glibc/disable-screensaver: glibc/restart-failed: * glibc/restart-services: exim4 cups cron atd apache2 -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#632798: libc6: broken LANGUAGE design
Hi Vincent, Vincent Lefevre wrote: LANGUAGE= [...] LC_MESSAGES=fr_FR [...] ypig% ssh localhost locale Connected to ypig (from ::1) LANG=POSIX LANGUAGE=en_US:en [...] LC_MESSAGES=fr_FR [...] LC_ALL= [...] The problem is that the system sets LANGUAGE in the user's back, so that the standard user's choice (LC_MESSAGES=fr_FR) is no longer taken into account. I agree that this is not very good. Before deciding how to fix it, let's describe the mechanism. 1. On my local machine, I could not reproduce the same effect. That's probably because no default locale is configured here. After making the default locale de_DE.UTF-8 using dpkg-reconfigure -plow locales, /etc/environment is empty and /etc/default/locale looks like this # File generated by update-locale LANG=de_DE.UTF-8 #LANGUAGE=en_US and ssh localhost locale still shows LANGUAGE=. Any idea what's different between your setup and mine? 2. openssh-server has the following in its default sshd/config: AcceptEnv LANG LC_* That is unfortunate. Independently of everything else, this list ought to be expanded to include LANGUAGE. 3. The locale C.UTF-8 does not work like C for the sake of this feature: $ LC_ALL=C LANGUAGE=de_DE cp cp: missing file operand Try `cp --help' for more information. $ LC_ALL=C.UTF-8 LANGUAGE=de_DE cp cp: Fehlendes Dateioperand „cp --help“ gibt weitere Informationen. If I understand the purpose of C.UTF-8 correctly, that's another bug (independent, though). 4. Despite what the gettext manual[*] says, setting LANG does not cause LANGUAGE to take effect. This is another bug, as far as I can tell. $ LANG=en_US LANGUAGE=de_DE cp cp: missing file operand Try `cp --help' for more information. 5. As you mentioned, when LANGUAGE has an effect, it takes precedence over LC_MESSAGES. The principle of least surprise might indicate that the locale in LC_MESSAGES should be used in preference to LANGUAGE, but that would make it impossible to express something like what is currently meant by LANGUAGE=fr:de LC_MESSAGES=de on an installation where most programs are translated to German. (It means: programs using catgets should use German, while programs using gettext should prefer French if possible. Intent: I'd prefer French, but barring that, please give me German instead of English). If one is willing to break with 10 years of gettext behavior, it should be possible to change this without that downside by prepending $LC_MESSAGES to LANGUAGE when and only when it is not already an item in $LANGUAGE. A nice side-effect would be the ability to partially work around (2), as you mentioned. What do you think? What pieces does the above description miss? Thanks for your attention to detail. Jonathan [*] http://www.gnu.org/software/gettext/manual/gettext.html#The-LANGUAGE-variable -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
