Bug#318923: xterm: localization hell again (even with UTF8 locales)
On Wed, Jul 20, 2005 at 09:02:13AM +0300, Wladimir Mutel wrote: > On Tue, Jul 19, 2005 at 06:09:36PM -0400, Thomas Dickey wrote: > > > Looking for a different way of seeing things, I find that xfd can display > > TrueType fonts (in a limited way - no size changes). If I run > > > > xfd -fa Terminus > > Oh yes. Running xfd in en_US.UTF-8 locale, I indeed get only > one glyph page. And running in in uk_UA.UTF-8, I get multiple > pages, with most of required Cyrillic in code points starting > from 0x400. I didn't notice that. But checking, it does indeed match. Looking at the trace (see below) from fontconfig, it appears that the weights given for the language support, without other parameters such as size to override it, make that select the Unicode font. Also, setting the locale before running xterm makes it select the Unicode font. I did consider last night whether the language feature was supposed to select a Unicode font, but did not see this in the source - the relation is indirect. > > It shows me a single page (0 to 255). Doing > > > > xfd -fa FreeMono > > > > shows several pages, populated with glyphs. Looking for the files that > > represent xfonts-terminus, I see only some bitmap fonts. > > Yes, Terminus is bitmapped. But on my system it is managed > through FontConfig and displayed in fc-list as well. > > As to FreeMono, in en_US.UTF-8 xfd catches 'Bitstream Vera Sans > Mono' that does not have Cyrillic page; and in uk_UA.UTF-8 - > 'Andale Mono' (Microsoft TTF) that does have it. fontconfig has a property for language, which is used to limit the fonts returned. But since the 8-bit and Unicode flavors for Terminus fonts both support your language, both are matched. > > However, as you report, I can tell gvim to load "Terminus" font and display > > the date in uk_UA.UTF-8 locale. > > Yes, gvim and Mozilla were rock-solid as to i18n, since long. > > > Here is another clue: I can see by the access times which font is actually > > being loaded. For whatever reason, both xterm and xfd are causing Xft > > to load the 8859-1 version of the font rather than the unicode version. > > Yes, indeed something may happen due to Xft/FontConfig. But why > should it limit code points in such way ? To bring more hell to > mixed-language users ? :> > Anyway, gvim and Mozilla managed to overcome that in some way... gvim is using pango, which interfaces to fontconfig. Reading through pango, I couldn't see that it used any features of fontconfig that xterm wasn't. So I went back to studying fontconfig. When I tested gvim, I set the locale outside the program. That would make fontconfig return the right font. > > The 8859-1 fonts of course are 8-bit, and the codes that you are attempting > > to display are not. > > I checked this too. I use strace rather that ls/find/etc > options. Lots of useful can be dug out of strace logs. I read/grep'd through the Xft and fontconfig source looking to see what parameters could be set. Neither has any documentation worth discussing - perhaps 5% of the interface is documented. But I did find (reading the source code) that I could set an environment variable to trace it, e.g., export FC_DEBUG=268435455 (a set of bits OR'd together - fontconfig uses atoi() which requires decimal, rather than using strtol() would could allow octal/decimal/hex). > > > So it does not appear to be a problem with the font-files themselves, > > but perhaps the font server (perhaps requiring some new or modified > > information). I see that /usr/bin/X11/xfs is dated June 1. > > I don't use xfs. I use client-side FontConfig and server-side > FontPath for 'classic' applications. It's still the same - the problem is how fontconfig handles the set of Terminus fonts. > Btw, I used XLFDs in app-defaults/XTerm and they worked for me > until some upgrade (don't remember exactly, either xterm from > 4.3.0 to 6.8.2, or between some 6.8.2s), when 'the hell' > returned again :> > > XTerm: ... that looks as if it should have worked. > After one upgrade, I have found that my xterm started to display > 'fixed' font. It started to skip these 'terminus' XLFDs in some way. > Then I did some juggling (copied XLFDs from XTerm to UXTerm ot vice > versa, don't remember exactly), and Terminus returned back. But > then it had bad boldification (artificial one, through > overstrike). Then I thrown '*faceName: Terminus' in, which > brought the proper boldification, but with bundled localization > hell as well :> Currently, all I can see to fix this is to set the uk_UA.UTF-8 locale before xterm is started. A better fix would be to either split Terminus fonts to reflect fontconfig's limitations, or modify fontconfig to allow the application to tell it to return a Unicode font. -- Thomas
Bug#318923: xterm: localization hell again (even with UTF8 locales)
On Wed, Jul 20, 2005 at 11:39:04AM +0200, David Martínez Moreno wrote: > El Miércoles, 20 de Julio de 2005 10:31, Wladimir Mutel escribió: > > > Ok, I decided to comment out this 'fontFace'. And know what ? > > > The localization hell has gone ! :> Now if only I could get back > > > my nice boldification that I had in the past ... :> > > > > Thanks to developers, in 6.8.2.dfsg.1-3 boldification hell was > > fixed as well. Now I just have to refrain from using faceName, > > and dear developers - from bringing us more hell again :> > > Is this a thumbs-up to close the bug? :-D Actually two bugs were reported. For this one, I think the problem lies outside xterm. I'll investigate the other one this evening. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net pgpp91cMydmnc.pgp Description: PGP signature
Re: Bug#318923: xterm: localization hell again (even with UTF8 locales)
El Miércoles, 20 de Julio de 2005 10:31, Wladimir Mutel escribió: > > Ok, I decided to comment out this 'fontFace'. And know what ? > > The localization hell has gone ! :> Now if only I could get back > > my nice boldification that I had in the past ... :> > > Thanks to developers, in 6.8.2.dfsg.1-3 boldification hell was > fixed as well. Now I just have to refrain from using faceName, > and dear developers - from bringing us more hell again :> Is this a thumbs-up to close the bug? :-D Regards, Ender. -- Yes I'm old. Old enough to remember when the MCP was just a chess program! -- Dumont (Tron). -- Debian developer pgpHZnAoU305I.pgp Description: PGP signature
Bug#318923: xterm: localization hell again (even with UTF8 locales)
> Ok, I decided to comment out this 'fontFace'. And know what ? > The localization hell has gone ! :> Now if only I could get back > my nice boldification that I had in the past ... :> Thanks to developers, in 6.8.2.dfsg.1-3 boldification hell was fixed as well. Now I just have to refrain from using faceName, and dear developers - from bringing us more hell again :> -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Bug#318923: xterm: localization hell again (even with UTF8 locales)
> After one upgrade, I have found that my xterm started to display > 'fixed' font. It started to skip these 'terminus' XLFDs in some way. > Then I did some juggling (copied XLFDs from XTerm to UXTerm ot vice > versa, don't remember exactly), and Terminus returned back. But > then it had bad boldification (artificial one, through > overstrike). Then I thrown '*faceName: Terminus' in, which > brought the proper boldification, but with bundled localization > hell as well :> Ok, I decided to comment out this 'fontFace'. And know what ? The localization hell has gone ! :> Now if only I could get back my nice boldification that I had in the past ... :> -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Bug#318923: xterm: localization hell again (even with UTF8 locales)
On Tue, Jul 19, 2005 at 06:09:36PM -0400, Thomas Dickey wrote: > Looking for a different way of seeing things, I find that xfd can display > TrueType fonts (in a limited way - no size changes). If I run > > xfd -fa Terminus Oh yes. Running xfd in en_US.UTF-8 locale, I indeed get only one glyph page. And running in in uk_UA.UTF-8, I get multiple pages, with most of required Cyrillic in code points starting from 0x400. > It shows me a single page (0 to 255). Doing > > xfd -fa FreeMono > > shows several pages, populated with glyphs. Looking for the files that > represent xfonts-terminus, I see only some bitmap fonts. Yes, Terminus is bitmapped. But on my system it is managed through FontConfig and displayed in fc-list as well. As to FreeMono, in en_US.UTF-8 xfd catches 'Bitstream Vera Sans Mono' that does not have Cyrillic page; and in uk_UA.UTF-8 - 'Andale Mono' (Microsoft TTF) that does have it. > However, as you report, I can tell gvim to load "Terminus" font and display > the date in uk_UA.UTF-8 locale. Yes, gvim and Mozilla were rock-solid as to i18n, since long. > Here is another clue: I can see by the access times which font is actually > being loaded. For whatever reason, both xterm and xfd are causing Xft > to load the 8859-1 version of the font rather than the unicode version. Yes, indeed something may happen due to Xft/FontConfig. But why should it limit code points in such way ? To bring more hell to mixed-language users ? :> Anyway, gvim and Mozilla managed to overcome that in some way... > Here's a cut-paste from my directory editor showing the access times: > > crayon:/4 of 191 > [atime] > 96 Tue 18:00:06 etc/fonts/conf.d/ > 54 Tue 18:00:06 etc/fonts/conf.d/50-enable-terminus.conf > 02 Tue 18:00:06 usr/X11R6/lib/X11/fonts/misc/ter-u12n_iso-8859-1.pcf.gz > 25 Tue 17:58:35 usr/X11R6/lib/X11/fonts/misc/ter-u14b_iso-8859-1.pcf.gz > 32 Tue 17:58:35 usr/X11R6/lib/X11/fonts/misc/ter-u14n_iso-8859-1.pcf.gz > 70 Tue 17:58:09 usr/X11R6/lib/X11/fonts/misc/ter-u16b_unicode.pcf.gz > 97 Tue 17:58:08 usr/X11R6/lib/X11/fonts/misc/ter-u16n_unicode.pcf.gz > 25 Tue 17:52:50 usr/share/doc/xfonts-terminus/README.Debian > 51 Tue 17:52:20 etc/X11/fonts/misc/xfonts-terminus.alias > > The "u12n" is loaded by xfd (running in uk_UA.UTF-8 locale). > The "u14b" files were loaded by "uxterm -fa Terminus" > The "u16n" files were loaded by gvim (running in uk_UA.UTF-8 locale). > > The 8859-1 fonts of course are 8-bit, and the codes that you are attempting > to display are not. I checked this too. I use strace rather that ls/find/etc options. Lots of useful can be dug out of strace logs. > So it does not appear to be a problem with the font-files themselves, > but perhaps the font server (perhaps requiring some new or modified > information). I see that /usr/bin/X11/xfs is dated June 1. I don't use xfs. I use client-side FontConfig and server-side FontPath for 'classic' applications. Btw, I used XLFDs in app-defaults/XTerm and they worked for me until some upgrade (don't remember exactly, either xterm from 4.3.0 to 6.8.2, or between some 6.8.2s), when 'the hell' returned again :> XTerm: *VT100.utf8Fonts.font2: -*-terminus-medium-*-*--14-*-*-*-*-*-iso10646-* *VT100.utf8Fonts.font: -*-terminus-medium-*-*--16-*-*-*-*-*-iso10646-* *VT100.utf8Fonts.font3: -*-terminus-medium-*-*--17-*-*-*-*-*-iso10646-* *VT100.utf8Fonts.font4: -*-terminus-medium-*-*--20-*-*-*-*-*-iso10646-* *VT100.utf8Fonts.font5: -*-terminus-medium-*-*--24-*-*-*-*-*-iso10646-* *VT100.utf8Fonts.font6: -*-terminus-medium-*-*--28-*-*-*-*-*-iso10646-* *VT100*font2: -*-terminus-medium-*-*--14-*-*-*-*-*-iso10646-* *VT100*font: -*-terminus-medium-*-*--16-*-*-*-*-*-iso10646-* *VT100*font3: -*-terminus-medium-*-*--17-*-*-*-*-*-iso10646-* *VT100*font4: -*-terminus-medium-*-*--20-*-*-*-*-*-iso10646-* *VT100*font5: -*-terminus-medium-*-*--24-*-*-*-*-*-iso10646-* *VT100*font6: -*-terminus-medium-*-*--28-*-*-*-*-*-iso10646-* UXTerm: *VT100*font2: -*-terminus-medium-*-*--14-*-*-*-*-*-iso10646-* *VT100*font: -*-terminus-medium-*-*--16-*-*-*-*-*-iso10646-* *VT100*font3: -*-terminus-medium-*-*--17-*-*-*-*-*-iso10646-* *VT100*font4: -*-terminus-medium-*-*--20-*-*-*-*-*-iso10646-* *VT100*font5: -*-terminus-medium-*-*--24-*-*-*-*-*-iso10646-* *VT100*font6: -*-terminus-medium-*-*--28-*-*-*-*-*-iso10646-* After one upgrade, I have found that my xterm started to display 'fixed' font. It started to skip these 'terminus' XLFDs in some way. Then I did some juggling (copied XLFDs from XTerm to UXTerm ot vice versa, don't remember exactly), and Terminus returned back. But then it had bad boldification (artificial one, through overstrike). Then I thr
Bug#318923: xterm: localization hell again (even with UTF8 locales)
I'm studied Xft/fontconfig well enough to guess what's going on. It appears that Xft/fontconfig don't provide a reliable mechanism for the application to say that they don't want an 8-bit font. (It is possible to use the FC_CHARSET property to do something like this, but it's slow and unreliable). I noticed that if I change the fontsize of the xterm, some fontsizes give the 8-bit font, some give the Unicode font. That's by chance, according to what fontconfig happened to find as a close match. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net pgpy8qDIh257A.pgp Description: PGP signature
Bug#318923: xterm: localization hell again (even with UTF8 locales)
On Tue, Jul 19, 2005 at 01:18:45PM +0300, Wladimir Mutel wrote: > > Very strange 'locale' printout above. Perhaps you should run > 'with-locale' with first parameter 'uk_UA.UTF-8', not the entire > assignment 'LANG=uk_UA.UTF-8'. Btw, your 'date' prints English > date, not Cyrillic/Ukrainian. And you see these > 'locale: Cannot set ...' diagnostics, which is not too good. > > > testit: > > #!/bin/sh > > with-locale LANG=uk_UA.UTF-8 locale > > with-locale LANG=uk_UA.UTF-8 date oops - I see what you are saying (very long day). The script should have said #!/bin/sh with-locale uk_UA.UTF-8 locale with-locale uk_UA.UTF-8 date When I do that, I do get the (un)expected result (the date rendered as n's). Looking for a different way of seeing things, I find that xfd can display TrueType fonts (in a limited way - no size changes). If I run xfd -fa Terminus It shows me a single page (0 to 255). Doing xfd -fa FreeMono shows several pages, populated with glyphs. Looking for the files that represent xfonts-terminus, I see only some bitmap fonts. However, as you report, I can tell gvim to load "Terminus" font and display the date in uk_UA.UTF-8 locale. Here is another clue: I can see by the access times which font is actually being loaded. For whatever reason, both xterm and xfd are causing Xft to load the 8859-1 version of the font rather than the unicode version. Here's a cut-paste from my directory editor showing the access times: crayon:/4 of 191 [atime] 96 Tue 18:00:06 etc/fonts/conf.d/ 54 Tue 18:00:06 etc/fonts/conf.d/50-enable-terminus.conf 02 Tue 18:00:06 usr/X11R6/lib/X11/fonts/misc/ter-u12n_iso-8859-1.pcf.gz 25 Tue 17:58:35 usr/X11R6/lib/X11/fonts/misc/ter-u14b_iso-8859-1.pcf.gz 32 Tue 17:58:35 usr/X11R6/lib/X11/fonts/misc/ter-u14n_iso-8859-1.pcf.gz 70 Tue 17:58:09 usr/X11R6/lib/X11/fonts/misc/ter-u16b_unicode.pcf.gz 97 Tue 17:58:08 usr/X11R6/lib/X11/fonts/misc/ter-u16n_unicode.pcf.gz 25 Tue 17:52:50 usr/share/doc/xfonts-terminus/README.Debian 51 Tue 17:52:20 etc/X11/fonts/misc/xfonts-terminus.alias The "u12n" is loaded by xfd (running in uk_UA.UTF-8 locale). The "u14b" files were loaded by "uxterm -fa Terminus" The "u16n" files were loaded by gvim (running in uk_UA.UTF-8 locale). The 8859-1 fonts of course are 8-bit, and the codes that you are attempting to display are not. So it does not appear to be a problem with the font-files themselves, but perhaps the font server (perhaps requiring some new or modified information). I see that /usr/bin/X11/xfs is dated June 1. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net pgpVfb7TwMaqv.pgp Description: PGP signature
Bug#318923: xterm: localization hell again (even with UTF8 locales)
On Tue, Jul 19, 2005 at 01:18:45PM +0300, Wladimir Mutel wrote: > > Very strange 'locale' printout above. Perhaps you should run > 'with-locale' with first parameter 'uk_UA.UTF-8', not the entire > assignment 'LANG=uk_UA.UTF-8'. Btw, your 'date' prints English > date, not Cyrillic/Ukrainian. And you see these > 'locale: Cannot set ...' diagnostics, which is not too good. I think that the shell was unhappy with the unset's. Of course I noticed that, which is why I ran "locale" to see what it would say. However, my problem getting date to show Cyrillic is not related to the bug reported. > nnn nnn 19 12:59:53 EEST 2005 > (with Terminus fontFace; with fixed/clean fontFaces it prints > dotted squares instead of 'n's) The dotted squares sound like missing glyphs. I'll try your output string to see how it looks for me (thanks). > Moreover, when I select 'nnn nnn' (or dotted squares) in > problem/English xterm and paste it into Cyrillized one, it is > pasted perfectly well, giving '÷ÔÒ ìÉÐ' there. It should. I looked as far last night to see that xterm is storing the data as expected, and making calls to Xft to display. I'll look further tonight to see what I can. xterm's select/paste is done on the data that xterm stores rather than on what Xft may display. > I was so glad working with xterm and Unicode locales. I thought > mixed-lang environment dream has been finally realized :> But > alas ... :> New changes returned old griefs :> -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net pgpejD2uf7j8R.pgp Description: PGP signature
Bug#318923: xterm: localization hell again (even with UTF8 locales)
On Mon, Jul 18, 2005 at 05:36:02PM -0400, Thomas Dickey wrote: > /usr/build/xterm/xterm-203b (101) ./testit > locale: Cannot set LC_CTYPE to default locale: No such file or directory > locale: Cannot set LC_MESSAGES to default locale: No such file or directory > locale: Cannot set LC_ALL to default locale: No such file or directory > LANG=LANG=uk_UA.UTF-8 > LC_CTYPE="LANG=uk_UA.UTF-8" > LC_NUMERIC="LANG=uk_UA.UTF-8" > LC_TIME="LANG=uk_UA.UTF-8" > LC_COLLATE="LANG=uk_UA.UTF-8" > LC_MONETARY="LANG=uk_UA.UTF-8" > LC_MESSAGES="LANG=uk_UA.UTF-8" > LC_PAPER="LANG=uk_UA.UTF-8" > LC_NAME="LANG=uk_UA.UTF-8" > LC_ADDRESS="LANG=uk_UA.UTF-8" > LC_TELEPHONE="LANG=uk_UA.UTF-8" > LC_MEASUREMENT="LANG=uk_UA.UTF-8" > LC_IDENTIFICATION="LANG=uk_UA.UTF-8" > LC_ALL=LANG=uk_UA.UTF-8 > Mon Jul 18 17:28:28 EDT 2005 > /usr/build/xterm/xterm-203b (102) Very strange 'locale' printout above. Perhaps you should run 'with-locale' with first parameter 'uk_UA.UTF-8', not the entire assignment 'LANG=uk_UA.UTF-8'. Btw, your 'date' prints English date, not Cyrillic/Ukrainian. And you see these 'locale: Cannot set ...' diagnostics, which is not too good. > testit: > #!/bin/sh > with-locale LANG=uk_UA.UTF-8 locale > with-locale LANG=uk_UA.UTF-8 date > > with-locale: > #!/bin/sh > unset LANG > unset LC_ALL > unset LC_CTYPE > export LANG=$1 > export LC_ALL=$1 > shift > $* > For the sake of discussion, a typescript file from your session, showing > the output of locale and the output of date would show enough information > to see if it is encoded in UTF-8, and allow me (by cat'ing to the terminal) > to see what xterm would display. Here is 'locale' printout from the 'problem' xterm : LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL= Here is what 'LANG=uk_UA.UTF-8 date' looks like : nnn nnn 19 12:59:53 EEST 2005 (with Terminus fontFace; with fixed/clean fontFaces it prints dotted squares instead of 'n's) Here is 'locale' printout from Cyrillized/Ukrainized xterm : LANG=uk_UA.UTF-8 LC_CTYPE="uk_UA.UTF-8" LC_NUMERIC="uk_UA.UTF-8" LC_TIME="uk_UA.UTF-8" LC_COLLATE="uk_UA.UTF-8" LC_MONETARY="uk_UA.UTF-8" LC_MESSAGES="uk_UA.UTF-8" LC_PAPER="uk_UA.UTF-8" LC_NAME="uk_UA.UTF-8" LC_ADDRESS="uk_UA.UTF-8" LC_TELEPHONE="uk_UA.UTF-8" LC_MEASUREMENT="uk_UA.UTF-8" LC_IDENTIFICATION="uk_UA.UTF-8" LC_ALL= How 'date' looks there : Втр Лип 19 13:02:49 EEST 2005 'date' passed through 'xxd' looks the same in both xterms : 000: d092 d182 d180 20d0 9bd0 b8d0 bf20 3139 .. ..19 I.e. Cyrillic letters are encoded as usual, with leading byte 'd0' or 'd1'. Moreover, when I select 'nnn nnn' (or dotted squares) in problem/English xterm and paste it into Cyrillized one, it is pasted perfectly well, giving 'Втр Лип' there. I was so glad working with xterm and Unicode locales. I thought mixed-lang environment dream has been finally realized :> But alas ... :> New changes returned old griefs :>
Bug#318923: xterm: localization hell again (even with UTF8 locales)
On Mon, Jul 18, 2005 at 07:20:06PM +0200, Wladimir Mutel wrote: > Package: xterm > Version: 6.8.2.dfsg.1-2 > Severity: important > > > How to reproduce : > > 1. Create/generate locales for en_US.UTF-8 and uk_UA.UTF-8 > 2. Add '*faceName: Terminus' into app-defaults/XTerm (Terminus has > enough Unicode characters for both Latin and Cyrillic). > 3. Start xterm (or uxterm) with LANG=en_US.UTF-8 > 4. Inside it, run 'date' with LANG=uk_UA.UTF-8 > 5. See 'nnn nnn' instead of weekday and month names. Actually on my machine, I'm not seeing a change for the output of 'date' (yes, I did regenerate locales): /usr/build/xterm/xterm-203b (101) ./testit locale: Cannot set LC_CTYPE to default locale: No such file or directory locale: Cannot set LC_MESSAGES to default locale: No such file or directory locale: Cannot set LC_ALL to default locale: No such file or directory LANG=LANG=uk_UA.UTF-8 LC_CTYPE="LANG=uk_UA.UTF-8" LC_NUMERIC="LANG=uk_UA.UTF-8" LC_TIME="LANG=uk_UA.UTF-8" LC_COLLATE="LANG=uk_UA.UTF-8" LC_MONETARY="LANG=uk_UA.UTF-8" LC_MESSAGES="LANG=uk_UA.UTF-8" LC_PAPER="LANG=uk_UA.UTF-8" LC_NAME="LANG=uk_UA.UTF-8" LC_ADDRESS="LANG=uk_UA.UTF-8" LC_TELEPHONE="LANG=uk_UA.UTF-8" LC_MEASUREMENT="LANG=uk_UA.UTF-8" LC_IDENTIFICATION="LANG=uk_UA.UTF-8" LC_ALL=LANG=uk_UA.UTF-8 Mon Jul 18 17:28:28 EDT 2005 /usr/build/xterm/xterm-203b (102) testit: #!/bin/sh with-locale LANG=uk_UA.UTF-8 locale with-locale LANG=uk_UA.UTF-8 date with-locale: #!/bin/sh unset LANG unset LC_ALL unset LC_CTYPE export LANG=$1 export LC_ALL=$1 shift $* For the sake of discussion, a typescript file from your session, showing the output of locale and the output of date would show enough information to see if it is encoded in UTF-8, and allow me (by cat'ing to the terminal) to see what xterm would display. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net pgpiZAJ3BCQeA.pgp Description: PGP signature
Bug#318923: xterm: localization hell again (even with UTF8 locales)
Package: xterm Version: 6.8.2.dfsg.1-2 Severity: important How to reproduce : 1. Create/generate locales for en_US.UTF-8 and uk_UA.UTF-8 2. Add '*faceName: Terminus' into app-defaults/XTerm (Terminus has enough Unicode characters for both Latin and Cyrillic). 3. Start xterm (or uxterm) with LANG=en_US.UTF-8 4. Inside it, run 'date' with LANG=uk_UA.UTF-8 5. See 'nnn nnn' instead of weekday and month names. Why, oh why English UTF8 locale/encoding excludes Cyrillic UTF8 characters out of printability ? (the same could be reproduced with 'fixed' or 'clean' fonts as well, (-fa option) with only difference - there dotted squares are seen). How this should be indeed : 3. Start xterm/uxterm with LANG=uk_UA.UTF-8 (or with any other UTF8 locale, as it worked before) 4. Run 'date' with LANG=uk_UA.UTF-8 5. See proper cyrillic letters for month and weekday. 6. Run 'date' wirh LANG=en_US.UTF-8 7. See proper English date. This behaviour started to occur after upgrading from XFree86 4.3.0 to X.Org 6.8.2 . Don't know if it lays in xterm itself or in supporting libs. Did not do any deep research. Just noticed that on one my system where locale was en_US.UTF-8, Cyrillic letters stopped to be printed after this upgrade. No similar effects were observed in gvim, neither in Mozilla. Only 'beloved' xterm was so overly sensitive ... -- System Information: Debian Release: testing/unstable APT prefers hoary-updates APT policy: (500, 'hoary-updates'), (500, 'hoary-security'), (500, 'hoary'), (500, 'unstable'), (1, 'experimental') Architecture: i386 (i686) Shell: /bin/sh linked to /bin/bash Kernel: Linux 2.6.12 Locale: LANG=uk_UA.UTF-8, LC_CTYPE=uk_UA.UTF-8 (charmap=UTF-8) Versions of packages xterm depends on: ii libc6 2.3.5-1ubuntu3 GNU C Library: Shared libraries an ii libexpat1 1.95.8-3 XML parsing C library - runtime li ii libfontconfig12.3.2-1generic font configuration library ii libfreetype6 2.1.10-1 FreeType 2 font engine, shared lib ii libice6 6.8.2.dfsg.1-2 Inter-Client Exchange library ii libncurses5 5.4-8 Shared libraries for terminal hand ii libsm66.8.2.dfsg.1-2 X Window System Session Management ii libxaw8 6.8.2.dfsg.1-2 X Athena widget set library ii libxext6 6.8.2.dfsg.1-2 X Window System miscellaneous exte ii libxft2 2.1.7-1FreeType-based font drawing librar ii libxmu6 6.8.2.dfsg.1-2 X Window System miscellaneous util ii libxp66.8.2.dfsg.1-2 X Window System printing extension ii libxpm4 6.8.2.dfsg.1-2 X pixmap library ii libxrender1 1:0.9.0-2 X Rendering Extension client libra ii libxt66.8.2.dfsg.1-2 X Toolkit Intrinsics ii xlibs 6.8.2.dfsg.1-2 X Window System client libraries m ii xlibs-data6.8.2.dfsg.1-2 X Window System client data Versions of packages xterm recommends: ii xutils6.8.2.dfsg.1-2 X Window System utility programs -- no debconf information -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
