Bug#219551: Unicode xterms should do some kind of substitution for missing characters
On Sun, Nov 09, 2003 at 11:15:04PM -0500, Branden Robinson wrote: | On Sat, Nov 08, 2003 at 12:09:16PM +0800, Cameron Patrick wrote: | > I nevertheless maintain that if the requested glyph /isn't/ available, | > it is more useful to display some approximation of the requested | > character than a little white box. | | Uh, you seem to be rather glibly overlooking the requirements involved | in "displaying some approximation of the requested character". Possibly I am. But if other programs can do it, at least for some subset of characters, there's no purely technical reason why xterm cannot. Whether or not it /should/ is another matter, and I admit that it may be ON MUCH shakier ground. | > Yes, that sounds like a good idea. I'd be surprised if there weren't | > libraries around that fall down the Unicode slippery slope already. | > Mozilla already handles this kind of thing. So do gvim and | > gnome-terminal, which suggests that it might be GTK in general that | > handles it. It is even conceivable that the Unicode-handling library | > might be separate from GTK itself; if that is the case, xterm may be | > able to link to it? | | That may be a bad idea. Yes, I suppose it may be - I was just throwing ideas around. I take it that you had some reasoning which led you to believe that the idea is bad, in which case it would be appreciated if you could explain why you think so rather than just vaguely suggesting so. | Even if it isn't, it might be a good idea if the fonts in question | troubled themselves to define a glyph for the codepoint. | | I trust you've filed a bug to that effect. AFAIK most of the Free fonts packaged in Debian have the particular glyph that this bug report mentioned. The one I'm using is from msttcorefonts, so I can't imagine anyone is likely to be able to do anything about a Debian bug files on it. Cheers, Cameron.
Bug#219551: Unicode xterms should do some kind of substitution for missing characters
On Sun, Nov 09, 2003 at 02:35:06PM +0100, Eduard Bloch wrote: | Okay, I see your problem now, the terminal emulator _may_ replace the | presented chars with equivalents in the font. But it's hell of work if | done right, it has to check every char, locate most similar character | in some encodings map and replace it. Presumably it only has to do that on missing characters, which one would think will usually be in the minority of the display at any given time. | I expect xterm to become slower and more memory consuming if this will | ever be implemented. I imagine it would. I concede that one reason that I don't use gnome terminal now is that it feels quite slow compared to xterm. On the other hand, some slowdown is acceptable, especially if the bulk of it would only affect those occasions when characters do need to be subtituted. | However, why do you not just set the locale to Latin1 if your | X-Terminal/X-Server does not support Unicode fonts? My X server is XFree86 4.3; it supports Unicode fonts. The font I am using contains a large number of characters not present in a Latin-1 encoding (although admittedly I would not miss most of them - Cyrillic, Greek, obscure accented letters, random mathematical symbols, the Euro sign, etc), which show up correctly in a uxterm. It does not, however, contain a hyphen in position U+2012. One could, I suppose, argue that the font is broken and should contain a hyphen. I accept this, but would nevertheless like to see xterm work around it, if at all possible. | As said, this would make them slower. Setting the locale to the one best | supported by the X-Server is easier. (As noted above, it's not an X server issue, but a font issue.) Yes, using Latin-1 would make things a lot easier. Indeed I haven't noticed this issue for long as I only switched to a UTF-8 locale a couple of weeks ago. For the most part, this hasn't caused major problems, and I'm interested in getting the minor niggles such as this one fixed up. Cheers, Cameron.
Bug#219551: Unicode xterms should do some kind of substitution for missing characters
On Sun, Nov 09, 2003 at 12:37:43PM +0100, Eduard Bloch wrote: | > That may be so, but gnome-terminal /does/ cope with this situation by | > displaying replacing the hyphen with a minus sign when the font doesn't | | Then gnome-terminal is broken and deservers a bug report. I disagree. gnome-terminal is behaving as intended, i.e., displaying the nearest character to a hyphen present in the font it was instructed to use. Incidentally, below are screenshots of "man rm" on three different terminal emulators. In order of increasing brokenness of display: http://cp.yi.org/cameron/hyphen/gnome-terminal.png http://cp.yi.org/cameron/hyphen/xterm.png http://cp.yi.org/cameron/hyphen/konsole.png | > contain the former. Konsole and xterm do not. xterm is my preferred | | They should not. They implement an UTF-8 terminal correctly, | gnome-terminal does not if I follow your explanation. gnome-terminal, like other GTK2 apps, deals gracefully with fonts with missing characters. Konsole, like other KDE/Qt apps, does not. | Though I could imagine this character replacement as an option, so the | user may enable it he likes it. Implementing it as optional would be possible, I suppose, but I fail to see why anyone would enable an option which gives them an inferior display. | > terminal emulator and it would be nice to see it cope gracefully with | > incomplete fonts---and after all, I would imagine that in the majority | > of fonts there exists some Unicode character which is missing and | > another similar character which could be substituted for it. | | Hyphens are not missing. They are meant to be hyphens and should be | presented as hyphens and not as some other char. They are missing in the font that I am using. Thus they are /not/ displayed as hyphens, they are displayed as little boxes. Presenting them as minus signs is certainly preferable to refusing to display them at all. | > | It was promised that groff will recode hyphen to minus sign in some | > | future version (maybe as an option) to work around broken manpages. | > | > That would be a workaround for this particular case - which is admittedly | > the only that I've noticed. | | That will be the only sane workaround except of fixing the actual | problems in the manpages. No. There is no problem with manpages using hyphens. If a font contains separate hyphens and minus signs, they should be presented as hyphens in UTF-8 locales, too. If a font does /not/ contain a hyphen character, displaying them as the closest approximation that is contained in that font, or perhaps even as a hyphen from another font, is the Right Thing to do. Cheers, Cameron.
Bug#219551: Unicode xterms should do some kind of substitution for missing characters
On Sat, Nov 08, 2003 at 09:03:49PM +0100, Eduard Bloch wrote: | > Many TrueType fonts don't have both a hyphen (0x2010) and a minus sign | > (0x2212); however, groff (and thus man pages) differentiates between the | > two in UTF-8 locales. This results in lots of man pages displaying ugly | > boxes where hyphens should be in a uxterm, but displaying fine on a | > 'normal' xterm where there is no distinction between the two characters. | | That's not a problem with the certain terminal emulator, that is a | problem with groff syntax not understood by many authors. Xterm works | just fine as UTF-8 terminal as well as mlterm/pterm/konsole/gnome-terminal | but the manpages simply specify the wrong char. That may be so, but gnome-terminal /does/ cope with this situation by displaying replacing the hyphen with a minus sign when the font doesn't contain the former. Konsole and xterm do not. xterm is my preferred terminal emulator and it would be nice to see it cope gracefully with incomplete fonts---and after all, I would imagine that in the majority of fonts there exists some Unicode character which is missing and another similar character which could be substituted for it. | It was promised that groff will recode hyphen to minus sign in some | future version (maybe as an option) to work around broken manpages. That would be a workaround for this particular case - which is admittedly the only that I've noticed. Cheers, Cameron.
Bug#219551: Unicode xterms should do some kind of substitution for missing characters
On Fri, Nov 07, 2003 at 10:49:16PM -0500, Branden Robinson wrote: | > If the default Unicode fonts are incomplete enough that they lack this | > character, then that is probably worthy of a bug (and this one could just | > be reassigned), The default bitmap fonts that come with XFree are fine for this particular character. | > but I fail to see why the application should be expected to | > do something fundamentally antithetical to the user's stated request for | > UTF-8, simply because some fonts claiming to be intended for Unicode fail | > to provide a useful set of Unicode entries. | | I am tempted to agree. *sniff* I nevertheless maintain that if the requested glyph /isn't/ available, it is more useful to display some approximation of the requested character than a little white box. | However: | | 1) The Unicode tables define decompositions and alternate forms for many |codepoints, so the slippery slope is at least defined. We probably |need a unicode-slippery-slope shared library rather than writing |this functionality into xterm directly. :) Yes, that sounds like a good idea. I'd be surprised if there weren't libraries around that fall down the Unicode slippery slope already. Mozilla already handles this kind of thing. So do gvim and gnome-terminal, which suggests that it might be GTK in general that handles it. It is even conceivable that the Unicode-handling library might be separate from GTK itself; if that is the case, xterm may be able to link to it? Cheers, Cameron.
Bug#219550: redraw problems with Xft
Package: xterm Version: 4.3.0-0pre1v4 Severity: normal With certain fonts and font sizes in Xft mode, xterm draws characters so that they partially lie in the character cell below; these characters are subsequently not erased properly, leading to an unsightly display. This is particularly noticeable with the "_" character in Andale Mono at large sizes. This is fixed in the latest upstream xterm; looking at the upstream changelog I believe the relevant change is patch #180: make height of TrueType fonts match ascent+descent (patch by Keith Packard). That upstream version also makes underlining work with Xft rendering, which would be nice to have too. Cheers, Cameron. -- System Information: Debian Release: testing/unstable Architecture: i386 Kernel: Linux erdos 2.4.21-cjp-erdos-3 #1 Sun Aug 24 01:18:50 WST 2003 i686 Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 Versions of packages xterm depends on: ii libc6 2.3.2.ds1-8GNU C Library: Shared libraries an ii libexpat1 1.95.6-6 XML parsing C library - runtime li ii libfontconfig12.2.1-8generic font configuration library ii libfreetype6 2.1.5-3FreeType 2 font engine, shared lib ii libncurses5 5.3.20030719-3 Shared libraries for terminal hand ii libxaw7 4.2.1-13 X Athena widget set library ii libxft2 2.1.2-4FreeType-based font drawing librar ii libxrender1 0.8.3-4X Rendering Extension client libra ii xlibs [libxpm4] 4.3.0-0pre1v4 X Window System client libraries -- no debconf information
Bug#219551: Unicode xterms should do some kind of substitution for missing characters
Package: xterm Version: 4.3.0-0pre1v4 Many TrueType fonts don't have both a hyphen (0x2010) and a minus sign (0x2212); however, groff (and thus man pages) differentiates between the two in UTF-8 locales. This results in lots of man pages displaying ugly boxes where hyphens should be in a uxterm, but displaying fine on a 'normal' xterm where there is no distinction between the two characters. uxterm should display hyphens and minus signs as a simple ASCII "-" (0x002D, "hyphen-minus") if the font does not contain the requested character. In fact, it would probably be useful if xterm had some kind of more general character substitution mechanism for this kind of problem. Cameron. -- System Information: Debian Release: testing/unstable Architecture: i386 Kernel: Linux erdos 2.4.21-cjp-erdos-3 #1 Sun Aug 24 01:18:50 WST 2003 i686 Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 Versions of packages xterm depends on: ii libc6 2.3.2.ds1-8GNU C Library: Shared libraries an ii libexpat1 1.95.6-6 XML parsing C library - runtime li ii libfontconfig12.2.1-8generic font configuration library ii libfreetype6 2.1.5-3FreeType 2 font engine, shared lib ii libncurses5 5.3.20030719-3 Shared libraries for terminal hand ii libxaw7 4.2.1-13 X Athena widget set library ii libxft2 2.1.2-4FreeType-based font drawing librar ii libxrender1 0.8.3-4X Rendering Extension client libra ii xlibs [libxpm4] 4.3.0-0pre1v4 X Window System client libraries -- no debconf information
