On Monday, 8 August 2016 14:51:01 UTC+2, Tony Mechelynck wrote: > Well, if you let Pango do glyph reshaping for U+0020 to U+007F you > might end up with what you said you didn't want, i.e. fi fl st ffi ffl > digraphs and trigraphs, which in my experience are actually uglier (in > monospaced fonts) than just letting the individual letters stand side > by side. Try the following in a gvim (even unpatched) with 'encoding' > set to utf-8 to see what I mean: > 1. Enter Insert mode (i or a or o or whatever) > 2. Type the following sequences (with no spaces). Each sequence should > output one glyph. > Ctrl-V u fb00 > Ctrl-V u fb01 > Ctrl-V u fb02 > Ctrl-V u fb03 > Ctrl-V u fb04 > Ctrl-V u fb05 > Ctrl-V u fb06 > You should see, in that order, the glyphs for ff fi fl ft st ffi ffl. > I absolutely don't like them (in the monospaced fonts that have them; > in serif or sans-serif fonts it's different, especially in serif > italic). OTOH, I think that expanding them to two or three character > cells would be ugly the opposite way.
Point taken, but this is not what the patch does. The sets [A-Z], [a-z], [0-9], and ' ' are still drawn without glyph shaping, when there's nothing else in the string being drawn that requires it - just as the old code did. (And the old code also fell back to full glyph shaping once you move above character code 128, i.e. into the range where you start to get UTF-8 encoded characters...) So in that sense, there's no substantial change in strategy. Also, I'm not changing the width of any glyph drawn, so there is no expansion of glyphs going on. It's just a question of how exactly the cache works that bypasses the glyph shaping (and which strings are allowed to bypass the glyph shaping process by going through the cache...). Are you proposing changes to the patch, or are you more concerned about how it'll look? In the former case, maybe you could outline which changes you'd like to see? In the latter case, why not just give the patch a try (e.g by typing one of the character combinations, and see if it does what you'd like it to do), and tell me which bits you don't like (which is much more productive than discussing in the abstract)? Kind regards, Manuel > Best regards, > Tony. > > On Mon, Aug 8, 2016 at 1:50 PM, manuelschiller.pimail via vim_dev > <vim_dev@googlegroups.com> wrote: > > On Monday, 8 August 2016 12:33:44 UTC+2, Tony Mechelynck wrote: > >> On Mon, Aug 8, 2016 at 11:15 AM, manuelschiller.pimail via vim_dev > >> <vim_dev@googlegroups.com> wrote: > >> > On Monday, 8 August 2016 08:31:13 UTC+2, Kazunobu Kuriyama wrote: > >> >> 2016-08-07 20:27 GMT+09:00 manuelschiller.pimail via vim_dev > >> >> <vim...@googlegroups.com>: > >> >> > >> >> > >> >> On Monday, 19 October 2015 08:41:36 UTC+2, Hörmetjan Yiltiz wrote: > >> >> > >> >> > On Sunday, 20 April 2014 11:55:22 UTC-4, François Gannaz wrote: > >> >> > >> >> > > Hello > >> >> > >> >> > > > >> >> > >> >> > > In a few words, here is a patch that makes gvim work better with > >> >> > > ligatures > >> >> > >> >> > > in fonts, which can be useful even for programmers. Details follow. > >> >> > >> >> > > > >> >> > >> >> > > I tried to use a Hasklig[^1], a font with ligatures intended for the > >> >> > >> >> > > Haskell language. It serves the same objective as the Haskell > >> >> > > Conceal > >> >> > >> >> > > script[^2], but with the added benefit that even a mouse copy-paste > >> >> > > works > >> >> > >> >> > > as intended. > >> >> > >> >> > > [^1]: https://github.com/i-tu/hasklig > >> >> > >> >> > > [^2]: http://www.vim.org/scripts/script.php?script_id=3200 > >> >> > >> >> > > > >> >> > >> >> > > Unfortunately, gvim doesn't support ligatures on ASCII characters. > >> >> > > The > >> >> > >> >> > > following assertion fails at run-time: > >> >> > >> >> > > > >> >> > >> >> > > ascii_glyph_table_init: assertion 'gui.ascii_glyphs->num_glyphs > >> >> > > == > >> >> > >> >> > > sizeof(ascii_chars)' failed > >> >> > >> >> > > > >> >> > >> >> > > and many characters are displayed with the wrong glyphs. > >> >> > >> >> > > The attached patch limits the function ascii_glyph_table_init() to > >> >> > >> >> > > spaces and alphanumeric chars. It solves the problem here. > >> >> > >> >> > > > >> >> > >> >> > > Yet I wonder if the current hack with ASCII characters is really > >> >> > > useful. > >> >> > >> >> > > Is there any performance test to check if a simpler behaviour > >> >> > > wouldn't be > >> >> > >> >> > > suitable, at least for modern desktop installations? > >> >> > >> >> > > As the code comment mentions spaces, maybe it should be restricted > >> >> > > to > >> >> > >> >> > > blank lines? > >> >> > >> >> > > > >> >> > >> >> > > Regards > >> >> > >> >> > > -- > >> >> > >> >> > > François > >> >> > >> >> > > >> >> > >> >> > This would be so great to see implemented. > >> >> > >> >> > >> >> > >> >> It certainly would (and apologies for reopening such an old discussion). > >> >> > >> >> > >> >> > >> >> On my system, the patch François sent doesn't work for the ligatures in > >> >> PragmataPro (but flawlessly for other fonts like Hasklig), > >> >> > >> >> > >> >> According to http://www.fsd.it/shop/fonts/pragmatapro/ , PragmataPro > >> >> has "One typeface, 3 families for the right need": > >> >> > >> >> > >> >> > >> >> - PragmataPro is a spaced modularly font family with working ligatures. > >> >> - PragmataPro Mono is a very monospaced without ligatures. > >> >> - Essential PragmataPro is the cheap version to coding for English only. > >> >> > >> >> > >> >> As you know, Vim is implemented by design only to support monospace > >> >> font families. > >> >> > >> >> > >> >> So it looks to me that, among those three font families, the only > >> >> possible and suitable choice for Vim to use with is PragmataPro Mono > >> >> alone. > >> > > >> > In PragmataPro, all ASCII glyphs etc. are monospaced. Just as they are > >> > in PragmatePro Mono or Courier, or whatever your favourite monospaced > >> > font is. The difference between the two is what happens during shaping > >> > into glyphs inside pango (or another library): For fonts like Hasklig or > >> > PragmataPro, some combinations of ASCII characters get assigned a new > >> > glyph (the ligature) which looks better (e.g. "<=" might be replaced > >> > with a new glyph that more closely mimics the mathematical notation), > >> > but fits into the normal monospaced character grid. These ligature > >> > characters are two and sometimes three times the width of a single ASCII > >> > glyph, depending on how many characters they replace (if it's a > >> > monospace font - in a proportional font like Times, the ligature for > >> > something like "fl" can be any width the font designer likes, and I'm > >> > not proposing to support this latter case). > >> > > >> > That kind of character/glyph juggling is okay because it fits with what > >> > vim is expecting to draw anyway (so vim's drawing code can largely > >> > remain unaware of the font shaping going on) - it's expecting to draw > >> > something that's e.g. 2 characters wide, and it draws (a single) glyph > >> > that wide. > >> > > >> > (In PragmataPro Mono, these ligature tables are absent, so shaping just > >> > returns the normal single-width glyphs.) > >> > > >> > The point of this patch is that it ensures that vim gets the right > >> > single width glyphs for all monospaced fonts, even in the code that > >> > bypasses the glyph shaping process and does its own thing. > >> > > >> > I've attached screenshots how unpatched vim (as shipped with the latest > >> > stable debian) and patched vim look with these fonts. In all cases, I do > >> > a ":version" before I take the screenshot. As you can see, it's totally > >> > garbled in case of the unpatched versions of the plots, despite the fact > >> > that the two fonts are monospaced (and are counted as such by all major > >> > terminal programs). > >> > > >> > This is a corner case where the optimisation to speed up rendering of > >> > ASCII glyphs in vim (by bypassing the shaping process) gets it wrong > >> > (namely the corner case where the font designer specified ligatures for > >> > a font). What happens in these cases is that pango returns a list with > >> > fewer glyphs than characters (because some character combinations return > >> > only a single glyph), and vim happily reads past the end of the glyph > >> > list and caches the result to render ascii glyphs. The effect is > >> > unreadable gibberish, as you can see. I think it should just be fixed, > >> > because it's ugly and needlessly astonishes unsuspecting users > >> > (neglecting for a moment the fact that there may also be security > >> > implications of such accesses past the end of a data structure)... > >> > > >> >> If that is the case, however, that font family is described as "without > >> >> ligatures," which makes your patch no sense. > >> > > >> > (See above.) > >> > > >> >> Accordingly, it appears my assumption was wrong; it might be better for > >> >> me to conclude that what you meant by PragmataPro was literally > >> >> PragmataPro, not its monospace version, or PragmataPro Mono. > >> > > >> > (See above again.) > >> > > >> >> Then, another question arises: What does "a spaced modularly font" > >> >> mean? Does it imply that the font family in question is technically > >> >> considered something equivalent to a monospaced font and hence deserve > >> >> Vim's support? > >> > > >> > I'll try and render it yet another way: If you ask pango/... to draw a > >> > single character in a monospaced font, it's always the same width. But, > >> > in some monospaced fonts, for some two (and sometimes three) letter > >> > combinations, the font rendering library substitutes a glyph that looks > >> > better, but occupies exactly the width of two (three) individual > >> > characters. While you still fit totally into the monospaced character > >> > grid, you draw fewer glyphs than input characters now... > >> > > >> > I think this implies that such modern programmer's fonts like Hasklig or > >> > PragmataPro are effectively a monospaced fonts, and people will > >> > increasingly want to use them. (In fact, you're talking to a former > >> > die-hard text mode console adept, who after 15+ years started to see the > >> > point of GUIs...) These are questions that people will keep asking > >> > (probably increasingly often). In my opinion, vim would do well to > >> > include support for this kind of effectively monospaced fonts, so the > >> > answer to those questions can become "update to the latest version", and > >> > you guys can get back to some serious hacking again... ;) > >> > > >> >> I'd appreciate it if you would give me some clarification or correct my > >> >> possible misunderstanding. > >> > > >> > I hope this somehow clarifies the issue. I'm not proposing to implement > >> > proportional font support or anything like that. But the assumption in > >> > the current code that one ascii character renders to a single glyph is > >> > no longer true in today's world, and I think that needs to be fixed > >> > somehow. The fact that it also happens to make Hasklig or PragmataPro > >> > render correctly is, for me, a welcome side effect (and also happens to > >> > be the reason why I got started with this). But the longer I think about > >> > this patch, the more I tend to think of the current (unpatched) drawing > >> > code as a liability for vim because it reads past the end of a data > >> > structure without any checks. We're lucky that only the graphical output > >> > is garbled. > >> > > >> > I hope this clarifies things a bit. > >> > > >> > Best regards, > >> > > >> > Manuel > >> > > >> >> Best regards, > >> >> Kazunobu Kuriyama > >> >> > >> >> apparently because the ASCII-to-glyph mapping won't quite work - I > >> >> suspect that's due to PragmataPro having ligatures in that range, too. > >> >> I solved this by inserting a space between characters that we hand to > >> >> pango for shaping. Sane fonts will not have ligatures between a space > >> >> and a printable character, this way we still get one glyph per ASCII > >> >> character. I've attached the modified patch. This will work with normal > >> >> fonts and ones with programming ligatures like Hasklig and PragmataPro; > >> >> I've tested it with the gtk2 and gtk3 front ends. > >> >> > >> >> > >> >> > >> >> Please let me know if it would be possible to include this, and if not, > >> >> at least the patch is public now where people can find it if they want > >> >> it. > >> >> > >> >> > >> >> > >> >> Best regards, > >> >> > >> >> > >> >> > >> >> Manuel > >> >> > >> >> > >> > >> Hm, the "unpatched hasklig" case seems to be just off-by-one > >> everywhere, while the other "unpatched" one looks pathologic. > >> > >> In current versions of Vim, there is no necessary equivalence between > >> the number of bytes and the number of screen cells used by a given > >> character. CJK "wide" characters take up two screen cells each > >> regardless of the number of bytes, and everything else takes up one > >> cell per character regardless of the number of bytes. (For instance, > >> in UTF-8 which is the recommended 'encoding' these days, codepoints > >> U+0080 to U+07FF use two bytes each, and anything from U+0800 up uses > >> at least three. FWIW, the "usual" Kangxi characters start at U+4E00 > >> with the single-line 一 character [Kangxi radical 1 plus no strokes] > >> meaning "one", but there are "extension CJK" characters as low as > >> U+3400, and the "Compatibility Ideographs Supplement" block starts at > >> U+2F800 and then goes up I don't know how high.) The horizontal > >> tabulation U+0009 is of course a special case, in a category by > >> itself. > >> > >> Best regards, > >> Tony. > > > > Hi Tony, > > > > your musings on UTF-8 sound reasonable, although I'm far from an expert. In > > fact for unicode characters with character code >= 128, even the current > > vim code base will always do the full-blown character shaping, and apply > > whatever magic it applies to display the text. And it'll hopefully get > > things correct even without my patch. > > > > To render strings with codes < 128, however, the current code will apply an > > optimisation for skipping the shaping step inside pango. The character code > > -> glyph mapping is no longer 1:1 for a font with ligatures in that range, > > and that is why the cache of that mapping appears to display gibberish. > > > > So it's less a question of the mapping of characters (whatever their > > encoding) to screen cells, but that a speed optimisation inside the gtk gui > > code can be fooled by some fonts... > > > > Best regards, > > > > Manuel > > > > -- > > -- > > You received this message from the "vim_dev" maillist. > > Do not top-post! Type your reply below the text you are replying to. > > For more information, visit http://www.vim.org/maillist.php > > > > --- > > You received this message because you are subscribed to the Google Groups > > "vim_dev" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to vim_dev+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
