[ft-devel] A question about freetype and harfbuzz
Hi all, I have a question about freetype and harfbuzz. FreeType1 includes an extension to support OpenType text layout processing. But this support hasn't become part of FreeType2. Why? Why does FreeType2 not use the codes of harfbuzz to support OpenType text layout processing? Thanks! :) Regards, Lingning Zhang ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
Re: [ft-devel] A question about freetype and harfbuzz
> I have a question about freetype and harfbuzz. FreeType1 includes > an extension to support OpenType text layout processing. But this > support hasn't become part of FreeType2. Why? Why does FreeType2 > not use the codes of harfbuzz to support OpenType text layout > processing? We have decided to let FreeType 2 do one job: Rendering glyphs. Producing correct text layout for more complicated scripts like Arabic or Indic requires the interpretation of the input text strings, in close connection with font data like OpenType or Apple's AAT. With other words, you need three levels to do this, for example pango -> handling of layout ^ | | v harfbuzz -> handling of OpenType features ^ | | V freetype -> handling of glyph rendering Both harfbuzz and pango could be replaced with ICU (which also can use FreeType). However, there must be some reason for your question -- you are actively working on both harfbuzz and pagno, AFAIK, and you know these things very well, I believe, so please tell us why you think that unifying harfbuzz and freetype would be a good thing. Werner ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
Re: [ft-devel] A question about freetype and harfbuzz
On Mon, 2007-05-28 at 01:48 -0400, LingNing Zhang wrote: > Hi all, > I have a question about freetype and harfbuzz. > FreeType1 includes an extension to support OpenType text layout > processing. But this support hasn't become part of FreeType2. Why? > Why does FreeType2 not use the codes of harfbuzz to support OpenType > text layout processing? > Thanks! > :) Hi Lingning, harfbuzz is in fact based on the OpenType Layout code salvaged from FreeType 1. The reason for not carrying it over to FreeType2 like Werner said is to let FreeType do one thing, and do it really good. behdad > Regards, > Lingning Zhang -- behdad http://behdad.org/ "Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin, 1759 ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
Re: [ft-devel] A question about freetype and harfbuzz
On Mon, 28 May 2007 13:48:20 +0800 LingNing Zhang <[EMAIL PROTECTED]> wrote: >Hi all, >I have a question about freetype and harfbuzz. >FreeType1 includes an extension to support OpenType text layout >processing. But this support hasn't become part of FreeType2. Why? >Why does FreeType2 not use the codes of harfbuzz to support OpenType >text layout processing? >Thanks! # Personally, I'm one of the people who want FT2 to have # support for text layout feature. I guess I feel sympathy # with you. But we are minority among FT2 developers :-). Excuse me, HarfBuzz developers want FT2 to include built-in OT table parsers? Or, there is non-ICU/ non-HarfBuzz/non-M17NLib/non-Pango/non-Qt library their developers want FT2 to have OT table parser and don't want to copy such from existing libraries? One of the disadvantage might be: if FT2 has OT table parser, the rasterizer in FT2 won't use it at all (because full featured text layout would be too big work to include in FT2). The integrity & relationship between FT2 rasterizer and OT support would be confusing. For example, "FT2 can parse BASE table, but FT2 does not reflect its content to rasterization" is confusing. Regards, mpsuzuki ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
Re: [ft-devel] A question about freetype and harfbuzz
> # Personally, I'm one of the people who want FT2 to have support for > # text layout feature. I guess I feel sympathy with you. But we > # are minority among FT2 developers :-). Perhaps a misunderstanding: I don't object to make FreeType handle OpenType tables (see the validating stuff which we have developed together). However, I object to let FreeType interpret them -- this is really the job of a higher-level library. Werner ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
Re: [ft-devel] A question about freetype and harfbuzz
On Mon, 2007-05-28 at 02:34 -0400, [EMAIL PROTECTED] wrote: > On Mon, 28 May 2007 13:48:20 +0800 > LingNing Zhang <[EMAIL PROTECTED]> wrote: > > >Hi all, > >I have a question about freetype and harfbuzz. > >FreeType1 includes an extension to support OpenType text layout > >processing. But this support hasn't become part of FreeType2. Why? > >Why does FreeType2 not use the codes of harfbuzz to support OpenType > >text layout processing? > >Thanks! > > # Personally, I'm one of the people who want FT2 to have > # support for text layout feature. I guess I feel sympathy > # with you. But we are minority among FT2 developers :-). > > Excuse me, HarfBuzz developers want FT2 to include > built-in OT table parsers? Or, there is non-ICU/ > non-HarfBuzz/non-M17NLib/non-Pango/non-Qt library > their developers want FT2 to have OT table parser > and don't want to copy such from existing libraries? No, no HarfBuzz developer wants that. The future is clear: HarfBuzz is the Linux OpenType Layout engine that Pango, Qt, and ICU will use. I'm currently in the process of (slowly) rewriting it to use mmap()ed font files instead of reading tables into memory. When I'm done with that, I'll revise the API and freeze it. There is further work to make HarfBuzz *the* shaper API for Linux systems. That is, again, Pango, Qt, ICU, Scribus, OO.o, etc all will be using HarfBuzz. This has been discussed extensively at the Text Layout Summit in Boston, and discussion is going on on the harfbuzz list and will be at the next Text Layout Summit at aKademy in a few weeks. This all probably takes a year or two to stabilize and released as a shared library, but what is two years anyway... On Mon, 2007-05-28 at 02:42 -0400, Werner LEMBERG wrote: > Perhaps a misunderstanding: I don't object to make FreeType handle > OpenType tables (see the validating stuff which we have developed > together). However, I object to let FreeType interpret them -- this > is really the job of a higher-level library. For the record, I think the ftvalid stuff is misplaced and doesn't belong in FreeType. Some of my concerns are similar to what David already raised. In short, I believe that any OpenType Layout engine should do its own validation, and has enough code to be able to do a validator very easily, and if mere font validation is desired, things like ttx already do it. I don't see where ftvalid is expected to be used. (that said, given that it's already in freetype, I don't mind it staying there.) Regards, -- behdad http://behdad.org/ "Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin, 1759 ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
Re: [ft-devel] A question about freetype and harfbuzz
> [...] I believe that any OpenType Layout engine should do its own > validation, Why? What is bad about letting FreeType doing that? Afterwards you can omit any error handling... Werner ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
Re: [ft-devel] A question about freetype and harfbuzz
On Mon, 2007-05-28 at 23:48 +0200, Werner LEMBERG wrote: > > [...] I believe that any OpenType Layout engine should do its own > > validation, > > Why? What is bad about letting FreeType doing that? Afterwards you > can omit any error handling... For one thing, FreeType is not necessarily available. We are removing FreeType dependency from HarfBuzz, and Qt wants to use HarfBuzz on Windows too. The other reason is that, for example, HarfBuzz is forgiving about some problems. A nonexistent lookup index for example is automatically ignored. Or a subtable that is not ever referenced can have an (invalid) offset of zero and it will still work. These kind of exceptions were added to make fonts that already worked with Windows work with Pango too. My plan for the rewrite I'm doing is, whenever I see a problem in a table, instead of rejecting the font, or having to check for validity all the time, just modify the data in place to make it valid. That will partically defeat the idea of using mmap()ed fonts, but it's not a big deal. > Werner -- behdad http://behdad.org/ "Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin, 1759 ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
Re: [ft-devel] A question about freetype and harfbuzz
Dear Sirs, Before all, Werner asked the question that I wanted to ask, thank you very much. On Mon, 28 May 2007 08:42:10 +0200 (CEST) Werner LEMBERG <[EMAIL PROTECTED]> wrote: >> # Personally, I'm one of the people who want FT2 to have support for >> # text layout feature. I guess I feel sympathy with you. But we >> # are minority among FT2 developers :-). > >Perhaps a misunderstanding: I don't object to make FreeType handle >OpenType tables (see the validating stuff which we have developed >together). I'm glad to hear that. In addition, I'm not sure if validating OT-tables only is sufficient. For example, otvalid checks whether the substituted glyph ID by "GSUB" is within the range declared by "maxp". I suppose HarfBuzz lets the handling of too-large glyph ID to the clients. I think it's reasonable because HarfBuzz parses and interprets only OT-tables. But from the client viewpoint, it's slightly inconvenient to check the glyphID by itself, after choosing a font to use. On Mon, 28 May 2007 17:37:03 -0400 Behdad Esfahbod <[EMAIL PROTECTED]> wrote: >There is further work to make HarfBuzz *the* shaper API for Linux >systems. That is, again, Pango, Qt, ICU, Scribus, OO.o, etc all will be >using HarfBuzz. This has been discussed extensively at the Text Layout >Summit in Boston, and discussion is going on on the harfbuzz list and >will be at the next Text Layout Summit at aKademy in a few weeks. Pango had already included HarfBuzz (as OpenType parser). I suppose Qt is also moving to adopt HarfBuzz. Although I could find the developers of other softwares (ICU, Scribus, OO.o) in Text Layout Summit 2006, I don't know the status and directions of them. Please let me know more about the migration movement. At present, Text Layout Summit 2007 page: http://www.freedesktop.org/wiki/TextLayout2007?highlight=%28%28HarfBuzz%29%29 has only 1 agenda of HarfBuzz. On Mon, 28 May 2007 18:05:19 -0400 Behdad Esfahbod <[EMAIL PROTECTED]> wrote: >On Mon, 2007-05-28 at 23:48 +0200, Werner LEMBERG wrote: >> > [...] I believe that any OpenType Layout engine should do its own >> > validation, >> >> Why? What is bad about letting FreeType doing that? Afterwards you >> can omit any error handling... > >For one thing, FreeType is not necessarily available. We are removing >FreeType dependency from HarfBuzz, and Qt wants to use HarfBuzz on >Windows too. I think it's slightly different issue, because FT2 does work on Windows. I suppose Qt wants to use Windows native rasterizer instead of FT2, but Qt wants to use HarfBuzz shaping engine? Using OTLS is difficult? >The other reason is that, for example, HarfBuzz is forgiving about some >problems. A nonexistent lookup index for example is automatically >ignored. Or a subtable that is not ever referenced can have an >(invalid) offset of zero and it will still work. These kind of >exceptions were added to make fonts that already worked with Windows >work with Pango too. Hmm, the severity of ftvalid is controlled by validation level, although the fine tuning of default level for otvalid is not finished. Considering the fact most Open Type layout engine follows the behaviour of OTLS (to share existing OpenType font resources), I think it's not disadvantage of layout-engine-independent validation approach, like ftvalid. Rather, I think the OT-table validation in HarfBuzz will be one side of its runtime failsafe mechanism. I remember what I wrote 2 years ago: http://lists.gnu.org/archive/html/freetype-devel/2005-11/msg00084.html >Another reason is that: if the text layout engine includes >its own validator, it will be runtime checking to avoid >from wrong behaviour and crashing, and won't validate >unaccessed parts. It's not easy to use text layout functions >to crawl all features declared in the OpenType/TrueTypeGX. Regards, mpsuzuki ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
Re: [ft-devel] A question about freetype and harfbuzz
On Tue, 2007-05-29 at 11:52 +0900, [EMAIL PROTECTED] wrote: > I'm glad to hear that. In addition, I'm not sure if > validating OT-tables only is sufficient. For example, > otvalid checks whether the substituted glyph ID by > "GSUB" is within the range declared by "maxp". > I suppose HarfBuzz lets the handling of too-large > glyph ID to the clients. I think it's reasonable > because HarfBuzz parses and interprets only OT-tables. > But from the client viewpoint, it's slightly > inconvenient to check the glyphID by itself, > after choosing a font to use. Most users don't have to deal with out-of-range glyphs at all. What happens is that they will see a box (glyph 0) instead. Font developers, ditto, they will see it and fix the font. Lets first identify who the target of the validator is. For validation-before-layout case, I really don't think using a validation code path separate from the actual layout path makes sense. > On Mon, 28 May 2007 17:37:03 -0400 > Behdad Esfahbod <[EMAIL PROTECTED]> wrote: > >There is further work to make HarfBuzz *the* shaper API for Linux > >systems. That is, again, Pango, Qt, ICU, Scribus, OO.o, etc all will be > >using HarfBuzz. This has been discussed extensively at the Text Layout > >Summit in Boston, and discussion is going on on the harfbuzz list and > >will be at the next Text Layout Summit at aKademy in a few weeks. > > Pango had already included HarfBuzz (as OpenType parser). > I suppose Qt is also moving to adopt HarfBuzz. Although > I could find the developers of other softwares (ICU, > Scribus, OO.o) in Text Layout Summit 2006, I don't know > the status and directions of them. Please let me know > more about the migration movement. At present, Text Layout > Summit 2007 page: > > http://www.freedesktop.org/wiki/TextLayout2007?highlight=%28%28HarfBuzz%29%29 > > has only 1 agenda of HarfBuzz. You can follow HarfBuzz activity on the harfbuzz mailing list. > On Mon, 28 May 2007 18:05:19 -0400 > Behdad Esfahbod <[EMAIL PROTECTED]> wrote: > >On Mon, 2007-05-28 at 23:48 +0200, Werner LEMBERG wrote: > >> > [...] I believe that any OpenType Layout engine should do its own > >> > validation, > >> > >> Why? What is bad about letting FreeType doing that? Afterwards you > >> can omit any error handling... > > > >For one thing, FreeType is not necessarily available. We are removing > >FreeType dependency from HarfBuzz, and Qt wants to use HarfBuzz on > >Windows too. > > I think it's slightly different issue, because FT2 > does work on Windows. I suppose Qt wants to use > Windows native rasterizer instead of FT2, but Qt > wants to use HarfBuzz shaping engine? Using OTLS > is difficult? No idea. OTLS's API is not publicly documented. > >The other reason is that, for example, HarfBuzz is forgiving about some > >problems. A nonexistent lookup index for example is automatically > >ignored. Or a subtable that is not ever referenced can have an > >(invalid) offset of zero and it will still work. These kind of > >exceptions were added to make fonts that already worked with Windows > >work with Pango too. > > Hmm, the severity of ftvalid is controlled by validation > level, although the fine tuning of default level for > otvalid is not finished. Considering the fact most Open > Type layout engine follows the behaviour of OTLS (to > share existing OpenType font resources), I think it's > not disadvantage of layout-engine-independent validation > approach, like ftvalid. Rather, I think the OT-table > validation in HarfBuzz will be one side of its runtime > failsafe mechanism. I remember what I wrote 2 years ago: Apparently our opinions are different, but I don't see any problem with that. I'm putting my touch-typing fingers where my mouth is and developing an OpenType Layout engine that I think is right. I'll come back with a comparison when I'm done. Before then, there's not much to talk about. I have the fundamental problem of doesn't-belong-in-a-rasterizer though. If one finds a font that crashes or doesn't work with the HarfBuzz Layout engine, they should be able to fix it in HarfBuzz. With the validator code in freetype, they may need to fix FreeType. To me, it's like, donno, putting an h&j engine in FreeType. It sure is text layout, and works nearly with fonts, but it's not rasterization. > http://lists.gnu.org/archive/html/freetype-devel/2005-11/msg00084.html > >Another reason is that: if the text layout engine includes > >its own validator, it will be runtime checking to avoid > >from wrong behaviour and crashing, and won't validate > >unaccessed parts. It's not easy to use text layout functions > >to crawl all features declared in the OpenType/TrueTypeGX. > > Regards, > mpsuzuki -- behdad http://behdad.org/ "Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin, 1759 ___ Freetype-devel mailing list Fr
