Re: [Bf-committers] Arabic support in text editing
I'm not sure if freetype2 alone can render Arabic/Bidi text perfectly. But I'm sure that fribdi is simply inadequate. As I said earlier, Arabic text is not just about converting letters to their presentation form. That approach will render a lot of fonts unusable. Let alone all the languages that use Arabic script and don't work with that simple approach. That problem can be avoided using a more 'proper' method to handle layouts and shaping. And let's not forget about all the other complex cases that can be solved with an OpenType-based solution (Indic, Thai, diacritics, vertical text .. etc). AFAIK, freetype2 alone doesn't deal with all of that. I might be wrong about this but this is how I've remembered it for quite a while. You'll need a developer's opinion to confirm that, though. Pango is great. But as Sergey pointed out, it requires GLib, which is quite a dependency. Also, I think its harfbuzz version is slightly out of date. Not that important but it's something to consider. But if you think using it instead of hooking up fribidi and harfbuzz directly is a better trade-off, then by all means :) On Thu, Oct 27, 2011 at 2:49 PM, Dalai Felinto wrote: > Shaping itself is already "kind-of" handled by freetype2. It may look > wrong in some corner cases (e.g. those words that take off the > ground). But overall the result of getting the text in their > presentation form was working ok (not that I can tell good arabic from > bad arabic, but Yousef himself seemed pleased with the result). > > Agree that fribidi may be buggy though. I'm my tests, at least the > standalone (known to be buggy) produced a different enough result from > our current _to_utf.py scripts. > > I know nothing about ligatures and how opentype (i.e. harfbuzz) does > it better than freetype2. So I take your word for that. > But what I heard (chatting on #harfbuzz) is that harfbuzz does not > handle bi-di. I believe this is the reason why fribidi still plays an > important role (or another bidi algorithm) and pango was suggested > (given that it already integrates fribidi and harfbuzz). > > -- > Dalai > > 2011/10/26 Majid AL-Dharrab : > > Actually, fribidi's shaper is very primitive and rudimentary and it has > lots > > of fundamental problems. It simply replaces letters with their Unicode > > presentation forms depending on their position in a word. That is not a > good > > approach because it completely ignores ligatures and other OpenType > > features, leading to broken support for most modern fonts that use > OpenType > > tables. That's why libass only uses a fribidi function that positions > text > > characters in the correct order, following rules from Unicode's bidi > > algorithm (http://unicode.org/reports/tr9/#The_Paragraph_Level). Shaping > is > > then handled by HarfBuzz, which a great OpenType shaper. > > > > Unfortunately, I'm not a developer, so I'm afraid I can't help any > further > > with this issue. But you can contact libass's developer if you face any > > problems with either lib. His GSoC project was supporting complex and > bidi > > scripts, so he probably knows a couple of tricks that might help. Good > luck. > > :) > > > > On Wed, Oct 26, 2011 at 9:57 AM, Dalai Felinto > wrote: > > > >> Thanks for the link. > >> > >> I looked at fribidi and found it interesting. Their standalone is > >> doing the same as we are with the python scripts (they call it log2vis > >> - to convert from logic to visual strings). > >> > >> Now, if we use this lib, do we still need harfbuzz? It seemed to me > >> that truetype2 can handle most of the layout, no? > >> > >> -- > >> Dalai > >> > >> 2011/10/25 Majid AL-Dharrab : > >> > I guess you can't go wrong with harfbuzz-ng on freetype2. After all, > that > >> is > >> > what's used to render text in Firefox. libass, the SSA subtitle > renderer > >> > used in VLC and MPlayer, has recently started to support complex and > bidi > >> > scripts using harfbuzz and fribidi on top of freetype2. That resulted > in > >> > very sophisticated rendering of complex texts. Implementing that > turned > >> out > >> > to be easier than the developer had thought. > >> > > >> > > >> > http://ssadev.blogspot.com/2011/05/getting-complex-text-layout-into-libass.html > >> > > >> > I hope his experience helps make the issue a bit easier for our > Blender > >> > devs. Good luck! > >> > ___ > >> > Bf-committers mailing list > >> > Bf-committers@blender.org > >> > http://lists.blender.org/mailman/listinfo/bf-committers > >> > > >> ___ > >> Bf-committers mailing list > >> Bf-committers@blender.org > >> http://lists.blender.org/mailman/listinfo/bf-committers > >> > > ___ > > Bf-committers mailing list > > Bf-committers@blender.org > > http://lists.blender.org/mailman/listinfo/bf-committers > > > ___ > Bf-committers mailing list > Bf-committers@
Re: [Bf-committers] Arabic support in text editing
Shaping itself is already "kind-of" handled by freetype2. It may look wrong in some corner cases (e.g. those words that take off the ground). But overall the result of getting the text in their presentation form was working ok (not that I can tell good arabic from bad arabic, but Yousef himself seemed pleased with the result). Agree that fribidi may be buggy though. I'm my tests, at least the standalone (known to be buggy) produced a different enough result from our current _to_utf.py scripts. I know nothing about ligatures and how opentype (i.e. harfbuzz) does it better than freetype2. So I take your word for that. But what I heard (chatting on #harfbuzz) is that harfbuzz does not handle bi-di. I believe this is the reason why fribidi still plays an important role (or another bidi algorithm) and pango was suggested (given that it already integrates fribidi and harfbuzz). -- Dalai 2011/10/26 Majid AL-Dharrab : > Actually, fribidi's shaper is very primitive and rudimentary and it has lots > of fundamental problems. It simply replaces letters with their Unicode > presentation forms depending on their position in a word. That is not a good > approach because it completely ignores ligatures and other OpenType > features, leading to broken support for most modern fonts that use OpenType > tables. That's why libass only uses a fribidi function that positions text > characters in the correct order, following rules from Unicode's bidi > algorithm (http://unicode.org/reports/tr9/#The_Paragraph_Level). Shaping is > then handled by HarfBuzz, which a great OpenType shaper. > > Unfortunately, I'm not a developer, so I'm afraid I can't help any further > with this issue. But you can contact libass's developer if you face any > problems with either lib. His GSoC project was supporting complex and bidi > scripts, so he probably knows a couple of tricks that might help. Good luck. > :) > > On Wed, Oct 26, 2011 at 9:57 AM, Dalai Felinto wrote: > >> Thanks for the link. >> >> I looked at fribidi and found it interesting. Their standalone is >> doing the same as we are with the python scripts (they call it log2vis >> - to convert from logic to visual strings). >> >> Now, if we use this lib, do we still need harfbuzz? It seemed to me >> that truetype2 can handle most of the layout, no? >> >> -- >> Dalai >> >> 2011/10/25 Majid AL-Dharrab : >> > I guess you can't go wrong with harfbuzz-ng on freetype2. After all, that >> is >> > what's used to render text in Firefox. libass, the SSA subtitle renderer >> > used in VLC and MPlayer, has recently started to support complex and bidi >> > scripts using harfbuzz and fribidi on top of freetype2. That resulted in >> > very sophisticated rendering of complex texts. Implementing that turned >> out >> > to be easier than the developer had thought. >> > >> > >> http://ssadev.blogspot.com/2011/05/getting-complex-text-layout-into-libass.html >> > >> > I hope his experience helps make the issue a bit easier for our Blender >> > devs. Good luck! >> > ___ >> > Bf-committers mailing list >> > Bf-committers@blender.org >> > http://lists.blender.org/mailman/listinfo/bf-committers >> > >> ___ >> Bf-committers mailing list >> Bf-committers@blender.org >> http://lists.blender.org/mailman/listinfo/bf-committers >> > ___ > Bf-committers mailing list > Bf-committers@blender.org > http://lists.blender.org/mailman/listinfo/bf-committers > ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Arabic support in text editing
> Harfbuzz can do all of this on its own (with freetype2). then this sums it up i think, implementing harfbuzz over the existence of freetype. i'll try to tackle the idea in the next days. ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Arabic support in text editing
@Sergey Kurdakov AFAICR, harfbuzz doesn't have any dependencies except for freetype2, of course. Fribidi doesn't require any dependencies either. Besides, pango's shaper is a little bit inferior to harfbuzz. @Yousef Hurfoush Harfbuzz can do all of this on its own (with freetype2). I think Greg just found it easier to use fribidi's function instead. Fribidi's part in this is the easiest anyway, so recoding it for blender won't be too much of a hassle. Although, I'm curious to see what other possible solutions you can come up with. Regards, Majid On Wed, Oct 26, 2011 at 6:11 PM, Sergey Kurdakov wrote: > Hi, > > I noticed pango being mentioned > > here is one possible implementation for graphics > http://code.google.com/p/osgpango/ > ( just for reference ) > > it requires though quite a lot dlls (cairo, pango, many gtk+ dlls ) to be > included, > so the size of Blender will keep growing > > Regards > Sergey > ___ > Bf-committers mailing list > Bf-committers@blender.org > http://lists.blender.org/mailman/listinfo/bf-committers > ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Arabic support in text editing
hi @dalai: > * Note 2: a python addon exclusively for the text3d could work for you as > well. Have you considered it? yes i do, and i'm trying to build one. > Let me know what you think. If someone is interested on tackling that we can > put branch up and running. i'm happey to help with that, but currently i'm totally occupied with my work, and i'm a .NET dev. i know basic c & c++ maybe more, but it's somewhat hard to do the implementation for know, i'll try to dig in if i had more time (if Allah wishes). @Majid AL-Dharrab: thanks for the info. why implement functions from different libs, i think there are other libs that could do it all i'm still searching for a better text stack libs. i'll report as i'm progressing. Yousef Harfoush ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Arabic support in text editing
Actually, fribidi's shaper is very primitive and rudimentary and it has lots of fundamental problems. It simply replaces letters with their Unicode presentation forms depending on their position in a word. That is not a good approach because it completely ignores ligatures and other OpenType features, leading to broken support for most modern fonts that use OpenType tables. That's why libass only uses a fribidi function that positions text characters in the correct order, following rules from Unicode's bidi algorithm (http://unicode.org/reports/tr9/#The_Paragraph_Level). Shaping is then handled by HarfBuzz, which a great OpenType shaper. Unfortunately, I'm not a developer, so I'm afraid I can't help any further with this issue. But you can contact libass's developer if you face any problems with either lib. His GSoC project was supporting complex and bidi scripts, so he probably knows a couple of tricks that might help. Good luck. :) On Wed, Oct 26, 2011 at 9:57 AM, Dalai Felinto wrote: > Thanks for the link. > > I looked at fribidi and found it interesting. Their standalone is > doing the same as we are with the python scripts (they call it log2vis > - to convert from logic to visual strings). > > Now, if we use this lib, do we still need harfbuzz? It seemed to me > that truetype2 can handle most of the layout, no? > > -- > Dalai > > 2011/10/25 Majid AL-Dharrab : > > I guess you can't go wrong with harfbuzz-ng on freetype2. After all, that > is > > what's used to render text in Firefox. libass, the SSA subtitle renderer > > used in VLC and MPlayer, has recently started to support complex and bidi > > scripts using harfbuzz and fribidi on top of freetype2. That resulted in > > very sophisticated rendering of complex texts. Implementing that turned > out > > to be easier than the developer had thought. > > > > > http://ssadev.blogspot.com/2011/05/getting-complex-text-layout-into-libass.html > > > > I hope his experience helps make the issue a bit easier for our Blender > > devs. Good luck! > > ___ > > Bf-committers mailing list > > Bf-committers@blender.org > > http://lists.blender.org/mailman/listinfo/bf-committers > > > ___ > Bf-committers mailing list > Bf-committers@blender.org > http://lists.blender.org/mailman/listinfo/bf-committers > ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Arabic support in text editing
Thanks for the link. I looked at fribidi and found it interesting. Their standalone is doing the same as we are with the python scripts (they call it log2vis - to convert from logic to visual strings). Now, if we use this lib, do we still need harfbuzz? It seemed to me that truetype2 can handle most of the layout, no? -- Dalai 2011/10/25 Majid AL-Dharrab : > I guess you can't go wrong with harfbuzz-ng on freetype2. After all, that is > what's used to render text in Firefox. libass, the SSA subtitle renderer > used in VLC and MPlayer, has recently started to support complex and bidi > scripts using harfbuzz and fribidi on top of freetype2. That resulted in > very sophisticated rendering of complex texts. Implementing that turned out > to be easier than the developer had thought. > > http://ssadev.blogspot.com/2011/05/getting-complex-text-layout-into-libass.html > > I hope his experience helps make the issue a bit easier for our Blender > devs. Good luck! > ___ > Bf-committers mailing list > Bf-committers@blender.org > http://lists.blender.org/mailman/listinfo/bf-committers > ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Arabic support in text editing
Hi Yousef, I talked about that with Campbell, and we got to some roadmap that should be a good start for someone willing to tackle the problem: (1) handle the drawing: 1.1.Implement bi-di (bi-directional support) in blf (Blender internal truetype2 wrapper+utility functions). 1.2.Implement complex scripting (the arabic replacement to the presentation form of the chars). This could be done by static linking an external library (e.g. ~pango). Or refactoring the blf to support this internally (unlikely to happen). If opting for an external lib complex-scripting should come "for free". (2) handle the editing: 2.1. that includes cursor positioning, backspace, copy/paste It may be a good idea to use part of 1.1. implementation for that. This should be simpler than (1). But without the drawing correct, it doesn't make sense to handle editing. Thus it comes after. I added those info here - http://wiki.blender.org/index.php/User:Dfelinto#UTF8_and_complex-scripting * Note to have text3d working for a 3d package you can write the text in the Text Editor, use the script in python to convert the text, and use the "Edit->Text to 3D Object". <> <> * Note 2: a python addon exclusively for the text3d could work for you as well. Have you considered it? Let me know what you think. If someone is interested on tackling that we can put branch up and running. Cheers, Dalai 2011/10/24 Yousef Hurfoush > > Hi dalai > > i know the fork i proposed is NOT the way it should be, i think going > the further mile is the best approach, but that WILL take a big amount of > time and commitment to revamp the text drawing aligorithm in blender. > > So, i think doing a simple procedure of implementing the arabic_to_utf > operator > is best for more internationalization as currently there are no 3D package > provide > RTL languages in the text object. > > Another approach i found is using special font's with replaced char map > with the language > you want to implement, which what i use now, but we still need to reverse > the order. > > here is a sample of the word "yousef" -> 'يوسف' : > http://www.pasteall.org/pic/19489 > > > > I would take a look at some library that we can used with freetype2 (our > > current text drawing low-level library). To re-create the wheels > ourselves > > seems a bit overkill to me. > > > > This text has a good overview, Panda caught my eyes there: > > http://behdad.org/text/ > > > > There is more to complex scripting than replacing the arabic letters by > > their presentation form. This is needed, but should happen under the hood > at > > drawing time only imho. > > > > That said, given that ui drawing is likely the most daily demanding usage > of > > text, I think "pre-baked" .po is still a great idea (I'm trying to > improve > > the py script btw) > > > ___ > Bf-committers mailing list > Bf-committers@blender.org > http://lists.blender.org/mailman/listinfo/bf-committers > ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Arabic support in text editing
Hi dalai i know the fork i proposed is NOT the way it should be, i think going the further mile is the best approach, but that WILL take a big amount of time and commitment to revamp the text drawing aligorithm in blender. So, i think doing a simple procedure of implementing the arabic_to_utf operator is best for more internationalization as currently there are no 3D package provide RTL languages in the text object. Another approach i found is using special font's with replaced char map with the language you want to implement, which what i use now, but we still need to reverse the order. here is a sample of the word "yousef" -> 'يوسف' : http://www.pasteall.org/pic/19489 > I would take a look at some library that we can used with freetype2 (our > current text drawing low-level library). To re-create the wheels ourselves > seems a bit overkill to me. > > This text has a good overview, Panda caught my eyes there: > http://behdad.org/text/ > > There is more to complex scripting than replacing the arabic letters by > their presentation form. This is needed, but should happen under the hood at > drawing time only imho. > > That said, given that ui drawing is likely the most daily demanding usage of > text, I think "pre-baked" .po is still a great idea (I'm trying to improve > the py script btw) ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] Arabic support in text editing
Hi Yousef, I would take a look at some library that we can used with freetype2 (our current text drawing low-level library). To re-create the wheels ourselves seems a bit overkill to me. This text has a good overview, Panda caught my eyes there: http://behdad.org/text/ There is more to complex scripting than replacing the arabic letters by their presentation form. This is needed, but should happen under the hood at drawing time only imho. That said, given that ui drawing is likely the most daily demanding usage of text, I think "pre-baked" .po is still a great idea (I'm trying to improve the py script btw) -- Dalai 2011/10/23 Yousef Hurfoush > > hello > > I was wondering after enabling UTF-8, how much work it needs to implement a > filtering function that replaces normal Arabic text with it's universal UTF > encoding, > as it is currently implemented in python forking a c++ version is trivial. > > ___ > Bf-committers mailing list > Bf-committers@blender.org > http://lists.blender.org/mailman/listinfo/bf-committers > ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
