[ft-devel] my Symbian OS glyph caching system
Some caveats about my glyph caching system: here is most of the text of my reply to Srinavasa: You are welcome to look at my cache implementation: it should be somewhere in this folder: http://developer.symbian.org/oss/FCL/sf/os/textandloc/file/030b3432fbe0/fontservices/fontstore However: 1. I can't provide support. I wrote this over 10 years ago. Since then many other people have worked on it and I no longer know exactly how it works. 2. As originally designed, the glyph cache was a binary tree that grew to a certain maximum size, after which no more entries could be added. For reasons of performance in a multi-processing system, glyphs could only be added, not deleted, addition of a glyph to a tree being an atomic action requiring no mutex. This design restriction may not exist any more; I don't know and don't have the time to find out. 3. Therefore you may be better advised, in a single-processing system, to create your own LRU glyph caching system. In principle it's simple (and I have used a variant of this design for my proprietary CartoType map rendering library): create a suitable database structure that stores key-value pairs where the key is the unique style of the glyph (font family, size, transform, emboldening, etc.) and the value is the glyph. Keep track of the least recently used glyph using a linked list; or simply rely on hashing, and remove glyphs where a conflict occurs. Best regards, Graham Asher Werner LEMBERG wrote: We are trying to achieve performance improvement either through parallelization/optimization in the freetype-2.3.11 library. Any help in this regard is highly appreciated. The solution is caching everything! Due to the nature of fonts I don't see much gain in parallelization while processing a single font. Graham Asher mentioned in a previous post that his FreeType caching stuff from the symbian OS is now open-source, so you might use this as a starting point. http://lists.gnu.org/archive/html/freetype-devel/2010-02/msg00033.html If you can optimize FreeType further (for example, using a profiler to isolate hot spots, then rewriting the code to make it faster) this is very welcomed. Werner ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
[ft-devel] RE: my Symbian OS glyph caching system
Thank for the information sir , it was very informative. _ From: Graham Asher [mailto:graham.as...@btinternet.com] Sent: Thursday, March 11, 2010 4:23 PM To: Werner LEMBERG Cc: srinivas.ka...@lge.com; freetype-devel@nongnu.org Subject: my Symbian OS glyph caching system Some caveats about my glyph caching system: here is most of the text of my reply to Srinavasa: You are welcome to look at my cache implementation: it should be somewhere in this folder: http://developer.symbian.org/oss/FCL/sf/os/textandloc/file/030b3432fbe0/font services/fontstore However: 1. I can't provide support. I wrote this over 10 years ago. Since then many other people have worked on it and I no longer know exactly how it works. 2. As originally designed, the glyph cache was a binary tree that grew to a certain maximum size, after which no more entries could be added. For reasons of performance in a multi-processing system, glyphs could only be added, not deleted, addition of a glyph to a tree being an atomic action requiring no mutex. This design restriction may not exist any more; I don't know and don't have the time to find out. 3. Therefore you may be better advised, in a single-processing system, to create your own LRU glyph caching system. In principle it's simple (and I have used a variant of this design for my proprietary CartoType map rendering library): create a suitable database structure that stores key-value pairs where the key is the unique style of the glyph (font family, size, transform, emboldening, etc.) and the value is the glyph. Keep track of the least recently used glyph using a linked list; or simply rely on hashing, and remove glyphs where a conflict occurs. Best regards, Graham Asher Werner LEMBERG wrote: We are trying to achieve performance improvement either through parallelization/optimization in the freetype-2.3.11 library. Any help in this regard is highly appreciated. The solution is caching everything! Due to the nature of fonts I don't see much gain in parallelization while processing a single font. Graham Asher mentioned in a previous post that his FreeType caching stuff from the symbian OS is now open-source, so you might use this as a starting point. http://lists.gnu.org/archive/html/freetype-devel/2010-02/msg00033.html If you can optimize FreeType further (for example, using a profiler to isolate hot spots, then rewriting the code to make it faster) this is very welcomed. Werner ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
Re: [ft-devel] Known problem with FT_StreamRec::base being non-NULL also for file-based streams?
On Wed, 10 Mar 2010 03:50:17 -0500 Behdad Esfahbod beh...@behdad.org wrote: Well, in short, all the hb_blob_t in HarfBuzz is about communicating to harfbuzz what it can do with the memory backing the font file. There are three different cases we are interested in: - The memory is read-only; harfbuzz will make a copy if it needs to modify it. - The memory is writable and it is ok to write to it. harfbuzz will not make a copy. - The memory is read-only, but can be made writable using mprotect() or similar (win32, ...) functionality. Currently the hb-ft glue layer assumes that font data is mmap()ed or are otherwise mprotect()able. Thanks. I understand the info needed by HB is whether HB can modify the memory image without duplication, or HB should copy the memory image before duplication. Just I've posted a proof of concept to know who allocated the buffer in FT_Stream, but it is not best solution for this task. I and Werner agreed that the easiest way to guarantee the origin of buffer as mmap()ed or malloc()ed is the font image preparation in HB/Pango side. But, taking a glance on Pango, I guess, there might be some delay between the invocation of FT_New_Face() and HB blob creation. The duplication of unwritable font image to writable buffer occurs for all faces? Or, the duplication occurs when the first modification is tried (to fix OpenType bug in runtime)? If latter scenario is correct - when Pango is going to create FT_Face object, Pango cannot know if the duplication will occur in future, so, my proposal (HB/Pango side font image preparation) will cause unwanted memory consumation by loading all faces to writable memory. It won't be good idea. # In Pango library, when PangoFT2Font-face is created once, # it should not be changed anymore? If replacing the face is # permitted, I want to create the earliest unwritable face # from mmap()ed image, then replace it by writable face with # malloc()-and-read() image when Pango/HB tries to modify it. # Pango has an API to expose PangoFT2Font-face to the client # (pango_ft2_font_get_face()), but it is classified as deprecated # interface. I wish Pango library is changing to hide raw # FT_Face object from Pango client. This fails for examples when: - Font data is in ROM. In this case mprotect() will fail and harfbuzz will make a copy of the memory. Not a huge problem. Indeed. If FT2 could mmap() readonly font file successfully, mprotect() will fail. - FreeType malloc()ed the font data. In this case, mprotect() is not necessary and will probably affect memory beyond the font data (since mprotect works on whole pages). Umm, I think, mprotect() for malloc()ed memory causes undefined result. http://www.opengroup.org/onlinepubs/95399/functions/mprotect.html The behavior of this function is unspecified if the mapping was not established by a call to mmap(). To avoid such ambiguity, we should know if the buffer is mmap()ed or malloc()ed, before mprotect() - am I misunderstanding? - Font data is coming from the user. In this case it may not be desirable to modify the data. Indeed. Could you tell me which function is used to push user-provided font data? Is it in cairo layer? Adding API to FTStream to be able to detect the above cases, specially the user-provided data, would be useful. Again thank you for comment about the additon of new API. As I've sketched, it is possible to get detailed info of FT_Stream object. My current sketch is huge, and I have a few issues to be discussed for further improvement of FT_Stream object. Another idea is an addition of the arguments for FT_Open_Face(), to specify 3 scenarios for font loading. 1) only mmap() is tried. 2) only malloc() + read() is tried. 3) mmap() is tried, then, malloc() + read() is tried (current behaviour) By using 1) and 2), HB/Pango can distinguish the buffer is mmap()ed or malloc()ed exactly. I will post a patch for FT2 and Pango for further discussion. Please wait a few days... Regards, mpsuzuki ___ Freetype-devel mailing list Freetype-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/freetype-devel
