Re: [HarfBuzz] Linking to libstdc++ in HarfBuzz
On Thu, 2 Aug 2018 02:38:00 -0700 Behdad Esfahbod wrote: > Just asking, does anybody still object about using C++ standard > library (ie. linking to it) in HarfBuzz? My personal experience makes me tremble at the thought. When I 'upgraded' from Ubuntu 12.04 to 14.04, I was without a libstdc++ library, and it took me a long time to clean up the resulting mess. The cause of the problem was that the version of C++ compiler I had been using was too far in advance of the distributions, and as I upgraded it was removed and I was left with nothing in its place. The installation scripts written in Python largely or entirely failed. The prospect of being without a text layout engine is terrifying. C++ was so far out of sync with the general distribution because I used to compile LibreOffice, and a key header file there wouldn't compile with the older versions of gcc. Richard. ___ HarfBuzz mailing list HarfBuzz@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/harfbuzz
[HarfBuzz] Linking to libstdc++ in HarfBuzz
Hi, Just asking, does anybody still object about using C++ standard library (ie. linking to it) in HarfBuzz? I know I've been one of the bigger opponents myself. But I can change too. :) -- behdad http://behdad.org/ ___ HarfBuzz mailing list HarfBuzz@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/harfbuzz
[HarfBuzz] harfbuzz: Branch 'master' - 6 commits
src/hb-face.cc |1 -
src/hb-graphite2.cc |2 +-
src/hb-machinery-private.hh |8 +++-
src/hb-ot-layout-private.hh | 15 +++
src/hb-uniscribe.cc |8
5 files changed, 19 insertions(+), 15 deletions(-)
New commits:
commit d4d1bf8177b127caa57b146c932f553dca1ad933
Author: Behdad Esfahbod
Date: Thu Aug 2 02:04:02 2018 -0700
Fix for recent rename
diff --git a/src/hb-graphite2.cc b/src/hb-graphite2.cc
index da7944b8..2ba905d6 100644
--- a/src/hb-graphite2.cc
+++ b/src/hb-graphite2.cc
@@ -106,7 +106,7 @@ _hb_graphite2_shaper_face_data_create (hb_face_t *face)
}
hb_blob_destroy (silf_blob);
- hb_graphite2_face_data_t *data = (hb_graphite2_face_data_t *) calloc (1,
sizeof (hb_graphite2_shaper_face_data_t));
+ hb_graphite2_face_data_t *data = (hb_graphite2_face_data_t *) calloc (1,
sizeof (hb_graphite2_face_data_t));
if (unlikely (!data))
return nullptr;
diff --git a/src/hb-uniscribe.cc b/src/hb-uniscribe.cc
index 6d579534..94e6bb55 100644
--- a/src/hb-uniscribe.cc
+++ b/src/hb-uniscribe.cc
@@ -442,7 +442,7 @@ _hb_rename_font (hb_blob_t *blob, wchar_t *new_name)
hb_uniscribe_face_data_t *
_hb_uniscribe_shaper_face_data_create (hb_face_t *face)
{
- hb_uniscribe_face_data_t *data = (hb_uniscribe_face_data_t *) calloc (1,
sizeof (hb_uniscribe_shaper_face_data_t));
+ hb_uniscribe_face_data_t *data = (hb_uniscribe_face_data_t *) calloc (1,
sizeof (hb_uniscribe_face_data_t));
if (unlikely (!data))
return nullptr;
@@ -520,7 +520,7 @@ _hb_uniscribe_shaper_font_data_create (hb_font_t *font)
{
if (unlikely (!hb_uniscribe_shaper_face_data_ensure (font->face))) return
nullptr;
- hb_uniscribe_font_data_t *data = (hb_uniscribe_font_data_t *) calloc (1,
sizeof (hb_uniscribe_shaper_font_data_t));
+ hb_uniscribe_font_data_t *data = (hb_uniscribe_font_data_t *) calloc (1,
sizeof (hb_uniscribe_font_data_t));
if (unlikely (!data))
return nullptr;
commit 91126aa11a5fa2bff72137df4768ad13fc9b7803
Author: Behdad Esfahbod
Date: Thu Aug 2 02:03:13 2018 -0700
[uniscribe] Update for recent changes
diff --git a/src/hb-uniscribe.cc b/src/hb-uniscribe.cc
index 7f7f10d0..6d579534 100644
--- a/src/hb-uniscribe.cc
+++ b/src/hb-uniscribe.cc
@@ -383,7 +383,7 @@ _hb_rename_font (hb_blob_t *blob, wchar_t *new_name)
memcpy(new_sfnt_data, orig_sfnt_data, length);
- OT::name = OT::StructAtOffset (new_sfnt_data,
name_table_offset);
+ OT::name = StructAtOffset (new_sfnt_data, name_table_offset);
name.format.set (0);
name.count.set (ARRAY_LENGTH (name_IDs));
name.stringOffset.set (name.get_size ());
@@ -399,7 +399,7 @@ _hb_rename_font (hb_blob_t *blob, wchar_t *new_name)
}
/* Copy string data from new_name, converting wchar_t to UTF16BE. */
- unsigned char *p = ::StructAfter (name);
+ unsigned char *p = (name);
for (unsigned int i = 0; i < name_str_len; i++)
{
*p++ = new_name[i] >> 8;
commit 66952ec47b5f09d88b83fb6a71b1cdb26c53668d
Author: Behdad Esfahbod
Date: Thu Aug 2 01:44:20 2018 -0700
Remove unused table reference
diff --git a/src/hb-ot-layout-private.hh b/src/hb-ot-layout-private.hh
index fc2e57e7..c8855723 100644
--- a/src/hb-ot-layout-private.hh
+++ b/src/hb-ot-layout-private.hh
@@ -179,11 +179,10 @@ struct hb_ot_layout_t
/* TODO Move the following out of this struct. */
hb_face_t *face; /* MUST be before the lazy loaders. */
- hb_table_lazy_loader_t<1, struct OT::BASE> base;
- hb_table_lazy_loader_t<2, struct OT::MATH> math;
- hb_table_lazy_loader_t<3, struct OT::fvar> fvar;
- hb_table_lazy_loader_t<4, struct OT::avar> avar;
- hb_table_lazy_loader_t<5, struct AAT::morx> morx;
+ hb_table_lazy_loader_t<1, struct OT::MATH> math;
+ hb_table_lazy_loader_t<2, struct OT::fvar> fvar;
+ hb_table_lazy_loader_t<3, struct OT::avar> avar;
+ hb_table_lazy_loader_t<4, struct AAT::morx> morx;
};
commit 443de26fa56dd1ef149d3ce4206f4495eceec2eb
Author: Behdad Esfahbod
Date: Thu Aug 2 01:41:19 2018 -0700
Minor
diff --git a/src/hb-face.cc b/src/hb-face.cc
index 75dc486e..9d17c4a5 100644
--- a/src/hb-face.cc
+++ b/src/hb-face.cc
@@ -33,7 +33,6 @@
#include "hb-open-file-private.hh"
-
/**
* hb_face_count: Get number of faces on the blob
* @blob:
diff --git a/src/hb-ot-layout-private.hh b/src/hb-ot-layout-private.hh
index 3a99937c..fc2e57e7 100644
--- a/src/hb-ot-layout-private.hh
+++ b/src/hb-ot-layout-private.hh
@@ -171,6 +171,12 @@ struct hb_ot_layout_t
const struct OT::GSUB *gsub;
const struct OT::GPOS *gpos;
+ unsigned int gsub_lookup_count;
+ unsigned int gpos_lookup_count;
+
+ hb_ot_layout_lookup_accelerator_t *gsub_accels;
+ hb_ot_layout_lookup_accelerator_t *gpos_accels;
+
/* TODO Move the following out of this struct. */
hb_face_t *face; /* MUST be before the lazy loaders. */
hb_table_lazy_loader_t<1, struct OT::BASE> base;
@@ -178,12 +184,6 @@ struct hb_ot_layout_t
hb_table_lazy_loader_t<3, struct
[HarfBuzz] harfbuzz: Branch 'master' - 7 commits
src/dump-emoji.cc |4
src/hb-aat-layout.cc |2
src/hb-blob-private.hh |5 -
src/hb-coretext.cc | 46 -
src/hb-directwrite.cc | 28 ++---
src/hb-face.cc | 21 ++--
src/hb-face.h | 16 +--
src/hb-fallback-shape.cc | 24 ++---
src/hb-graphite2.cc| 26 ++---
src/hb-machinery-private.hh| 137 ++---
src/hb-ot-font.cc | 19 ++--
src/hb-ot-layout-private.hh| 11 +-
src/hb-ot-layout.cc| 17 +--
src/hb-ot-shape-complex-arabic-fallback.hh | 18 +--
src/hb-ot-shape.cc | 20 ++--
src/hb-shaper-private.hh | 10 +-
src/hb-static.cc |4
src/hb-subset-glyf.cc |2
src/hb-subset.cc |8 -
src/hb-uniscribe.cc| 36 +++
20 files changed, 231 insertions(+), 223 deletions(-)
New commits:
commit ff7826e90bce46985651015059872d1d8559b6ce
Author: Behdad Esfahbod
Date: Thu Aug 2 01:27:40 2018 -0700
Reduce storage by sharing face amongst lazy_loaders
diff --git a/src/hb-machinery-private.hh b/src/hb-machinery-private.hh
index dbabeadd..081cbbfd 100644
--- a/src/hb-machinery-private.hh
+++ b/src/hb-machinery-private.hh
@@ -590,93 +590,106 @@ struct BEInt
* Lazy struct and blob loaders.
*/
-template
struct hb_base_lazy_loader_t
{
+ static_assert (WheresFace > 0, "");
+
/* https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern */
inline const Subclass* thiz (void) const { return static_cast (this); }
+ inline Subclass* thiz (void) { return static_cast (this); }
- inline void init (hb_face_t *face_)
+ inline void init (void)
{
-face = face_;
instance = nullptr;
}
-
- inline const Returned * operator-> (void) const
+ inline void fini (void)
{
-return thiz ()->get ();
+if (instance)
+ thiz ()->destroy (instance);
}
- protected:
- hb_face_t *face;
- mutable Stored *instance;
-};
-
-template
-struct hb_lazy_loader_t : hb_base_lazy_loader_t, T>
-{
- inline void fini (void)
+ inline const Returned * operator-> (void) const
{
-if (this->instance && this->instance != (T))
-{
- this->instance->fini();
- free (this->instance);
-}
+return thiz ()->get ();
}
- inline const T* get (void) const
+ inline Stored * get_stored (void) const
{
retry:
-T *p = (T *) hb_atomic_ptr_get (>instance);
+Stored *p = (Stored *) hb_atomic_ptr_get (>instance);
if (unlikely (!p))
{
- p = (T *) calloc (1, sizeof (T));
- if (unlikely (!p))
-p = const_cast ((T));
- else
- p->init (this->face);
- if (unlikely (!hb_atomic_ptr_cmpexch (const_cast(>instance),
nullptr, p)))
+ hb_face_t *face = *(((hb_face_t **) this) - WheresFace);
+ p = thiz ()->create (face);
+ if (unlikely (!hb_atomic_ptr_cmpexch (const_cast(>instance), nullptr, p)))
{
- if (p != (T))
- p->fini ();
+thiz ()->destroy (p);
goto retry;
}
}
return p;
}
+
+ inline const Returned * get (void) const
+ {
+return thiz ()->convert (get_stored ());
+ }
+
+ static inline const Returned* convert (const Stored *p)
+ {
+return p;
+ }
+
+ private:
+ /* Must only have one pointer. */
+ mutable Stored *instance;
};
-template
-struct hb_table_lazy_loader_t :
hb_base_lazy_loader_t, T, hb_blob_t>
+template
+struct hb_lazy_loader_t : hb_base_lazy_loader_t, T>
{
- inline void fini (void)
+ static inline T *create (hb_face_t *face)
{
-hb_blob_destroy (this->instance);
+T *p = (T *) calloc (1, sizeof (T));
+if (unlikely (!p))
+ p = const_cast ((T));
+else
+ p->init (face);
+return p;
}
-
- inline hb_blob_t* get_blob (void) const
+ static inline void destroy (T *p)
{
- retry:
-hb_blob_t *b = (hb_blob_t *) hb_atomic_ptr_get (>instance);
-if (unlikely (!b))
+if (p != (T))
{
- b = hb_sanitize_context_t ().reference_table (this->face);
- if (!hb_atomic_ptr_cmpexch (>instance, nullptr, b))
- {
- hb_blob_destroy (b);
- goto retry;
- }
- this->instance = b;
+ p->fini();
+ free (p);
}
-return b;
}
+};
- inline const T* get (void) const
+template
+struct hb_table_lazy_loader_t : hb_base_lazy_loader_t, T, hb_blob_t>
+{
+ static inline hb_blob_t *create (hb_face_t *face)
+ {
+return hb_sanitize_context_t ().reference_table (face);
+ }
+ static inline void destroy (hb_blob_t *p)
+ {
+hb_blob_destroy (p);
+ }
+ static inline const T* convert (const hb_blob_t *blob)
+ {
+return blob->as ();
+ }
+
+ inline hb_blob_t*
Re: [HarfBuzz] HB_CLOSURE_MAX_STAGES (was: harfbuzz: Branch 'master')
On Wed, 1 Aug 2018 17:31:06 -0700 Behdad Esfahbod wrote: > On Mon, Jul 30, 2018 at 6:21 PM, Richard Wordingham < > richard.wording...@ntlworld.com> wrote: > > > On Mon, 30 Jul 2018 17:04:42 -0700 > > Behdad Esfahbod wrote: > > > It's impossible to hit that limit... Ok, it would be impossible > > > if we increase it to 32. I'll do that. > > That'll probably work, but I'm now intrigued. Why have a limit that > > will never be hit? Are you just catering for HarfBuzz's logic > > simply going badly wrong in very unusual circumstances? > Yes, simply as defense against malicious fonts and how the subsetter's > glyph-closure routine can be tricked to collect (way) more glyphs than > shaper can actually reach. But if the limit is never hit, then the defence is never used, and so it may as well not be there. Or is it meant to initimidate designers of malicious fonts who study Harfbuzz? Richard. ___ HarfBuzz mailing list HarfBuzz@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/harfbuzz
[HarfBuzz] Fwd: On atomic ops and text stack thread-safety
Since I've been hacking on the atomic stuff past few days (It's over!) and
Werner asked, I wrote this. Might be of interest to others.
-- Forwarded message --
From: Behdad Esfahbod
Date: Wed, Aug 1, 2018 at 10:16 PM
Subject: On atomic ops and text stack thread-safety
To: Werner LEMBERG
Ok let's see if I can explain this succinctly.
For what I'm doing, performance is definitely of utmost important, but only
while maintaining correctness, and portability has been a huge part of it.
There's different issues at play. Let's discuss them in three parts:
1. Atomic operations in presence of multiple threads,
2. Memory ordering in regard to compiler and CPU optimizations,
3. Memory / cache coherency in presence of multiple CPUs.
== Atomic operations ==
By default, read and write operations on properly-aligned char, short, int,
and long on all current CPUs are atomic. C11 and C++11 guarantee that
IIRC. What we mean by atomic, is that, eg, if we have "static x = 0" and
one thread sets it to 42, other threads reading this same variable will
only either see 0 or 42, but no other value. That's atomicity of read /
write (also known as load / store).
Now, when we get to more involved operations, namely, "read-modify-write",
atomicity is not guaranteed. Take FT_Reference_Library:
FT_EXPORT_DEF( FT_Error )
FT_Reference_Library( FT_Library library )
{
if ( !library )
return FT_THROW( Invalid_Library_Handle );
library->refcount++;
return FT_Err_Ok;
}
As long as there's only one thread of execution, this code is fine and
correct. Compiler and CPU make sure that what gets executed has the same
semantics of what you wrote. But what if there are multiple threads
involved? Things get out of hand and programmer needs to intervene. In
this case, the "refcount++" is a read-modify-write operation. That is, in
machine code / pseudo-assembler, it involves:
load r, refcount
inc r, 1
store refcount, r
Now, what happens if the first thread calling this function performs the
load, reading, eg. refcount of 1, but before performing the rest of the
operations it's pre-empted and second thread runs the same code, also
reading refcount of 1, incrementing it, and storing 2. Now the first
thread resumes, incrementing r to 2 and store it into refcount. What just
happened is that we started with a refcount of 1, two threads incremented
refcount by one each, but as a result we got 2 stored in refcount instead
of 3.
That's the first issue. To resolve it, we need an atomic "fetch_and_add"
operation. That is, one that does the "load, inc, store" atomically
without another thread interrupting. That's exactly the kind of thing that
atomic operations in CPUs, compilers, and standard libraries provide.
You can read more at Linux's documentation:
https://github.com/torvalds/linux/blob/master/
Documentation/core-api/atomic_ops.rst
== Memory ordering and optimizations ==
Imagine this pseudo code:
static int x = 0;
static bool ready = false;
One thread running:
x = expensive_computation_producing_42 ();
ready = true;
Other thread:
if (ready)
use (x);
Now, as discussed, we already know that read and write of int and bool
types *are* atomic. So *that* is not the issue here. However, you would
expect that if the second thread get into the body of the "if", then the
value of x read must be 42. However, that's not what always will happen,
because of legitimate compiler and CPU optimizations.
In particular, the compiler "is within its rights" to reorder the
assignment of x and ready in the first thead. The CPU, is within its
rights to execute them out of order, or in parallel. These freedoms /
rights allow for tons of optimizations / parallelism. They are allowed
because to single-threaded programs they don't make any observable
difference. The program results are always the same. But if you have
another thread reading those, this can wreak havoc. What's worse, the
second thread's compiler / CPU is also within its rights to, eg, read "x"
before / in parallel to reading "ready". Because to a single-threaded
program those do NOT make a visible difference, but allow for faster code.
Imagine the code above being transformed in machine code to:
Thread 1:
ready = true;
x = expensive_computation_producing_42 ();
Other thread:
r = x;
if (ready)
use (r);
Obviously there's a race condition there now. We might be reading old
value of x and using it because "ready" is true. That's a problem.
To fix this problem, we use what's called memory-barriers. There's many
types of them, offering different guarantees. For now, let's just discuss
what's known as a full memory barrier, or simply memory-barrier. What a
memory barrier does, in this, context, is to make sure compiler / CPU do
NOT reorder reads / writes from before and after it. Ie. a read / write
after the memory barrier will be executed *after* a read / write before the
memory
