Re: [HarfBuzz] Building and testing HarfBuzz 2.3.0 on MinGW

[Richard Wordingham]

On Fri, 08 Feb 2019 11:22:03 +0200
Eli Zaretskii  wrote:

>   . Graphite2 is becoming less and less important, as fonts which
> require that are rare, and their importance for minority scripts
> is diminishing with recent OpenType developments.

But note that the Universal Script Engine (USE) forced at least one font
from HarfBuzz OpenType to Graphite.  It's now very tricky to render Tai
Tham using OpenType.  At present the USE is also a threat to 19th
century Khmer spellings.

Richard.
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Building and testing HarfBuzz 2.3.0 on MinGW

[Ebrahim Byagowi]

On Fri, Feb 8, 2019 at 5:24 PM Eli Zaretskii  wrote:

> > From: Ebrahim Byagowi 
> > Date: Fri, 8 Feb 2019 16:02:46 +0330
> > Cc: Nathan Willis , Harfbuzz <
> harfbuzz@lists.freedesktop.org>
> >
> > > My conclusion was that ICU is not needed, but maybe it has some
> advantages,
> >
> > It will be a good idea if someone ships ICU anyway, they use their ICU
> (or glib, which can provide unicode
> > callbacks also) instead having extra a harfbuzz buildin UCDN, at least
> for size reduction reasons.
> > [...]
> > > Glib is needed for running a large part of the test suite
> >
> > It can provide unicode callbacks also as just said before.
>
> Thanks, but I still don't think I understand: given that Unicode
> character properties are all derived from the same UCD database, what
> would be the motivation to use Glib or ICU for these purposes, even if
> these libraries are already linked into a program?  Do ICU/Glib
> support some extensions that UCDN doesn't, or are more likely to
> support the latest Unicode Standard?
>
This is my understanding at least: Binary size reduction, if an application
uses ICU already, harfbuzz can use it also, if it uses glib already,
harbuzz can use that instead, and if none, harfbuzz can bring its own.
Chrome for example uses ICU callbacks so they won't need harfbuzz's ucdn.
AFAIK they provide roughly similar quality.

>
> > > It is not clear to me what are GObject and Introspection needed for;
> it would be good to clarify that.
> >
> > Roughly, gnome way of writing language bindings, ie. make non C/C++
> language users able to interact with
> > the library with Gnome provided facilities. Not needed for C/C++ or
> users don't use gobject introspection
> > anyway.
>
> Thanks, this part is now clear, I think.
>
> > > Btw, the information about "Building on Windows" is IMO outdated:
> > > nowadays one can use the "normal" Unix configure/make steps assuming
> > > one has MSYS and MinGW installed.  That's what I did.  There should be
> > > no need anymore for any Windows-specific build procedures.
> >
> > Not everyone will agree with you on that I guess, maybe different
> use-cases or something, as you see vcpkg
> > project https://github.com/Microsoft/vcpkg/graphs/contributors is still
> a pretty busy project, that's why I
> > suggest vcpkg for non-msys Windows users, even instead directly using
> our cmake on Windows. Vcpkg
> > itself uses our cmake but can switch to meson if needed and it can
> target Linux in addition to Windows, for
> > use-cases I am not aware of.
>
> So maybe that section should be extended to mention both methods?
> People who build HarfBuzz on Windows are likely to have MSYS installed
> anyway, because building the dependencies mostly does require it.  And
> even if they don't have it already installed, it's good to mention
> that, just so that the reader would know such a method is supported.
> When I first read that, I was left wondering whether the normal
> configure && make paradigm will get me a port as functional as the
> method described on that page.
>
> Thanks.
>
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Building and testing HarfBuzz 2.3.0 on MinGW

[Eli Zaretskii]

> From: Ebrahim Byagowi 
> Date: Fri, 8 Feb 2019 16:02:46 +0330
> Cc: Nathan Willis , Harfbuzz 
> 
> 
> > My conclusion was that ICU is not needed, but maybe it has some advantages,
> 
> It will be a good idea if someone ships ICU anyway, they use their ICU (or 
> glib, which can provide unicode
> callbacks also) instead having extra a harfbuzz buildin UCDN, at least for 
> size reduction reasons.
> [...]
> > Glib is needed for running a large part of the test suite
> 
> It can provide unicode callbacks also as just said before.

Thanks, but I still don't think I understand: given that Unicode
character properties are all derived from the same UCD database, what
would be the motivation to use Glib or ICU for these purposes, even if
these libraries are already linked into a program?  Do ICU/Glib
support some extensions that UCDN doesn't, or are more likely to
support the latest Unicode Standard?

> > It is not clear to me what are GObject and Introspection needed for; it 
> > would be good to clarify that.
> 
> Roughly, gnome way of writing language bindings, ie. make non C/C++ language 
> users able to interact with
> the library with Gnome provided facilities. Not needed for C/C++ or users 
> don't use gobject introspection
> anyway.

Thanks, this part is now clear, I think.

> > Btw, the information about "Building on Windows" is IMO outdated:
> > nowadays one can use the "normal" Unix configure/make steps assuming
> > one has MSYS and MinGW installed.  That's what I did.  There should be
> > no need anymore for any Windows-specific build procedures.
> 
> Not everyone will agree with you on that I guess, maybe different use-cases 
> or something, as you see vcpkg
> project https://github.com/Microsoft/vcpkg/graphs/contributors is still a 
> pretty busy project, that's why I
> suggest vcpkg for non-msys Windows users, even instead directly using our 
> cmake on Windows. Vcpkg
> itself uses our cmake but can switch to meson if needed and it can target 
> Linux in addition to Windows, for
> use-cases I am not aware of.

So maybe that section should be extended to mention both methods?
People who build HarfBuzz on Windows are likely to have MSYS installed
anyway, because building the dependencies mostly does require it.  And
even if they don't have it already installed, it's good to mention
that, just so that the reader would know such a method is supported.
When I first read that, I was left wondering whether the normal
configure && make paradigm will get me a port as functional as the
method described on that page.

Thanks.
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Building and testing HarfBuzz 2.3.0 on MinGW

[Ebrahim Byagowi]

> My conclusion was that ICU is not needed, but maybe it has some
advantages,

It will be a good idea if someone ships ICU anyway, they use their ICU (or
glib, which can provide unicode callbacks also) instead having extra a
harfbuzz buildin UCDN, at least for size reduction reasons.

> graphite2, is becoming less and less important [...] with recent OpenType
developments.

And the implementation of AAT on HarfBuzz, I believe

> Glib is needed for running a large part of the test suite

It can provide unicode callbacks also as just said before.

> It is not clear to me what are GObject and Introspection needed for; it
would be good to clarify that.

Roughly, gnome way of writing language bindings, ie. make non C/C++
language users able to interact with the library with Gnome provided
facilities. Not needed for C/C++ or users don't use gobject introspection
anyway.

> Btw, the information about "Building on Windows" is IMO outdated:
> nowadays one can use the "normal" Unix configure/make steps assuming
> one has MSYS and MinGW installed.  That's what I did.  There should be
> no need anymore for any Windows-specific build procedures.

Not everyone will agree with you on that I guess, maybe different use-cases
or something, as you see vcpkg project
https://github.com/Microsoft/vcpkg/graphs/contributors is still a pretty
busy project, that's why I suggest vcpkg for non-msys Windows users, even
instead directly using our cmake on Windows. Vcpkg itself uses our cmake
but can switch to meson if needed and it can target Linux in addition to
Windows, for use-cases I am not aware of.

On Fri, Feb 8, 2019 at 12:52 PM Eli Zaretskii  wrote:

> > From: Nathan Willis 
> > Date: Mon, 4 Feb 2019 12:28:02 +
> > Cc: harfbuzz@lists.freedesktop.org
> >
> > On Sat, Jan 26, 2019 at 5:35 PM Eli Zaretskii  wrote:
> >
> > > 1) It would be good to have some guidance in some README or in the
> > > HTML docs regarding the optional dependencies and configuration
> > > options, and their significance.  For example, it turns out Glib is
> > > needed to run a large portion of the test suite, something that wasn't
> > > clear (I initially concluded that I didn't need Glib at all).  Also,
> > > hb-shape is not built if Glib isn't available.  Similarly, hb-view is
> > > not build unless both Cairo and cairo-ft are available.
> > >
> > >
> > I added https://harfbuzz.github.io/building.html#configuration a few
> weeks
> > ago; would you mind elaborating on what is missing there from your POV?
>
> Thanks for adding this, and sorry for the long delay in responding.
>
> The information you added tells when to use the optional configure
> switches.  That is important, but there's a more general issue of what
> optional dependencies are needed for which parts of HarfBuzz's
> functionalities.  This is important for someone who wants to build
> HarfBuzz with the minimal set of dependencies, but without losing any
> functionality important for one's use case.  Without a good
> understanding of these issues, one cannot easily decide on which of
> the configure switches to use, and more importantly what packages need
> to be installed before building HarfBuzz.
>
> In response to my questions, Khaled once provided some of the
> information about that.  I now combine that below with what I learned
> while building HarfBuzz:
>
>   . ICU is needed for accessing Unicode character properties; UCDN is
> the built-in alternative to that which has no external
> dependencies.  My conclusion was that ICU is not needed, but maybe
> it has some advantages, in which case it would be good to describe
> them.
>
>   . Cairo is needed for command-line tools (so can be skipped if one
> only wants the library).  Note that Cairo alone is not enough for
> building the command-line tools, you also need cairo-ft, and for
> hb-shape one also needs Glib.
>
>   . Freetype is one of two font callbacks; the other is built-in and
> has no external dependencies.  The decision whether to use
> Freetype largely depends on whether the program(s) to be linked
> against HarfBuzz already use Freetype.
>
>   . Fontconfig is only needed for command-line tools.
>
>   . Graphite2 is becoming less and less important, as fonts which
> require that are rare, and their importance for minority scripts
> is diminishing with recent OpenType developments.
>
>   . Glib is needed for running a large part of the test suite, so if
> one decides not to build with Glib, a separate build with Glib
> just for running the test suite is a good idea.
>
>   . Python is required (and should be on PATH) for most of the test
> suite.
>
>   . It is not clear to me what are GObject and Introspection needed
> for; it would be good to clarify that.
>
> Btw, the information about "Building on Windows" is IMO outdated:
> nowadays one can use the "normal" Unix configure/make steps assuming
> one has MSYS and MinGW installed.  That's what I did.  There sho

Re: [HarfBuzz] Building and testing HarfBuzz 2.3.0 on MinGW

[Eli Zaretskii]

> From: Nathan Willis 
> Date: Mon, 4 Feb 2019 12:28:02 +
> Cc: harfbuzz@lists.freedesktop.org
> 
> On Sat, Jan 26, 2019 at 5:35 PM Eli Zaretskii  wrote:
> 
> > 1) It would be good to have some guidance in some README or in the
> > HTML docs regarding the optional dependencies and configuration
> > options, and their significance.  For example, it turns out Glib is
> > needed to run a large portion of the test suite, something that wasn't
> > clear (I initially concluded that I didn't need Glib at all).  Also,
> > hb-shape is not built if Glib isn't available.  Similarly, hb-view is
> > not build unless both Cairo and cairo-ft are available.
> >
> >
> I added https://harfbuzz.github.io/building.html#configuration a few weeks
> ago; would you mind elaborating on what is missing there from your POV?

Thanks for adding this, and sorry for the long delay in responding.

The information you added tells when to use the optional configure
switches.  That is important, but there's a more general issue of what
optional dependencies are needed for which parts of HarfBuzz's
functionalities.  This is important for someone who wants to build
HarfBuzz with the minimal set of dependencies, but without losing any
functionality important for one's use case.  Without a good
understanding of these issues, one cannot easily decide on which of
the configure switches to use, and more importantly what packages need
to be installed before building HarfBuzz.

In response to my questions, Khaled once provided some of the
information about that.  I now combine that below with what I learned
while building HarfBuzz:

  . ICU is needed for accessing Unicode character properties; UCDN is
the built-in alternative to that which has no external
dependencies.  My conclusion was that ICU is not needed, but maybe
it has some advantages, in which case it would be good to describe
them.

  . Cairo is needed for command-line tools (so can be skipped if one
only wants the library).  Note that Cairo alone is not enough for
building the command-line tools, you also need cairo-ft, and for
hb-shape one also needs Glib.

  . Freetype is one of two font callbacks; the other is built-in and
has no external dependencies.  The decision whether to use
Freetype largely depends on whether the program(s) to be linked
against HarfBuzz already use Freetype.

  . Fontconfig is only needed for command-line tools.

  . Graphite2 is becoming less and less important, as fonts which
require that are rare, and their importance for minority scripts
is diminishing with recent OpenType developments.

  . Glib is needed for running a large part of the test suite, so if
one decides not to build with Glib, a separate build with Glib
just for running the test suite is a good idea.

  . Python is required (and should be on PATH) for most of the test
suite.

  . It is not clear to me what are GObject and Introspection needed
for; it would be good to clarify that.

Btw, the information about "Building on Windows" is IMO outdated:
nowadays one can use the "normal" Unix configure/make steps assuming
one has MSYS and MinGW installed.  That's what I did.  There should be
no need anymore for any Windows-specific build procedures.

Thanks, and let me know if I can help more with this documentation
effort.
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Building and testing HarfBuzz 2.3.0 on MinGW

[Eli Zaretskii]

> From: Nathan Willis 
> Date: Mon, 4 Feb 2019 12:28:02 +
> Cc: harfbuzz@lists.freedesktop.org
> 
> On Sat, Jan 26, 2019 at 5:35 PM Eli Zaretskii  wrote:
> 
>  1) It would be good to have some guidance in some README or in the
>  HTML docs regarding the optional dependencies and configuration
>  options, and their significance.  For example, it turns out Glib is
>  needed to run a large portion of the test suite, something that wasn't
>  clear (I initially concluded that I didn't need Glib at all).  Also,
>  hb-shape is not built if Glib isn't available.  Similarly, hb-view is
>  not build unless both Cairo and cairo-ft are available.
> 
> I added https://harfbuzz.github.io/building.html#configuration a few weeks 
> ago; would you mind elaborating on
> what is missing there from your POV?

Hi,

I didn't forget, I just have my plate full.  Will respond in a few
days.
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Building and testing HarfBuzz 2.3.0 on MinGW

[Nathan Willis]

Hi Eli,

On Sat, Jan 26, 2019 at 5:35 PM Eli Zaretskii  wrote:

>
> 1) It would be good to have some guidance in some README or in the
> HTML docs regarding the optional dependencies and configuration
> options, and their significance.  For example, it turns out Glib is
> needed to run a large portion of the test suite, something that wasn't
> clear (I initially concluded that I didn't need Glib at all).  Also,
> hb-shape is not built if Glib isn't available.  Similarly, hb-view is
> not build unless both Cairo and cairo-ft are available.
>
>
I added https://harfbuzz.github.io/building.html#configuration a few weeks
ago; would you mind elaborating on what is missing there from your POV?

And thanks for the note about hb-view and hb-shape; that is a wise addition.
Nate
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Building and testing HarfBuzz 2.3.0 on MinGW

[Behdad Esfahbod]

We should port the test suite runner to C, and test suite to gtest (Google
Test) instead of glib.  For hb-shape / hb-view, requirements stay the same,
and they are hinted at in the configure report.

On Sun, Jan 27, 2019 at 4:48 PM Eli Zaretskii  wrote:

> > From: Ebrahim Byagowi 
> > Date: Sun, 27 Jan 2019 01:02:01 +0330
> > Cc: Harfbuzz 
> >
> > 1) Agreed
>
> Btw, one other prerequisite for running the test suite is Python.  I
> suggest that to be mentioned as well.  In my case, Python was not on
> PATH, and most tests failed.
>
> > 2) Something feels wrong as we compile all these in our msys2 CI already
> and that shouldn't be that different
> > from your setup
>
> I saw that similar failures were reported here:
>
>   https://github.com/harfbuzz/harfbuzz/issues/1560
>
> So I upgraded my Freetype 2.5.0.1 to the latest 2.9.1, and then all
> the tests passed.  Therefore, I suggest that the oldest version of
> Freetype that is considered "good enough" for the test suite be
> referenced in the documentation of prerequisites for running the
> tests.
>
> > 3) Uniscribe and DirectWrite backends and now CoreText, are mostly for
> comparison while development, so
> > developers can check what can be expected behavior while development,
> and are not used in the test suit at
> > least which tends to be platform agnostic so don't use them at all if
> you can.
>
> Got it, thanks.
> ___
> HarfBuzz mailing list
> HarfBuzz@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/harfbuzz
>


-- 
behdad
http://behdad.org/
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Building and testing HarfBuzz 2.3.0 on MinGW

[Eli Zaretskii]

> From: Ebrahim Byagowi 
> Date: Sun, 27 Jan 2019 01:02:01 +0330
> Cc: Harfbuzz 
> 
> 1) Agreed

Btw, one other prerequisite for running the test suite is Python.  I
suggest that to be mentioned as well.  In my case, Python was not on
PATH, and most tests failed.

> 2) Something feels wrong as we compile all these in our msys2 CI already and 
> that shouldn't be that different
> from your setup

I saw that similar failures were reported here:

  https://github.com/harfbuzz/harfbuzz/issues/1560

So I upgraded my Freetype 2.5.0.1 to the latest 2.9.1, and then all
the tests passed.  Therefore, I suggest that the oldest version of
Freetype that is considered "good enough" for the test suite be
referenced in the documentation of prerequisites for running the
tests.

> 3) Uniscribe and DirectWrite backends and now CoreText, are mostly for 
> comparison while development, so
> developers can check what can be expected behavior while development, and are 
> not used in the test suit at
> least which tends to be platform agnostic so don't use them at all if you can.

Got it, thanks.
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

Re: [HarfBuzz] Building and testing HarfBuzz 2.3.0 on MinGW

[Ebrahim Byagowi]

Hey there.
1) Agreed
2) Something feels wrong as we compile all these in our msys2 CI

already and that shouldn't be that different from your setup
3) Uniscribe and DirectWrite backends and now CoreText, are mostly for
comparison while development, so developers can check what can be expected
behavior while development, and are not used in the test suit at least
which tends to be platform agnostic so don't use them at all if you can.

On Sat, Jan 26, 2019 at 9:05 PM Eli Zaretskii  wrote:

> I'm resending this after subscribing to the list, since my original
> message, send a month ago, only got an automated response that it's
> waiting for a moderator.  (Does someone actually tend to the
> moderator's tasks of this list?)
>
> I've built HarfBuzz 2.3.0 on MS-Windows using mingw.org's MinGW
> (https://osdn.net/projects/mingw/, different from MinGW64).  In
> general, the build was successful, with a small number of changes that
> I will soon report to the issue tracker.
>
> I have a few questions/suggestions as result of this experience, which
> I'd like to voice.  Thanks in advance for any responses.
>
> 1) It would be good to have some guidance in some README or in the
> HTML docs regarding the optional dependencies and configuration
> options, and their significance.  For example, it turns out Glib is
> needed to run a large portion of the test suite, something that wasn't
> clear (I initially concluded that I didn't need Glib at all).  Also,
> hb-shape is not built if Glib isn't available.  Similarly, hb-view is
> not build unless both Cairo and cairo-ft are available.
>
> 2) Several tests fail.  For example, "indic-joiners" and "use" in
> shaping/data/in-house, CVAR-1 and CVAR-2 in
> shaping/data/text-rendering-tests, most of the gpos_* tests in
> shaping/data/aots:, etc.  I also built HarfBuzz on GNU/Linux, and I
> see failures in almost the same tests.  The reason for the failures
> are some differences between the expected and the actual outputs.  Are
> these real problems, for which you'd like me to report issues, or is
> this a known problem?  Did someone succeed to run the entire test
> suite without a single failure?
>
> 3) I'm uncertain about the use of Uniscribe in the Windows build.  I
> was told that it was only used "for comparison", which I interpreted
> to mean it was used in the test suite.  But I don't think it's the
> case, since the Uniscribe dependent functions of HarfBuzz are in the
> library, so it seems like Uniscribe is used by the library itself.
> What is the purpose of using Uniscribe (and DirectWrite, when that is
> compiled in)?
> ___
> HarfBuzz mailing list
> HarfBuzz@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/harfbuzz
>
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz

[HarfBuzz] Building and testing HarfBuzz 2.3.0 on MinGW

[Eli Zaretskii]

I'm resending this after subscribing to the list, since my original
message, send a month ago, only got an automated response that it's
waiting for a moderator.  (Does someone actually tend to the
moderator's tasks of this list?)

I've built HarfBuzz 2.3.0 on MS-Windows using mingw.org's MinGW
(https://osdn.net/projects/mingw/, different from MinGW64).  In
general, the build was successful, with a small number of changes that
I will soon report to the issue tracker.

I have a few questions/suggestions as result of this experience, which
I'd like to voice.  Thanks in advance for any responses.

1) It would be good to have some guidance in some README or in the
HTML docs regarding the optional dependencies and configuration
options, and their significance.  For example, it turns out Glib is
needed to run a large portion of the test suite, something that wasn't
clear (I initially concluded that I didn't need Glib at all).  Also,
hb-shape is not built if Glib isn't available.  Similarly, hb-view is
not build unless both Cairo and cairo-ft are available.

2) Several tests fail.  For example, "indic-joiners" and "use" in
shaping/data/in-house, CVAR-1 and CVAR-2 in
shaping/data/text-rendering-tests, most of the gpos_* tests in
shaping/data/aots:, etc.  I also built HarfBuzz on GNU/Linux, and I
see failures in almost the same tests.  The reason for the failures
are some differences between the expected and the actual outputs.  Are
these real problems, for which you'd like me to report issues, or is
this a known problem?  Did someone succeed to run the entire test
suite without a single failure?

3) I'm uncertain about the use of Uniscribe in the Windows build.  I
was told that it was only used "for comparison", which I interpreted
to mean it was used in the test suite.  But I don't think it's the
case, since the Uniscribe dependent functions of HarfBuzz are in the
library, so it seems like Uniscribe is used by the library itself.
What is the purpose of using Uniscribe (and DirectWrite, when that is
compiled in)?
___
HarfBuzz mailing list
HarfBuzz@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/harfbuzz