Re: gs 10.04 will break doc builds again

[Masamichi Hosoda]

> `gs` compiled from the current 'master' branch of the 'ghostpdl'
> repository makes a doc build fail if 'extractpdfmark' gets used: In
> the created `notation.pdf`, Emmentaler fonts are missing in all
> included snippets.
> 
[...snip...]
> 
> Sigh.  Masamichi-san, please have a look!

It seems that the Ghostscript generated PDF is broken
regardless of LilyPond or extractpdfmark.

I tested Ghostscript GIT PRERELEASE at
commit 99b48f1992844e30ab4b05dbb2764ba99f7b66f4
Date:   Mon Feb 19 08:33:23 2024 +
.
Here's results.

```
$ cat test.ps
/NimbusSans-Regular findfont 14 scalefont setfont
50 100 moveto
(Hello World) show

$ ./gs -q -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile=test.pdf test.ps

$ qpdf --check test.pdf
checking test.pdf
PDF Version: 1.7
File is not encrypted
WARNING: test.pdf: object 6/0: error reading object: integer out of range 
converting 889058230272 from a 8-byte signed type to a 4-byte signed type
WARNING: test.pdf (object 9 0, offset 100): stream dictionary lacks /Length key
WARNING: test.pdf (object 9 0, offset 146): attempting to recover stream length
WARNING: test.pdf (object 9 0, offset 146): recovered stream length: 82
File is not linearized
qpdf: operation succeeded with warnings

$
```

Re: NR: no working links in included PDFs

[Masamichi Hosoda]

>> As with the Java part of `pax`, it may be possible to extract
>> annotations from PDFs with a C/C++ program.  But, it looks like
>> `latex-pax` is adding the extracted annotations to the PDF using
>> `pax.sty`.  The `pax.sty` is for LaTeX, and it looks like it is for
>> pdfLaTeX only.  It would be quite difficult to get it to work with
>> Texinfo on XeTeX/pdfTeX.
> 
> Oh, I don't mean that we should exactly follow the `pax` algorithm.  I
> only mentioned it because it provides a solution.
> 
> Isn't it possible to add links with the `pdfmark` operator in the same
> way as is already done for annotations by `extractpdfmark`?  In
> Adobe's pdfmark reference manual (`5150.Pdfmark.pdf`) I see a section
> 2.1.2 called 'Links', which appears to be exactly what's needed.

The PDF output by pdfTeX/XeTeX has no information about
the file name or position (page number, x-y coordinates, size, etc.)
of the PDFs it includes.
It means that even if we extract the link information from the included PDFs,
we don't know where to apply it in the positions
of the PDF output by pdfTeX/XeTeX.

`pax.sty` seems to recognize the positions of the PDFs included
by the LaTeX layer and reproduce the links.
If something similar to this can be done with Texinfo,
link reproduction can be realized.
However, I have no idea how to do this in Texinfo.

Re: NR: no working links in included PDFs

[Masamichi Hosoda]

Sorry, I'm too late.

As with the Java part of `pax`,
it may be possible to extract annotations from PDFs with a C/C++ program.
But, it looks like `latex-pax` is adding the extracted annotations
to the PDF using `pax.sty`.
The `pax.sty` is for LaTeX, and it looks like it is for pdfLaTeX only.
It would be quite difficult to get it to work with Texinfo on XeTeX/pdfTeX.

Alternatively, you could ask the xdvipdfmx developer
to add an option to keep the links and the annotations.

> Look at the documention of `\with-url` in the PDF version of the NR:
> The link in the included snippet (to 'lilypond.org') doesn't work.
> This is a limitation of pdfTeX (or XeTeX), which drops all links and
> annotations for included PDF files.
> 
> Masamichi-san, is it possible to extend `extractpdfmark` to cover
> this, too?  Looking around it seems that the tool 'pax'
> ('PDFAnnotExtractor') in TeXLive can do that.  However, it is based on
> Java...
> 
>   https://github.com/bastien-roucaries/latex-pax
> 
> The reason why I ask is the planned inclusion of the Visual Grob Index
> in the NR – all grobs shown have links to the corresponding
> documentation.  If these links are lost the value of the index is
> reduced enormously, and it would be probably better to not include it
> at all but to have a stand-alone, simple, two-page document instead.

Re: Cairo plans

[Masamichi Hosoda]

> I analyzed more closely in the issue
> https://gitlab.com/lilypond/lilypond/-/issues/6172

The garbling sample I have shown has two different issues.
#6172 is one of them.

I created #6173 for the other one.
https://gitlab.com/lilypond/lilypond/-/issues/6173

Re: Cairo plans

[Masamichi Hosoda]

 Most PDF readers have a mapping from Adobe-Japan1 CID to Unicode
 code points as follows.
 https://github.com/adobe-type-tools/mapping-resources-pdf/blob/master/pdf2unicode/Adobe-Japan1-UCS2
>>>
>>> Is it impossible to discover this mapping from the OTF file alone?
>> 
>> If I understand correctly, yes.
> 
> I guess Masamichi means that the mapping is not part of the font
> itself.

Yes. Thanks.

>>> Where does the OTF file say it is using encoding Adobe-Japan1-UCS
>> 
>> It is described in ROS in the CFF table.
>> https://docs.microsoft.com/en-us/typography/opentype/spec/cff
>> http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/font/pdfs/5176.CFF.pdf
>> 
>> ROS means `Registry-Ordering-Supplement`.
>> For the current HaranoAjiMincho-Regular.otf,
>> Registry is `Adobe`, Ordering is `Japan1`, and Supplement is `7`.
>> So it is an `Adobe-Japan1-7` font.
>> 
>> PDF readers use Adobe-Japan1-UCS2 for `Adobe-Japan1-*` fonts.
> 
> Hopefully there is a solution that makes Cairo do the mapping work for
> us...

If I understand correctly, it is PDF readers that use Adobe-Japan1-UCS,
not PDF generators such as cairo.
If you use the PS backend and Ghostscript outputs the PDF,
it does not use Adobe-Japan1-UCS, but it works fine.

Re: Cairo plans

[Masamichi Hosoda]

>> Most PDF readers have a mapping
>> from Adobe-Japan1 CID to Unicode code points as follows.
>> https://github.com/adobe-type-tools/mapping-resources-pdf/blob/master/pdf2unicode/Adobe-Japan1-UCS2
> 
> Is it impossible to discover this mapping from the OTF file alone?

If I understand correctly, yes.

> Where does the OTF file say it is using encoding Adobe-Japan1-UCS

It is described in ROS in the CFF table.
https://docs.microsoft.com/en-us/typography/opentype/spec/cff
http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/font/pdfs/5176.CFF.pdf

ROS means `Registry-Ordering-Supplement`.
For the current HaranoAjiMincho-Regular.otf,
Registry is `Adobe`, Ordering is `Japan1`, and Supplement is `7`.
So it is an `Adobe-Japan1-7` font.

PDF readers use Adobe-Japan1-UCS2 for `Adobe-Japan1-*` fonts.

Re: Cairo plans

[Masamichi Hosoda]

> I analyzed more closely in the issue
> https://gitlab.com/lilypond/lilypond/-/issues/6172
> 
> Would you know how the mapping of character index => codepoint is
> supposed to work for this font with character 3056 (辻) ?

Most PDF readers have a mapping
from Adobe-Japan1 CID to Unicode code points as follows.
https://github.com/adobe-type-tools/mapping-resources-pdf/blob/master/pdf2unicode/Adobe-Japan1-UCS2

In the case of Acrobat Reader on Windows, it is stored in the following path.
C:\Program Files (x86)\Adobe\Acrobat Reader DC\Resource\CMap\Adobe-Japan1-UCS2

The number 3056 in the Adobe-Japan1 CID is 0xbf0 in hexadecimal.
The Adobe-Japan1-UCS2 file contains the following line,
which shows that CID 0xbf0 is U+8FBB.
https://github.com/adobe-type-tools/mapping-resources-pdf/blob/20190401/pdf2unicode/Adobe-Japan1-UCS2#L2329

```
<0bf0> <8fbb>
```

In the case of poppler, the file format is different,
but the following file has a similar mapping.
https://gitlab.freedesktop.org/poppler/poppler-data/-/blob/master/cidToUnicode/Adobe-Japan1

There is a Unicode code point at line 3056 + 1 (i.e. line 3057),
which shows U+8FBB.
https://gitlab.freedesktop.org/poppler/poppler-data/-/blob/POPPLER_DATA_0_4_10/cidToUnicode/Adobe-Japan1#L3057

Re: Cairo plans

[Masamichi Hosoda]

> I installed your Harano fonts in ~/.fonts. Fontconfig sees them, but I
> still get DroidSans when I compile the snippet. How do I reproduce the
> problem?

Would you show me the results of the follwing commands?

```
find ~/.fonts -name "HaranoAjiMincho-*.otf"
fc-list "HaranoAjiMincho"
lilypond -dshow-available-fonts 2>&1 | grep "Harano Aji Mincho"
```

Re: Cairo plans

[Masamichi Hosoda]

> * when could/would we drop the PS backend altogether?

In my humble opinion,
the PS backend is still needed for CJK character handling.
When you copy and paste text from a PDF generated by the cairo backend,
some CJK characters are garbled.
By the PS backend, no such garbling occurs.

Here's a sample.

```
\version "2.22.0"
\markup {
  \override #'(font-name . "HaranoAjiMincho")
  { 初見はハ長調で高音の白玉があった } }
\markup {
  \override #'(font-name . "HaranoAjiMincho")
  \override #'(font-features . ("jp90"))
  { 辻井 逗子 飴玉 } }
```

Re: NotoSansCJKjp-Regular cannot be loaded

[Masamichi Hosoda]

> I've noticed that utf-8.ly confused the font names.

I've created a merge request.
https://gitlab.com/lilypond/lilypond/-/merge_requests/595

It solves the confusion of Japanese font names in utf-8.ly
but does not solve the `--dgs-load-fonts` warning.

Re: NotoSansCJKjp-Regular cannot be loaded

[Masamichi Hosoda]

> Note: NotoSansJP-Regular and NotoSansCJKjp-Regular are similar
> but different fonts.

I've noticed that utf-8.ly confused the font names.

It requires the font `Noto Serif JP`.
https://gitlab.com/lilypond/lilypond/-/blob/d6b62def753b268fc5adfbe8443aa477d38395ef/input/regression/utf-8.ly#L30

And, it requires fonts-noto-cjk package for Debian/Ubuntu.
https://gitlab.com/lilypond/lilypond/-/blob/d6b62def753b268fc5adfbe8443aa477d38395ef/input/regression/utf-8.ly#L23

However, fonts-noto-cjk package contains `Noto Serif CJK JP`
instead of `Noto Serif JP`.
https://packages.debian.org/buster/fonts-noto-cjk

`Noto Serif CJK JP` and `Noto Serif JP` are similar but different fonts.
In other words, Debian/Ubuntu with fonts-noto-cjk package
does not have `Noto Serif JP` that utf-8.ly requires.

Then, I think fontconfig found NotoSansCJKjp-Regular
by searching for other fonts that can be used in Japanese
because the specified font is not available.

On the other hand, for Fedora,
utf-8.ly requires google-noto-serif-jp-fonts package.
https://gitlab.com/lilypond/lilypond/-/blob/d6b62def753b268fc5adfbe8443aa477d38395ef/input/regression/utf-8.ly#L18

It contains non-OTC version of exactly `Noto Serif JP`.
https://fedora.pkgs.org/33/fedora-x86_64/google-noto-serif-jp-fonts-20190416-7.fc33.noarch.rpm.html
Therefore, there is no problem.



There are many variations of the Noto font that can be used for Japanese.

* Region-specific Subset OpenType/CFF (Subset OTF)
+ Font name: Noto{Sans|Serif}JP-*
+ File name: Noto{Sans|Serif}JP-*.otf
* Language-specific OpenType/CFF (OTF)
+ Font name: Noto{Sans|Serif}CJKjp-*
+ File name: Noto{Sans|Serif}CJKjp-*.otf
* OpenType/CFF Collection (OTC)
+ Font name: Noto{Sans|Serif}CJKjp-* and non-Japanese font names
+ File name: Noto{Sans|Serif}CJK-*.ttc
* Super OpenType/CFF Collection (Super OTC)
+ Font name: Noto{Sans|Serif}CJKjp-* and non-Japanese font names
+ File name: Noto{Sans|Serif}CJK.ttc

Noto's OTF, OTC, and SuperOTC have the same font name, the same font content,
and only the file format is different.
Noto's Subset OTF is similar but different font names
and font content from the other fonts.

Re: NotoSansCJKjp-Regular cannot be loaded

[Masamichi Hosoda]

>> ```
>> Layout output to `./1a/lily-d0aeef0b.eps'...
>> fatal error: Font NotoSansCJKjp-Regular cannot be loaded via Ghostscript 
>> because it is an OpenType/CFF Collection (OTC) font.
>> ```

> /usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc: Noto Sans CJK 
> JP:style=Regular

This file contains NotoSansCJKjp-Regular
but it is an OpenType/CFF Collection (OTF) font.

LilyPond can handle OTC fonts but Ghostscript cannot.
https://bugs.ghostscript.com/show_bug.cgi?id=696808

If you use LilyPond without `--dgs-load-fonts`,
OTC fonts don't raise the error because LilyPond processes them.
Instead, with `--dgs-load-fonts`,
LilyPond does not process font files
and passes through the file name to Ghostscript.
But Ghostscript cannot handle the OTC fonts.

Another solution is to not use OTC fonts.

Non-OTC version of NotoSansCJKjp-Regular is contained
by `NotoSansCJKjp-Regular.otf`.

Note: NotoSansJP-Regular and NotoSansCJKjp-Regular are similar
but different fonts.

Unfortunately, Debian/Ubuntu does not seem to have a package
that contains `NotoSansCJKjp-Regular.otf`.
https://packages.debian.org/search?searchon=contents=NotoSansCJK=filename=stable=any

Re: Problem with OTF support

[Masamichi Hosoda]

> [lilypond dcd531b0f, gs 9.52]
> 
> Processing the following sample code
> 
>   \paper {
> #(define fonts
>   (set-global-fonts
>#:roman "Source Han Serif TW"
>#:sans "sans-serif"
>#:typewriter "monospace"
>  ))
>   }
> 
>   \markup { "ろ" }   % This is character U+308D.
> 
> causes the attached image, which is obviously wrong (inspite of the
> name, the 'TW' variant of the 'Source Han Serif' family contains
> Hiragana syllables).  Running `pdffonts` on the output PDF
> demonstrates that the right font gets used:
> 
>   UOIEOY+SourceHanSerifTW-Regular  CID Type 0C  Identity-H  yes yes no   9  0
> 
> Looking into the `--eps` output of LilyPond I see the following:
> 
>   /SourceHanSerifTW-Regular /CIDFont findresource [...]
>   2.1852 0. 0. 1277
>   1 print_glyphs
> 
> The main question is where the value 1277 comes from.
> 
> In the (OTF) font's `cmap` table, U+308D maps to glyph 'cid01536',
> which has index 1277 in the font.
> 
> However, the glyph shown in the image is U+2609, glyph 'cid01277'.
> index 1021.
> 
> If I extract the (raw) CFF file from the EPS file, glyph index 1277 is
> U+2609 (so CID numbers are one-to-one mapped to glyph indices, as
> indicated by the 'Identity-H' entry above).  The correct value should
> be – as can be guessed now – 1536.
> 
> In other words, CID values (this is, glyph indices in the raw CFF
> font) and OpenType glyph indices are mixed up.  Masamichi-san, do you
> have a quick fix for this?

If I understand correctly,
"Source Han Serif TW" is one of "Region-specific Subset OTFs".
They are subset fonts, so CID != GID.

"Language-specific OTFs" (e.g. "Source Han Serif TC") are not subset fonts,
so CID==GID.
I think these non-subset fonts would be fine.

Of course, if you use Hiragana,
it's better to use "Source Han Serif" (for Japanese)
instead of "Source Han Serif TC" (for Traditional Chinese).

Re: font problems

[Masamichi Hosoda]

>>> For Japanese fonts, I suggest HaranoAji.
>>> It is the default Japanese font from TeX Live 2020.
>>> 
>>> https://www.ctan.org/pkg/haranoaji
>> 
>> OK, thanks for the suggestion.
> 
> After some thinking I'm not sure whether HaranoAji is the best
> solution.  Given that both LilyPond and ghostscript need the fonts, a
> TeXLive package doesn't help much – usually, FontConfig doesn't scan
> TeXLive font directories.  Additionally, older GNU/Linux distributions
> don't provide a separate package for this font.
> 
> I'm not sure how to tackle both the building of the Japanese texinfo
> PDFs and the Japanese demo stuff in the LilyPond example files at the
> same time.
> 
> Suggestions?  Ideas?

If I understand correctly, LilyPond needs to find the HaranoAji font,
but Ghostscript does not need to find it.

If LilyPond uses the HaranoAji font,
the font will be embedded in the generated Postscript file.
Even if you use `-dgs-never-embed-fonts`,
you can still pass the font to Ghostscript with `-dfont-ps-resdir`.

In Texinfo PDF building, you don't need Japanese fonts
if only the included figure PDFs use the Japanese fonts.
If you want to use Japanese characters in the text instead of PDF figures
of Texinfo, you need the Japanese font.

Of course, it is required that Fontconfig can find the HaranoAji font.

Most environments do not have Japanese fonts.
How about using generic font names (e.g serif and sans-serif etc.)
for Japanese font fall back?

This selects the generic serif font
if the environment that doesn't have the HaranoAji font.

```
\override #'(font-name . "HaranoAjiMincho-Regular,serif")
```

Re: font problems

[Masamichi Hosoda]

> * Documentation/snippets/utf-8.ly:
> 
> [for Japanese]  →  Source Han Serif

For Japanese fonts, I suggest HaranoAji.
It is the default Japanese font from TeX Live 2020.

https://www.ctan.org/pkg/haranoaji

Re: 2,21,4 released

[Masamichi Hosoda]

> That was also my understanding, but "dir /x" already shows a short name
> of "D15A~1.LY" for "ä.ly". Maybe I'm doing something wrong? (not used
> Windows at that technical level for a few years now...)

Sorry, I have no idea because I'm not familiar with non-Japanese Windows.
At least in Japanese Windows, it is easy to create the file that `stat ()`
in msvcrt.dll raises the error.

Re: 2,21,4 released

[Masamichi Hosoda]

>> Do you have an example of a file name that should not work? I now have
>> three versions from GUB (one with MoveFileExW; one without but with
>> wstat; and one without wstat) and all work correctly on a recent
>> Windows 10. Does that mean the issue is gone with a recent update?
> 
> Might it be dependent on which VC++ package you have installed (or
> not) that determines which definition is used by Windows?
> 
> i.e. (from Hosoda-san)
> 
>> `stat ()` in newer UCRTs than msvcrt.dll does not have the problem. 
> 
> I won't pretend to know the internals here but my work laptop has a
> number of different iterations of the older VC++ components installed,
> some of them go back to 2013 (for old internally made tools we still
> use now and again) and they seem to co-exist but I don't know what it
> is LP calls in this case. Some were installed manually and some by the
> various software installers I have run.

If I understand correctly,
lilypond.exe generated by GUB uses msvcrt.dll instead of UCRT.

The recent Visual C++ generated executables use UCRT.
The executables generated by MinGW normally use msvcrt.dll.
The recent MinGW can generate executables that use UCRT
by a special configuration.
However, GUB contains older MinGW that can not be configured as such.

Re: 2,21,4 released

[Masamichi Hosoda]

> Thanks for all the explanation, I think I'm slowly starting to
> understand the problem. I've tried to get short names with special
> characters, but failed so far (maybe you need a setting for this?).
> I'm giving up on this for now and will keep the code path with
> MoveFileExW unless you feel strongly about removing it.

Do you use German Windows?
Is its ANSI encodings CP1252?

If so, maybe `.ly` should not work with `stat ()` in msvcrt.dll.
If I understand correctly, `Ä` is 1 byte in CP1252 and is 2 bytes in UTF-8.
`.LY` is 8 bytes + 2 bytes in CP1252 and 16 bytes + 2 bytes in UTF-8.

Re: 2,21,4 released

[Masamichi Hosoda]

>> `stat ()` in msvcrt.dll has a problem with some Unicode file names.
>> So we use `_wstat ()` instead of `stat ()` if msvcrt.dll is used.
>> To use `_wstat ()`, we need a wide string,
>> so we use `MultiByteToWideChar ()` to convert it.
> 
> Do you have an example of a file name that should not work? I now have
> three versions from GUB (one with MoveFileExW; one without but with
> wstat; and one without wstat) and all work correctly on a recent
> Windows 10. Does that mean the issue is gone with a recent update?

The file name that should not work depends on the language of Windows.
If a short file name exceeds 8 bytes + 3 bytes when written in UTF-8,
`stat ()` in msvcrt.dll cause an error.

In Windows, most files have a short file name
in addition to the normal file name.
The short file names are up to 8 bytes + 3 bytes in length
and are stored in a Windows language-dependent encoding.

For example, on Japanese Windows,
the short file name `インスト.LY` is stored in CP932 encoding
with 8 bytes + 2 bytes.
If the name is written in UTF-8 encoding, it is 12 bytes + 2 bytes.
So the short file name `インスト.LY` in Japanese Windows should not work.
However, in other languages Windows, it may work.

If the normal file name, such as `☃.ly`,
cannot be written in CP932, the problem does not occur
because the short file name is given in US-ASCII only.

Re: 2,21,4 released

[Masamichi Hosoda]

> Testing with Frescobaldi would be appreciated, and also with special
> characters in file names. I tried to model this after the code
> Masamichi-san wrote in March, but I'm not sure I understand what it's
> really needed for?

`stat ()` in msvcrt.dll has a problem with some Unicode file names.
So we use `_wstat ()` instead of `stat ()` if msvcrt.dll is used.
To use `_wstat ()`, we need a wide string,
so we use `MultiByteToWideChar ()` to convert it.

`stat ()` in newer UCRTs than msvcrt.dll does not have the problem.
Therefore, if `_UCRT` is defined,
we do not use `_wstat ()` and `MultiByteToWideChar ()`.

If I understand correctly,
`MoveFileExA ()` and `CopyFileA ()` do not have the problem.
So we do not necessarily need to use `MoveFileExW ()` and `CopyFileW ()`.

Re: NR pdf is larger with current git by 15%

[Masamichi Hosoda]

>> `selectdevice` does `setdevice` and then `.setdefaultscreen`.
>> If we use only `setdevice`, then `.setdefaultscreen` is not done.
> 
> Aha. So should we just call .setdefaultscreen ourselves? (calling
> "[...] setdevice (pdfwrite) finddevice setdevice" is duplicate) Looks
> like it was introduced long ago, so we can safely rely on it.

It seems that `.setdefaultscreen` is not described
at current Ghostscript documents.
Ghostscript sometimes removes old or undocumented features,
so it's better not to use them in my humble opinion.
e.g. https://ghostscript.com/pipermail/gs-devel/2020-May/010295.html

To avoid duplicate `setdevice`, we can do the following.

```
mark [...snip...] (pdfwrite) finddevice putdeviceprops pop (pdfwrite) 
selectdevice
```

I've created merge request !242.

Re: NR pdf is larger with current git by 15%

[Masamichi Hosoda]

>>> I've found the documented way to specify Ghostscript devices
>>> without `-sDEVICE=pdfwrite`.  It is using the operator
>>> `selectdevice`.
> 
> Thanks for investigating.  The main question remains why gs produces
> suboptimal results without `selectdevice`.

`selectdevice` is defined in gs_init.ps as follows.

```
/selectdevice
{ finddevice setdevice .setdefaultscreen } bind def
```

`selectdevice` does `setdevice` and then `.setdefaultscreen`.
If we use only `setdevice`, then `.setdefaultscreen` is not done.

I've found that `.setdefaultscreen` is described here.
https://www.ghostscript.com/doc/current/History3.htm#V3.61_Interpreter

Here's a quote.

-
Defines a .setdefaultscreen procedure that sets the default halftone screen,
transfer function, and stroke adjustment for the current device.  Changes
selectdevice and setpagedevice so that they call .setdefaultscreen;
setpagedevice calls .setdefaultscreen before calling the Install procedure,
so that Install can set different parameters if it wishes.  Note that
setdevice does not do this.  (gs_init.ps, gs_setpd.ps)
-

It seems to set some defaults for the current device.

This means that if we use only `setdevice`, those defaults are not set.
In that case, many drawing operators need to be written,
including the defaults that can be omitted if I understand correctly.

Re: NR pdf is larger with current git by 15%

[Masamichi Hosoda]

> Ghostscript seems to need `-sDEVICE=pdfwrite` to produce the PDF we expect.
> 
> Is the device specifying way on the new method (without `-sDEVICE=pdfwrite`)
> documented by Ghostscript documents?
> If not, it is not a bug in my humble opinion.

I've found the documented way to specify Ghostscript devices
without `-sDEVICE=pdfwrite`.
It is using the operator `selectdevice`.

https://www.ghostscript.com/doc/9.52/Use.htm#Output_device
https://www.ghostscript.com/doc/9.26/Use.htm#Output_device

Here is my experiment.

```
$ echo "mark /OutputFile (foo.new-documented.pdf) /PageSize [595.28 841.89] 
(pdfwrite) finddevice putdeviceprops setdevice (pdfwrite) selectdevice newpath 
fill (foo.ps) run" > gs-command-documented.ps

$ gs -dNODISPLAY -dNOSAFER -dNOPAUSE -dBATCH -dAutoRotatePages=/None 
-dPrinted=false gs-command-documented.ps
...snip...

$ qpdf --qdf foo.new-documented.pdf foo.new-documented.qdf

$ diff -u foo.old-method.qdf foo.new-documented.qdf > old-to-documented.diff

$
```

As a result, the drawing operators in PDF are the same as the old method.

Re: NR pdf is larger with current git by 15%

[Masamichi Hosoda]

>> From the same PostScript file, the new method generates a PDF of
>> 7081 bytes and the old method generates a PDF of 6423 bytes.  The
>> new method is more than 10 % larger than the old method.
> 
> Do you consider this a bug in gs or a feature?
> 
>> From `old-to-new.diff`, it seems that the amount of drawing
>> operators in the /Contents has increased significantly.
> 
> OK.  What do you think about opening a ghostscript issue, asking the
> developers for an explanation?  Maybe our new method can be slightly
> adapted to avoid the size increase.

I tried another method that adds `-sDEVICE=pdfwrite -sOutputFile=/dev/null`
and removes `-dNODISPLAY` from the new method.

```
$ gs -dNOSAFER -dNOPAUSE -dBATCH -dAutoRotatePages=/None -dPrinted=false 
-sDEVICE=pdfwrite -sOutputFile=/dev/null gs-command.ps
...snip...

$ qpdf --qdf foo.new-method.pdf foo.new-method.qdf

$ diff -u foo.old-method.qdf foo.new-method.qdf > old-to-new.diff

$
```

As a result, the drawing operators in PDF are the same as the old method.
Ghostscript seems to need `-sDEVICE=pdfwrite` to produce the PDF we expect.

Is the device specifying way on the new method (without `-sDEVICE=pdfwrite`)
documented by Ghostscript documents?
If not, it is not a bug in my humble opinion.

Re: NR pdf is larger with current git by 15%

[Masamichi Hosoda]

> your fix for gs works fine; the resulting PDFs are small again.
> Thanks a lot!
> 
> BUT: They are not as small as previously.  For example, with commit
> 21a20de3, the NR has 10 pages more and is now 8.3MByte (I've tested
> compilation with both gs 9.21 and 9.52) instead of 7.1MByte (with gs
> 9.21, produced April 20th using commit f86f7705).  As far as I can
> see, my used xetex binary is the same, as is `texinfo.tex` (version
> 2019-02-16.14), and the included fonts.
> 
> Do you have any explanation for this?  Is this expected, probably a
> side effect of your latest changes?  I uncompressed the PDFs using
> `pdftk` and did a comparison; unfortunately I couldn't see big
> differences in the diff file, which probably hints at small but many
> changes that accumulate to the 15% difference.

If I understand correctly,
it is due to the difference in the method of invoking Ghostscript.
The new method was introduced from commit 017927b4.

For example, here is my experiment with commit 21a20de3 and gs-9.52.

```
$ cat foo.ly
{ c d e f g a b }

$ lilypond -dgs-never-embed-fonts --ps --pdf foo.ly
...snip...

$ echo "mark /OutputFile (foo.new-method.pdf) /PageSize [595.28 841.89] 
(pdfwrite) finddevice putdeviceprops setdevice newpath fill (foo.ps) run" > 
gs-command.ps

$ gs -dNODISPLAY -dNOSAFER -dNOPAUSE -dBATCH -dAutoRotatePages=/None 
-dPrinted=false gs-command.ps
...snip...

$ gs -dSAFER -dNOPAUSE -dBATCH -dAutoRotatePages=/None -dPrinted=false 
-sDEVICE=pdfwrite -dDEVICEHEIGHTPOINTS=841.89 -dDEVICEWIDTHPOINTS=595.28 
-sOutputFile=foo.old-method.pdf -f foo.ps
...snip...

$ ls -l *.pdf
-rw-r--r--+ 1 trueroad none 7081 Jul 11 14:41 foo.new-method.pdf
-rw-r--r--+ 1 trueroad none 6423 Jul 11 14:41 foo.old-method.pdf
-rw-r--r--+ 1 trueroad none 7084 Jul 11 14:41 foo.pdf

$
```

>From the same PostScript file, the new method generates a PDF of 7081 bytes
and the old method generates a PDF of 6423 bytes.
The new method is more than 10 % larger than the old method.

Next, I tried to find out what the increase was.

```
$ qpdf --qdf foo.pdf foo.qdf

$ qpdf --qdf foo.new-method.pdf foo.new-method.qdf

$ qpdf --qdf foo.old-method.pdf foo.old-method.qdf

$ diff -u foo.qdf foo.new-method.qdf > direct-to-new.diff

$ diff -u foo.old-method.qdf foo.new-method.qdf > old-to-new.diff

$
```

>From `old-to-new.diff`, it seems that the amount of drawing operators
in the /Contents has increased significantly.

Re: Huge PDF doc files

[Masamichi Hosoda]

> The very problem seems to be that the PDF snippets in 'lybook-db' are
> no longer created with `-dgs-never-embed-fonts=#t`.

It seems `-dgs-never-embed-fonts=#t` does not work in current LilyPond.
I've created merge request !218 that fixes it.

Re: new procedure with GitLab CI

[Masamichi Hosoda]

> I can't tell you for sure either. It could be that the fork at 
> https://gitlab.com/trueroad/lilypond has Pipelines enabled, but only
> visible to project members. Hosoda-san, could you check this in your
> project? The drop-down is at Settings > General > Visibility ... >
> Pipelines. I'm guessing here, could be something else...

I've changed the Pipelines settings
from `Only Project Members' to `Everyone With Access'.

Re: Inline assembler fallback for _FPU_SETCW() missing in MINGW libraries (issue 577450043 by thomasmorle...@gmail.com)

[Masamichi Hosoda]

>> $ ~gub/gub/target/linux-x86/root/usr/cross/bin/i686-linux-g++ -c -msse2 
>> -mfpmath=sse -I include -I .. rational.cc
>> $ ~gub/gub/target/freebsd-x86/root/usr/cross/bin/i686-freebsd6-g++ -c -msse2 
>> -mfpmath=sse -I include -I .. rational.cc
> 
> So this only affects darwin-x86 which confirms my observation that
> linux-x86 still works. AFAICS darwin-x86 never had a fpu_control.h, so
> I'd propose to drop -msse2 -mfpmath=sse for i686-apple-darwin8.
> This unblocks the release and the situation on darwin-x86 remains the
> same as before, but we have the fix for linux-x86 and mingw.
> 
> My fear is that not only rational.cc is affected, but also many other
> files. Did you test if a full compile works? If no, I'm against adding
> workarounds for all files that suffer from the compiler error.

I didn't test full compiler works.
However, If I understand correctly,
it seems that static cast from `unsigned long long` to `double`
is only in rational.cc.

Re: Inline assembler fallback for _FPU_SETCW() missing in MINGW libraries (issue 577450043 by thomasmorle...@gmail.com)

[Masamichi Hosoda]

>>>>> We currently have the problem that the compiler used in GUB for
>>>>> compiling 32bit binaries gets an internal compiler fault for those
>>>>> options.
>>>>> 
>>>>> We'll need to figure out whether a newer compiler does the trick, and if
>>>>> it does, update GUB.  Or find a different way of proceeding.
>>>>> 
>>>>> I'll see whether I can convince my current compilers to generate 32bit
>>>>> code and see whether those are currently up to using those options.  If
>>>>> current compilers don't want them, we'll need to revert to a different
>>>>> plan.
>>>>
>>>> It seems that static cast from `unsigned long long` to `double`
>>>> by x86 SSE2 raises the internal compile error.
>>>> However, static cast from `signed long long` to `double`
>>>> does not raise the errir.
>>>> I think it can be a workaround for rational.cc.
>>>>
>>>> i.e.
>>>> First, static cast from `unsigned long long` to `signed long long`
>>>> Then, static cast from `singed long long` to `double`
>>> 
>>> Oh wow.  I would never have thought one could identify something as
>>> specific.  I think I have convinced my system to build a 32bit Guile,
>>> but have problems convincing LilyPond to do the same.  So I cannot help
>>> with debugging this situation yet or even finding out whether it
>>> persists into newer compiler versions.
>>
>> I've attached the workaround patch for stable/2.20 branch.
>> The results of my experiment is here.
> 
> 
>>>From 2a3816e49067e026c7180dc6a3b2d5614d9c20d6 Mon Sep 17 00:00:00 2001
>> From: Masamichi Hosoda 
>> Date: Tue, 4 Feb 2020 23:30:29 +0900
>> Subject: [PATCH] Add workaround for avoiding GUB darwin-x86 error
>>
>> In GUB, g++ 4.9.4 for darwin-x86 (macOS x86),
>> it seems that static cast from  `unsigned long long` to `double`
>> by x86 SSE2 raises an internal compile error.
>> However, static cast from `signed long long` to `double`
>> does not raise the error.
>> So we use it for a workaround.
>>
>> i.e.
>> First, static cast from `unsigned long long` to `signed long long`.
>> Then, static cast from `singed long long` to `double`.
>> ---
>>  flower/rational.cc | 16 
>>  1 file changed, 16 insertions(+)
>>
>> diff --git a/flower/rational.cc b/flower/rational.cc
>> index 559e1646a0..9435edbb8f 100644
>> --- a/flower/rational.cc
>> +++ b/flower/rational.cc
>> @@ -31,7 +31,23 @@ double
>>  Rational::to_double () const
>>  {
>>if (sign_ == -1 || sign_ == 1 || sign_ == 0)
>> +// FIXME: workaround: In GUB, g++ 4.9.4 for darwin-x86,
>> +// it seems that static cast from `unsigned long long` to `double`
>> +// by x86 SSE2 raises an internal compile error.
>> +// However, static cast from `signed long long` to `double`
>> +// does not raise the error.
>> +// So we use it for a workaround.
>> +#if defined (__i386__) && defined (__APPLE__) && \
>> +  defined (__SSE2_MATH__) && __GNUC__ < 5
> 
> Wouldn't the same problem occur on Windows?  We have 32bit executables
> there as well.  Or is the compiler version we use there not afflicted?

Sorry, I forgot MinGW.
The internal compiler error seems to raise only in darwin-x86.
I don't have newer g++ for darwin-x86, so I'm not sure
if it was fixed in the newer g++.

without the patch:

```
$ ~gub/gub/target/darwin-x86/root/usr/cross/bin/i686-apple-darwin8-g++ -c 
-msse2 -mfpmath=sse -I include -I .. rational.cc
rational.cc:36:5: warning: floating constant exceeds range of 'double' 
[-Woverflow]
 return -HUGE_VAL;
 ^
rational.cc:38:5: warning: floating constant exceeds range of 'double' 
[-Woverflow]
 return HUGE_VAL;
 ^
rational.cc: In member function 'double Rational::to_double() const':
rational.cc:43:1: internal compiler error: in gen_reg_rtx, at emit-rtl.c:838
 }
 ^
0x773934 gen_reg_rtx(machine_mode)
/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/emit-rtl.c:838
0xc53c53 gen_split_4130(rtx_def*, rtx_def**)

/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/config/i386/sse.md:884
0x7772a7 try_split(rtx_def*, rtx_def*, int)
/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/emit-rtl.c:3444
0x8feb71 split_insn
/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/recog.c:2897
0x9034e4 split_all_insns()
/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/recog.c:2987
0x903578 rest_of_handle_split_after_reload
/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4

Re: Inline assembler fallback for _FPU_SETCW() missing in MINGW libraries (issue 577450043 by thomasmorle...@gmail.com)

[Masamichi Hosoda]

>>> We currently have the problem that the compiler used in GUB for
>>> compiling 32bit binaries gets an internal compiler fault for those
>>> options.
>>> 
>>> We'll need to figure out whether a newer compiler does the trick, and if
>>> it does, update GUB.  Or find a different way of proceeding.
>>> 
>>> I'll see whether I can convince my current compilers to generate 32bit
>>> code and see whether those are currently up to using those options.  If
>>> current compilers don't want them, we'll need to revert to a different
>>> plan.
>>
>> It seems that static cast from `unsigned long long` to `double`
>> by x86 SSE2 raises the internal compile error.
>> However, static cast from `signed long long` to `double`
>> does not raise the errir.
>> I think it can be a workaround for rational.cc.
>>
>> i.e.
>> First, static cast from `unsigned long long` to `signed long long`
>> Then, static cast from `singed long long` to `double`
> 
> Oh wow.  I would never have thought one could identify something as
> specific.  I think I have convinced my system to build a 32bit Guile,
> but have problems convincing LilyPond to do the same.  So I cannot help
> with debugging this situation yet or even finding out whether it
> persists into newer compiler versions.

I've attached the workaround patch for stable/2.20 branch.
The results of my experiment is here.

Current stable/2.20 bcad34e31d7866eb126e73abc89eeeb01faf081f without the patch:

```
$ ~gub/gub/target/darwin-x86/root/usr/cross/bin/i686-apple-darwin8-g++ -c 
-msse2 -mfpmath=sse -I include -I .. rational.cc
rational.cc: In member function 'double Rational::to_double() const':
rational.cc:48:1: internal compiler error: in gen_reg_rtx, at emit-rtl.c:838
 }
 ^
0x773934 gen_reg_rtx(machine_mode)
/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/emit-rtl.c:838
0xc53c53 gen_split_4130(rtx_def*, rtx_def**)

/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/config/i386/sse.md:884
0x7772a7 try_split(rtx_def*, rtx_def*, int)
/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/emit-rtl.c:3444
0x8feb71 split_insn
/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/recog.c:2897
0x9034e4 split_all_insns()
/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/recog.c:2987
0x903578 rest_of_handle_split_after_reload
/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/recog.c:3938
0x903578 execute
/home/gub/gub/target/darwin-x86/src/cross/gcc-4.9.4/gcc/recog.c:3967
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
$ ~gub/gub/target/linux-x86/root/usr/cross/bin/i686-linux-g++ -c -msse2 
-mfpmath=sse -I include -I .. rational.cc
$ ~gub/gub/target/freebsd-x86/root/usr/cross/bin/i686-freebsd6-g++ -c -msse2 
-mfpmath=sse -I include -I .. rational.cc
$
```

Current stable/2.20 bcad34e31d7866eb126e73abc89eeeb01faf081f with the patch:

```
$ ~gub/gub/target/darwin-x86/root/usr/cross/bin/i686-apple-darwin8-g++ -c 
-msse2 -mfpmath=sse -I include -I .. rational.cc
rational.cc:52:5: warning: floating constant exceeds range of 'double' 
[-Woverflow]
 return -HUGE_VAL;
 ^
rational.cc:54:5: warning: floating constant exceeds range of 'double' 
[-Woverflow]
 return HUGE_VAL;
 ^
$ ~gub/gub/target/linux-x86/root/usr/cross/bin/i686-linux-g++ -c -msse2 
-mfpmath=sse -I include -I .. rational.cc
$ ~gub/gub/target/freebsd-x86/root/usr/cross/bin/i686-freebsd6-g++ -c -msse2 
-mfpmath=sse -I include -I .. rational.cc
$
```
>From 2a3816e49067e026c7180dc6a3b2d5614d9c20d6 Mon Sep 17 00:00:00 2001
From: Masamichi Hosoda 
Date: Tue, 4 Feb 2020 23:30:29 +0900
Subject: [PATCH] Add workaround for avoiding GUB darwin-x86 error

In GUB, g++ 4.9.4 for darwin-x86 (macOS x86),
it seems that static cast from  `unsigned long long` to `double`
by x86 SSE2 raises an internal compile error.
However, static cast from `signed long long` to `double`
does not raise the error.
So we use it for a workaround.

i.e.
First, static cast from `unsigned long long` to `signed long long`.
Then, static cast from `singed long long` to `double`.
---
 flower/rational.cc | 16 
 1 file changed, 16 insertions(+)

diff --git a/flower/rational.cc b/flower/rational.cc
index 559e1646a0..9435edbb8f 100644
--- a/flower/rational.cc
+++ b/flower/rational.cc
@@ -31,7 +31,23 @@ double
 Rational::to_double () const
 {
   if (sign_ == -1 || sign_ == 1 || sign_ == 0)
+// FIXME: workaround: In GUB, g++ 4.9.4 for darwin-x86,
+// it seems that static cast from `unsigned long long` to `double`
+// by x86 SSE2 raises an internal compile error.
+// However, static cast from `signed long long` to `double`
+// does not raise the erro

Re: Inline assembler fallback for _FPU_SETCW() missing in MINGW libraries (issue 577450043 by thomasmorle...@gmail.com)

[Masamichi Hosoda]

> We currently have the problem that the compiler used in GUB for
> compiling 32bit binaries gets an internal compiler fault for those
> options.
> 
> We'll need to figure out whether a newer compiler does the trick, and if
> it does, update GUB.  Or find a different way of proceeding.
> 
> I'll see whether I can convince my current compilers to generate 32bit
> code and see whether those are currently up to using those options.  If
> current compilers don't want them, we'll need to revert to a different
> plan.

It seems that static cast from `unsigned long long` to `double`
by x86 SSE2 raises the internal compile error.
However, static cast from `signed long long` to `double`
does not raise the errir.
I think it can be a workaround for rational.cc.

i.e.
First, static cast from `unsigned long long` to `signed long long`
Then, static cast from `singed long long` to `double`

Re: Inline assembler fallback for _FPU_SETCW() missing in MINGW libraries (issue 577450043 by thomasmorle...@gmail.com)

[Masamichi Hosoda]

> Nothing?  I am currently proposing that we compile with
> 
> -mfpmath=sse -msse2
> 
> as options which should shift arithmetic off to the SSE2 instruction set
> which doesn't work with 80-bit arithmetic.  That would be a default way
> of compilation.

I've created a patch for master.

https://sourceforge.net/p/testlilyissues/issues/5725/
https://codereview.appspot.com/565600045/

Re: Inline assembler fallback for _FPU_SETCW() missing in MINGW libraries (issue 577450043 by thomasmorle...@gmail.com)

[Masamichi Hosoda]

>>> You provided a lot of good information in your post, but the
>>> conclusion was not entirely clear.
>>> Are you suggesting requiring SSE2 at this time?
>> 
>> Yes.  It appears to get used anyway for 64bit executables, and it seems
>> safe enough to demand it for 32bit executables.
> 
> +1
> far better than inline asm poking control registers

The cause of the issues is that
the unexpected precision conversions raises small difference
in floating-point calculation.

If libguile uses x87 FPU, high-precision calculation
and low-precision calculation coexist inside GUILE,
and precision conversion occurs between them.

Even if C++ uses SSE2,
libguile and other shared libraries still may use x87 FPU.

I wonder this may cause problems.

Re: Inline assembler fallback for _FPU_SETCW() missing in MINGW libraries (issue 577450043 by thomasmorle...@gmail.com)

[Masamichi Hosoda]

>> This sounds promising.
>>
>>>   -fexcess-precision=standard is not implemented for languages
>>>   other than C.
>>
>> Never mind.
> 
> Hm.  Maybe
> 
> -mfpmath=sse
> 
> instead?  The problem with switching the whole FPU to reduced precision
> is that some library functions might not expect that, so it is making me
> queasy.  On the other hand, using SSE might have a negative performance?
> I just don't have a good idea what we are dealing with here.

In my experiment, `-fexcess-precision=standard` cannot be used for C++.

```
$ g++ -fexcess-precision=standard test.cc
cc1plus: sorry, unimplemented: -fexcess-precision=standard for C++

$ g++ --version
g++ (GCC) 7.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


$
```

On the other hand, `-msse -mfpmath=sse` uses SSE,
which can only perform single-precision floating-point calculation.
SSE is too low precision.

`-msse2 -mfpmath=sse` uses SSE2,
which can perform double-precision floating-point calculation.
Precision is sufficient, but older 32-bit x86 CPUs do not have SSE2.

Re: Inline assembler fallback for _FPU_SETCW() missing in MINGW libraries (issue 577450043 by thomasmorle...@gmail.com)

[Masamichi Hosoda]

>>   -fexcess-precision=style
>>   This option allows further control over excess precision on
>>   machines where floating-point operations occur in a format
>>   with more precision or range than the IEEE standard and
>>   interchange floating-point types.  By default,
>>   -fexcess-precision=fast is in effect; this means that
>>   operations may be carried out in a wider precision than the
>>   types specified in the source if that would result in faster
>>   code, and it is unpredictable when rounding to the types
>>   specified in the source code takes place.
> 
> This sounds promising.

If I understand correctly, such compiler options are not promising.

Even if LilyPond C++ source codes are compiled with such option,
libguile without the option may calculate in a wider precision.
In this case, the floating point calculation
inside GUILE causes these issues.

Re: 2.91.84 - windows-only bugs

[Masamichi Hosoda]

> I ask because, in the german forum Arnold found a method to cure some
> windows-only bugs., about mis-predicted force and probably several
> assertion-failures:
> https://lilypondforum.de/index.php/topic,609.msg3463.html#msg3463

The patch is very interesting.

I've tried x87mant53.dll showed in the german forum.
It solved not only Issue 4943 but also Issue 4975.

I guess that these issues do not only occur on Windows,
but on all x86 32 bit platforms except Linux.
Therefore, I think that `defined (__MINGW32__)` is not necessary.

Re: build problem

[Masamichi Hosoda]

> Hello,
> 
> Since I can't get a successful doc-build on my Fedora 30 box, I'm
> unable to check doc-changes and merge translation into stable.
> 
> It's the same with extractpdfmark 1.0.2 1.0.3 and 1.1.0
> 
> log spits:
> 
> extractpdfmark -o ./out-www/collated-files.pdfmark
> ./out-www/collated-files.tmp.pdf
> extractpdfmark: error while loading shared libraries:
> libpoppler.so.78: cannot open shared object file: No such file or
> directory
> 
> And I won't try to package poppler-0.78. There are too many
> dependencies for me (Fedora is stuck to 0.73).

In Fedora 30, I suggest to use extractpdfmark from package,
rather than self-built extractpdfmark.
https://apps.fedoraproject.org/packages/extractpdfmark

If I understand correctly,
Fedora 30 has popler-0.73.0 package that has libpoppler.so.84.
https://koji.fedoraproject.org/koji/rpminfo?rpmID=17656985
Fedora 29 has popler-0.67.0 package that has libpoppler.so.78.
https://koji.fedoraproject.org/koji/rpminfo?rpmID=17657086

Did you build extractpdfmark on Fedora 29?
If so, it depends on libpoppler.so.78.
However, Fedora 30 does not have libpoppler.so.78.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Please test new lilypond installers

[Masamichi Hosoda]

> $ uname -a
> FreeBSD freebsd-hyperv 12.0-RELEASE FreeBSD 12.0-RELEASE r341666 GENERIC  
> amd64

> ld-elf.so.1: Shared object "libm.so.4" not found, required by "lilypond"
> 
> Are there any further efforts anyone would like to see made here?

Maybe, installing compat6x package is required.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: I cannot run make check since Issue 5450: relocate.cc: Introduce new command `set?'

[Masamichi Hosoda]

>>> Is dir guaranteed to be a relative path?
>>
>> It seems that there is no guarantee.
>> But, `make check` invokes `output-distance` with relative paths.
>> http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=blob;f=GNUmakefile.in;h=da1f6f64e088756e07d20f8e1603ccc3a102053e;hb=HEAD#l340
>>
>> Therefore, `dir` is also a relative path.
>> There seems to be no place invoking `output-distance` other than that.
> 
> Python has os.path.join that should allow to combine cur_dir with either
> relative or absolute paths.

Thank you for your advice.
But, Werner's Issue 5466 uses `abs_dir` by `os.path.abspath ()`.
So I've uploaded Patch Set 2 using `abs_dir`.
It requires Issue 5466.

https://sourceforge.net/p/testlilyissues/issues/5467/
https://codereview.appspot.com/349110043

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: I cannot run make check since Issue 5450: relocate.cc: Introduce new command `set?'

[Masamichi Hosoda]

>>> This is a gs 9.26 issue and I cannot see how this might be related to what 
>>> we have hit here, so maybe Hosoda-san will be abel to figure why make check 
>>> is breaking without extractpdfmark installed.
>>
>> If I understand correctly, this patch solves the error.
>>
>> ```
>> --- a/scripts/build/output-distance.py
>> +++ b/scripts/build/output-distance.py
>> @@ -652,7 +652,7 @@ class SignatureFileLink (FileLink):
>>  data_option = ''
>>  if options.local_data_dir:
>>  data_option = 
>> ('-slilypond-datadir=%s/share/lilypond/current '
>> -   % dir)
>> +   % (cur_dir + '/' + dir))
>>
> 
> Is dir guaranteed to be a relative path?

It seems that there is no guarantee.
But, `make check` invokes `output-distance` with relative paths.
http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=blob;f=GNUmakefile.in;h=da1f6f64e088756e07d20f8e1603ccc3a102053e;hb=HEAD#l340

Therefore, `dir` is also a relative path.
There seems to be no place invoking `output-distance` other than that.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: I cannot run make check since Issue 5450: relocate.cc: Introduce new command `set?'

[Masamichi Hosoda]

> This is a gs 9.26 issue and I cannot see how this might be related to what we 
> have hit here, so maybe Hosoda-san will be abel to figure why make check is 
> breaking without extractpdfmark installed.

If I understand correctly, this patch solves the error.

```
--- a/scripts/build/output-distance.py
+++ b/scripts/build/output-distance.py
@@ -652,7 +652,7 @@ class SignatureFileLink (FileLink):
 data_option = ''
 if options.local_data_dir:
 data_option = 
('-slilypond-datadir=%s/share/lilypond/current '
-   % dir)
+   % (cur_dir + '/' + dir))

 cmd = ('gs'
' -sDEVICE=png16m'
```

Without extractpdfmark,
each eps file has relative font path from `lilypond-datadir`.
e.g.

```
%%BeginProlog
/lilypond-datadir where {pop} {userdict /lilypond-datadir 
(/path/to/lilypond/build-without-extractpdfmark/out/share/lilypond/current) put 
} ifelse
%%BeginFont: Emmentaler-20
lilypond-datadir (/fonts/otf/emmentaler-20.otf) concatstrings (r) file .loadfont
%%EndFont
```

Then, `output-distance.py` invoked gs with commandline option
`-slilypond-datadir=input/regression/out-test-baseline/share/lilypond/current`.

It specified `lilypond-datadir` as relative path
from `/path/to/lilypond/build-without-extractpdfmark`.
However, current directory was
`/path/to/lilypond/build-without-extractpdfmark/input/regression/out-test-baseline/share/lilypond/current`
because 
http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=blob;f=scripts/build/output-distance.py;h=ba787931a5edb6f1f2f46e6c53e851241ba4b9d1;hb=HEAD#l648
changes current directory.
Since current directory was different from the origin of the relative path,
gs could not find the font file.

My patch shown above fixes the wrong relative path specification.

On the other hand, with extractpdfmark,
each eps file has absolute font path.
e.g.

```
%%BeginProlog
/lilypond-datadir where {pop} {userdict /lilypond-datadir 
(/path/to/lilypond/build-with-extractpdfmark/out/share/lilypond/current) put } 
ifelse
%%BeginFont: Emmentaler-20
(/path/to/lilypond/build-with-extractpdfmark/out/share/lilypond/current/fonts/otf/emmentaler-20.otf)
 (r) file .loadfont
%%EndFont
```

In this case,
even if `lilypond-datadir` is specified as such a wrong relative directory,
gs can find the font file.
However, I think absolute font path in eps is undesirable behavior.
So, I've created Issue 5467.
https://sourceforge.net/p/testlilyissues/issues/5467/
https://codereview.appspot.com/349110043/

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: gub: I can now completely build lilypond

[Masamichi Hosoda]

 But building of lilypond-test fails.

 ./target/linux-64/build/lilypond-git.sv.gnu.org--lilypond.git-master/input/regression/lilypond-book/suffix-tely.texi2pdf.log:

 /home/gub/NewGub/gub/target/tools/root/usr/bin/texi2dvi: texinfo.tex
 appears to be broken.
 This may be due to the environment variable TEX set to something
 other than (plain) tex, a corrupt texinfo.tex file, or
 to tex itself simply not working.
 etex:
 /home/gub/NewGub/gub/target/linux-64/root/usr/lib/libstdc++.so.6:
 version `CXXABI_1.3.9' not found (required by etex)
>>> Obviously another linking path issue: While executing the external TeX
>>> binaries, DLL stuff from `target/...' must not be used.
>> 
>> The problem here is that our texi2* stuff needs LD_LIBRARY_PATH to
>> find our DLLs in target/..., but the system's etex binary, executed by
>> texi2*, fails on openSuSE Tumbleweed because it is incompatible to
>> those libs in target/
>> 
>> I think the easiest solution is a simple etex wrapper script in
>> tools/root that clears LD_LIBRARY_PATH and then executes the system's
>> etex.
> 
> texi2dvi / texi2pdf can be specified external program name.
> Previously, there was a similar error in XeTeX with texi2pdf.
> 
> For XeTeX, we use wrapper scripts.
> http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=blob;f=scripts/build/xetex-with-options.sh
> http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=blob;f=scripts/build/xelatex-with-options.sh
> 
> In the LilyPond's makefiles, it is specified as follows.
> 
> ```
> PDFTEX=$(PDFTEX) PDFLATEX=$(PDFLATEX)  ... $(LILYPOND_BOOK)
> ```
> 
> The valiables, PDFTEX and PDFLATEX are set to be
> xetex-with-options.sh and xelatex-with-options.sh at configure script.
> 
> To avoid the error,
> I've added a line to clear LD_LIBRARY_PATH to the scripts in GUB.
> https://github.com/gperciva/gub/commit/57d66d1262551c062a3edc85e27ac69748f2f136#diff-94f19c1786c0853ef47371aa55ba2b21
> 
> If I understand correctly,
> it may be resolved simply by changing LilyPond's makefiles as follows.
> 
> ```
> TEX=$(PDFTEX) PDFTEX=$(PDFTEX) PDFLATEX=$(PDFLATEX)  ... $(LILYPOND_BOOK)
> ```

I noticed I was wrong...
texi2dvi / texi2pdf should not invoke etex in GUB.
xetex should be invoked.

Does your openSuSE system have both xetex and xelatex?
LilyPond's configure script sets PDFTEX=/path/to/xetex-with-option.sh
if your openSuSE system has both xetex and xelatex.
Then, texi2pdf invokes xetex-with-option.sh instead of etex.
In this case, etex does not be invoked.
The test succeeds because GUB makes
xetex-with-option.sh clears LD_LIBRARY_PATH.

On the other hand,
LilyPond's configure script sets PDFTEX=etex
if your openSuSE system does not have xelatex, pdfetex, and pdftex.
In this case, texi2pdf invokes etex. LD_LIBRARY_PATH is not cleared.
GUB has older libc and older libstdc++.
LD_LIBRARY_PATH makes etex uses the old libc and the old libstdc++.
But, openSuSE system's etex requires newer libc and newer libstdc++.
This is the cause of the error.

Again, if I understand correctly,
etex is not invoked if your openSuSE system have both xetex and xelatex.
If your openSuSE system has both xetex and xelatex,
GUB clears LD_LIBRARY_PATH and the test will succeed.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: gub: I can now completely build lilypond

[Masamichi Hosoda]

> Hi Werner!
>>> But building of lilypond-test fails.
>>>
>>> ./target/linux-64/build/lilypond-git.sv.gnu.org--lilypond.git-master/input/regression/lilypond-book/suffix-tely.texi2pdf.log:
>>>
>>> /home/gub/NewGub/gub/target/tools/root/usr/bin/texi2dvi: texinfo.tex
>>> appears to be broken.
>>> This may be due to the environment variable TEX set to something
>>> other than (plain) tex, a corrupt texinfo.tex file, or
>>> to tex itself simply not working.
>>> etex:
>>> /home/gub/NewGub/gub/target/linux-64/root/usr/lib/libstdc++.so.6:
>>> version `CXXABI_1.3.9' not found (required by etex)
>> Obviously another linking path issue: While executing the external TeX
>> binaries, DLL stuff from `target/...' must not be used.
> 
> The problem here is that our texi2* stuff needs LD_LIBRARY_PATH to
> find our DLLs in target/..., but the system's etex binary, executed by
> texi2*, fails on openSuSE Tumbleweed because it is incompatible to
> those libs in target/
> 
> I think the easiest solution is a simple etex wrapper script in
> tools/root that clears LD_LIBRARY_PATH and then executes the system's
> etex.

texi2dvi / texi2pdf can be specified external program name.
Previously, there was a similar error in XeTeX with texi2pdf.

For XeTeX, we use wrapper scripts.
http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=blob;f=scripts/build/xetex-with-options.sh
http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=blob;f=scripts/build/xelatex-with-options.sh

In the LilyPond's makefiles, it is specified as follows.

```
PDFTEX=$(PDFTEX) PDFLATEX=$(PDFLATEX)  ... $(LILYPOND_BOOK)
```

The valiables, PDFTEX and PDFLATEX are set to be
xetex-with-options.sh and xelatex-with-options.sh at configure script.

To avoid the error,
I've added a line to clear LD_LIBRARY_PATH to the scripts in GUB.
https://github.com/gperciva/gub/commit/57d66d1262551c062a3edc85e27ac69748f2f136#diff-94f19c1786c0853ef47371aa55ba2b21

If I understand correctly,
it may be resolved simply by changing LilyPond's makefiles as follows.

```
TEX=$(PDFTEX) PDFTEX=$(PDFTEX) PDFLATEX=$(PDFLATEX)  ... $(LILYPOND_BOOK)
```

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: gub's `make bootstrap' is useless

[Masamichi Hosoda]

> So it seems that you can save some hours of compilation time if you
> don't build the `bootstrap' target.  I now wonder whether this is true
> on my openSuSE GNU/Linux box only ...

Sorry, I have not worked on GUB because I have no time for the time being.
Thank you for your work on GUB.

In my experience, it seems that `make lilypond' succeeded
even if I forgot `make bootstrap'.
My GUB environment is Ubuntu 14.04 64 bit.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Ghostscript 9.26 cannot embed CID fonts

[Masamichi Hosoda]

I've noticed that gs-9.26 cannot embed CID fonts
for the extractpdfmark method.

In the method, we defined fonts in PostScript, and use it in PDF.
gs-9.26 cannot define the fonts for PDF in PostScript.
https://bugs.ghostscript.com/show_bug.cgi?id=700367#c4

To define fonts for ghostscript,
root privileges seems to be required on most systems.
https://bugs.ghostscript.com/show_bug.cgi?id=700367#c6

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB

[Masamichi Hosoda]

 I found a possibility that the wrong PDF is output
 if the build directory is not clean.
>>
>> I've created the patch.
>> https://sourceforge.net/p/testlilyissues/issues/5380/
>> https://codereview.appspot.com/355750043
> 
> Thanks a lot!

If I understand correctly,
there is a possibility that intermediate PDFs are output as final PDFs
since the intermediate filename is the same as the final filename.

So I've created a patch that changes the intermediate PDF filename.
https://sourceforge.net/p/testlilyissues/issues/5381/
https://codereview.appspot.com/357760043/

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB

[Masamichi Hosoda]

> Masamichi Hosoda  writes:
> 
>>>> GUB worked for a long time, but we have a) an unsolved problem
>>>> building the pdfs of the english documentation of stable/2.20,
>>> 
>>> That sounds like more of a race condition to me, so it's likely
>>> unrelated to GUB but may be related to building in a separate directory
>>> or to cross-compilation.
>>
>> I found a possibility that the wrong PDF is output
>> if the build directory is not clean.
> 
> Like when a build aborted, maybe.
> 
>> I'll create a patch which fixes this issue.
> 
> Let's hope that this will avoid this problem in future.

I've created the patch.
https://sourceforge.net/p/testlilyissues/issues/5380/
https://codereview.appspot.com/355750043

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB

[Masamichi Hosoda]

>> GUB worked for a long time, but we have a) an unsolved problem
>> building the pdfs of the english documentation of stable/2.20,
> 
> That sounds like more of a race condition to me, so it's likely
> unrelated to GUB but may be related to building in a separate directory
> or to cross-compilation.

I found a possibility that the wrong PDF is output
if the build directory is not clean.
This issue is unrelated to GUB.
With GUB, without GUB, it might happen in both.

You can reproduce by the following procedure without GUB.

```
$ git checkout release/2.19.82-1
$ ./autogen.sh --noconf
$ rm -fr build
$ mkdir build
$ mkdir -p out-www/offline-root/Documentation/
$ echo "wrong file" > out-www/offline-root/Documentation/notation.pdf
$ mkdir -p out-www/online-root/Documentation/
$ echo "wrong file" > out-www/online-root/Documentation/notation.pdf
$ ../configure
$ make -j 8
$ make -j 8 CPU_COUNT=8 WEB_TARGETS='offline online' LANGS='' doc
```

The result is as follows.

```
$ find . -name 'notation.pdf' -exec ls -lah {} \;
-rw-r--r-- 1 trueroad none 6.5M Jul 13 22:37 
./Documentation/out-www/notation.pdf
-rw-r--r-- 1 trueroad none 11 Jul 13 21:39 
./out-www/offline-root/Documentation/notation.pdf
-rw-r--r-- 1 trueroad none 11 Jul 13 21:39 
./out-www/online-root/Documentation/notation.pdf

$ cat out-www/offline-root/Documentation/notation.pdf
wrong file

$ cat out-www/online-root/Documentation/notation.pdf
wrong file

$
```

I'll create a patch which fixes this issue.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB

[Masamichi Hosoda]

>> Ubuntu 14 for both.  I would expect Ubuntu 16 to work but have not tried it.
> 
> Not sure about that.  We had several incompatibilities in the LilyPond
> code base since then (2.18.2 does not compile on current GCC versions)
> and GCC is written in C++ these days.  Though the bootstrap process
> might actually go through C-written GCC to get to its C++ compiler: not
> sure about that.

I tried to use GUB in Ubuntu 16.04 LTE 64 bit.
But It failed with some errors.

So I use GUB in Ubuntu 14.04 LTE 64 bit.
I have not tried Ubuntu 18.04 LTE yet.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: speed of fontconfig

[Masamichi Hosoda]

 [... replacing] 2.12.6 [... with] 2.12.93 indeed provides ~100x
 improvement and builds the cache on my system in 600ms.
>>>
> 
>>> This was a report of using fontconfig on Windows, IIRC.  So my
>>> question: Does the Windows tarball of lilypond contain a recent
>>> fontconfig version?
>> 
>> According to
>> https://github.com/gperciva/gub/blob/master/gub/specs/fontconfig.py
>> it's 2.12.1.
> 
> Should we upgrade?  Masamichi-san?

For LilyPond 2.21, we should upgrade fontconfig, in my humble opinion.
But, for LilyPond 2.20, I'm wondering whether we should upgrade or not.

If I understand correctly, the new fontconfig has significant changes.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Ghostscript/GhostPDL 9.22 Release Candidate 1

[Masamichi Hosoda]

> Note that I didn't attempt this on 9.21, just the 9.22 release
> candidate. While the fonts are properly dropped from the individual
> PDF files, none of the text was visible in the final PDF file, when
> rebuilding with Emmentaler supplied as an external font in fontmap.GS.
> 
> Its interesting that the OTF worked for you because I would expect it
> to; OTF fonts with CFF outlines are handled simply be extracting the
> CFF, which is usable directly in PostSceript. TrueType fonts are not
> directly supported by PostScript and that's where the heuristics come
> back again, we have to guess at some aspects of the encodings.

In my experiment, I used gs-9.22rc1 but I didn't use fontmap.GS.

Instead, I created a PostScript file
that contains only font resource in the same format as EPSs.
That is, for the  Emmentaler-20 font,
I created a file `fonts/Emmentaler-20.font.ps` with the following contents.

```
%%BeginFont: Emmentaler-20
%%BeginResource: font Emmentaler-20
%!PS-Adobe-3.0 Resource-FontSet
%%DocumentNeededResources: ProcSet (FontSetInit)
%%Title: (FontSet/Emmentaler-20)
%%Version: 0
%%EndComments
%%IncludeResource: ProcSet (FontSetInit)
%%BeginResource: FontSet (Emmentaler-20)
/FontSetInit /ProcSet findresource begin
%%BeginData: 67355 Binary Bytes
[...snip...]
%%EndData
%%EndResource
%%EndResource
%%EndFont
```

Then, I invoked Ghostscipt with the following command.

$ gs -dBATCH -dNOPAUSE -sDEVICE=pdfwrite \
 -sOutputFile=include-bigpdfs-noembed-gse.pdf \
 -c ".setpdfwrite" \
 -f fonts/*.font.ps include-bigpdfs-noembed.pdf

`fonts/Emmentaler-20.font.ps` and other font files
`fonts/TeXGyreSchola-Regular.font.ps` etc. are in my sample tarball.

In this method, for TrueType fonts,
I think that the encoding is confusing and broken as you mentioned.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: [gs-devel] Ghostscript/GhostPDL 9.22 Release Candidate 1

[Masamichi Hosoda]

This discussion concerns two mailing lists,
some mails exist in both archives,
but other mail seems to exist only in one archive.
For convenience of people subscribing to only one mailing list,
both mailing list archive URLs are as follows.

http://lists.gnu.org/archive/html/lilypond-devel/2017-09/index.html
https://ghostscript.com/pipermail/gs-devel/2017-September/date.html

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: [gs-devel] Ghostscript/GhostPDL 9.22 Release Candidate 1

[Masamichi Hosoda]

> In my impression, if
> 
> $ gs -dSAFER -dEPSCrop -dCompatibilityLevel=1.4 -dNOPAUSE -dBATCH
> -r1200
> -dSubsetFonts=false -sDEVICE=pdfwrite -dAutoRotatePages=/None
> -sOutputFile=filename.pdf -c.setpdfwrite -ffilename.eps
> 
> does not embed the completed font into PDF, Ghostscript developers
> would
> regard it as a bug and they might be willing to fix it (at least, try
> to
> fix it).
> 
> Could you provide some sample EPS files to reproduce the issue,
> and the resulted PDF in your environment?

Here is sample files `20170922_lilypond_eps_pdf_examples.tar.xz`
https://drive.google.com/file/d/0ByGBX3PDrqjsSFhVdXJfbjFjRlk/view?usp=sharing

It contains `README.md`.
Would you read the file?

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Ghostscript/GhostPDL 9.22 Release Candidate 1

[Masamichi Hosoda]

>>If there is a full font embedded (non-subset) PDF,
>>does the bigpdfs trick work effectively?
> 
> Its still, in my opinion, a risky thing to do. However, if the font
> were fully embedded, you wouldn't need to use Ghostscript and the
> PDFDontUseFontObjectNum bug approach (which is the risky part).
> 
> Because the fonts would be genuinely identical, MuPDF would be able to
> spot the font streams at least as being the same and would be able to
> reliably remove the duplicates.
> 
> The Font and FontDescriptor dictionaries might not be possible to
> remove, so the effect wouldn't be quite as good as the current
> approach, but these dictionaries run to a few tens or at most a few
> hundred bytes. The FontFile streams are where most of the space is
> going, and those would be possible to remove, if they were truly
> duplicates.

I understand that the bigpdfs trick should to be fixed
to use MuPDF instead of PDFDontUseFontObjectNum bug.

>>Or, even so, should we take other methods (e.g. using non-embedded
>>PDFs)?
> 
> I'm not sure what other methods there would be. Using EPS inclusions
> would have the same effect as PDF. Rendering to bitmaps would be (I'd
> guess) as large as the PDF files, and would suffer from
> non-scalability.
> 
> Converting to TeX format would probably work, but apparently there
> were problems with that.
> 
> Is there some other approach available ?

There is a method of using font non-embedded PDF.
In my experiment, it seems to work fine except TrueType fonts.

It uses like
`-c ".setpdfwrite << /NeverEmbed [ /Emmentaler-20 /TeXGyreSchola-Regular ] >> 
setdistillerparams"`
instead of
`-sOutputFile=filename.pdf`.

I've made sample files `20170922_lilypond_eps_pdf_examples.tar.xz`.
https://drive.google.com/file/d/0ByGBX3PDrqjsSFhVdXJfbjFjRlk/view?usp=sharing

In the tarball:

`fonts?-bigpdfs.eps`: EPS that is generated by LilyPond.
`fonts?-bigpdfs-noembed.pdf`: No font embedded PDF
  that is generated by Ghostscript
  with `/NeverEmbed` parameter like above.
`include-bigpdfs-noembed.pdf`: TeX outputted PDF that lacks LilyPond fonts.
`include-bigpdfs-noembed-gse.pdf`: Finally PDF that is font embedded
   by Ghostscript.

In `include-bigpdfs-noembed-gse.pdf`,
OTF seems to be fine. However, some glyphs of TTF are broken.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Ghostscript/GhostPDL 9.22 Release Candidate 1

[Masamichi Hosoda]

>>We use the following command to convert from EPS to PDF.
>>
>>$ gs -dSAFER -dEPSCrop -dCompatibilityLevel=1.4 -dNOPAUSE -dBATCH
>>-r1200 -dSubsetFonts=false -sDEVICE=pdfwrite -dAutoRotatePages=/None
>>-sOutputFile=filename.pdf -c.setpdfwrite -ffilename.eps
>>
>>We believed that Ghostscript generates full font embedded (non-subset)
>>PDF
>>when `-dSubsetFonts=false` is specified.
> 
> It should. In this case, I believe it does not. No clue why at the
> moment though.

Thank you for your answer.

If there is a full font embedded (non-subset) PDF,
does the bigpdfs trick work effectively?
Or, even so, should we take other methods (e.g. using non-embedded PDFs)?

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Ghostscript/GhostPDL 9.22 Release Candidate 1

[Masamichi Hosoda]

> So the question then becomes 'why are the fonts subset ?' That's a
> really good question, and the answer is that I don't know. Its
> possible that there is a genuine pdfwrite bug here. The piece of
> information I'm missing is the step used to create the PDF files from
> the EPS files, I don't know how you are doing that.

We use the following command to convert from EPS to PDF.

$ gs -dSAFER -dEPSCrop -dCompatibilityLevel=1.4 -dNOPAUSE -dBATCH -r1200 
-dSubsetFonts=false -sDEVICE=pdfwrite -dAutoRotatePages=/None 
-sOutputFile=filename.pdf -c.setpdfwrite -ffilename.eps

We believed that Ghostscript generates full font embedded (non-subset) PDF
when `-dSubsetFonts=false` is specified.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: [gs-devel] Ghostscript/GhostPDL 9.22 Release Candidate 1

[Masamichi Hosoda]

>> > Please give them a try on your system if you're interested in helping
>> > test the release-in-progress. Your feedback is appreciated.
>>
>>It seems that `-dPDFDontUseFontObjectNum` option does not work.
> 
> It has been removed, as documented in History9.htm:
> 
> 2017-08-02 13:41:59 +0100
> Ken Sharp 
> ca1ec9b486ddba3f921355fd1d775f27f4871356
> 
> PDF interpreter - remove the PDFDontUseObjectNum switch
> 
> This was implemented to allow us to restore the default behaviour if
> it caused problems. No real problems reported, so lets get rid of
> (yet another) of our many, many command line switches.
> 
> 
>>`-dPDFDontUseFontObjectNum` is useful to unify duplicate fonts.
>>So I would like to use `-dPDFDontUseFontObjectNum`.
> 
> The fonts can't realistically be described as duplicates, if they have
> different object numbers. Or if they are then the original PDF file
> (this only works with PDF input) was poorly constructed. Using this
> option isn't really sensible, it was only ever present in case the
> Font numbering usage went awry. Consider creating the original PDF
> files more efficiently instead.
> 
> I'll discuss it with the other developers but I am not inclined to
> restore this. Obviously if you feel its important you can revert the
> change locally.

When you create a PDF document using something like a TeX system
you may include many small PDF files in the main PDF file.
It is common for each of the small PDF files to use the same fonts.

If the small PDF files contain embedded full font sets,
the TeX system includes all of them in the main PDF.
The main PDF contains duplicates of the same full sets of fonts.
Therefore, `PDFDontUseFontObjectNum` can remove the duplicates.
This may considerably reduce the main PDF-file's size.

There is a tool for using this method of removing duplicate fonts.
https://www.ctan.org/pkg/extractpdfmark
https://packages.debian.org/stretch/extractpdfmark
http://packages.ubuntu.com/zesty/extractpdfmark

LilyPond has option `--bigpdfs` for unifying duplicate fonts in this method.
http://lilypond.org/doc/v2.19/Documentation/usage/command_002dline-usage#basic-command-line-options-for-lilypond

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: ghostscript 9.22 will remove PDFDontUseFontObjectNum option

[Masamichi Hosoda]

>> I've noticed that ghostscript 9.22
>> will remove `PDFDontUseFontObjectNum` option.
>>
>> http://git.ghostscript.com/?p=ghostpdl.git;a=commit;h=ca1ec9b486ddba3f921355fd1d775f27f4871356
>>
>> So `--bigpdfs` will not work with gs-9.22.
>> It already did not work with gs-9.22rc1.
> 
> Did they give a rationale?  What kind of PostScript code would have a
> chance to work here instead?

I have no idea.
I've send a request that
I'd like to use `PDFDontUseFontObjectNum` option to gs-devel.
https://ghostscript.com/pipermail/gs-devel/2017-September/009987.html

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

ghostscript 9.22 will remove PDFDontUseFontObjectNum option

[Masamichi Hosoda]

I've noticed that ghostscript 9.22
will remove `PDFDontUseFontObjectNum` option.

http://git.ghostscript.com/?p=ghostpdl.git;a=commit;h=ca1ec9b486ddba3f921355fd1d775f27f4871356

So `--bigpdfs` will not work with gs-9.22.
It already did not work with gs-9.22rc1.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

translation branch and stable/2.20 branch

[Masamichi Hosoda]

2.19.65 has been released,
it seems that it does not contain some recent translations.

e.g.
eb38c33a95 Doc-ja: fix cindex in NR
eccad9a9f8 Doc-ja: update Usage
1c1a932358 Doc-es: update one texidoc.
e7385f6e58 Web-es: update Introduction.
2cfed6cc93 Doc-es: update Notation/Simultaneous.
9f53d74eaf Web: Add a new document to what We Wrote list.
ce1c9d0ea4 Doc-ca: Catalan translation of the notation manual: expressive.itely 
and text.itely
etc.

Is the translation branch not contained in the release?
Or will it be contained in the next release?

On the other hand,
the current "translation" branch does not seem to contain
recent document updates.

e.g.
0712559601 Issue 5164: Doc: Add usage of OpenType font feature
97956152eb Issue 5166/2: Doc: Delete note for svg backend which become 
unnecessary
f40aa7dc74 Issue 5166/1: Doc: Update default font description
0e23f12caa Issue 5163: Update URW++ OTF files
68dbc841e3 Doc: CG, explain other git prompt variables
54a35cdc57 Web: Authors.itexi update
2dcca6dacd Doc: add comment about automatically generated documentation
etc.

Which branch should the translators translate?
e.g. translate the English document in the master branch,
commit to the translation branch, etc.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB Ghostscript 9.21

[Masamichi Hosoda]

>> Phil Holmes:
>> ...
>>> /usr/bin/ld: cannot find -lfreetype
>> ...
>> ld cannot find libfreetype.so
>> Do you have it installed ?
>> Try this:
>> $ locate libfreetype.so
>> ...
>> /usr/lib/i386-linux-gnu/libfreetype.so
>> /usr/lib/i386-linux-gnu/libfreetype.so.6
>> /usr/lib/i386-linux-gnu/libfreetype.so.6.8.1
>> ...
>> 
> 
> I get:
> 
> /home/gub/NewGub/gub/target/freebsd-64/root/usr/lib/libfreetype.so
> /home/gub/NewGub/gub/target/freebsd-64/root/usr/lib/libfreetype.so.18
> /home/gub/NewGub/gub/target/freebsd-x86/root/usr/lib/libfreetype.so
> /home/gub/NewGub/gub/target/freebsd-x86/root/usr/lib/libfreetype.so.18
> /home/gub/NewGub/gub/target/linux-64/root/usr/lib/libfreetype.so
> /home/gub/NewGub/gub/target/linux-64/root/usr/lib/libfreetype.so.6
> /home/gub/NewGub/gub/target/linux-64/root/usr/lib/libfreetype.so.6.12.5
> /home/gub/NewGub/gub/target/linux-ppc/root/usr/lib/libfreetype.so
> /home/gub/NewGub/gub/target/linux-ppc/root/usr/lib/libfreetype.so.6
> /home/gub/NewGub/gub/target/linux-ppc/root/usr/lib/libfreetype.so.6.12.5
> /home/gub/NewGub/gub/target/linux-x86/root/usr/lib/libfreetype.so
> /home/gub/NewGub/gub/target/linux-x86/root/usr/lib/libfreetype.so.6
> /home/gub/NewGub/gub/target/linux-x86/root/usr/lib/libfreetype.so.6.12.5
> /home/gub/NewGub/gub/target/tools/root/usr/lib/libfreetype.so
> /home/gub/NewGub/gub/target/tools/root/usr/lib/libfreetype.so.6
> /home/gub/NewGub/gub/target/tools/root/usr/lib/libfreetype.so.6.12.5
> /usr/lib/i386-linux-gnu/libfreetype.so.6
> /usr/lib/i386-linux-gnu/libfreetype.so.6.11.1

If I understand correctly,
your system is Ubuntu 14.04 LTS 32 bit (i386-linux-gnu).
It seems that your system missing this file:
/usr/lib/i386-linux-gnu/libfreetype.so

So `ld` cannot find `-lfreetype`.

My system (Ubuntu 14.04 LTS 64 bit, x86_64-linux-gnu) has this file:
/usr/lib/x86_64-linux-gnu/libfreetype.so

So `ld` can find `-lfreetype`.
There is no problem.

I cannot reproduce the issue. However, I've created a patch.
Would you try it?
https://github.com/gperciva/gub/pull/40

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Releasing 2.20

[Masamichi Hosoda]

> Hans Aikema <hans.aik...@aikebah.net> writes:
> 
>> David,
>>
>> Make sure that your main focus will be getting back into good shape.
>>
>> If schedule and energy levels allow for starting of a 2.20 branch I
>> think it would be good to update ghostscript to a version that has the
>> PNG-transparency glitches fixed that surfaced in 2.19.51.
>> Ghostscript has made a release with the fix for the issue that was
>> created by Masamichi Hosoda as a result of the discovery of the
>> PNG-transparency bug in the 2.19.51 Lilypond build)
> 
> That's more a matter of GUB than of LilyPond.  GhostScript is more of a
> runtime dependency so it's not really something we would refuse a build
> for.

I've sent a pull request that updates GUB's Ghostscript to 9.21.
https://github.com/gperciva/gub/pull/39

However, on FreeBSD 32 bit, both Ghostscript 9.20 and 9.21 crash.
So only for FreeBSD 32 bit, Ghostscript 9.15 is used.

Linux 64 bit, Linux 32 bit, Linux PPC, FreeBSD 64 bit,
and Windows are no problem.

In my humble opinion, after or before releasing 2.20,
it is better to discontinue some platforms' binary release.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Add guile-config-1.8 etc. searching (issue 320830043 by truer...@gmail.com)

[Masamichi Hosoda]

> On 05/04/2017 10:27, thomasmorle...@gmail.com wrote:
>> LGTM
>>
>> https://codereview.appspot.com/320830043/
>>
> 
> Detection is fine, but another piece is missing (tested on 2.19.58) :
> 
> checking for guile... guile-1.8
> checking guile-1.8 version... 1.8.8
> checking for guile-1.8... guile-1.8
> checking for guile-1.8... /usr/bin/guile-1.8
> ...
> config.status: creating config.make
> config.status: creating config.hh
> 
> ERROR: Please install required programs: guile-config (guile-devel,
> guile-dev or libguile-dev package) GUILE-with-rational-bugfix
> 
> 
> Of course when both guile-config from 2.0.14 and guile-config-1.8 are
> present:
> 
> ERROR: Please install required programs: guile-config < 1.9.0
> (installed: 2.0.14) (guile-devel, guile-dev or libguile-dev package)
> GUILE-with-rational-bugfix

If I understand correctly, it has been fixed by the following commit.

http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=commit;h=e9ae1cb3b093498ccb9d5f7348c755a3243bbccc

It will be contained in the next release 2.19.59.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Need some help - build doc fails for me

[Masamichi Hosoda]

> Hello,
> 
> I appear no longer to be able to make doc.
> 
> I do not think it is for any particular checkin as I went back to
> before 2.19.57 was released and still have problems.
> 
> The 'only' thing I did prior to that was to install
> texlive-lang-japanese to see if I could test Hosoda-san's recent
> checkin and builld the Japanese Docs following his instructions in the
> GNUmakefile.
> 
> (note: I only needed to install that one package, the other
> requirements were already met by whatever I had installed previously).
> 
> I have also deleted by lilypong-git dir and re-fetched it but I still
> get the same issue. It seems to be around the typography.ly demo
> (where we typeset a lot of non-standard character sets) so it could be
> something inadvertently pulled down while installing the japanese
> package, but I removed it and it made no difference.
> 
> If someone can just confirm that doc builds so I know if it is my
> environment or something else I would appreciate it.
> 
> I am at work now so will take some time to set up another build env
> 'quickly' (i.e. before the end of the day), as there is no countdown
> it is not so bad, but I cannot test any patches at the moment.
> 
> 
> Thanks for your help.

I cannot reproduce the error.
Would you show me the error messages?

For the following commit, I've succeed make and make doc.

commit eca3d05456383fc0b451468ce1b21ed08229821f
Issue 5083: Add making web.$(ISOLANG).pdf

My environment is Cygwin64.
TeX Live Japanese environment is installed.
I use Japanese locale.

$ locale
LANG=ja_JP.UTF-8
LC_CTYPE="ja_JP.UTF-8"
LC_NUMERIC="ja_JP.UTF-8"
LC_TIME="ja_JP.UTF-8"
LC_COLLATE="ja_JP.UTF-8"
LC_MONETARY="ja_JP.UTF-8"
LC_MESSAGES="ja_JP.UTF-8"
LC_ALL=

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Fontconfig cache file update fails on Windows

[Masamichi Hosoda]

>>> I've created a patch and reported it to fontconfig Bugzilla.
>>> https://bugs.freedesktop.org/show_bug.cgi?id=99360
>> 
>> I notice that Behdad is mentioned on this bugzilla page as the contact
>> person.  However, he is no longer the maintainer of fontconfig, so I
>> ask you to also write an e-mail to the fontconfig mailing list,
>> telling them that you have provided a fix just to be sure that the
>> right persons see your patch.
> 
> done
> https://lists.freedesktop.org/archives/fontconfig/2017-January/005898.html
> 
> Thanks

The patch has not yet been merged into the fontconfig repository.
However, in order to solve an issue, such as the following, I would like to 
apply the patch using GUB.
http://lists.gnu.org/archive/html/lilypond-user/2017-02/msg00364.html

I've created a pull request for GUB.
https://github.com/gperciva/gub/pull/37

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB fail

[Masamichi Hosoda]

> GUB is not building correctly for me.  The end of the command line
> output is:

I've created a patch.
https://github.com/gperciva/gub/pull/36

Sorry, I've not tested it.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Fontconfig cache file update fails on Windows (also Cygwin)

[Masamichi Hosoda]

>> I've created a patch and reported it to fontconfig Bugzilla.
>> https://bugs.freedesktop.org/show_bug.cgi?id=99360
> 
> I notice that Behdad is mentioned on this bugzilla page as the contact
> person.  However, he is no longer the maintainer of fontconfig, so I
> ask you to also write an e-mail to the fontconfig mailing list,
> telling them that you have provided a fix just to be sure that the
> right persons see your patch.

done
https://lists.freedesktop.org/archives/fontconfig/2017-January/005898.html

Thanks

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Fontconfig cache file update fails on Windows (also Cygwin)

[Masamichi Hosoda]

I've found an issue of fontconfig on Windows.
If you have an old cache file and need updating,
the updating fails on Windows (both MinGW and Cygwin).

I've created a patch and reported it to fontconfig Bugzilla.
https://bugs.freedesktop.org/show_bug.cgi?id=99360

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: linux-ppc harfbuzz GUB build error

[Masamichi Hosoda]

 In my environments, autoconf does not raise such error.
 Do you set the "set -u" or "set -o nounset" somewhere?
 If so, would you remove the setting?
>>> The set -u (or to be complete set -ux) I discovered inside
>>> 
>>> /home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master/smart-configure.sh
>>> 
>>> which was the command that was called when the error was signalled
>> 
>> I've noticed that smart-autogen.sh has "set -ux" and it is invoked from GUB.
>> If I understand correctly, "set -u" and "set -x" etc. do not carry over
>> into child processes.
>> 
>> However, at least in your log file,
>> "set -x" seems to be carried over to the child process.
>> From smart-autogen.sh through autogen.sh to autoconf,
>> all scripts are setted trace mode.
>> 
> [...snip...]
>> 
>> Perhaps, also "set -u" is carried over in your environment
>> and it is not carried over in my environment.
>> 
>> I have no idea.
>> But, in CentOS, /bin/sh is symbolic link to bash,
>> whereas in Ubuntu, /bin/sh is dash.
>> I think that this difference is influenced.
> 
> Anyway, I've noticed that if the environment variable SHELLOPTS exists,
> "set -ux" and "set -e" carry over to the child processes.
> There may be other conditions that it carries over.
> 
> In order to avoid this issue,
> there is a way to "set +ux" before invoking the child process.
> So I've created a patch for LilyPond's smart-autogen.sh and 
> smart-configure.sh.
> 
> https://sourceforge.net/p/testlilyissues/issues/5013/
> https://codereview.appspot.com/319870043/

It has been merged.
Probably the issue on CentOS has been solved.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: linux-ppc harfbuzz GUB build error

[Masamichi Hosoda]

> I've updated my GUB environment and the build fails.  Please see the
> attached zipped logfile.

Perhaps gcc has upgraded but libtool has not been updated.
Does the folowing command

$ /home/gub/NewGub/gub/target/mingw/root/usr/cross/bin/i686-mingw32-gcc 
--version

show 4.9.4?
And, does the following command

$ grep "/gcc/i686-mingw32/4\.9\.2/" 
/home/gub/NewGub/gub/target/mingw/root/usr/bin/libtool

output some variables?
If both are yes, the following commands may solve the issue.

$ bin/gpkg -p mingw remove libtool
$ rm -fr target/mingw/packages/libtool*
$ bin/gub mingw::libtool

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: linux-ppc harfbuzz GUB build error

[Masamichi Hosoda]

>>> In my environments, autoconf does not raise such error.
>>> Do you set the "set -u" or "set -o nounset" somewhere?
>>> If so, would you remove the setting?
>> The set -u (or to be complete set -ux) I discovered inside
>> 
>> /home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master/smart-configure.sh
>> 
>> which was the command that was called when the error was signalled
> 
> I've noticed that smart-autogen.sh has "set -ux" and it is invoked from GUB.
> If I understand correctly, "set -u" and "set -x" etc. do not carry over
> into child processes.
> 
> However, at least in your log file,
> "set -x" seems to be carried over to the child process.
> From smart-autogen.sh through autogen.sh to autoconf,
> all scripts are setted trace mode.
> 
[...snip...]
> 
> Perhaps, also "set -u" is carried over in your environment
> and it is not carried over in my environment.
> 
> I have no idea.
> But, in CentOS, /bin/sh is symbolic link to bash,
> whereas in Ubuntu, /bin/sh is dash.
> I think that this difference is influenced.

Anyway, I've noticed that if the environment variable SHELLOPTS exists,
"set -ux" and "set -e" carry over to the child processes.
There may be other conditions that it carries over.

In order to avoid this issue,
there is a way to "set +ux" before invoking the child process.
So I've created a patch for LilyPond's smart-autogen.sh and smart-configure.sh.

https://sourceforge.net/p/testlilyissues/issues/5013/
https://codereview.appspot.com/319870043/

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: linux-ppc harfbuzz GUB build error

[Masamichi Hosoda]

>>> Maybe the issue is to use ICU of CentOS system.
>>> 
>>> I've created a patch.
>>> https://github.com/trueroad/gub/commit/081cc91f698186795dca45e8d6db8af6616826d4
>>> 
>>> If this patch solves the issue, I will send the pull request.
>> 
>> New gub bootstrapped from you repo; running a trialbuild tonight… more to 
>> follow tomorrow
>> 
> New build from Masamichi’s branch passed the linux ppc harfbuzz, so the ICU 
> patch worked. Now looking into a breaking freebsd-x86 lilypond error: 

I've created the harfbuzz ICU pull request.
https://github.com/gperciva/gub/pull/33

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: linux-ppc harfbuzz GUB build error

[Masamichi Hosoda]

>> In my environments, autoconf does not raise such error.
>> Do you set the "set -u" or "set -o nounset" somewhere?
>> If so, would you remove the setting?
> The set -u (or to be complete set -ux) I discovered inside
> 
> /home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master/smart-configure.sh
> 
> which was the command that was called when the error was signalled

I've noticed that smart-autogen.sh has "set -ux" and it is invoked from GUB.
If I understand correctly, "set -u" and "set -x" etc. do not carry over
into child processes.

However, at least in your log file,
"set -x" seems to be carried over to the child process.
>From smart-autogen.sh through autogen.sh to autoconf,
all scripts are setted trace mode.

Your log file:

 *** Stage: autoupdate (lilypond, freebsd-x86)
invoking cd 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master
 && ./smart-autogen.sh --noconfigure
++ cat 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master/configure.ac
 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master/aclocal.m4
++ md5sum
++ cut -b 1-32
+ AUTOGEN_INPUT_CHECKSUM=6d044a282ff778474007c3d61d131ebc
+ CHECKSUM_FILE=autogen.checksum
++ cat autogen.checksum
cat: autogen.checksum: No such file or directory
+ test '' = 6d044a282ff778474007c3d61d131ebc
+ set -e
+ 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master/autogen.sh
 --noconfigure
++ dirname 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master/autogen.sh
+ 
srcdir=/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master
+ case $1 in
+ NOCONFIGURE=true
+ for i in '$srcdir/configure.ac'
++ dirname 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master/configure.ac
+ 
dir=/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master
+ echo processing 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master
processing 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master
+ cd 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master
+ echo 'Running autoconf ...'
Running autoconf ...
+ autoconf
+ DUALCASE=1
[...snip...]

In my log file, "set -x" does not seem to be carried over to the child process.
Only smart-autogen.sh is setted tracemode.
autogen.sh and autoconf are not setted trace mode.

My log file:

 *** Stage: autoupdate (lilypond, freebsd-x86)
invoking cd 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master
 && ./smart-autogen.sh --noconfigure
+ cat 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master/configure.ac
 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master/aclocal.m4
+ cut -b 1-32
+ md5sum
+ AUTOGEN_INPUT_CHECKSUM=6d044a282ff778474007c3d61d131ebc
+ CHECKSUM_FILE=autogen.checksum
+ cat autogen.checksum
cat: autogen.checksum: No such file or directory
+ test  = 6d044a282ff778474007c3d61d131ebc
+ set -e
+ 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master/autogen.sh
 --noconfigure
processing 
/home/gub/gub/target/freebsd-x86/src/lilypond-git.sv.gnu.org--lilypond.git-master
Running autoconf ...
Skipping configure process.
+ printf %s 6d044a282ff778474007c3d61d131ebc
[...snip...]

Perhaps, also "set -u" is carried over in your environment
and it is not carried over in my environment.

I have no idea.
But, in CentOS, /bin/sh is symbolic link to bash,
whereas in Ubuntu, /bin/sh is dash.
I think that this difference is influenced.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: linux-ppc harfbuzz GUB build error

[Masamichi Hosoda]

>>> New build from Masamichi’s branch passed the linux ppc harfbuzz, so the ICU 
>>> patch worked. Now looking into a breaking freebsd-x86 lilypond error: 
>> 
>> Would you show me the whole log file?
> Went into a bit of experimentation afterwards, still failed attempts only, 
> after discovering that the message is basically a side-effect of 
> set -u
> combined with autoconf apparently trying to use unbound variable CONFIG_SHELL
> 
> On a cleanly booted docker-image recreated the issue with 
> bin/gub freebsd-x86::lilypond 

In my environments, autoconf does not raise such error.
Do you set the "set -u" or "set -o nounset" somewhere?
If so, would you remove the setting?
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: linux-ppc harfbuzz GUB build error

[Masamichi Hosoda]

> New build from Masamichi’s branch passed the linux ppc harfbuzz, so the ICU 
> patch worked. Now looking into a breaking freebsd-x86 lilypond error: 

Would you show me the whole log file?
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: linux-ppc harfbuzz GUB build error

[Masamichi Hosoda]

> Meanwhile took a look on the rest of the logfile and spotted a bunch of 
> “checking <….>… no”… any of those a concern in your experience?
> 
> checking for powerpc-linux-dlltool... no
> checking for dlltool... no
> checking for sysroot... no
> checking for powerpc-linux-mt... no
> checking for mt... no
> checking if : is a manifest tool... no
> checking if powerpc-linux-gcc  supports -fno-rtti -fno-exceptions... no
> checking whether -lc should be explicitly linked in... no
> checking whether to build static libraries... no
> checking for gtk-doc... no
> checking for gtkdoc-check... no
> checking for gtkdoc-check... no
> checking for gtkdoc-rebase... no
> checking for gtkdoc-mkpdf... no
> checking whether to build gtk-doc documentation... no
> checking for native Win32... no
> checking for the pthreads library -lpthreads... no
> checking whether pthreads work without any flags... no
> checking whether pthreads work with -Kthread... no
> checking whether pthreads work with -kthread... no
> checking for the pthreads library -llthread... no
> checking if more special flags are required for pthreads... no
> checking for PTHREAD_PRIO_INHERIT... no
> checking for FONTCONFIG... no
> checking for ICU... no
> checking for powerpc-linux-icu-config... no

I've noticed the difference between your log and my log.

Your first log file:

checking for FONTCONFIG... no
checking for ICU... no
checking for powerpc-linux-icu-config... no
checking for icu-config... icu-config
configure: WARNING: using cross tools not prefixed with host triplet
checking for ICU by using icu-config fallback... yes
checking for FREETYPE... yes
[...snip...]
Unicode callbacks (you want at least one):
Glib:   true
ICU:true
UCDN:   true

My log file:

checking for FONTCONFIG... no
checking for ICU... no
checking for powerpc-linux-icu-config... no
checking for icu-config... no
checking for ICU by using icu-config fallback... no
checking for FREETYPE... yes
[...snip...]
Unicode callbacks (you want at least one):
Glib:   true
ICU:false
UCDN:   true

Maybe the issue is to use ICU of CentOS system.

I've created a patch.
https://github.com/trueroad/gub/commit/081cc91f698186795dca45e8d6db8af6616826d4

If this patch solves the issue, I will send the pull request.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: linux-ppc harfbuzz GUB build error

[Masamichi Hosoda]

> I hope someone can shed a light on what I’m doing wrong here.
> 
> librestrict:error:/home/gub/gub/target/linux-ppc/root/usr/cross/libexec/gcc/powerpc-linux/4.9.4/cc1plus:
>  tried to open () file /usr/include/stdc-predef.h
> librestrict:allowed:
>   /home/gub/gub/target/linux-ppc
>   /tmp
>   /dev/null
>   /dev/urandom
>   /proc/self
> :0:0: internal compiler error: Aborted
> 
> 
> Full harfbuzz.log attached

My GUB environments are Ubuntu 14.04 LTS 64 bit and Ubuntu 16.04 LTS 64bit.
I have never used GUB on CentOS.
But, your log file has the following line.

/home/gub/gub/target/linux-ppc/src/harfbuzz-1.3.0/configure: line 7264: 
/usr/bin/file: No such file or directory

Does your system have /usr/bin/file ?
My system has it and my log file does not have that line.
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB fail

[Masamichi Hosoda]

>> I've had my last 2 tries to compile a GUB build today fail with the
>> same error.  The last relevant part of the log is:
> 
> Would you show me the whole log file?

If I understand correctly, I've found out the cause.
Is there the following line in the log file?

GPL Ghostscript 9.20: Can't find initialization file gs_init.ps.

I've created a pull request.
https://github.com/gperciva/gub/pull/32

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB fail

[Masamichi Hosoda]

> I've had my last 2 tries to compile a GUB build today fail with the
> same error.  The last relevant part of the log is:

Would you show me the whole log file?

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: guile-2.0 and debian

[Masamichi Hosoda]

> Not related to the current guile2-problem, because it happens in the
> 2.19.51-docs as well:
> (1)
> The Japanese docs always display a Yen-sign instead of the backslash
> in the browser, although copying it into a utf-8-aware editor works
> well.
> See attached screenshot.
> Masamichi-San cc-ed.

It is a well-known phenomenon in the Japanese environment.
Due to historical reasons,
most Japanese fonts have Yen-sign glyphs for U+005C.

https://en.wikipedia.org/wiki/JIS_X_0201

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: meta-data with guile2

[Masamichi Hosoda]

> Hi,
> 
> this is a single problem from
> http://lists.gnu.org/archive/html/lilypond-devel/2016-11/msg00090.html
> about pdf-meta-data with guile2
> 
> In framework-ps.scm we have `metadata-encode' defined as part of
> `handle-metadata'
> There's the comment:
> ;; First, call ly:encode-string-for-pdf to encode the string (latin1 or
> ;; utf-16be), then escape all parentheses and backslashes
> ;; FIXME guile-2.0: use (string->utf16 str 'big) instead
> 
> `handle-metadata' finally returns
> (ps-quote (ly:encode-string-for-pdf val))
> 
> Why did we do so at all?
> In my (ofcourse limited) testings I see no difference doing directly:
> (ps-quote val)
> instead.

Non-Latin-1 strings (like Japanese strings)
in PDF metadata must be expressed in UTF-16BE encoding.
`ly:encode-string-for-pdf` converts such strings
to UTF-16BE encoding strings.

If you use directly `(ps-quote val)`, non-Latin-1 strings will broken.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Add installing optional URW++ fonts (issue 315850043 by truer...@gmail.com)

[Masamichi Hosoda]

> But, I think that the simplest way is to distribute the tarball at the 
> LilyPond site.
> e.g.
> http://lilypond.org/downloads/gub-sources/urw-fonts/urw-core35-fonts-79bcdfb.tar.gz
> 
> And change the warning message of configure script.
> e.g.
> (download tarball from
> http://lilypond.org/downloads/gub-sources/urw-fonts/urw-core35-fonts-79bcdfb.tar.gz
> and extract it under '~/.local/share/fonts' etc., or use --with-urwotf-dir)

I've made `urw-core35-fonts-79bcdfb.tar.xz`.
https://drive.google.com/file/d/0ByGBX3PDrqjsbm9TOWktWjFCRmM/view?usp=sharing

If someone uploads it as
`http://lilypond.org/downloads/gub-sources/urw-fonts/urw-core35-fonts-79bcdfb.tar.xz`,
I will upload Patch Set 2 to Rietveld for changing the warning message.

Or should I push the current Patch Set 1?

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Add installing optional URW++ fonts (issue 315850043 by truer...@gmail.com)

[Masamichi Hosoda]

>> I attached the shell script which can download the 12 font files from
> 
> 'http://git.ghostscript.com/?p=urw-core35-fonts.git;a=commit;h=79bcdfb34fbce12b592cce389fa7a19da6b5b018'.
> 
>> Its each download URL is just link of each font file in above URL.
>> It works fine on my environment.
> 
>> Would you try it and put the downloaded files to ~/.local/share/fonts
> etc?
> 
> Yes this works, but is this the most elegant/simplest way to get these
> files?

I've found a more simple way.
The following commands can install the font files.

$ wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=snapshot;h=79bcdfb34fbce12b592cce389fa7a19da6b5b018;sf=tgz;
$ mkdir -p ~/.local/share/fonts
$ tar xfvz urw-core35-fonts-79bcdfb.tar.gz -C ~/.local/share/fonts

But, I think that the simplest way is to distribute the tarball at the LilyPond 
site.
e.g.
http://lilypond.org/downloads/gub-sources/urw-fonts/urw-core35-fonts-79bcdfb.tar.gz

And change the warning message of configure script.
e.g.
(download tarball from
http://lilypond.org/downloads/gub-sources/urw-fonts/urw-core35-fonts-79bcdfb.tar.gz
and extract it under '~/.local/share/fonts' etc., or use --with-urwotf-dir)

> 1. If you look here:
> http://lilypond.org/doc/v2.19/Documentation/contributor-big-page.html#requirements-for-running-lilypond
> 
> we already list a number of 'font requirements'. Does including
> TexGyre
> and now these URW++ fonts mean that all/some/none of these font
> pacakges
> we currently document can now be removed as a requirement? (e.g.
> texlive-lang-cyrillic).
> 
> 2. The same question but for
> http://lilypond.org/doc/v2.19/Documentation/contributor-big-page.html#requirements-for-building-documentation

If I understand correctly, there is no requirement that can be deleted.

`texlive-lang-cyrillic` is for Texinfo. It is not for LilyPond snippets.
The URW++ fonts are for LilyPond snippets.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Add installing optional URW++ fonts (issue 315850043 by truer...@gmail.com)

[Masamichi Hosoda]

> I am sorry to still talk about this, but I have been unable to work
> out
> how I could install just these fonts without having to clone the
> entire
> repo. I have looked in places like stackoverflow and the various 'git'
> sites but no one seems to be able to clearly state how to install just
> this commit (with for example git fetch). I also have not been able to
> 'download' the raw blobls from the repo either using 'right click/save
> as'.

I attached the shell script which can download the 12 font files from
'http://git.ghostscript.com/?p=urw-core35-fonts.git;a=commit;h=79bcdfb34fbce12b592cce389fa7a19da6b5b018'.

Its each download URL is just link of each font file in above URL.
It works fine on my environment.

Would you try it and put the downloaded files to ~/.local/share/fonts etc?
#!/bin/sh

wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=C059-BdIta.otf;h=0f59bb96119f289eb348c511764c9d2ceafb7f80;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;
wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=C059-Bold.otf;h=f8718a8d481b4483aa38667dba64e4949533945f;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;
wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=C059-Italic.otf;h=4abe846c09f3aaf8fafa2a4efa94484e842fd100;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;
wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=C059-Roman.otf;h=fcd5e4c18ae04338570e25fa3fb6eb1839afa7bf;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;

wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=NimbusMonoPS-Bold.otf;h=624871f6291b0346092ca0b6a571b6d7904e0f02;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;
wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=NimbusMonoPS-BoldItalic.otf;h=a8698a89d9d27dfc78431e89c930b35c5d003464;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;
wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=NimbusMonoPS-Italic.otf;h=a8fa428ec66b42e0f6d35599268050abbaad3de9;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;
wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=NimbusMonoPS-Regular.otf;h=f94f61a02a0deeba22fe41c2882c717d5aac1385;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;

wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=NimbusSans-Bold.otf;h=941f882eaf17d626a9960db75c32fc8296e81211;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;
wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=NimbusSans-BoldOblique.otf;h=91cf2e3529d456358fccc3d1006dfc910439fe11;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;
wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=NimbusSans-Oblique.otf;h=f47d0651c35849044142739aad1cf3b07822e8c3;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;
wget --content-disposition 
"http://git.ghostscript.com/?p=urw-core35-fonts.git;a=blob;f=NimbusSans-Regular.otf;h=bdeb5c275eac2dc7b95655b55405f84489e2;hb=79bcdfb34fbce12b592cce389fa7a19da6b5b018;
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Replace `-dgs-load-fonts' to `--bigpdfs' in lilypond-book (issue 300280043 by truer...@gmail.com)

[Masamichi Hosoda]

>>> Do you have time to integrate this into lilypond in the near future?
>> 
>> I would like so.  My GUB environment can compile it for GUB inner
>> using.  (v1.0.0 could not compile.)
>> 
>> But, I think that the integration requires a lot of time.
> 
> Why?  Please elaborate.

At least, in order to avoid broken glyphs,
we will need to change whether or not embedding by the font type.

i.e.
TrueType   : no embedding
Type1 / OTF: embedding

So `-dEmbedAllFonts=false` cannot be used.
Perhaps, `/NeverEmbed` and `/AlwaysEmbed` can be used.
However, I've never tried it. It is just an idea.

Moreover, we will need to change font resource directory
depending on the font type.

i.e.
Japanese OTF: /CIDFont/
Other fonts?: 

But, I have another idea.
It does not need the font resource directory.
To use a PostScript file which only contains font resources.
It is one PostScript file and includes

`%%BeginFont`
~
`%%EndFont`

blocks of all necessary fonts.
I'm trying it.

Anyway, I think there will be a lot of things to consider.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Replace `-dgs-load-fonts' to `--bigpdfs' in lilypond-book (issue 300280043 by truer...@gmail.com)

[Masamichi Hosoda]

>>> I've just tested successfully the following method, except item 1,
>>> which I've executed manually.  [...]
>>
>> I've tried this way.  [...]
> 
> Thank you for your thorough testing, which obviously covers more
> situations than my limited tries.
> 
> BTW, I see that your `extractpdfmark' tool has already reached version
> 1.0.1 – congratulations!
> 
>   https://github.com/trueroad/extractpdfmark

Thank you.

> Do you have time to integrate this into lilypond in the near future?

I would like so.
My GUB environment can compile it for GUB inner using.
(v1.0.0 could not compile.)

But, I think that the integration requires a lot of time.
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Gub is failing to build fontconfig-2.12.1

[Masamichi Hosoda]

> The host machine is 64 bit debian stable I believe.
> This is the error I am getting:
> 
> ail of target/mingw/log/fontconfig.log 
> Alternatively, you may set the environment variables FREETYPE_CFLAGS
> and FREETYPE_LIBS to avoid the need to call pkg-config.
> See the pkg-config man page for more details.
[...snip...]
> 
> I checked and freetype was in fact installed. I don't know why it can't see
> the .pc file.

If I understand correctly, it is due to building order.

If you build zlib before pkg-config, zlib does not generate .pc file.
FreeType's .pc file depends on zlib's .pc file.
pkg-config found FreeType but did not found zlib.
So it failed for FreeType.

Would you see the whole of log file `target/mingw/log/fontconfig.log` ?

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Help with GUB, or manual cross-compile

[Masamichi Hosoda]

>> Following up a conversation on the user list, as this might be a
>> better
>> place to ask... I think we've established that the only way to compile
>> for
>> Windows is to use GUB on Linux to cross-compile for mingw.
> 
> Well I *think* Masamichi Hosoda compiles on Windows - I only say this
> as I had some private email conversations last year about this to
> possibly document this in the Contributor's Guide when I had some
> spare time to try it out for myself (I always like to make sure that I
> can do something before I document it), and that by using the windows
> based GNU Utils (i.e. cygwin) he was able to make and make doc without
> having to use GUB.

Yes, I compile LilyPond on Windows.
But, I use cygwin environment.
So built binary is for cygwin instead of mingw.

If you accept cygwin, you can build LilyPond like Linux way.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: v2.19.47 on Mac x86 (Jacques Menu Muzhic)

[Masamichi Hosoda]

>>>   https://bugs.freedesktop.org/show_bug.cgi?id=97546
>>> 
>>> The proposed fix hasn't been applied yet to the fontconfig git
>>> repository – maybe we should downgrade GUB's fontconfig version to
>>> 2.12.0 until this is fixed.
>> 
>> I think that there is another possible solution.  To use the
>> proposed patch in GUB.
> 
> Certainly.  However, I can't estimate whether there aren't side
> effects...  On the other hand, given that the patch is from the
> fontconfig maintainer, there are high chances that it is the right
> one :-)

I've created pull request for GUB.
https://github.com/gperciva/gub/pull/29

The patch is only applied to building binary for mac.
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: v2.19.47 on Mac x86 (Jacques Menu Muzhic)

[Masamichi Hosoda]

>> Most likely cause of the issue will be the 2.12.1 change that
>> improved cache validation logic (and for yet unknown reasons always
>> invalidates the cache of the Mac OS System fonts)
>> 
>> https://www.freedesktop.org/software/fontconfig/release/ChangeLog-2.12.1
> 
> Aah, this rings a bell.
> 
>   https://bugs.freedesktop.org/show_bug.cgi?id=97546
> 
> The proposed fix hasn't been applied yet to the fontconfig git
> repository – maybe we should downgrade GUB's fontconfig version to
> 2.12.0 until this is fixed.

I think that there is another possible solution.
To use the proposed patch in GUB.
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: 2.19.47 long compile times

[Masamichi Hosoda]

>> Hi,
>>
>> The new 2.19.47 is taking about 40 seconds to compile a one-note test
>> file.
>> (MacOS X x86.)
>>
>> In my past experience, new LilyPond builds sometimes take 30 or 40
>> seconds
>> to compile *on the very first run after install* (presumably to update
>> the
>> font cache, or something similar). But 2.19.47 is taking 40 seconds
>> *on
>> every run*.
>>
>> Trevor.
> 
> 
> There was something _similar_ reported on Windows 10, although the
> slow-down occurred between 19.42 and 19.43.  In this case, it appeared
> that the problem was fixed by deleting all the files in
> c:\users\yourusername\.lilypond-fonts.cache-2.  Perhaps a similar
> approach will work for the Mac?
> 
> --
> Phil Holmes 

If you would like to delete the fontconfig cache files in OSX,
cache directory might be `~/.lilypond-fonts.cache-2`.
http://lists.gnu.org/archive/html/bug-lilypond/2016-07/msg00050.html

So If I understand correctly, the following command can delete them.

$ rm -fr ~/.lilypond-fonts.cache-2

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB problem

[Masamichi Hosoda]

> Thank you for your answers.
> Is there
> /home/gub/NewGub/gub/target/tools/build/gmp-6.0.0.a/config.log
> ?
> Would you show me config.log in gmp-6.0.0.a ?
>
 As far as I can see, that file does not exist on my system.
>>> Would you show me result of the following commands?
>>> $ cat /proc/cpuinfo
>>> $ cd /home/gub/NewGub/gub/target/tools
>>> $ find . -name "config.log"
>> 
>> gub@gub-VirtualBox:~$ cd NewGub/gub/target/tools/
>> gub@gub-VirtualBox:~/NewGub/gub/target/tools$ find . -name
>> "config.log"
>> ./build/gmp-6.0.0.a/config.log
>> 
>> 
>> 
>> I have attached config.log and the output of cpuinfo.
> 
> Thank you for your files.
> 
> If I understand correctly,
> my environment and your environment difference is found.
> I'm making a patch for GUB.
> 
> In my 32 bit VM,
> 
> /proc/cpuinfo flags has `lm' (long mode).
> So GUB sets build_hardware_bits=64.
> https://github.com/gperciva/gub/blob/7e12e0ab779b11db32b471875ab31d93aabb7f7d/gub/settings.py#L230
> 
> And, GUB sets build_bits=32 because the OS is 32 bit.
> https://github.com/gperciva/gub/blob/7e12e0ab779b11db32b471875ab31d93aabb7f7d/gub/settings.py#L65
> 
> Thus, build_bits != build_hardware_bits.
> So GUB sets environment variable ABI=32.
> https://github.com/gperciva/gub/blob/7e12e0ab779b11db32b471875ab31d93aabb7f7d/gub/settings.py#L243
> 
> GMP's configure script detects the 64 bit capable CPU but ABI=32 is set.
> So the GMP building succeed.
> 
> 
> On the other hands,
> in your 32 bit VM,
> 
> /proc/cpuinfo flags does not have `lm' (long mode).
> Ofcource, your CPU is 64 bit capable.
> It originally would have the flag.
> Perhaps virtualization software drops the flag.
> So GUB sets build_hardware_bits=32.
> 
> Thus, build_bits == build_hardware_bits.
> So GUB does not set environment variable ABI.
> 
> GMP's configure script detects the 64 bit capable CPU
> because it uses CPUID instead of `/proc/cpuinfo'.
> Then, it try to use 64 bit but it cannot use 64 bit because the OS is 32 bit.

I've created pull request.
https://github.com/gperciva/gub/pull/28

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB problem

[Masamichi Hosoda]

 Thank you for your answers.
 Is there
 /home/gub/NewGub/gub/target/tools/build/gmp-6.0.0.a/config.log
 ?
 Would you show me config.log in gmp-6.0.0.a ?

>>> As far as I can see, that file does not exist on my system.
>> Would you show me result of the following commands?
>> $ cat /proc/cpuinfo
>> $ cd /home/gub/NewGub/gub/target/tools
>> $ find . -name "config.log"
> 
> gub@gub-VirtualBox:~$ cd NewGub/gub/target/tools/
> gub@gub-VirtualBox:~/NewGub/gub/target/tools$ find . -name
> "config.log"
> ./build/gmp-6.0.0.a/config.log
> 
> 
> 
> I have attached config.log and the output of cpuinfo.

Thank you for your files.

If I understand correctly,
my environment and your environment difference is found.
I'm making a patch for GUB.

In my 32 bit VM,

/proc/cpuinfo flags has `lm' (long mode).
So GUB sets build_hardware_bits=64.
https://github.com/gperciva/gub/blob/7e12e0ab779b11db32b471875ab31d93aabb7f7d/gub/settings.py#L230

And, GUB sets build_bits=32 because the OS is 32 bit.
https://github.com/gperciva/gub/blob/7e12e0ab779b11db32b471875ab31d93aabb7f7d/gub/settings.py#L65

Thus, build_bits != build_hardware_bits.
So GUB sets environment variable ABI=32.
https://github.com/gperciva/gub/blob/7e12e0ab779b11db32b471875ab31d93aabb7f7d/gub/settings.py#L243

GMP's configure script detects the 64 bit capable CPU but ABI=32 is set.
So the GMP building succeed.


On the other hands,
in your 32 bit VM,

/proc/cpuinfo flags does not have `lm' (long mode).
Ofcource, your CPU is 64 bit capable.
It originally would have the flag.
Perhaps virtualization software drops the flag.
So GUB sets build_hardware_bits=32.

Thus, build_bits == build_hardware_bits.
So GUB does not set environment variable ABI.

GMP's configure script detects the 64 bit capable CPU
because it uses CPUID instead of `/proc/cpuinfo'.
Then, it try to use 64 bit but it cannot use 64 bit because the OS is 32 bit.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB problem

[Masamichi Hosoda]

>>> Would you show me the following log files?
>>>
>>> /home/gub/NewGub/gub/target/tools/log/gmp.log
>>> /home/gub/NewGub/gub/target/tools/src/gmp-6.0.0.a/config.log
> 
> 
> Sorry - forgot to say.  gmp-6.0.0.a/config.log is not present on my
> system. The directory is there but no file of that name.

Thank you for your answers.

Is there
/home/gub/NewGub/gub/target/tools/build/gmp-6.0.0.a/config.log
?

Would you show me config.log in gmp-6.0.0.a ?

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB problem

[Masamichi Hosoda]

> I get the following:
> 
> checking size of mp_limb_t... 4
> configure: error: Oops, mp_limb_t is 32 bits, but the assembler code
> in this configuration expects 64 bits.
> You appear to have set $CFLAGS, perhaps you also need to tell GMP the
> intended ABI, see "ABI and ISA" in the manual.
> Command barfed: cd /home/gub/NewGub/gub/target/tools/build/gmp-6.0.0.a
> && chmod +x
> /home/gub/NewGub/gub/target/tools/src/gmp-6.0.0.a/configure && sh
> /home/gub/NewGub/gub/target/tools/src/gmp-6.0.0.a/configure
> --prefix=/home/gub/NewGub/gub/target/tools/root/usr --enable-shared
> --enable-static --disable-silent-rules
> CFLAGS=-I/home/gub/NewGub/gub/target/tools/root/usr/include
> LDFLAGS='-L/home/gub/NewGub/gub/target/tools/root/usr/lib -Wl,-rpath
> -Wl,\$$ORIGIN/../lib -Wl,-rpath
> -Wl,/home/gub/NewGub/gub/target/tools/root/usr/lib '
> CPPFLAGS=-D_GNU_SOURCE

Did you delete target/ ?

Would you show me the following log files?

/home/gub/NewGub/gub/target/tools/log/gmp.log
/home/gub/NewGub/gub/target/tools/src/gmp-6.0.0.a/config.log

Would you show me the result of the follwing commands?

$ cd /home/gub/NewGub/gub
$ git status
$ git diff
$ git log -5
$ env

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB problem

[Masamichi Hosoda]

>> I'd like to suggest deleting `target/` such as in the GUB's basic
>> document.
>> But, after you delete it, GUB's full build takes a long long time.
>>
> 
> Yes, I know :-(
> 
> Should I revert my ABI change before trying the rebuild?

Yes, please.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB problem

[Masamichi Hosoda]

>> GUB's basic document
>> https://github.com/gperciva/gub/blob/master/web/basics.html
>> says
>>
>> ```
>>SHARING DIRECTORIES
>>
>>GUB uses an ABI environment variable to work around some build
>>bugs in some packages.  This variable is not checked by the
>>environment-changed function, so problems will probably occur if
>>you share your GUB build directory between a 32-bit and 64-bit OS.
>>If you change your operating system, delete the target/ directory
>>of GUB before trying to build it again.
> 
> 
> I think this is quite a difficult one to answer.
> 
> My main machine was (IIRC) 64 bit originally - that's partly why I
> used a VM for GUB, since GUB used not to work in a 64 bit environment.
> When I upgraded from Ubuntu 10.04, via 12 to 14.04, the machine became
> 32 bit, since I selected that option because it seemed that the 64 bit
> was not deigned for Intel CPUs.  I think I may have misunderstood
> that, but whatever the reason, I now run a 32 bit main machine.  As
> part of the move to Ubuntu 14, I needed to upgrade the Oracle VM
> software, but did not change the VM spec at all.  So I think the
> answer is that the actual VM disk has never seen a 64 bit environment,
> but has run within a 64 bit environment.

Thank you for your answer.

I'd like to suggest deleting `target/` such as in the GUB's basic document.
But, after you delete it, GUB's full build takes a long long time.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB problem

[Masamichi Hosoda]

> Apologies for the delay in replying.  I knew it would take me a little
> time to gather the information you needed, so had to wait for some
> free time. Responses are below.

Thank you very much.

I suspect that environment variable ABI problem.

Have you ever mounted the disk to 64 bit OS?
Have you changed virtualization software?

GUB's basic document
https://github.com/gperciva/gub/blob/master/web/basics.html
says

```
SHARING DIRECTORIES

GUB uses an ABI environment variable to work around some build
bugs in some packages.  This variable is not checked by the
environment-changed function, so problems will probably occur if
you share your GUB build directory between a 32-bit and 64-bit OS.
If you change your operating system, delete the target/ directory
of GUB before trying to build it again.
```

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB problem

[Masamichi Hosoda]

>>> FYI I had to make a small change to the GMP spec to accommodate 32 bit
>>> - my patch shows what I had to do.
>>
>> If you don't use the patch, do any errors occur?
> 
> Yes:  the terminal output has this:
> 
> Tail of target/tools/log/gmp.log 
> in this configuration expects 64 bits.
> You appear to have set $CFLAGS, perhaps you also need to tell GMP the
> intended ABI, see "ABI and ISA" in the manual.

I've prepared Ubuntu 14.04 LTS 32 bit VM.
In the environment, I've succeed GMP compiling without your patch.

Would you show me the whole log file target/tools/log/gmp.log?

>> $ LIBRESTRICT_ALLOW=/ bin/gub --fresh freebsd-64::fontconfig
> 
> Do I issue that command and then restart the GUB compile?

No.
If I understand correctly, that command will fail.
I'd like to see config.log error message.

In addition, I've succeed freebsd-64::fontconfig compiling
in my Ubuntu 32 bit VM.
I'd like to investigate what is different between
my environment and your environment.

Would you show me the result of the following command?

$ uname -a
$ uname -m
$ uname -p
$ uname -i
$ dpkg -l

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: GUB problem

[Masamichi Hosoda]

>> In my environment (Ubuntu 14.04 LTS 64 bit), the error does not occur.
>> Would you tell me the environment which the error occurs?
> 
> Ubuntu 14.04 LTS 32 bit.
>
> FYI I had to make a small change to the GMP spec to accommodate 32 bit
> - my patch shows what I had to do.

If you don't use the patch, do any errors occur?

>> Anyway, this error means that
>> cross-compiler gcc (for freebsd-64) tries to open "librestrict.so".
>>
>> "librestrict.so" is for host OS (perhaps Linux)
>> instead of for target OS (freebsd-64).
>>
>> If I understand correctly,
>> cross-compiler gcc does not need to open "librestrict.so".
>> I don't know why cross-compiler gcc tries to open "librestrict.so".
>>
>> Moreover, at least a few weeks ago, the error did not occur.
>> My recent pull request does not change gcc.
>> I wonder why cross-compiler gcc began to open "librestrict.so".
> 
> Perhaps your fix to librestrict made something work that was not
> previously, and so an error is now detected?

I've noticed that this librestrict-error is secondary-effect.

If I understand correctly,
the first error is

> x86_64-freebsd6-gcc: internal compiler error: Illegal instruction (program 
> cc1)

Then, gcc tried to open "librestrict.so" for showing debugging information.
So secondary-effect librestrict-error was occured.

To confirm this hypothesis,
would you try the following command and show me the result (config.log etc.)?

$ LIBRESTRICT_ALLOW=/ bin/gub --fresh freebsd-64::fontconfig

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel