Re: xdvipdfmx fails with some regtests (“Invalid object”)
> Where can I find the source code for xdvipdfmx? Here: https://www.tug.org/svn/texlive/trunk/Build/source/texk/dvipdfm-x/ This is the code for a binary that gets stored under five different names (either as links or as exact copies): xdvipdfmx dvipdfm dvipdfmx ebb extractbb The binary checks which name was used to call itself and uses this info to behave accordingly: For example, if it is called as `xdvipdfmx`, it acts as `xdvipdfmx`. To build this program, have a look at https://www.tug.org/texlive/build.html Builds are always with srcdir != builddir. For example, attached you can find my script to build `patgen`; I needed this some time ago because the limits in the distributed binary were too small for my needs. Note that `patgen` is part of the 'web2c' group, so you actually get a bunch of (small) binaries, since the TeXLive build doesn't allow a finer-grained resolution. You probably have to change --enable-web2c→ --disable-web2c --disable-xetex → --enable-xetex in the script to get an `xdvipdfmx` binary (among others). >> Maybe slight changes in the ghostscript output or options... > > Wouldn’t that be what we used to call a regression? :-) I would rather call it unveiling a bug, since chances are high that the problem is not within LilyPond. Werner export LANG= /my/path/to/TeXLive/Build/source/configure \ --disable-all-pkgs \ --enable-web2c \ --disable-tex \ --disable-etex-synctex \ --disable-ptex \ --disable-ptex-synctex \ --disable-eptex \ --disable-eptex-synctex \ --disable-uptex \ --disable-uptex-synctex \ --disable-euptex \ --disable-euptex-synctex \ --disable-aleph \ --disable-pdftex \ --disable-pdftex-synctex \ --disable-luatex \ --disable-luatex53 \ --disable-luajittex \ --disable-xetex \ --disable-synctex \ --disable-mp \ --disable-pmp \ --disable-upmp \ --disable-mfluajit \ --disable-mf \ --disable-mf-nowin \ --disable-mflua \ CFLAGS="-O3" &> configure.log \ && make &> make.log
Re: xdvipdfmx fails with some regtests (“Invalid object”)
> Git bisect actually tells me that xdvipdfmx started misbehaving from > the same commit that caused gs issues: > > 017927b4d63c317e1fc450be2537ccc058072538 (HEAD) > Author: Han-Wen Nienhuys > Date: Fri Jun 5 20:36:42 2020 +0200 > Unify calling convention for command-line and API GS use > > Jonas reverted some parts of that with MR !148, but that particular > issue was left unaddressed. That affects xdvipdfmx svn/20190225, > which is shipped with Fedora 32, and with a more recent build from > the 2020 texlive release (20200315). (Now, I haven’t been able to > reproduce it on LilyDev so other libraries may be a factor as well.) > > It already took me many hours to find the eight regtests that > trigger this, but there are also many snippets and NR ly blocks, > which I won’t be able to track down :-/ Could you send me off-list an example, this is, all necessary input files that allow me to directly call xetex (and thus xdvipdfmx), before and after the problematic commit? I can then start to reduce the example and contact the xdvipdfmx author, asking for further analysis. Or maybe you have produced such a minimum example already? Werner
Re: xdvipdfmx fails with some regtests (“Invalid object”)
On 6/17/20, Valentin Villenave wrote: > `make doc’ has been broken for nearly a week on my system (even with a > clean git clone), with the following error: > > xdvipdfmx:fatal: typecheck: Invalid object type: -1 7 (line 2161) Git bisect actually tells me that xdvipdfmx started misbehaving from the same commit that caused gs issues: 017927b4d63c317e1fc450be2537ccc058072538 (HEAD) Author: Han-Wen Nienhuys Date: Fri Jun 5 20:36:42 2020 +0200 Unify calling convention for command-line and API GS use Jonas reverted some parts of that with MR !148, but that particular issue was left unaddressed. That affects xdvipdfmx svn/20190225, which is shipped with Fedora 32, and with a more recent build from the 2020 texlive release (20200315). (Now, I haven’t been able to reproduce it on LilyDev so other libraries may be a factor as well.) It already took me many hours to find the eight regtests that trigger this, but there are also many snippets and NR ly blocks, which I won’t be able to track down :-/ Cheers, -- V.
Re: -Werror
On Jun 17, 2020, at 17:18, Han-Wen Nienhuys wrote: > > My proposal is to use -Werror only in CI, so we can keep code free of > warnings. > I'd go along with that if I could have a single switch or environment variable to configure my local build with all the options that will be enabled in CI, without having to go look up what they are or keep up to date when they change. — Dan
Re: LilyPond | Pipeline #156017999 has failed for dev/hanwen/doc-build | 47b263b1 in !84
On Jun 14, 2020, at 07:04, Han-Wen Nienhuys wrote: > > Hey Dan, > > does your CI runner have limitations on sending data back to Gitlab by any > chance? It looks like this hung while uploading the failure logs > > -- Forwarded message - > From: GitLab mailto:git...@mg.gitlab.com>> > Date: Sun, Jun 14, 2020 at 12:49 PM > Subject: LilyPond | Pipeline #156017999 has failed for dev/hanwen/doc-build | > 47b263b1 in !84 Would you like me to take my runner offline for a bit and trigger a rebuild of this pipeline to see whether it will succeed on another runner? It would be nice to advance this MR since it seems to take about 25% off the time required for the doc stage (on my runner). — Dan
Re: -Werror
On Mon, Jun 15, 2020 at 2:43 PM Jonas Hahnfeld wrote: > > Am Montag, den 15.06.2020, 07:50 -0400 schrieb Dan Eble: > > Han-Wen proposed building with -Werror in a merge request. What do you > > think? > > > > I like -Werror, but I've only ever used it where there were very few (or > > one) supported build environments, all of which were covered in CI. A > > dimension of the CI coverage was optimization level, which can change what > > the compiler discovers. > > > > We don't have that here, so we might regret turning warnings into errors > > all at once. How about we turn particular warnings into errors, starting > > with narrowing conversions and printf formatting, and wait to see if anyone > > reports problems? (Am I being too conservative?) > > As I'm not sure what this proposal is about, I'd like to express that > I'm not a fan of enabling -Werror in the default configuration. That's > a pain when using a newer compiler, and you'll eventually get there > when bisecting an old bug. > > I'm much more in favor of enabling -Werror in CI because the setting is > pretty fixed. However this assumes that the compiler in Ubuntu 16.04 > doesn't throw stupid warnings that we don't want to fix, and it makes > future upgrades of the CI system a bigger pain than without. Not > necessarily bad to fix warnings for the newer compiler version, but > could be daunting. My proposal is to use -Werror only in CI, so we can keep code free of warnings. For CI upgrades, we could adapt the flags to switch off individual warnings that trigger newly after an upgrade (we already do this for -Wsequence-point, for example.) -- Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen
Re: GSoC 2020 update, June 15
On Wed, Jun 17, 2020 at 5:26 AM Owen Lamb wrote: > > On Tue, Jun 16, 2020 at 12:16 AM Han-Wen Nienhuys wrote: >> >> On Tue, Jun 16, 2020 at 9:06 AM Werner LEMBERG wrote: >> >> [...] Now we have external >> > metadata files again – not a single one, but a bunch of them... >> >> They're putting the metadata in separate files? > > > If they are, I'm not aware of it. The specification only mentions one > metadata file, [font name]-metadata.json. I guess you mean "they are using only one separate json file", but to me that is still one file too many. -- Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen
Re: GSoC 2020 update, June 15
On 6/16/2020 10:26 PM, Owen Lamb wrote: I wasn't sure what to say as to my "affiliation" with LilyPond and/or GSoC I've also had that feeling when interacting with other entities on behalf of the LilyPond. This has happened maybe twice. In the past, I've said something like "I am a volunteer doing research and testing for the GNU LilyPond project (lilypond.org, sheet music engraving software) and I'm working on [insert topic here.]" -- Karlin High Missouri, USA
Re: Texinfo - manual line breaks in URLs?
Michael Käppler writes: > Am 11.06.2020 um 15:22 schrieb Werner LEMBERG: >>> I can prepare a version of the docs with all occurences of '@/' >>> removed and '@urefbreakstyle before' set, so that everyone can >>> compare and build an opinion on this. >> Great, thanks in advance. I suggest that you use the `texinfo.tex` >> file directly from the 'texinfo' git repository so that we test the >> most recent version. >> > This turned out to be difficult. > As a first step I tried to upgrade texinfo.tex to current master > of the official Texinfo git repo. This broke 'make doc' > with a XeTeX error, handed over by 'texi2pdf' with the pretty > useless message: "XeTeX exited with bad exit status" > This happened just after finishing the first XeTeX pass. > > I appended various options to texi2pdf to make the output > more verbose, but I did not get useful output regarding > the XeTeX termination. > > Bisecting yielded that the first texinfo commit that introduced > the error is > http://git.savannah.gnu.org/cgit/texinfo.git/commit/?id=aa18f519e091b6ada0fb2b0a65a51880031d5014 Uh oh. This looks like it introduces (modifies?) a command named @seealso but it would appear that we define our own version of it. -- David Kastrup
Re: xdvipdfmx fails with some regtests (“Invalid object”)
On 6/17/20, Werner LEMBERG wrote: > I suggest upgrading. The current maintainer of xdvipdfmx fixes bugs > in TeXLive directly – however, this doesn't automatically update the > binaries that are in TeXLive's SVN repository. It's not that > complicated to get the TeXLive sources and compile a private version > of xdvipdfmx by yourself, BTW. Where can I find the source code for xdvipdfmx? They only have a binary blob on https://tug.org/svn/texlive/trunk/Master/bin/x86_64-linux/xdvipdfmx and the only other thing I could find was that comment: https://sourceforge.net/p/xetex/code/ci/master/tree/build.sh#l129 BTW, having tested the latest texlive 2020 distro and even their blob from svn, I can confirm that the bug still happens (it’s actually the same version than the one provided in Fedora repos). >> Besides, I haven’t changed anything on my OS recently and this >> started happening only a week or so ago; isn’t it possible that this >> is triggered by something introduced in our build system? > > Maybe slight changes in the ghostscript output or options... Wouldn’t that be what we used to call a regression? :-) On 6/17/20, Jean Abou Samra wrote: > you can delete all untracked and ignored files in your repository using > git clean -xdf Well, I was using git ls-files -o | xargs rm, which probably amounts to the same, but thanks for that more subtle alternative. Cheers, -- V.
Re: xdvipdfmx fails with some regtests (“Invalid object”)
>> It seems that you probably have to update your TeX system... > > Update or downgrade? I suggest upgrading. The current maintainer of xdvipdfmx fixes bugs in TeXLive directly – however, this doesn't automatically update the binaries that are in TeXLive's SVN repository. It's not that complicated to get the TeXLive sources and compile a private version of xdvipdfmx by yourself, BTW. > Besides, I haven’t changed anything on my OS recently and this > started happening only a week or so ago; isn’t it possible that this > is triggered by something introduced in our build system? Maybe slight changes in the ghostscript output or options... Werner
Re: xdvipdfmx fails with some regtests (“Invalid object”)
Hi, Le 17/06/2020 à 09:53, Valentin Villenave a écrit : Hey everybody, `make doc’ has been broken for nearly a week on my system (even with a clean git clone), with the following error: No clue what your problem is, but as an aside, I can give a hint: instead of a fresh git clone (which must have taken pretty long!), you can delete all untracked and ignored files in your repository using git clean -xdf Add -n for a dry run. Remove -x to remove untracked files only but not ignored files or change it to -X to remove ignored files only, which is often what you want to clean build artifacts. Enjoy :) Perhaps worth addition to http://lilypond.org/doc/v2.19/Documentation/contributor/advanced-git-procedures . Cheers, Jean
Re: “make doc” broken because of gs eps->pdf failing
On 6/17/20, Valentin Villenave wrote: > Thanks for giving me hope, even for a minute :-) I spoke too soon; running doc-clean and _then_ “make doc” indeed fixes the problem. (And Jonas has just merged it onto master so that problem is fixed.) Thanks! Cheers, -- V.
Re: “make doc” broken because of gs eps->pdf failing
On 6/17/20, Michael Käppler wrote: > Do you have > https://gitlab.com/lilypond/lilypond/-/merge_requests/148/diffs?commit_id=c002d5e8c2e737cefbafa84f4b66e4b3bbc6d111 > in your repo? Otherwise that may well be the problem. Jonas wanted to > push this one yesterday, > but it seems that did not happen yet. Nope, even with this applied, it still fails at the same point. Thanks for giving me hope, even for a minute :-) Cheers, -- V.
Re: “make doc” broken because of gs eps->pdf failing
Am Mittwoch, den 17.06.2020, 10:59 +0200 schrieb Michael Käppler: > Am 17.06.2020 um 10:52 schrieb Valentin Villenave: > > Greetings everyone, > > I’ve noticed a recent regression with `make doc’: when it reaches > > Documentation/ly-examples, gs fails on a bunch of files with the > > following error message: > > > > [...] > > Do you have > https://gitlab.com/lilypond/lilypond/-/merge_requests/148/diffs?commit_id=c002d5e8c2e737cefbafa84f4b66e4b3bbc6d111 > in your repo? Otherwise that may well be the problem. Jonas wanted to > push this one yesterday, but it seems that did not happen yet. Sorry about that, got caught up with my work on texi2html. Merging right now. > > Which makes it very weird that I seem to be the only one getting > > that problem :-) You're not, but it took some time until I understood the full breakage and nobody made me push the fixes faster... Jonas > Cheers, > Michael > > > Cheers, > > -- V. > > > > > signature.asc Description: This is a digitally signed message part
Re: “make doc” broken because of gs eps->pdf failing
Am 17.06.2020 um 10:52 schrieb Valentin Villenave: Greetings everyone, I’ve noticed a recent regression with `make doc’: when it reaches Documentation/ly-examples, gs fails on a bunch of files with the following error message: Command: /home/work/lilypond/out/bin/lilypond -dpreview -dresolution=150 -o ./out-www sesto-violin.ly Changing working directory to: `./out-www' Processing `sesto-violin.ly' Parsing... Interpreting music...[8] Preprocessing graphical objects... Finding the ideal number of pages... Fitting music on 1 page... Drawing systems... Layout output to `/tmp/lilypond-7lZ5so'... Converting to `sesto-violin.pdf'... Deleting `/tmp/lilypond-7lZ5so'... Layout output to `sesto-violin.preview.eps'... Converting to `sesto-violin.preview.pdf'... warning: `(gs -q -dNODISPLAY -dNOSAFER -dNOPAUSE -dBATCH -dEPSCrop -dAutoRotatePages=/None -dPrinted=false /tmp/lilypond-4CZcrq)' failed (256) fatal error: failed files: "sesto-violin.ly" make[3]: *** [GNUmakefile:18: out-www/sesto-violin.png] Error 1 This happens not only with sesto-violin.ly, but also orchestra.ly, bach-bwv610.ly, sesto-piano.ly, tab-example.ly, stockhausen-klavierstueckII.ly, bach-schenker.ly, aucun-snippet.ly, theory.ly, granados.ly, cary.ly or chart.ly (take your pick). Unlike my other recent regtest/xdvipdfmx problem, I was able to reproduce this with both my native Fedora system and LilyDev-2/Ubuntu -- even with a pristing git clone. (Which makes it very weird that I seem to be the only one getting that problem :-) Do you have https://gitlab.com/lilypond/lilypond/-/merge_requests/148/diffs?commit_id=c002d5e8c2e737cefbafa84f4b66e4b3bbc6d111 in your repo? Otherwise that may well be the problem. Jonas wanted to push this one yesterday, but it seems that did not happen yet. Cheers, Michael Cheers, -- V.
Re: xdvipdfmx fails with some regtests (“Invalid object”)
On 6/17/20, Werner LEMBERG wrote: > It seems that you probably have to update your TeX system... Update or downgrade? I’m using the latest fc32 repos. Besides, I haven’t changed anything on my OS recently and this started happening only a week or so ago; isn’t it possible that this is triggered by something introduced in our build system? Cheers, -- V.
“make doc” broken because of gs eps->pdf failing
Greetings everyone, I’ve noticed a recent regression with `make doc’: when it reaches Documentation/ly-examples, gs fails on a bunch of files with the following error message: Command: /home/work/lilypond/out/bin/lilypond -dpreview -dresolution=150 -o ./out-www sesto-violin.ly Changing working directory to: `./out-www' Processing `sesto-violin.ly' Parsing... Interpreting music...[8] Preprocessing graphical objects... Finding the ideal number of pages... Fitting music on 1 page... Drawing systems... Layout output to `/tmp/lilypond-7lZ5so'... Converting to `sesto-violin.pdf'... Deleting `/tmp/lilypond-7lZ5so'... Layout output to `sesto-violin.preview.eps'... Converting to `sesto-violin.preview.pdf'... warning: `(gs -q -dNODISPLAY -dNOSAFER -dNOPAUSE -dBATCH -dEPSCrop -dAutoRotatePages=/None -dPrinted=false /tmp/lilypond-4CZcrq)' failed (256) fatal error: failed files: "sesto-violin.ly" make[3]: *** [GNUmakefile:18: out-www/sesto-violin.png] Error 1 This happens not only with sesto-violin.ly, but also orchestra.ly, bach-bwv610.ly, sesto-piano.ly, tab-example.ly, stockhausen-klavierstueckII.ly, bach-schenker.ly, aucun-snippet.ly, theory.ly, granados.ly, cary.ly or chart.ly (take your pick). Unlike my other recent regtest/xdvipdfmx problem, I was able to reproduce this with both my native Fedora system and LilyDev-2/Ubuntu -- even with a pristing git clone. (Which makes it very weird that I seem to be the only one getting that problem :-) Cheers, -- V.
Re: xdvipdfmx fails with some regtests (“Invalid object”)
> xdvipdfmx:fatal: typecheck: Invalid object type: -1 7 (line 2161) > > This is on a Fedora system with xdvipdfmx 20190225 and gs 9.52. It seems that you probably have to update your TeX system... https://tex.stackexchange.com/questions/490616/xdvipdfmx-typecheck-problem If this is not a feasible solution please contact the TeX maintainers of your distribution. An alternative is to directly use TeXLive together with its binaries, which use static linking as much as possible. Werner
xdvipdfmx fails with some regtests (“Invalid object”)
Hey everybody, `make doc’ has been broken for nearly a week on my system (even with a clean git clone), with the following error: Making input/regression/out-www/collated-files.list < 1368 files Making input/regression/out-www/collated-files.texi < tely Making input/regression/out-www/collated-files.pdf < texi Command: cd ./out-www; texi2pdf --batch -q -I input/regression \ -o collated-files.tmp.pdf collated-files.texi < /dev/null xdvipdfmx:fatal: typecheck: Invalid object type: -1 7 (line 2161) No output PDF file written. /usr/bin/texi2dvi: scripts/build/xetex-with-options.sh exited with bad status, quitting. The "line 2161" doesn’t vary; it actually refers to xdvipdfmx itself and not to our source files: https://duckduckgo.com/?q="line+2161"+"xdvipdfmx; This is on a Fedora system with xdvipdfmx 20190225 and gs 9.52. It took me hours to track it down to the exact regtests that trigger this; this happens iff any of the following files is found in input/regression: ├── header-score-multiple.ly ├── header-score-reordered.ly ├── header-toplevel-multiple.ly ├── markup-cyclic-reference.ly ├── skiptypesetting-all-true.ly ├── tag-filter.ly ├── tag-group.ly └── tag-multiple.ly If these files have anything in common (and that isn’t shared by any of the other regtests), I don’t have a clue as to what it might be. Meanwhile, I’ll have to remove these regtests of all my git branches if I have any chance of being able to `make doc’ again (actually there’s another error I need to track down, but it seems unrelated; I’ll open another thread about it). Cheers, -- V.
