On Sun, Jul 28, 2024 at 10:44 PM Michael Käppler <xmichae...@web.de> wrote: > I would like to bring up the question if it is really necessary that we > do all testing > "end-to-end", i.e. from input ly code to pixel-based graphical output. > IIUC, we have an intermediate graphical language, consisting of the > various stencil commands, > that is mostly backend-agnostic. (modulo such things like `\image` or > `\postscript`) > > What if we resuscitate the SCM backend in a different form, say, > outputting all stencils in serialized form > as JSON or XML and first compare these files? ... > IMHO, such an approach would also scale better - with the current > approach we have low coverage e.g. for the SVG backend. > To improve this, we would have to test all files in all backends, don't > we? Let's imagine we add an MusicXML backend...
We have had this approach for a long time, and I removed it quite recently. Have a look at commit 22d10325e61c8a4d2edf6d7b08eec931c82a8167 and 9f115c4017d2ea1db73f07c79823f86b321af654. The commit messages explain why this isn't a great idea. It also doesn't really solve the problem that Werner alludes to: the XML you produce still has geometric information, and you'll still have to come up with a sensible algorithm to score that data, which is what Werner was asking for in the first place. The SVG backend is not maintained and should really be put out of its misery. The Cairo backend supports SVG output that is guaranteed pixel-perfect rendering of the same PDF/PNG images. IMO It is a more fruitful approach to push down the concept of "backend" into Cairo, retire the PS backend, and let Cairo deal with ensuring different formats look the same and all the minutiae of producing PS/PDF (loading font files, font subsetting, etc.) As long as we don't have a MusicXML backend, it makes no sense to take technical decisions to facilitate one. I wonder why we are not exploring the other side of the failure that Werner pointed out: you could have a much smaller test that exercises the same feature where a symbol went missing (was it a Tablature test?). If the test image has just a few symbols, a missing symbol will be a comparatively larger change, and will stand out in the current regression testing framework. It also provides a more granular test for debugging purposes. -- Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen
