UTF-8-aware backwards string searching in Guile, or: fixing centered lyrics ignoring punctuation

[Alexander Kobel]

Dear all,

I'm happily using center-lyrics-ignoring-punctuation.ily in the version from https://lists.gnu.org/archive/html/lilypond-user/2016-12/msg00382.html for a while now. (It's attached as well.)

Now I stumbled across the following unexpected warning:

\version "2.19.82"
\include "center-lyrics-ignoring-punctuation.ily"
{ d4 }
\addlyrics { à }

GNU LilyPond 2.19.82
Processing `test.ly'
Parsing...
Interpreting music...
Preprocessing graphical objects...
(process:2528): Pango-WARNING **: 17:49:34.874: Invalid UTF-8 string passed to 
pango_layout_set_text()

warning: no glyph for character U+FFFD in font 
`/usr/share/fonts/tex-gyre/texgyreschola-regular.otf'

Observations:

2. There is no U+FFFD (REPLACEMENT CHARACTER) anywhere in my input.








Thanks,
Alex

\version "2.19.50"  %% and higher

%% https://lists.gnu.org/archive/html/lilypond-user/2016-12/msg00382.html
%% http://lsr.di.unimi.it/LSR/Item?id=888

#(define space-set
  (list->char-set
    (string->list ".?-;,:„“‚‘«»‹›『』「」“”‘’–— */()[]{}|<>!`~&…†‡")))

#(define (width grob text)
   (let* ((X-extent
           (ly:stencil-extent (grob-interpret-markup grob text) X)))
     (if (interval-empty? X-extent)
         0
         (cdr X-extent))))

#(define (remove-suspended-note-heads stem note-heads)
   (let* ((nc (ly:grob-common-refpoint stem (car note-heads) X))
          (stem-coord
           (ly:grob-relative-coordinate stem stem X))
          (half-stem-thick
           (/ (ly:grob-property stem 'thickness) 2))
          (stem-dir (ly:grob-property stem 'direction)))
     (remove
      (lambda (nh)
        (if (positive? stem-dir)
            (> (ly:grob-relative-coordinate nh nc X)
               stem-coord)
            (< (ly:grob-relative-coordinate nh nc X)
               (- stem-coord half-stem-thick))))
      note-heads)))

#(define (make-center-on-word-callback grob center-on-chords)
   (let* ((text (ly:grob-property-data grob 'text))
          (syllable (markup->string text))
          (word-position
           (if (string-skip syllable space-set)
               (string-skip syllable space-set)
               0))
          (word-end
           (if (string-skip-right syllable space-set)
               (+ (string-skip-right syllable space-set) 1)
               (string-length syllable)))
          (preword (substring syllable 0 word-position))
          (word (substring syllable word-position word-end))
          (preword-width (width grob preword))
          (word-width (width grob (if (string-null? syllable) text word)))
          (note-column (ly:grob-parent grob X))
          (stem (ly:grob-object note-column 'stem))
          (stem-dir (ly:grob-property stem 'direction))
          (sys (ly:grob-system grob))
          (nh-ls
           (if (ly:grob-array? (ly:grob-object note-column 'note-heads))
               (ly:grob-array->list (ly:grob-object note-column 'note-heads))
               '()))
          (full-column-width
           (interval-length (ly:relative-group-extent nh-ls note-column X)))
          (note-column-width
           (interval-length
            (ly:relative-group-extent
             (remove-suspended-note-heads stem nh-ls) note-column X))))
     (-
      (*
       (/ (if center-on-chords
              (if (positive? stem-dir)
                  (- full-column-width word-width)
                  (- (* 2 note-column-width) full-column-width word-width))
              (- note-column-width word-width))
         2)
       (1+ (ly:grob-property-data grob 'self-alignment-X)))
      preword-width)))

#(define (center-on-word grob) (make-center-on-word-callback grob #f))
#(define (center-on-word-on-chords grob) (make-center-on-word-callback grob #t))

\layout {
  \context {
    \Lyrics
    \override LyricText #'X-offset = #center-on-word-on-chords
  }
}

\version "2.19.82"

\include "center-lyrics-ignoring-punctuation.ily"

{ d4 }
\addlyrics { à }

1. The warning vanishes as soon as I replace the lyrics by àa, but not by aà. This makes me wonder whether the problem is in the backwards string-search in string-skip-right or in the substring routine used in the make-center-on-word-callback; the only reason I can imagine why this pops up is that some blind-to-Unicode slicing cuts some string in the middle of a multi-byte Unicode character. The warning is merely annoying, but harmless; but I suspect that there are unfortunate combinations where trailing "to-be-ignored" characters are missed, or "not-to-be-ignored" characters are split into an ill-defined character and a false "to-be-ignored". Does anyone have a hint how to approach this one? (Or is the answer just: be patient and hope for Guile v2?)

smime.p7s
Description: S/MIME Cryptographic Signature

_______________________________________________
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user