Hello,
While doing some drums transcription, I've been missing a feature to
synchronize the display and midi playback. So I decided to give it a go,
and made draft patches for both Lilypond and Frescobaldi.
The idea is to add point-and-click-like links inside midi files, that
players can parse and use to synchronize their display. Practically
speaking, those links are generated as midi meta events, using the
sequencer-specific 0x7F code. There are two kinds of events (start and
end), as follows:
FF 7F sequencer-specific
ll* event data length
4C 59 xx message identifier: "LYS" for start, "LYE" for end
dd* textedit link
In the current implementation, those are generated only for notes (LYS
at the same time as note on, and LYE as note off), but they could be
extended for any other element. The idea being that a midi player able
to display the score (or source) would highlight elements while they are
playing, and that you could click on the score to seek the player to the
corresponding position.
Here is a demo of this feature in action:
https://player.vimeo.com/video/1132810999
You can find the draft implementation here:
- Lilypond: https://gitlab.com/lilypond/lilypond/-/merge_requests/2790
- Frescobaldi: https://github.com/frescobaldi/frescobaldi/pull/2113
These are merely proof-of-concept implementations, as I'm not at all
familiar with both codebases, and have limited experience in C++ and Qt.
There are many open questions, among which:
- Is this feature of interest to anyone else?
- Should this be optional, or should it just follow existing
point-and-click options?
- Are there better alternatives for midi messages for this kind of
events? Any existing solution?
- Those messages make midi files much bigger, should we try to
deduplicate data (eg. sending filenames once and referencing them later
with an ID)?
- Are my implementations correct?
Thank you for your comments.
--
Nicolas Joyard
