Re: Fixing regressions and serious issues

[Jean Abou Samra]

Le 18/09/2022 à 18:35, Jonas Hahnfeld a écrit :

On Sun, 2022-09-18 at 18:17 +0200, Jean Abou Samra wrote:

By the way, if you happen to still have a binary of what is now
2.23.13 with debugging symbols, that would save me time.

Yes, I always try to keep around the latest build, for accessing logs
and debug symbols. I uploaded the lilypond.exe before stripping here:
https://cloud.hahnjo.de/index.php/s/ZB8mG82TNpLwi3b Let me know if you
also need the Dlls.


I'd like to investigate the crash when loading the edition-engraver,
which is 100% reproducible for me, but at a glance doesn't seem related
to this GC Heisenbug.

I saw your comment, but you need an account to even read the
openlilylib forum, so I didn't bother...




OK, I've tracked this one down, and sadly it's not going to be fun either.

https://gitlab.com/lilypond/lilypond/-/issues/6430

Re: Fixing regressions and serious issues

[Andrew Bernard]

Gosh, to both.

All are welcome to the OLL forum should they so feel like it.

One reason not to open a forum is that there are people who may not want 
their postings disseminated and indexed, for whatever reason, even 
though this instance is purely a technical discussion platform.


If you want to contribute to the OLL development then I'm afraid you 
need to sign up for a github account.


Andrew


On 19/09/2022 6:41 am, anthony wrote:

On 18/09/2022 20:38, Jean Abou Samra wrote:

Requiring an account to post is understandable, but I'd make
any forum public for reading by default unless there are reasons
not to. Subscribing to read posts is not a big barrier, but people
are driven by curiosity, and if you need to take administrative
steps before reading a post it can turn you away.


From my point of view, the FEWER accounts I have the happier I am. 
Having to create an account is a massive turn-off that, if I don't 
have a burning desire for what's on offer, I just won't bother.


(And what happens if I create an account, forget all about it, and 
then try and create a new account six months later? The friction is 
greatly increased because I can't create a new account, but can't get 
into the old account either ...)


The other thing is, how much personal information does it want? The 
best internet security is just not to be on the net - if a site does 
not have your information, it can't lose it! Even when I *want* 
something, I regularly abandon any attempt half-way through because I 
don't want to provide the information they're demanding.


Cheers,
Wol

Re: Fixing regressions and serious issues

[anthony]

On 18/09/2022 20:38, Jean Abou Samra wrote:

Requiring an account to post is understandable, but I'd make
any forum public for reading by default unless there are reasons
not to. Subscribing to read posts is not a big barrier, but people
are driven by curiosity, and if you need to take administrative
steps before reading a post it can turn you away.


From my point of view, the FEWER accounts I have the happier I am. 
Having to create an account is a massive turn-off that, if I don't have 
a burning desire for what's on offer, I just won't bother.


(And what happens if I create an account, forget all about it, and then 
try and create a new account six months later? The friction is greatly 
increased because I can't create a new account, but can't get into the 
old account either ...)


The other thing is, how much personal information does it want? The best 
internet security is just not to be on the net - if a site does not have 
your information, it can't lose it! Even when I *want* something, I 
regularly abandon any attempt half-way through because I don't want to 
provide the information they're demanding.


Cheers,
Wol

Re: Fixing regressions and serious issues

[Jean Abou Samra]

Le 18/09/2022 à 18:45, Andrew Bernard a écrit :

Hi Jonas,

I run the OpenLilyLib Discourse forum and have taken over from Urs 
managing OLL (where more work will be happening soon!).



Interesting, would you mind sharing about the work you
plan? (Preferably in a separate thread)



Is it too onerous to register with a username and password? It's 
totally free. One cannot make forums totally open because of spammers, 
although I may consider making it readable to the public, but given 
it's a group of about 20 so far there does not seem much point.




Requiring an account to post is understandable, but I'd make
any forum public for reading by default unless there are reasons
not to. Subscribing to read posts is not a big barrier, but people
are driven by curiosity, and if you need to take administrative
steps before reading a post it can turn you away.


If you register, your personal data is kept totally private and not 
disseminated or used in any way apart from providing a login to the 
service.


You are very welcome to join.

Re: Fixing regressions and serious issues

[Andrew Bernard]

Hi Jonas,

I run the OpenLilyLib Discourse forum and have taken over from Urs 
managing OLL (where more work will be happening soon!).


Is it too onerous to register with a username and password? It's totally 
free. One cannot make forums totally open because of spammers, although 
I may consider making it readable to the public, but given it's a group 
of about 20 so far there does not seem much point. If you register, your 
personal data is kept totally private and not disseminated or used in 
any way apart from providing a login to the service.


You are very welcome to join.

Andrew


On 19/09/2022 2:35 am, Jonas Hahnfeld via Discussions on LilyPond 
development wrote:

I saw your comment, but you need an account to even read the
openlilylib forum, so I didn't bother...

Re: Fixing regressions and serious issues

[Jonas Hahnfeld via Discussions on LilyPond development]

On Sun, 2022-09-18 at 18:17 +0200, Jean Abou Samra wrote:
> By the way, if you happen to still have a binary of what is now
> 2.23.13 with debugging symbols, that would save me time.

Yes, I always try to keep around the latest build, for accessing logs
and debug symbols. I uploaded the lilypond.exe before stripping here:
https://cloud.hahnjo.de/index.php/s/ZB8mG82TNpLwi3b Let me know if you
also need the Dlls.

> I'd like to investigate the crash when loading the edition-engraver,
> which is 100% reproducible for me, but at a glance doesn't seem related
> to this GC Heisenbug.

I saw your comment, but you need an account to even read the
openlilylib forum, so I didn't bother...


signature.asc
Description: This is a digitally signed message part

Re: Fixing regressions and serious issues

[Jean Abou Samra]

Le 18/09/2022 à 14:46, Jonas Hahnfeld a écrit :

On Fri, 2022-09-16 at 13:36 +0200, Jonas Hahnfeld via Discussions on
LilyPond development wrote:

On Wed, 2022-09-14 at 22:35 +0200, Jean Abou Samra wrote:

Le 14/09/2022 à 22:16, Jonas Hahnfeld a écrit :

On Wed, 2022-07-20 at 11:39 +0200, Jonas Hahnfeld via Discussions on
LilyPond development wrote:
What do we do about this one? Over the past couple of weeks, I tried
quite a number of ideas, with no success so far.

Thanks a lot for working on this even if it didn't succeed so far.

Just in case for others: Jonas shared some details about what he tried
in https://github.com/ivmai/bdwgc/issues/454#issuecomment-1244375504

I've tried some more and probably developed an understanding of what's
happening - I will post on the upstream issue later. For our use case,
however, we can "cheat" a bit because we statically link both bdwgc and
libguile; https://gitlab.com/lilypond/lilypond/-/merge_requests/1627
should work around the crashes (at least they do for me in Wine).

Jean thankfully tested the binaries I produced with that change, and
they don't seem to completely break while fixing the crashes with the
originally reported scores.





I'm sorry for not having been clear on this point: yes, I
haven't observed crashes on the original reports; on the other
hand, I tried 2.23.10 (supposed to be buggy), and didn't get
crashes either, so I dunno :(  This is the glory of Heisenbugs.

The MR at least fixed the crashes you observed with Wine, so
hopefully it should at least reduce the frequency of crashes.
At this point, there seems to be a good chance that we are dealing
with a bug in BDWGC itself, so I think we can call it good enough
if it doesn't pop up for real-world use cases on our end, and
allow the 2.24 in that case, even if the underlying issue in
BDWGC is not fully understood.

The conclusion is that I am happy this change was included in
2.23.13, as we will be able to ask Windows users if these binaries
seem to fix the issue for them.

By the way, if you happen to still have a binary of what is now
2.23.13 with debugging symbols, that would save me time. I'd like
to investigate the crash when loading the edition-engraver, which
is 100% reproducible for me, but at a glance doesn't seem related
to this GC Heisenbug.

Re: Fixing regressions and serious issues

[Jonas Hahnfeld via Discussions on LilyPond development]

On Fri, 2022-09-16 at 13:36 +0200, Jonas Hahnfeld via Discussions on
LilyPond development wrote:
> On Wed, 2022-09-14 at 22:35 +0200, Jean Abou Samra wrote:
> > Le 14/09/2022 à 22:16, Jonas Hahnfeld a écrit :
> > > On Wed, 2022-07-20 at 11:39 +0200, Jonas Hahnfeld via Discussions on
> > > LilyPond development wrote:
> > > What do we do about this one? Over the past couple of weeks, I tried
> > > quite a number of ideas, with no success so far.
> > 
> > Thanks a lot for working on this even if it didn't succeed so far.
> > 
> > Just in case for others: Jonas shared some details about what he tried
> > in https://github.com/ivmai/bdwgc/issues/454#issuecomment-1244375504
> 
> I've tried some more and probably developed an understanding of what's
> happening - I will post on the upstream issue later. For our use case,
> however, we can "cheat" a bit because we statically link both bdwgc and
> libguile; https://gitlab.com/lilypond/lilypond/-/merge_requests/1627
> should work around the crashes (at least they do for me in Wine).

Jean thankfully tested the binaries I produced with that change, and
they don't seem to completely break while fixing the crashes with the
originally reported scores. Note that there are apparently still
crashes with debugging scores that Jean produced, calling (gc) on every
translator slot...

With the fix of the user scores merged though, I've now started to
produce 2.23.13, I'll announce it once ready later today.

What are the updated feelings on the plan of branching next week? If we
agree to stick to that plan, I will bump master to 2.23.80 in
preparation of the first release candidate. If not, I'll go with
2.23.14 with the possibility that this version will never see the light
of the world...

Jonas


signature.asc
Description: This is a digitally signed message part

Re: Fixing regressions and serious issues

[Werner LEMBERG]

>> I've tried some more and probably developed an understanding of
>> what's happening - I will post on the upstream issue later. For our
>> use case, however, we can "cheat" a bit because we statically link
>> both bdwgc and libguile;
>> https://gitlab.com/lilypond/lilypond/-/merge_requests/1627 should
>> work around the crashes (at least they do for me in Wine).
> 
> This is absolutely fantastic! Huge congrats and thank you.

+1


Werner

Re: Fixing regressions and serious issues

[Jean Abou Samra]

Le 16/09/2022 à 13:36, Jonas Hahnfeld a écrit :

I've tried some more and probably developed an understanding of what's
happening - I will post on the upstream issue later. For our use case,
however, we can "cheat" a bit because we statically link both bdwgc and
libguile; https://gitlab.com/lilypond/lilypond/-/merge_requests/1627
should work around the crashes (at least they do for me in Wine).




This is absolutely fantastic! Huge congrats and thank you.



It's not fully clear to me what "next" means here, but I really want to
do an unstable release this weekend (probably on Sunday) unless there
are very good arguments not to. The reason is simply that I don't have
much time the weekend of the 24th/25th and no time at all the week
after for an eventual branching.
For !1510 I'd argue that it's simply too late for this weekend.




OK, so be it.

Re: Fixing regressions and serious issues

[Jonas Hahnfeld via Discussions on LilyPond development]

On Wed, 2022-09-14 at 22:35 +0200, Jean Abou Samra wrote:
> Le 14/09/2022 à 22:16, Jonas Hahnfeld a écrit :
> > On Wed, 2022-07-20 at 11:39 +0200, Jonas Hahnfeld via Discussions on
> > LilyPond development wrote:
> > What do we do about this one? Over the past couple of weeks, I tried
> > quite a number of ideas, with no success so far.
> 
> Thanks a lot for working on this even if it didn't succeed so far.
> 
> Just in case for others: Jonas shared some details about what he tried
> in https://github.com/ivmai/bdwgc/issues/454#issuecomment-1244375504

I've tried some more and probably developed an understanding of what's
happening - I will post on the upstream issue later. For our use case,
however, we can "cheat" a bit because we statically link both bdwgc and
libguile; https://gitlab.com/lilypond/lilypond/-/merge_requests/1627
should work around the crashes (at least they do for me in Wine).


> > Questions:
> > a) Do we stick to the plan of branching next week, after the planned
> > release of 2.23.13 this weekend?
> > b) If we decide to branch and eventually arrive in December without a
> > fix, do we block the release?
> > 
> > At the current moment, branching without a "guaranteed" release date
> > bears a certain risk that we will end up with something half-finished
> > while blocking progress before resuming a new cycle of development
> > releases. What do people think?
> 
> b) I would say yes. It would be sad, but for better or worse, our
> significant part of our user base is on Windows, as far as I know.
> 
> a) I don't know.
> 
> One thing I can say is that finalizing !1510 (-dcompile-scheme-code)
> is going to take me a day or two (not helped by being sick of working
> on that problem), and it wouldn't be unreasonable to have it in the
> unstable release before branching, as it changes the execution of Scheme
> code in some significant respects (compiling is optional, but the new
> error handling isn't). So I'd consider it reasonable to delay the release
> to next week-end for now, and see what happens for the Windows crashes.

It's not fully clear to me what "next" means here, but I really want to
do an unstable release this weekend (probably on Sunday) unless there
are very good arguments not to. The reason is simply that I don't have
much time the weekend of the 24th/25th and no time at all the week
after for an eventual branching.
For !1510 I'd argue that it's simply too late for this weekend.

Jonas


signature.asc
Description: This is a digitally signed message part

Re: Fixing regressions and serious issues

[Jean Abou Samra]

Le 14/09/2022 à 22:16, Jonas Hahnfeld a écrit :

On Wed, 2022-07-20 at 11:39 +0200, Jonas Hahnfeld via Discussions on
LilyPond development wrote:
What do we do about this one? Over the past couple of weeks, I tried
quite a number of ideas, with no success so far.




Thanks a lot for working on this even if it didn't succeed so far.

Just in case for others: Jonas shared some details about what he tried
in https://github.com/ivmai/bdwgc/issues/454#issuecomment-1244375504



Questions:
a) Do we stick to the plan of branching next week, after the planned
release of 2.23.13 this weekend?
b) If we decide to branch and eventually arrive in December without a
fix, do we block the release?

At the current moment, branching without a "guaranteed" release date
bears a certain risk that we will end up with something half-finished
while blocking progress before resuming a new cycle of development
releases. What do people think?




b) I would say yes. It would be sad, but for better or worse, our
significant part of our user base is on Windows, as far as I know.

a) I don't know.

One thing I can say is that finalizing !1510 (-dcompile-scheme-code)
is going to take me a day or two (not helped by being sick of working
on that problem), and it wouldn't be unreasonable to have it in the
unstable release before branching, as it changes the execution of Scheme
code in some significant respects (compiling is optional, but the new
error handling isn't). So I'd consider it reasonable to delay the release
to next week-end for now, and see what happens for the Windows crashes. I
might manage to spend some time on that next week, but I'm not sure,
and it's not like I'm particularly good at that kind of low-level work.

One thing we could try is biting the bullet and attempting to
cross-compile Guile 3 for MinGW. We might have a slight chance that
the problem goes away, or becomes different in a way that makes
it easier to understand. And this is something we want to do
on the long term anyway.

Jean

Re: Fixing regressions and serious issues

[Jonas Hahnfeld via Discussions on LilyPond development]

On Wed, 2022-07-20 at 11:39 +0200, Jonas Hahnfeld via Discussions on
LilyPond development wrote:
> On Thu, 2022-07-14 at 17:38 +0200, Jean Abou Samra wrote:
> 
> > * “Heisen-crashes on Windows with large scores”
> > 
> >    https://gitlab.com/lilypond/lilypond/-/issues/6361
> > 
> >    A nasty and poorly understood GC problem on Windows,
> >    needs some tough debugging.
> 
> This is currently the only issue marked ~Critical, and I agree this
> must be addressed before a stable release. I hope I can come back to
> this in the next weeks.

What do we do about this one? Over the past couple of weeks, I tried
quite a number of ideas, with no success so far.

Questions:
a) Do we stick to the plan of branching next week, after the planned
release of 2.23.13 this weekend?
b) If we decide to branch and eventually arrive in December without a
fix, do we block the release?

At the current moment, branching without a "guaranteed" release date
bears a certain risk that we will end up with something half-finished
while blocking progress before resuming a new cycle of development
releases. What do people think?

Jonas


signature.asc
Description: This is a digitally signed message part

Re: Fixing regressions and serious issues

[Jean Abou Samra]

Le 24/07/2022 à 17:20, Jonas Hahnfeld a écrit :

I didn't want to imply this problem isn't important, but it's also
difficult to address (at least from my attempts).


https://gitlab.com/lilypond/lilypond/-/merge_requests/1510

Re: Fixing regressions and serious issues

[Jonas Hahnfeld via Discussions on LilyPond development]

On Wed, 2022-07-20 at 13:56 +0200, Thomas Morley wrote:
> Am Mi., 20. Juli 2022 um 11:40 Uhr schrieb Jonas Hahnfeld via
> Discussions on LilyPond development :
> > 
> > On Thu, 2022-07-14 at 17:38 +0200, Jean Abou Samra wrote:
> > > * “GUILE 2.2 doesn't provide source locations”
> > > 
> > >https://gitlab.com/lilypond/lilypond/-/issues/5992
> > > 
> > >Needs figuring out if executing Scheme code with
> > >`compile` is OK performance-wise and how to get
> > >it to display source location info.
> > 
> > I agree this is annoying, and it would be great to improve the
> > situation. I'm not sure if this should block the release though.
> 
> I beg to differ.
> For me it's the most pressing and annoying issue. A stable release
> without good error messages is unusable for huge custom scheme
> codings.
> Granted, recently some workarounds were discussed on -user. Alas, I
> can't imagine a stable version needing those workarounds right from
> it's release date.
> Even it does not crash lilypond, I'd label it Critical as well.

I didn't want to imply this problem isn't important, but it's also
difficult to address (at least from my attempts). So the real question
is: If this is the only remaining issue, would we rather still have a
release or no stable release at all until it can be fixed. That's what
~Critical means in the end.

Jonas


signature.asc
Description: This is a digitally signed message part

Re: Fixing regressions and serious issues

[Jean Abou Samra]

Le 20/07/2022 à 11:39, Jonas Hahnfeld a écrit :

On Thu, 2022-07-14 at 17:38 +0200, Jean Abou Samra wrote:

Hi,

In https://lists.gnu.org/archive/html/lilypond-devel/2022-05/msg00099.html,
we talked about targeting the end of this year for the next
stable release. That probably means branching around
the beginning of September, right?

Possibly, yes. If wanted, I can try and prepare a "draft timeline"
based on what we did two years ago and what needs to happen until
around what deadline...



For me at least, that would be helpful.



I'd like to call for attention on the regressions we have

as well as some other serious things that should go
into this release: basically, (assuming branching in September)
there is a month and a half left to take care of those.
This is not all that long if not very short, especially since
some people (like me) tend to be less active in August.

I agree, even though I'd like to note that only those marked ~Critical
actually block the release. There are a number of ~Regression issues
that have been there for 2.22.x or even 2.20.0. Whether we want to re-
classify some of the more recent ones as ~Critical, is of course up for
discussion. And fixing issues is obviously still a good idea 



You have a good point that which of these should block the
release is not very clear. I guess it would need discussion,
and the energy spent discussing will have been wasted if the
issue just gets fixed before the release, so for now I think
the best investment of energy (or my energy at least) is in
fixing as many as possible of these issues. If some remain by
the time we want to make the release, it will be time to discuss
whether they should block it.

Re: Fixing regressions and serious issues

[Thomas Morley]

Am Mi., 20. Juli 2022 um 11:40 Uhr schrieb Jonas Hahnfeld via
Discussions on LilyPond development :
>
> On Thu, 2022-07-14 at 17:38 +0200, Jean Abou Samra wrote:

> I agree, even though I'd like to note that only those marked ~Critical
> actually block the release. There are a number of ~Regression issues
> that have been there for 2.22.x or even 2.20.0. Whether we want to re-
> classify some of the more recent ones as ~Critical, is of course up for
> discussion. And fixing issues is obviously still a good idea 
>
> > [...]
> > * “Heisen-crashes on Windows with large scores”
> >
> >https://gitlab.com/lilypond/lilypond/-/issues/6361
> >
> >A nasty and poorly understood GC problem on Windows,
> >needs some tough debugging.
>
> This is currently the only issue marked ~Critical, and I agree this
> must be addressed before a stable release.

+1

> I hope I can come back to
> this in the next weeks.
>
> > * “GUILE 2.2 doesn't provide source locations”
> >
> >https://gitlab.com/lilypond/lilypond/-/issues/5992
> >
> >Needs figuring out if executing Scheme code with
> >`compile` is OK performance-wise and how to get
> >it to display source location info.
>
> I agree this is annoying, and it would be great to improve the
> situation. I'm not sure if this should block the release though.

I beg to differ.
For me it's the most pressing and annoying issue. A stable release
without good error messages is unusable for huge custom scheme
codings.
Granted, recently some workarounds were discussed on -user. Alas, I
can't imagine a stable version needing those workarounds right from
it's release date.
Even it does not crash lilypond, I'd label it Critical as well.

Cheers,
  Harm

Re: Fixing regressions and serious issues

[Jonas Hahnfeld via Discussions on LilyPond development]

On Thu, 2022-07-14 at 17:38 +0200, Jean Abou Samra wrote:
> Hi,
> 
> In https://lists.gnu.org/archive/html/lilypond-devel/2022-05/msg00099.html,
> we talked about targeting the end of this year for the next
> stable release. That probably means branching around
> the beginning of September, right?

Possibly, yes. If wanted, I can try and prepare a "draft timeline"
based on what we did two years ago and what needs to happen until
around what deadline...

> I'd like to call for attention on the regressions we have
> 
> as well as some other serious things that should go
> into this release: basically, (assuming branching in September)
> there is a month and a half left to take care of those.
> This is not all that long if not very short, especially since
> some people (like me) tend to be less active in August.

I agree, even though I'd like to note that only those marked ~Critical
actually block the release. There are a number of ~Regression issues
that have been there for 2.22.x or even 2.20.0. Whether we want to re-
classify some of the more recent ones as ~Critical, is of course up for
discussion. And fixing issues is obviously still a good idea 

> [...]
> 
> * Cairo anyone? What's the status of
>    https://gitlab.com/lilypond/lilypond/-/merge_requests/913?
>    Not really a blocker, but I thought the plan was to
>    integrate Cairo in the official binaries for this stable
>    release.

The merge request needs work; on Linux it currently finds system libs:
https://gitlab.com/lilypond/lilypond/-/merge_requests/913#note_678654750
I expect this will also need work for FreeBSD, macOS and Windows; first
getting it to build and then some serious testing.

> * “Heisen-crashes on Windows with large scores”
> 
>    https://gitlab.com/lilypond/lilypond/-/issues/6361
> 
>    A nasty and poorly understood GC problem on Windows,
>    needs some tough debugging.

This is currently the only issue marked ~Critical, and I agree this
must be addressed before a stable release. I hope I can come back to
this in the next weeks.

> * “GUILE 2.2 doesn't provide source locations”
> 
>    https://gitlab.com/lilypond/lilypond/-/issues/5992
> 
>    Needs figuring out if executing Scheme code with
>    `compile` is OK performance-wise and how to get
>    it to display source location info.

I agree this is annoying, and it would be great to improve the
situation. I'm not sure if this should block the release though.


Jonas


signature.asc
Description: This is a digitally signed message part

Re: Fixing regressions and serious issues

[Jean Abou Samra]

Le 14/07/2022 à 17:38, Jean Abou Samra a écrit :

Hi,

In 
https://lists.gnu.org/archive/html/lilypond-devel/2022-05/msg00099.html,

we talked about targeting the end of this year for the next
stable release. That probably means branching around
the beginning of September, right? I'd like to call
for attention on the regressions we have
 


as well as some other serious things that should go
into this release: basically, (assuming branching in September)
there is a month and a half left to take care of those.
This is not all that long if not very short, especially since
some people (like me) tend to be less active in August.




I should have added that of course some of these can be fixed
after branching and cherry-picked. But it's better if cherry-picks
are not too invasive, e.g., changing the way Scheme code is executed
might be a little too much.

There are also \fine issues, but I guess Dan is on track with those.

Jean

Fixing regressions and serious issues

[Jean Abou Samra]

Hi,

In https://lists.gnu.org/archive/html/lilypond-devel/2022-05/msg00099.html,
we talked about targeting the end of this year for the next
stable release. That probably means branching around
the beginning of September, right? I'd like to call
for attention on the regressions we have

as well as some other serious things that should go
into this release: basically, (assuming branching in September)
there is a month and a half left to take care of those.
This is not all that long if not very short, especially since
some people (like me) tend to be less active in August.

Dan and Jonas deserve thanks for having both worked on some of
these recently. As you might have noticed, much of my work lately
was also dedicated to fixing regressions. But there are quite
a number of them remaining, and a they make a lot of work in
total. So, if you're wondering what to contribute next, please
consider putting in some time into one of these.

Specifically, these seem significant to me:


* Cairo anyone? What's the status of
  https://gitlab.com/lilypond/lilypond/-/merge_requests/913?
  Not really a blocker, but I thought the plan was to
  integrate Cairo in the official binaries for this stable
  release.


* “Heisen-crashes on Windows with large scores”

  https://gitlab.com/lilypond/lilypond/-/issues/6361

  A nasty and poorly understood GC problem on Windows,
  needs some tough debugging.


* “GUILE 2.2 doesn't provide source locations”

  https://gitlab.com/lilypond/lilypond/-/issues/5992

  Needs figuring out if executing Scheme code with
  `compile` is OK performance-wise and how to get
  it to display source location info.


* “\markup \score in \paper can no longer have \layout block”

  https://gitlab.com/lilypond/lilypond/-/issues/6364

  Needs investigating what broke with Guile 2 and its
  module system.


* “No cropping when using lilypond-book-preamble include”

  https://gitlab.com/lilypond/lilypond/-/issues/6235

  Needs either reverting to the old behavior or a Changes
  entry and a convert-ly rule.


* “\bar creates a spurious staff”

  https://gitlab.com/lilypond/lilypond/-/issues/6341


* “Unterminated beam causes intimidating programming errors"

  https://gitlab.com/lilypond/lilypond/-/issues/6372


* “Changed behavior of staves with differing repeat structures
  breaks doc example"

  https://gitlab.com/lilypond/lilypond/-/issues/6373

  Needs at least amending a doc example, but first checking
  that behavior changes are unlikely to be noticed often,
  and if not, add a Changes entry and/or a compilation warning
  if staves have different repeat structures.



Again, if you have some energy, fixing one of these
won't be wasted time.

Thanks,
Jean