subject:"Re\: branching"

>> [...] if I delete a PDF file, say, `notation.pdf`, right now it
>> gets *not* rebuilt!
> 
> The new documentation build has much more accurate dependency
> tracking, so if you want to rebuild notation.pdf, you can just say
> so:
> 
>   make out=www out-www/en/notation.pdf

Aah, I tried without `out=www`.  This incantation is good enough for
me, thanks.  No further action needed.

>> Similarly, if I change a documentation string in an SCM file like
>> `define-markup-commands.scm`, the documentation doesn't get
>> rebuilt, either.
> 
> I can look at reintroducing the SCM -> texi dependencies.

Please do so.  Due to the auto-generation process, it is far from
trivial to find out which command has to be called.

>>  I consider this fundamental flaws.
> 
> I disagree. These flaws might be a bother for developers, but
> branching stable/2.22 is about not having user-visible regressions
> of lilypond itself, relative to 2.20, which has nothing to do with
> how developers experience working on lilypond.

OK.

>> For me, a freeze can only start if we agree that nothing
>> fundamental has to be changed or added.  IMHO, we are far away from
>> such a state.
> 
> Could you enumerate the issues you think are fundamental?

Well, I referred mainly to the build problems.  Since your goal to cut
stable/2.22 is a bit different my arguments are moot.


Werner

Re: branching stable/2.22?

2020-09-11 Thread Dan Eble

On Sep 11, 2020, at 15:19, James Lowe  wrote:
> 
> On 11/09/2020 20:13, Han-Wen Nienhuys wrote:
>>>  I consider this fundamental flaws.
>> I disagree. These flaws might be a bother for developers, but
>> branching stable/2.22 is about not having user-visible regressions of
>> lilypond itself, relative to 2.20, which has nothing to do with how
>> developers experience working on lilypond.
>> 
> I'm with Han-Wen on this.
> 
> PDF building has nothing whatsoever to do with what most users need. This is 
> the 'build' stuff that frankly can wait until we have fixed the 
> regressions/documentation of features.

I lean toward letting the people who want to freeze/stabilize/release get on 
with it.  I would hold back my \volta changes for the month or so that it takes 
to do this, work on related new features on my own, and contribute bug fixes as 
I find them.
— 
Dan

Re: branching stable/2.22?

2020-09-11 Thread James Lowe


On 11/09/2020 20:13, Han-Wen Nienhuys wrote:

  I consider this fundamental flaws.

I disagree. These flaws might be a bother for developers, but
branching stable/2.22 is about not having user-visible regressions of
lilypond itself, relative to 2.20, which has nothing to do with how
developers experience working on lilypond.


I'm with Han-Wen on this.

PDF building has nothing whatsoever to do with what most users need. 
This is the 'build' stuff that frankly can wait until we have fixed the 
regressions/documentation of features.


James

Re: branching stable/2.22?

2020-09-11 Thread Han-Wen Nienhuys

On Fri, Sep 11, 2020 at 7:08 PM Werner LEMBERG  wrote:
>
>
> >> > I don't see that in the current stage of upheaval of both
> >> > internals and build system and infrastructure, there is a point
> >> > in freezing off some half-baked intermediate state that hasn't
> >> > seen significant exposure to extensive testing.
> >>
> >> +1  It's too early IMHO.  Let's wait at laest a month.
> >
> > To be clear here: You mean at least one month before starting a
> > freeze?
>
> Yes.  The build infrastructure is still not ok.  For example, if I
> delete a PDF file, say, `notation.pdf`, right now it gets *not*
> rebuilt!

There is a comment about this:

# Do not specify phony targets (pdf, html) as dependencies to avoid building the
# webdoc tree every time.

We can make this accurate so your pdf file gets rebuilt, but it comes
at the cost of always building the webdoc tree even if there is no
work to do. This adds 5 to 10 seconds to a null-build, so I think it's
not a good tradeoff.

The new documentation build has much more accurate dependency
tracking, so if you want to rebuild notation.pdf, you can just say so:

  make out=www out-www/en/notation.pdf

> Similarly, if I change a documentation string in an SCM file
> like `define-markup-commands.scm`, the documentation doesn't get
> rebuilt, either.

I can look at reintroducing the SCM -> texi dependencies.

>  I consider this fundamental flaws.

I disagree. These flaws might be a bother for developers, but
branching stable/2.22 is about not having user-visible regressions of
lilypond itself, relative to 2.20, which has nothing to do with how
developers experience working on lilypond.

> > This is not my proposal anymore to just branch, but Han-Wen's idea of
> > having a freeze of 3-4 weeks before branching.
>
> For me, a freeze can only start if we agree that nothing fundamental
> has to be changed or added.  IMHO, we are far away from such a state.

Could you enumerate the issues you think are fundamental?

--
Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen

Re: branching stable/2.22?



>> This is not my proposal anymore to just branch, but Han-Wen's idea
>> of having a freeze of 3-4 weeks before branching.
> 
> For me, a freeze can only start if we agree that nothing fundamental
> has to be changed or added.  IMHO, we are far away from such a
> state.

What we can start, however, is a feature freeze, especially
syntax-wise.  So if you mean exactly that, I support that idea.


Werner

Re: branching stable/2.22?



>> > I don't see that in the current stage of upheaval of both
>> > internals and build system and infrastructure, there is a point
>> > in freezing off some half-baked intermediate state that hasn't
>> > seen significant exposure to extensive testing.
>> 
>> +1  It's too early IMHO.  Let's wait at laest a month.
> 
> To be clear here: You mean at least one month before starting a
> freeze?

Yes.  The build infrastructure is still not ok.  For example, if I
delete a PDF file, say, `notation.pdf`, right now it gets *not*
rebuilt!  Similarly, if I change a documentation string in an SCM file
like `define-markup-commands.scm`, the documentation doesn't get
rebuilt, either.  I consider this fundamental flaws.

> This is not my proposal anymore to just branch, but Han-Wen's idea of
> having a freeze of 3-4 weeks before branching.

For me, a freeze can only start if we agree that nothing fundamental
has to be changed or added.  IMHO, we are far away from such a state.


Werner

Re: branching stable/2.22?

2020-09-11 Thread Jonas Hahnfeld

Am Freitag, den 11.09.2020, 16:18 +0100 schrieb James Lowe:
> On 11/09/2020 15:22, David Kastrup wrote:
> > Jonas Hahnfeld  writes:
> > 
> > > Am Sonntag, den 06.09.2020, 13:40 +0200 schrieb Jonas Hahnfeld:
> > > > Am Sonntag, den 06.09.2020, 12:33 +0200 schrieb Han-Wen Nienhuys:
> > > > > Here is my proposal for how to go ahead:
> > > > > 
> > > > > * we build a 2.21.6 from master, and announce it widely as a 2.22
> > > > > pre-release version.
> > > > 
> > > > Adding Phil. I did a full test build of current master yesterday and it
> > > > worked fine with https://github.com/gperciva/gub/pull/80 in place.
> > > > With respect to wording, every 2.21.x is some kind of pre-release for
> > > > 2.22. If this proposal gets consensus, I'd emphasize that feature
> > > > development is over, but I'll leave that to native speakers 
> > > > 
> > > > > * we institute a X week freeze; during the freeze, we only merge
> > > > >- fixes for regressions
> > > > >- updates to the documentation
> > > > >- cleanups with no functional changes, with little risk (ie. 
> > > > > refrain
> > > > > from build system changes, for example).
> > > > > * after the X week freeze, if we still have open regressions, we tack
> > > > > on another few weeks of freeze to fix them.
> > > > > * if there are no regressions left, we branch stable/2.22, and release
> > > > > a new pre-release.
> > > > > 
> > > > > X could be up for discussion, but 3 or 4 weeks seems long enough to
> > > > > gather some feedback, but short enough that experimental/feature work
> > > > > should not be affected.
> > > > > 
> > > > > The objective of the freeze is to focus developer energy on fixing
> > > > > regressions rather than causing new regressions.
> > > > 
> > > > Sounds good to me.
> > > 
> > > To arrive at a decision, what do others think about this? Having a
> > > large silent mass is not really helpful for this kind of discussion (as
> > > it wasn't for the switch to GitLab).
> > 
> > Frankly I don't see the point in repeating points I already made and
> > call that "discussion".
> > 
> > I don't see that in the current stage of upheaval of both internals and
> > build system and infrastructure, there is a point in freezing off some
> > half-baked intermediate state that hasn't seen significant exposure to
> > extensive testing.
> > 
> 
> I cannot comment on 'Internals' but 'build system' and 'infrastructure' 
> are not, I think, issues that really concern 'average Joe User' who 
> wants to download and install a version of LP that works. That doesn't 
> mean they don't matter but I don't see us getting all our proverbial 
> waterfowl in a row any time soon with all the mix of MRs I am seeing of 
> build/Infra and LP-specific. So we have to draw a line in the sand 
> somewhere if only for those developers that just want to do 'build 
> system and infrastructure' stuff so unless anything is broken 'now' for 
> the build, why not just stop accepting any more MRs for it at this time?
> 
> In effect I agree with Jonas' plan

To keep things together, it was Han-Wen's proposal that I just happened
to quote. (My original question didn't have a freeze).

> but slightly modified, I suggest that 
> unless anything is really broken with the build system 'today' (and I am 
> not getting that vibe from the conversations I read) we have one more 
> countdown for 'anything' build-related to get in the patch countdown and 
> after that, we only focus on Regressions and docs, anything currently in 
> the patch countdown that is build related gets counted down normally (if 
> it gets pulled/needs more changes it cannot come back in) and eventually 
> the countdown should only contain LP/Doc MRs and possibly even peter out 
> to nothing which would be a good indicator of when to revisit a 'when 
> shall we freeze'.
> 
> I don't do any development so feel a bit brash saying all this but 
> personally I don't think we will need 'weeks' to decide a freeze date, I 
> think it might become very obvious, very quickly once we prohibit 
> build/infra MRs and focus on regressions/docs. And until someone 'says' 
> an actual date - and gets feed back on that - I think we may keep on 
> procrastinating.
> 
> As for 'extensive testing' well all that I can see we can do here I 
> think is announce this to the user group and ask for help testing - even 
> just running 'a build' on some big and/or complex scores and ask for 
> feedback (perhaps put something on the website). I don't know what else 
> we can do for testing.

Other than that, I fully agree with all you say (which was the reason I
didn't have a freeze initially and would have just picked whatever
necessary into the created branch, but that's off the table now).
Basically everything that gives an actable plan with consensus is fine
for me, I just want to avoid doing nothing until it's urgent.

Jonas


signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

2020-09-11 Thread Jonas Hahnfeld

Am Freitag, den 11.09.2020, 17:14 +0200 schrieb Werner LEMBERG:
> > I don't see that in the current stage of upheaval of both internals and
> > build system and infrastructure, there is a point in freezing off some
> > half-baked intermediate state that hasn't seen significant exposure to
> > extensive testing.
> 
> +1  It's too early IMHO.  Let's wait at laest a month.

To be clear here: You mean at least one month before starting a freeze?
This is not my proposal anymore to just branch, but Han-Wen's idea of
having a freeze of 3-4 weeks before branching.


signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

2020-09-11 Thread James Lowe


On 11/09/2020 15:22, David Kastrup wrote:

Jonas Hahnfeld  writes:


Am Sonntag, den 06.09.2020, 13:40 +0200 schrieb Jonas Hahnfeld:

Am Sonntag, den 06.09.2020, 12:33 +0200 schrieb Han-Wen Nienhuys:

Here is my proposal for how to go ahead:

* we build a 2.21.6 from master, and announce it widely as a 2.22
pre-release version.

Adding Phil. I did a full test build of current master yesterday and it
worked fine with https://github.com/gperciva/gub/pull/80 in place.
With respect to wording, every 2.21.x is some kind of pre-release for
2.22. If this proposal gets consensus, I'd emphasize that feature
development is over, but I'll leave that to native speakers 


* we institute a X week freeze; during the freeze, we only merge
   - fixes for regressions
   - updates to the documentation
   - cleanups with no functional changes, with little risk (ie. refrain
from build system changes, for example).
* after the X week freeze, if we still have open regressions, we tack
on another few weeks of freeze to fix them.
* if there are no regressions left, we branch stable/2.22, and release
a new pre-release.

X could be up for discussion, but 3 or 4 weeks seems long enough to
gather some feedback, but short enough that experimental/feature work
should not be affected.

The objective of the freeze is to focus developer energy on fixing
regressions rather than causing new regressions.

Sounds good to me.

To arrive at a decision, what do others think about this? Having a
large silent mass is not really helpful for this kind of discussion (as
it wasn't for the switch to GitLab).

Frankly I don't see the point in repeating points I already made and
call that "discussion".

I don't see that in the current stage of upheaval of both internals and
build system and infrastructure, there is a point in freezing off some
half-baked intermediate state that hasn't seen significant exposure to
extensive testing.

I cannot comment on 'Internals' but 'build system' and 'infrastructure' 
are not, I think, issues that really concern 'average Joe User' who 
wants to download and install a version of LP that works. That doesn't 
mean they don't matter but I don't see us getting all our proverbial 
waterfowl in a row any time soon with all the mix of MRs I am seeing of 
build/Infra and LP-specific. So we have to draw a line in the sand 
somewhere if only for those developers that just want to do 'build 
system and infrastructure' stuff so unless anything is broken 'now' for 
the build, why not just stop accepting any more MRs for it at this time?


In effect I agree with Jonas' plan but slightly modified, I suggest that 
unless anything is really broken with the build system 'today' (and I am 
not getting that vibe from the conversations I read) we have one more 
countdown for 'anything' build-related to get in the patch countdown and 
after that, we only focus on Regressions and docs, anything currently in 
the patch countdown that is build related gets counted down normally (if 
it gets pulled/needs more changes it cannot come back in) and eventually 
the countdown should only contain LP/Doc MRs and possibly even peter out 
to nothing which would be a good indicator of when to revisit a 'when 
shall we freeze'.


I don't do any development so feel a bit brash saying all this but 
personally I don't think we will need 'weeks' to decide a freeze date, I 
think it might become very obvious, very quickly once we prohibit 
build/infra MRs and focus on regressions/docs. And until someone 'says' 
an actual date - and gets feed back on that - I think we may keep on 
procrastinating.


As for 'extensive testing' well all that I can see we can do here I 
think is announce this to the user group and ask for help testing - even 
just running 'a build' on some big and/or complex scores and ask for 
feedback (perhaps put something on the website). I don't know what else 
we can do for testing.


Thanks for listening.

James

Re: branching stable/2.22?

> I don't see that in the current stage of upheaval of both internals and
> build system and infrastructure, there is a point in freezing off some
> half-baked intermediate state that hasn't seen significant exposure to
> extensive testing.

+1  It's too early IMHO.  Let's wait at laest a month.


Werner

Re: branching stable/2.22?

2020-09-11 Thread David Kastrup

Jonas Hahnfeld  writes:

> Am Sonntag, den 06.09.2020, 13:40 +0200 schrieb Jonas Hahnfeld:
>> Am Sonntag, den 06.09.2020, 12:33 +0200 schrieb Han-Wen Nienhuys:
>> > Here is my proposal for how to go ahead:
>> > 
>> > * we build a 2.21.6 from master, and announce it widely as a 2.22
>> > pre-release version.
>> 
>> Adding Phil. I did a full test build of current master yesterday and it
>> worked fine with https://github.com/gperciva/gub/pull/80 in place.
>> With respect to wording, every 2.21.x is some kind of pre-release for
>> 2.22. If this proposal gets consensus, I'd emphasize that feature
>> development is over, but I'll leave that to native speakers 
>> 
>> > * we institute a X week freeze; during the freeze, we only merge
>> >   - fixes for regressions
>> >   - updates to the documentation
>> >   - cleanups with no functional changes, with little risk (ie. refrain
>> > from build system changes, for example).
>> > * after the X week freeze, if we still have open regressions, we tack
>> > on another few weeks of freeze to fix them.
>> > * if there are no regressions left, we branch stable/2.22, and release
>> > a new pre-release.
>> > 
>> > X could be up for discussion, but 3 or 4 weeks seems long enough to
>> > gather some feedback, but short enough that experimental/feature work
>> > should not be affected.
>> > 
>> > The objective of the freeze is to focus developer energy on fixing
>> > regressions rather than causing new regressions.
>> 
>> Sounds good to me.
>
> To arrive at a decision, what do others think about this? Having a
> large silent mass is not really helpful for this kind of discussion (as
> it wasn't for the switch to GitLab).

Frankly I don't see the point in repeating points I already made and
call that "discussion".

I don't see that in the current stage of upheaval of both internals and
build system and infrastructure, there is a point in freezing off some
half-baked intermediate state that hasn't seen significant exposure to
extensive testing.

-- 
David Kastrup

Re: branching stable/2.22?

2020-09-11 Thread Jonas Hahnfeld

Am Sonntag, den 06.09.2020, 13:40 +0200 schrieb Jonas Hahnfeld:
> Am Sonntag, den 06.09.2020, 12:33 +0200 schrieb Han-Wen Nienhuys:
> > Here is my proposal for how to go ahead:
> > 
> > * we build a 2.21.6 from master, and announce it widely as a 2.22
> > pre-release version.
> 
> Adding Phil. I did a full test build of current master yesterday and it
> worked fine with https://github.com/gperciva/gub/pull/80 in place.
> With respect to wording, every 2.21.x is some kind of pre-release for
> 2.22. If this proposal gets consensus, I'd emphasize that feature
> development is over, but I'll leave that to native speakers 
> 
> > * we institute a X week freeze; during the freeze, we only merge
> >   - fixes for regressions
> >   - updates to the documentation
> >   - cleanups with no functional changes, with little risk (ie. refrain
> > from build system changes, for example).
> > * after the X week freeze, if we still have open regressions, we tack
> > on another few weeks of freeze to fix them.
> > * if there are no regressions left, we branch stable/2.22, and release
> > a new pre-release.
> > 
> > X could be up for discussion, but 3 or 4 weeks seems long enough to
> > gather some feedback, but short enough that experimental/feature work
> > should not be affected.
> > 
> > The objective of the freeze is to focus developer energy on fixing
> > regressions rather than causing new regressions.
> 
> Sounds good to me.

To arrive at a decision, what do others think about this? Having a
large silent mass is not really helpful for this kind of discussion (as
it wasn't for the switch to GitLab).

Jonas


signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

2020-09-06 Thread Han-Wen Nienhuys

On Sun, Sep 6, 2020 at 1:40 PM Jonas Hahnfeld  wrote:
>
> Am Sonntag, den 06.09.2020, 12:33 +0200 schrieb Han-Wen Nienhuys:
> > On Sat, Sep 5, 2020 at 10:52 AM Jonas Hahnfeld  wrote:
> > > > I think the real problem is that we don't know exactly how many
> > > > problems there are that would be unacceptable in a stable release. So
> > > > we need a way to coax people normally on stable releases to try out
> > > > our current master, so we know where we are. Could we rename the
> > > > current 2.21 as 2.22-RC1 or 2.21.90 and hopefully get some feedback?
> > >
> > > Well yes, that's the point of the whole thread, but I don't think we'll
> > > get much feedback when continuing to break core functionality by usual
> > > development. IMHO the bump to 2.21.90 (or some other high value) should
> > > happen after branching, ie when it's really a release candidate.
> > > We could of course have some smaller bump to advertise as alpha while
> > > still working on master, but I doubt that would help much without some
> > > kind of freeze...
> > >
> > > To reiterate the initial question: What would it take to get there? In
> > > my opinion, that's equivalent to asking about critical issues that need
> > > large changes to fix them.
> >
> > I am not aware of critical issues at the moment. (While the doc build
> > refactoring has been disruptive, and still hasn't completely
> > stabilized, it doesn't affect end-users directly.)
> >
> > I read over the commits with potential end-user impact since branching
> > 2.21,
>
> Did you mean to say "branching stable/2.20" (commit 0712559601)? While

Yes. I browsed through at a lot of commits.

-- 
Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen

Re: branching stable/2.22?

2020-09-06 Thread Thomas Morley

Am So., 6. Sept. 2020 um 12:33 Uhr schrieb Han-Wen Nienhuys :
> I am not aware of critical issues at the moment.

At least
https://gitlab.com/lilypond/lilypond/-/issues?scope=all=%E2%9C%93=opened_name[]=Critical
returns empty.

Some issues are labeled "crash" though:
https://gitlab.com/lilypond/lilypond/-/issues?scope=all=%E2%9C%93=opened_name[]=Crash

I did not check whether those issues are still valid, though.

Cheers,
  Harm

Re: branching stable/2.22?

2020-09-06 Thread Jonas Hahnfeld

Am Sonntag, den 06.09.2020, 12:33 +0200 schrieb Han-Wen Nienhuys:
> On Sat, Sep 5, 2020 at 10:52 AM Jonas Hahnfeld  wrote:
> > > I think the real problem is that we don't know exactly how many
> > > problems there are that would be unacceptable in a stable release. So
> > > we need a way to coax people normally on stable releases to try out
> > > our current master, so we know where we are. Could we rename the
> > > current 2.21 as 2.22-RC1 or 2.21.90 and hopefully get some feedback?
> > 
> > Well yes, that's the point of the whole thread, but I don't think we'll
> > get much feedback when continuing to break core functionality by usual
> > development. IMHO the bump to 2.21.90 (or some other high value) should
> > happen after branching, ie when it's really a release candidate.
> > We could of course have some smaller bump to advertise as alpha while
> > still working on master, but I doubt that would help much without some
> > kind of freeze...
> > 
> > To reiterate the initial question: What would it take to get there? In
> > my opinion, that's equivalent to asking about critical issues that need
> > large changes to fix them.
> 
> I am not aware of critical issues at the moment. (While the doc build
> refactoring has been disruptive, and still hasn't completely
> stabilized, it doesn't affect end-users directly.)
> 
> I read over the commits with potential end-user impact since branching
> 2.21,

Did you mean to say "branching stable/2.20" (commit 0712559601)? While
some later commits might have made it into 2.20, I think there's still
more than the last few months.

> and got the following list of things to consider:
> 
> * lots of changes to GS calling conventions updated; changes to default fonts?
> This has lots of area for unexpected interactions between platforms
> (windows!) and GS versions. At the same time, we control the GS
> version through GUB, right?

Yes, calling conventions and atomic replacement resulted in some
breakage, but the fixes in 2.21.4 and 2.21.5 seem to work (at least no
new complaints and reports on the mailing list that functionality has
been restored).

> * General: C++ changes triggering segfault due to optimized out GC
> references. We are probably well covered, though, because we run GL CI
> often; we haven't seen flakes recently, have we?
> 
> * New output backend calling conventions may break user-supplied
> stencil expressions. This should probably be documented better in
> changes.tely. Also, I would like to hear from Scorio about the new
> backend API (ly:make-paper-outputter) before we commit to it in a
> stable version.
> 
> * We have a lot of coverage for the Python3 migration for
> lilypond-book, musicxml2ly, and midi2ly, but what about other scripts?
> 
> * Doc build rewrite: is the translation tooling working correctly now?

I fixed one wrongly referenced variable (TRANSLATION_FILES instead
of CHECKED_FILES) when updating the German web pages, but other than
that the instructions worked for me (who never did this before).


> * Move to gitlab: did all docs get updated?

Probably most, but maybe good to do another pass. One thing to consider
is issue verification (which hasn't happened for 2.21.2 and later
versions yet), but I'll start a separate thread about this.


> Here is my proposal for how to go ahead:
> 
> * we build a 2.21.6 from master, and announce it widely as a 2.22
> pre-release version.

Adding Phil. I did a full test build of current master yesterday and it
worked fine with https://github.com/gperciva/gub/pull/80 in place.
With respect to wording, every 2.21.x is some kind of pre-release for
2.22. If this proposal gets consensus, I'd emphasize that feature
development is over, but I'll leave that to native speakers 

> * we institute a X week freeze; during the freeze, we only merge
>   - fixes for regressions
>   - updates to the documentation
>   - cleanups with no functional changes, with little risk (ie. refrain
> from build system changes, for example).
> * after the X week freeze, if we still have open regressions, we tack
> on another few weeks of freeze to fix them.
> * if there are no regressions left, we branch stable/2.22, and release
> a new pre-release.
> 
> X could be up for discussion, but 3 or 4 weeks seems long enough to
> gather some feedback, but short enough that experimental/feature work
> should not be affected.
> 
> The objective of the freeze is to focus developer energy on fixing
> regressions rather than causing new regressions.

Sounds good to me.


signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

2020-09-06 Thread Han-Wen Nienhuys

On Sat, Sep 5, 2020 at 10:52 AM Jonas Hahnfeld  wrote:
> > I think the real problem is that we don't know exactly how many
> > problems there are that would be unacceptable in a stable release. So
> > we need a way to coax people normally on stable releases to try out
> > our current master, so we know where we are. Could we rename the
> > current 2.21 as 2.22-RC1 or 2.21.90 and hopefully get some feedback?
>
> Well yes, that's the point of the whole thread, but I don't think we'll
> get much feedback when continuing to break core functionality by usual
> development. IMHO the bump to 2.21.90 (or some other high value) should
> happen after branching, ie when it's really a release candidate.
> We could of course have some smaller bump to advertise as alpha while
> still working on master, but I doubt that would help much without some
> kind of freeze...
>
> To reiterate the initial question: What would it take to get there? In
> my opinion, that's equivalent to asking about critical issues that need
> large changes to fix them.

I am not aware of critical issues at the moment. (While the doc build
refactoring has been disruptive, and still hasn't completely
stabilized, it doesn't affect end-users directly.)

I read over the commits with potential end-user impact since branching
2.21, and
got the following list of things to consider:

* lots of changes to GS calling conventions updated; changes to default fonts?
This has lots of area for unexpected interactions between platforms
(windows!) and GS versions. At the same time, we control the GS
version through GUB, right?

* General: C++ changes triggering segfault due to optimized out GC
references. We are probably well covered, though, because we run GL CI
often; we haven't seen flakes recently, have we?

* New output backend calling conventions may break user-supplied
stencil expressions. This should probably be documented better in
changes.tely. Also, I would like to hear from Scorio about the new
backend API (ly:make-paper-outputter) before we commit to it in a
stable version.

* We have a lot of coverage for the Python3 migration for
lilypond-book, musicxml2ly, and midi2ly, but what about other scripts?

* Doc build rewrite: is the translation tooling working correctly now?

* Move to gitlab: did all docs get updated?

Here is my proposal for how to go ahead:

* we build a 2.21.6 from master, and announce it widely as a 2.22
pre-release version.
* we institute a X week freeze; during the freeze, we only merge
  - fixes for regressions
  - updates to the documentation
  - cleanups with no functional changes, with little risk (ie. refrain
from build system changes, for example).
* after the X week freeze, if we still have open regressions, we tack
on another few weeks of freeze to fix them.
* if there are no regressions left, we branch stable/2.22, and release
a new pre-release.

X could be up for discussion, but 3 or 4 weeks seems long enough to
gather some feedback, but short enough that experimental/feature work
should not be affected.

The objective of the freeze is to focus developer energy on fixing
regressions rather than causing new regressions.

-- 
Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen

Re: branching stable/2.22?

2020-09-05 Thread Jonas Hahnfeld

Am Freitag, den 04.09.2020, 22:49 +0200 schrieb Han-Wen Nienhuys:
> On Thu, Sep 3, 2020 at 11:16 AM Jonas Hahnfeld  wrote:
> > Am Mittwoch, den 26.08.2020, 00:05 +0200 schrieb Han-Wen Nienhuys:
> > > On Tue, Aug 25, 2020 at 11:17 PM Jonas Hahnfeld  wrote:
> > > > > I think the stabilization effort could be a joint effort by the entire
> > > > > dev team, by agreeing with the team to hold off on new features and
> > > > > invasive changes for a period of time (say, 1 to 2 months).
> > > > 
> > > > My feeling is that we should prioritize on bug fixes, but not actively
> > > > block the development of features.
> > > 
> > > Right - but we should refrain from changes that will make the
> > > backporting process more difficult, so it is still a freeze of sorts.
> > 
> > So what do we do here?
> > 0) Do nothing and continue to break things
> > 1) Prioritize fixes over features
> > 2) Agree on some sort of freeze
> > 3) Branch off stable/2.22 and cherry-pick only fixes
> 
> I think the real problem is that we don't know exactly how many
> problems there are that would be unacceptable in a stable release. So
> we need a way to coax people normally on stable releases to try out
> our current master, so we know where we are. Could we rename the
> current 2.21 as 2.22-RC1 or 2.21.90 and hopefully get some feedback?

Well yes, that's the point of the whole thread, but I don't think we'll
get much feedback when continuing to break core functionality by usual
development. IMHO the bump to 2.21.90 (or some other high value) should
happen after branching, ie when it's really a release candidate.
We could of course have some smaller bump to advertise as alpha while
still working on master, but I doubt that would help much without some
kind of freeze...

To reiterate the initial question: What would it take to get there? In
my opinion, that's equivalent to asking about critical issues that need
large changes to fix them.

signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

2020-09-04 Thread Han-Wen Nienhuys

On Thu, Sep 3, 2020 at 11:16 AM Jonas Hahnfeld  wrote:
>
> Am Mittwoch, den 26.08.2020, 00:05 +0200 schrieb Han-Wen Nienhuys:
> > On Tue, Aug 25, 2020 at 11:17 PM Jonas Hahnfeld  wrote:
> > > > I think the stabilization effort could be a joint effort by the entire
> > > > dev team, by agreeing with the team to hold off on new features and
> > > > invasive changes for a period of time (say, 1 to 2 months).
> > >
> > > My feeling is that we should prioritize on bug fixes, but not actively
> > > block the development of features.
> >
> > Right - but we should refrain from changes that will make the
> > backporting process more difficult, so it is still a freeze of sorts.
>
> So what do we do here?
> 0) Do nothing and continue to break things
> 1) Prioritize fixes over features
> 2) Agree on some sort of freeze
> 3) Branch off stable/2.22 and cherry-pick only fixes

I think the real problem is that we don't know exactly how many
problems there are that would be unacceptable in a stable release. So
we need a way to coax people normally on stable releases to try out
our current master, so we know where we are. Could we rename the
current 2.21 as 2.22-RC1 or 2.21.90 and hopefully get some feedback?

-- 
Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen

Re: branching stable/2.22?

2020-09-03 Thread David Kastrup

Jonas Hahnfeld  writes:

> Am Mittwoch, den 26.08.2020, 00:05 +0200 schrieb Han-Wen Nienhuys:
>> On Tue, Aug 25, 2020 at 11:17 PM Jonas Hahnfeld  wrote:
>> > > I think the stabilization effort could be a joint effort by the entire
>> > > dev team, by agreeing with the team to hold off on new features and
>> > > invasive changes for a period of time (say, 1 to 2 months).
>> > 
>> > My feeling is that we should prioritize on bug fixes, but not actively
>> > block the development of features.
>> 
>> Right - but we should refrain from changes that will make the
>> backporting process more difficult, so it is still a freeze of sorts.
>
> So what do we do here?
> 0) Do nothing and continue to break things
> 1) Prioritize fixes over features

The breakage we currently get was not because of features.  Mainly it
was because of overhauling code base and build system.  There is very
little payoff for the end user, apart from moderate performance
improvements.  And of course a higher likelihood that their distribution
will not discontinue distribution of LilyPond.

> 2) Agree on some sort of freeze
> 3) Branch off stable/2.22 and cherry-pick only fixes

I think that "some sort of freeze" makes sense only in light of defining
goals to achieve for the next stable release.  As long as those goals
still necessitate disruptive/extensive changes, there is not much of a
point in branching.

-- 
David Kastrup

Re: branching stable/2.22?

2020-09-03 Thread Jonas Hahnfeld

Am Mittwoch, den 26.08.2020, 00:05 +0200 schrieb Han-Wen Nienhuys:
> On Tue, Aug 25, 2020 at 11:17 PM Jonas Hahnfeld  wrote:
> > > I think the stabilization effort could be a joint effort by the entire
> > > dev team, by agreeing with the team to hold off on new features and
> > > invasive changes for a period of time (say, 1 to 2 months).
> > 
> > My feeling is that we should prioritize on bug fixes, but not actively
> > block the development of features.
> 
> Right - but we should refrain from changes that will make the
> backporting process more difficult, so it is still a freeze of sorts.

So what do we do here?
0) Do nothing and continue to break things
1) Prioritize fixes over features
2) Agree on some sort of freeze
3) Branch off stable/2.22 and cherry-pick only fixes

Am Dienstag, den 25.08.2020, 07:51 -0600 schrieb Carl Sorensen:
> Rather than a date, I'd prefer a criterion.
> 
> [...]
> 
> I recommend against creating a stable branch too soon.  In my experience,
> we get much more testing on the unstable branch than on the stable branch.
> People are likely to use the most recent unstable release as a matter of
> course; they are much less likely to go check out the most recent stable
> (but not yet released) branch.

Note that some more unstable releases would follow from the stable
branch, and that's likely what end-users test. Developers will continue
to build master, but any problem seen in the stable branch should
initially also occur in master, right? (unless there's some magical fix
or the issue disappears with some refactoring)

Jonas

signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

2020-08-25 Thread David Kastrup

Jonas Hahnfeld  writes:

> Am Dienstag, den 25.08.2020, 07:51 -0600 schrieb Carl Sorensen:
>> On Tue, Aug 25, 2020 at 7:13 AM Jonas Hahnfeld  wrote:
>> > I know I'll regret it because I still don't know what objective
>> > criteria others have, but as you really insist on a statement:
>> > in the week of 14th of September (this year, 2020, just to be clear)
>> > or put differently: right after 2.21.7, from that very tag
>> > As said before, I wouldn't make any commitment on the date of the final
>> > release.
>> 
>> Rather than a date, I'd prefer a criterion.
>> 
>> Once we have an unstable release with the build system in good shape
>> (all the auxiliary scripts work well, the website builds correctly,
>> the MacOS build is functional at least on MacPorts), I'd be in favor
>> of creating a pre-release candidate (which in the past has *not*
>> been a stable branch, but a big bump in the version number; the CG
>> refers to this as an alpha) and implementing the Build-frozen state
>> listed in the CG.
>
> Are you sure this is what happened in the past? I mean I didn't
> participate, so all I can rely on is git and the mailing list archives.
>
> For 2.18, I see
>  - release/2.17.29-1 tagged on 20 October 2013
>  - 
> https://lists.gnu.org/archive/html/lilypond-devel/2013-10/msg00189.html
>  on 22 October 2013
>  - release/2.17.95-1 tagged on 04 November 2013
>  - 
> https://lists.gnu.org/archive/html/lilypond-devel/2013-11/msg00077.html
>  on 05 November 2013
> As far as I understand, stable/2.18 was created before 2.17.95 (which
> was incorrectly released from master, see above). The merge base
> between current master and stable/2.18 points to April 2014 which is
> probably due to a partial merge? Certainly after 2.18.0 had been
> released in December 2013.
>
> For 2.20, I see
>  - considerations in December 2016:
> https://lists.gnu.org/archive/html/lilypond-devel/2016-12/msg00348.html
>  - 
> https://lists.gnu.org/archive/html/lilypond-devel/2017-06/msg00030.html
>  on 06 June 2017
>  - release/2.19.65-1 tagged on 06 August 2017
>  - merge-base HEAD stable/2.20: 0712559601, committed 08 August 2017
> and merged into stable/2.20 on 16 August 2017
>  - 
> https://lists.gnu.org/archive/html/lilypond-devel/2017-08/msg00068.html
>  on 16 August 2017
>  - release/2.19.80-1 tagged on 16 October 2017
>
> This looks like the release candidates (coinciding with the big bump)
> have (or should have been) released from the stable/ branches?

Yes.

> That said, I'm not opposed to doing a pre-release candidate from
> master and git history tells little about possible freezes.

The freezes mostly consisted in telling people to _please_ stop making
life harder for the stable release manager, with moderate success.
However, as release stoppers were accumulating and the release process
got stuck up, the "don't do development" appeals stopped making a whole
lot of sense.

> Just trying to understand what might have worked in the past...

I sure wished I had been able to get an understanding about what might
have worked...

-- 
David Kastrup

Re: branching stable/2.22?

Am Dienstag, den 25.08.2020, 07:51 -0600 schrieb Carl Sorensen:
> On Tue, Aug 25, 2020 at 7:13 AM Jonas Hahnfeld  wrote:
> > I know I'll regret it because I still don't know what objective
> > criteria others have, but as you really insist on a statement:
> > in the week of 14th of September (this year, 2020, just to be clear)
> > or put differently: right after 2.21.7, from that very tag
> > As said before, I wouldn't make any commitment on the date of the final
> > release.
> 
> Rather than a date, I'd prefer a criterion.
> 
> Once we have an unstable release with the build system in good shape (all the 
> auxiliary scripts work well, the website builds correctly, the MacOS build is 
> functional at least on MacPorts), I'd be in favor of creating a pre-release 
> candidate (which in the past has *not* been a stable branch, but a big bump 
> in the version number; the CG refers to this as an alpha) and implementing 
> the Build-frozen state listed in the CG. 

Are you sure this is what happened in the past? I mean I didn't
participate, so all I can rely on is git and the mailing list archives.

For 2.18, I see
 - release/2.17.29-1 tagged on 20 October 2013
 - 
https://lists.gnu.org/archive/html/lilypond-devel/2013-10/msg00189.html
 on 22 October 2013
 - release/2.17.95-1 tagged on 04 November 2013
 - 
https://lists.gnu.org/archive/html/lilypond-devel/2013-11/msg00077.html
 on 05 November 2013
As far as I understand, stable/2.18 was created before 2.17.95 (which
was incorrectly released from master, see above). The merge base
between current master and stable/2.18 points to April 2014 which is
probably due to a partial merge? Certainly after 2.18.0 had been
released in December 2013.

For 2.20, I see
 - considerations in December 2016:
https://lists.gnu.org/archive/html/lilypond-devel/2016-12/msg00348.html
 - 
https://lists.gnu.org/archive/html/lilypond-devel/2017-06/msg00030.html
 on 06 June 2017
 - release/2.19.65-1 tagged on 06 August 2017
 - merge-base HEAD stable/2.20: 0712559601, committed 08 August 2017
and merged into stable/2.20 on 16 August 2017
 - 
https://lists.gnu.org/archive/html/lilypond-devel/2017-08/msg00068.html
 on 16 August 2017
 - release/2.19.80-1 tagged on 16 October 2017

This looks like the release candidates (coinciding with the big bump)
have (or should have been) released from the stable/ branches?

That said, I'm not opposed to doing a pre-release candidate from master
and git history tells little about possible freezes. Just trying to
understand what might have worked in the past...

> After we've had some time to get the alpha release tested, we announce a beta 
> release and branch stable.
> 
> I concur with the idea that a properly functioning full conversion to Python 
> 3 and workable (though not required) Guile 2 constitutes sufficient change 
> for the next stable version.  No other features are needed.
> 
> I recommend against creating a stable branch too soon.  In my experience, we 
> get much more testing on the unstable branch than on the stable branch.  
> People are likely to use the most recent unstable release as a matter of 
> course; they are much less likely to go check out the most recent stable (but 
> not yet released) branch.
> 
> THanks,
> 
> Carl
> 
> 


signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

Am Dienstag, den 25.08.2020, 22:56 +0200 schrieb Han-Wen Nienhuys:
> On Tue, Aug 25, 2020 at 8:31 AM Jonas Hahnfeld  wrote:
> > I don't understand why you would want to backport features? IMO that's
> > got nothing to do with how far the stable branch diverges.
> > 
> > > Whatever the option, we will need people to manage the release (yes, I 
> > > could possibly help next summer ... though I'm afraid I'd be NOT_SMART).
> > 
> > If it's just missing people to do the work, I obviously volunteer and
> > am willing to cherry-pick fixes from master as needed.
> 
> I am happy to help with fixing bugs for the stable release.
> 
> > From my experience in the LLVM project, there is no such thing as
> > "naturally stabilizing code". Either you create a branch and pick fixes
> > or you have a strict policy that allows only fixes to master before
> > branching.
> >  That's basically the model GCC is using, and I don't think
> > it fits the community.
> 
> I agree that we have to do something, either branching and
> cherry-picking, or holding off on work that destabilizes the
> situation.
> 
> I don't understand why you think the GCC model doesn't fit, though?

If followed strictly, it means no feature commits in master during the
freeze which leaves two options: Either features build up in private
and it'll be serious work to integrate them after the freeze ends, or
they're not developed at all which is also bad.

> I think the branching model has the problem that someone has to do the
> work of backporting, which is probably less fun to do. If in parallel
> cowboys like me keep submitting experimental things to master,
> backporting the fixes is made all the more difficult.

Right, that is the trade-off. So really no easy solution to the
problem.

> I think the stabilization effort could be a joint effort by the entire
> dev team, by agreeing with the team to hold off on new features and
> invasive changes for a period of time (say, 1 to 2 months).

My feeling is that we should prioritize on bug fixes, but not actively
block the development of features.

Jonas

> BTW- aside from GUILE 2 and Python 3 support, I think users will also
> be happy with the speedups that I've been working on in this cycle.


signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

2020-08-25 Thread Han-Wen Nienhuys

On Tue, Aug 25, 2020 at 8:31 AM Jonas Hahnfeld  wrote:
> I don't understand why you would want to backport features? IMO that's
> got nothing to do with how far the stable branch diverges.
>
> > Whatever the option, we will need people to manage the release (yes, I 
> > could possibly help next summer ... though I'm afraid I'd be NOT_SMART).
>
> If it's just missing people to do the work, I obviously volunteer and
> am willing to cherry-pick fixes from master as needed.

I am happy to help with fixing bugs for the stable release.

> From my experience in the LLVM project, there is no such thing as
> "naturally stabilizing code". Either you create a branch and pick fixes
> or you have a strict policy that allows only fixes to master before
> branching.

>  That's basically the model GCC is using, and I don't think
> it fits the community.

I agree that we have to do something, either branching and
cherry-picking, or holding off on work that destabilizes the
situation.

I don't understand why you think the GCC model doesn't fit, though?

I think the branching model has the problem that someone has to do the
work of backporting, which is probably less fun to do. If in parallel
cowboys like me keep submitting experimental things to master,
backporting the fixes is made all the more difficult.

I think the stabilization effort could be a joint effort by the entire
dev team, by agreeing with the team to hold off on new features and
invasive changes for a period of time (say, 1 to 2 months).

BTW- aside from GUILE 2 and Python 3 support, I think users will also
be happy with the speedups that I've been working on in this cycle.

-- 
Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen

Re: branching stable/2.22?

2020-08-25 Thread Kevin Barry

> I concur with the idea that a properly functioning full conversion to
> Python 3 and workable (though not required) Guile 2 constitutes sufficient
> change for the next stable version.  No other features are needed.

I agree with this.

Kevin

Re: branching stable/2.22?

2020-08-25 Thread Dan Eble

On Aug 25, 2020, at 09:51, Carl Sorensen  wrote:
> 
> Once we have an unstable release with the build system in good shape (all
> the auxiliary scripts work well, the website builds correctly, the MacOS
> build is functional at least on MacPorts), I'd be in favor of creating a
> pre-release candidate (which in the past has *not* been a stable branch,
> but a big bump in the version number; the CG refers to this as an alpha)
> and implementing the Build-frozen state listed in the CG.
> 
> After we've had some time to get the alpha release tested, we announce a
> beta release and branch stable.
> 
> I concur with the idea that a properly functioning full conversion to
> Python 3 and workable (though not required) Guile 2 constitutes sufficient
> change for the next stable version.  No other features are needed.
> 
> I recommend against creating a stable branch too soon.  In my experience,
> we get much more testing on the unstable branch than on the stable branch.
> People are likely to use the most recent unstable release as a matter of
> course; they are much less likely to go check out the most recent stable
> (but not yet released) branch.

I could get behind all of this.
— 
Dan

Re: branching stable/2.22?

2020-08-25 Thread Jean Abou Samra



> Now go off tearing me apart.

That one was a blow in my stomach.(In fact, I was about to write something 
similar in my last message.)

Taking stock on the thread, my reactions were not appropriate, especially the 
fourth one which is rubbish. I should have anticipated that personal issues 
wouldn't help either.

Jonas and other readers, I sincerely apologize for the harm caused.

As I currently appear unable to express my views in an actionable manner that 
is not offensive or annoying to others, I am going to step down from this 
discussion. I simply shouldn't have joined it in the first place.

Cheers,
Jean

Re: branching stable/2.22?

2020-08-25 Thread Carl Sorensen

On Tue, Aug 25, 2020 at 7:13 AM Jonas Hahnfeld  wrote:

>
> I know I'll regret it because I still don't know what objective
> criteria others have, but as you really insist on a statement:
> in the week of 14th of September (this year, 2020, just to be clear)
> or put differently: right after 2.21.7, from that very tag
> As said before, I wouldn't make any commitment on the date of the final
> release.
>
>
>
Rather than a date, I'd prefer a criterion.

Once we have an unstable release with the build system in good shape (all
the auxiliary scripts work well, the website builds correctly, the MacOS
build is functional at least on MacPorts), I'd be in favor of creating a
pre-release candidate (which in the past has *not* been a stable branch,
but a big bump in the version number; the CG refers to this as an alpha)
and implementing the Build-frozen state listed in the CG.

After we've had some time to get the alpha release tested, we announce a
beta release and branch stable.

I concur with the idea that a properly functioning full conversion to
Python 3 and workable (though not required) Guile 2 constitutes sufficient
change for the next stable version.  No other features are needed.

I recommend against creating a stable branch too soon.  In my experience,
we get much more testing on the unstable branch than on the stable branch.
People are likely to use the most recent unstable release as a matter of
course; they are much less likely to go check out the most recent stable
(but not yet released) branch.

THanks,

Carl

Re: branching stable/2.22?

Am Dienstag, den 25.08.2020, 14:43 +0200 schrieb Jean Abou Samra:
> > Le 25 août 2020 à 12:29, Jonas Hahnfeld  a écrit :
> > 
> > Am Dienstag, den 25.08.2020, 12:06 +0200 schrieb Jean Abou Samra:
> > > > Le 25 août 2020 à 08:30, Jonas Hahnfeld  a écrit :
> > > > For me, creating the branch is nothing more than saying "feature
> > > > development is over and the current set is worth making stable". Which
> > > > I'm arguing is already there with Python 3 and the possibility to use
> > > > Guile 2. See my very initial message.
> > > 
> > > At the same time, you're saying that branching is not going to happen 
> > > next week. Please elaborate on your mind: when should that happen?
> > 
> > After below points have happened and after gathering agreement that
> > there are no open blockers to branching. IMO that would be something
> > fundamentally broken which can be expected to hit every user. AFAICT
> > that's not the case. Other problems can be addressed by picking fixes
> > into the branch.
> 
> That can happen in a week, can't it?

It could, but discussion takes time. Only because I don't see open
blockers doesn't mean it's true and others agree right away, as
evidenced here.

> I can't follow your mind anymore. You previously agreed with David that the 
> code base was in too much of a destabilized state for branching soon.

I said "in the next week" (note the singular form) which is basically
proven by this discussion. "soon" for me would be anything within the
next month.

> We're talking about bugs that we don't yet know but could pop up in the 
> months to come given this state.
> 
> > (It probably makes sense to branch right after making some future
> > unstable release, which implies that GUB is mostly happy, but that's
> > some minor detail I would say.)
> > 
> > > > On the administrative side, I think
> > > > * there should be another reformatting for all C++ and Scheme
> > > > * docs should be updated, including authors.itexi
> > > > Everything else can and should be picked as needed.
> > > 
> > > [...]
> > > 
> > > We're having, in fact, similar views. You say that we need to stabilize 
> > > the code base through branching, which I  entirely agree with -- except 
> > > that right now is not the right time.
> > 
> > So what objective function would you use to set an agreeable date? Just
> > time,
> 
> Yes, that is basically the idea. I think schedules help people work together 
> even if later deviated from.
> 
> > January 2021 being a shot in the dark?
> 
> Do speak up about what you would consider a more reasonable time.

I know I'll regret it because I still don't know what objective
criteria others have, but as you really insist on a statement:
in the week of 14th of September (this year, 2020, just to be clear)
or put differently: right after 2.21.7, from that very tag
As said before, I wouldn't make any commitment on the date of the final
release.

Now go off tearing me apart.

Jonas

> Also, I value the actual date less than I value agreement on the date.
> 
> Best regards.
> Jean


signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

Am Dienstag, den 25.08.2020, 12:06 +0200 schrieb Jean Abou Samra:
> > Le 25 août 2020 à 08:30, Jonas Hahnfeld  a écrit :
> > 
> > Am Montag, den 24.08.2020, 22:10 +0200 schrieb Jean Abou Samra:
> > > > > > > As sort of a shot in the dark, how about planning the 2.22 
> > > > > > > release for May 2021, for example?
> > > > > > 
> > > > > > Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? 
> > > > > > In
> > > > > > my understanding, the past process includes the release of beta
> > > > > > versions from the branch. That makes it close to impossible to 
> > > > > > predict
> > > > > > the final date of the stable version, and that's not the purpose of
> > > > > > this thread.
> > > > > 
> > > > > I mean releasing 2.22.0 in May 2021. This is not about predictions, 
> > > > > but objectives. I think that instead of planning each small step on 
> > > > > the fly with no idea how the future looks like, we should settle an 
> > > > > expected date for the release and plan backwards accordingly. Sure, 
> > > > > there could be critical bugs that delay the actual release, but 
> > > > > setting the expectation enables more resources to focus on the 
> > > > > release by the time it is due to happen. In my opinion, this is the 
> > > > > way we can avoid things like the 2.14 release that is documented in 
> > > > > the CG.
> > > > > 
> > > > > So, an expected release in May 2021 would mean branching release/2.20 
> > > > > around January (?) and beta releases at a monthly cadence until the 
> > > > > release is out.
> > > > > 
> > > > > I'm curious about what others think of that. In fact it looks like 
> > > > > you already proposed something along these lines:
> > > > > https://www.mail-archive.com/lilypond-devel@gnu.org/msg72997.html
> > > > > 
> > > > 
> > > > And it didn't get much support. Which is why I don't see what's
> > > > different today. Asking what it would take to branch is really the only
> > > > sensible thing I think we could possibly agree on.
> > > 
> > > As I see it, you're asking something nobody, apparently, can give you. We 
> > > need to create the process instead of finding it out: what do you think 
> > > it should take to branch?
> > 
> > For me, creating the branch is nothing more than saying "feature
> > development is over and the current set is worth making stable". Which
> > I'm arguing is already there with Python 3 and the possibility to use
> > Guile 2. See my very initial message.
> 
> At the same time, you're saying that branching is not going to happen next 
> week. Please elaborate on your mind: when should that happen?

After below points have happened and after gathering agreement that
there are no open blockers to branching. IMO that would be something
fundamentally broken which can be expected to hit every user. AFAICT
that's not the case. Other problems can be addressed by picking fixes
into the branch.
(It probably makes sense to branch right after making some future
unstable release, which implies that GUB is mostly happy, but that's
some minor detail I would say.)

> > On the administrative side, I think
> > * there should be another reformatting for all C++ and Scheme
> > * docs should be updated, including authors.itexi
> > Everything else can and should be picked as needed.
> 
> [...]
> 
> We're having, in fact, similar views. You say that we need to stabilize the 
> code base through branching, which I  entirely agree with -- except that 
> right now is not the right time.

So what objective function would you use to set an agreeable date? Just
time, January 2021 being a shot in the dark?

> What I'm trying to convey here is that postponing decisions on the ground of 
> them being controversial is damaging to team members' morale.
> 
> To me, for the above-mentioned reasons, settling a date for branching 2.22 
> amounts to scheduling the 2.22 release, which is why I think we should 
> explicitely discuss that schedule, instead of making short-term decisions 
> that only have consensus because the consequences weren't discussed, with no 
> longer term perspectives. The contrary would let the community split into 
> small groups of like-minded persons that avoid each other because don't want 
> to go the trouble of convincing each other. Given that you're ready to 
> endorse the release manager role and responsibility, I no longer see any 
> blocker to scheduling 2.20, except getting agreement about the schedule.
> 
> So we better start arguing about the schedule.
> 
> Cheers,
> Jean
> 


signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

2020-08-25 Thread Jean Abou Samra




> Le 25 août 2020 à 08:30, Jonas Hahnfeld  a écrit :
> 
> Am Montag, den 24.08.2020, 22:10 +0200 schrieb Jean Abou Samra:
>> As sort of a shot in the dark, how about planning the 2.22 release for 
>> May 2021, for example?
> 
> Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? In
> my understanding, the past process includes the release of beta
> versions from the branch. That makes it close to impossible to predict
> the final date of the stable version, and that's not the purpose of
> this thread.
 
 I mean releasing 2.22.0 in May 2021. This is not about predictions, but 
 objectives. I think that instead of planning each small step on the fly 
 with no idea how the future looks like, we should settle an expected date 
 for the release and plan backwards accordingly. Sure, there could be 
 critical bugs that delay the actual release, but setting the expectation 
 enables more resources to focus on the release by the time it is due to 
 happen. In my opinion, this is the way we can avoid things like the 2.14 
 release that is documented in the CG.
 
 So, an expected release in May 2021 would mean branching release/2.20 
 around January (?) and beta releases at a monthly cadence until the 
 release is out.
 
 I'm curious about what others think of that. In fact it looks like you 
 already proposed something along these lines:
 https://www.mail-archive.com/lilypond-devel@gnu.org/msg72997.html
>>> 
>>> And it didn't get much support. Which is why I don't see what's
>>> different today. Asking what it would take to branch is really the only
>>> sensible thing I think we could possibly agree on.
>> 
>> As I see it, you're asking something nobody, apparently, can give you. We 
>> need to create the process instead of finding it out: what do you think it 
>> should take to branch?
> 
> For me, creating the branch is nothing more than saying "feature
> development is over and the current set is worth making stable". Which
> I'm arguing is already there with Python 3 and the possibility to use
> Guile 2. See my very initial message.

At the same time, you're saying that branching is not going to happen next 
week. Please elaborate on your mind: when should that happen?

> On the administrative side, I think
> * there should be another reformatting for all C++ and Scheme
> * docs should be updated, including authors.itexi
> Everything else can and should be picked as needed.
> 
>> Branching means collectively commiting to creating a stable release a 
>> handful of months later, otherwise we get to the situation that David 
>> described in his last reply: the stable branch comes so far from master that 
>> features need to be ported to it; clearly, that's not a desirable workflow.
> 
> I don't understand why you would want to backport features? IMO that's
> got nothing to do with how far the stable branch diverges.

I was talking about what happened in 2.20:

> It's worth stressing that the 2.20 branch persisted much longer before
> the 2.20 release than planned.  So there were also some feature
> cherry-picks in order to avoid the situation that the 2.20 release would
> be "outdated from the start" with regard to some "must-have" features
> that would be expected to be common in suggested code on the mailing
> lists.


Now, you might say that if we do more frequent releases, the problem will 
disappear. This means, however, that we're planning to make releases more 
often, which is a decision that extends far beyond simply planning the 
branching time.


>> Whatever the option, we will need people to manage the release (yes, I could 
>> possibly help next summer ... though I'm afraid I'd be NOT_SMART).
> 
> If it's just missing people to do the work, I obviously volunteer and
> am willing to cherry-pick fixes from master as needed.

Nice!

>> So, I think the question is essentially wether we try to plan the release 
>> now or just wait for the essential features we'd like in it to be 
>> implemented, e.g., full Guile 2 support. Personally, I think it's better to 
>> plan it so hopefully developers will naturally organize their respective 
>> works accordingly. In this perspective, if full Guile 2 support is not 
>> implemented by the deadline, it just waits for another release. But that's 
>> just my opinion.
> 
> From my experience in the LLVM project, there is no such thing as
> "naturally stabilizing code". Either you create a branch and pick fixes
> or you have a strict policy that allows only fixes to master before
> branching. That's basically the model GCC is using, and I don't think
> it fits the community.

I was not meaning that the code base would naturally stabilize but instead that 
developers would be able to plan what features they can implement in the time 
that runs from now to branching. Predictability makes people happy.

We're having, in fact, similar views. You say that we

Re: branching stable/2.22?

Am Montag, den 24.08.2020, 22:10 +0200 schrieb Jean Abou Samra:
> > > > > As sort of a shot in the dark, how about planning the 2.22 release 
> > > > > for May 2021, for example?
> > > > 
> > > > Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? In
> > > > my understanding, the past process includes the release of beta
> > > > versions from the branch. That makes it close to impossible to predict
> > > > the final date of the stable version, and that's not the purpose of
> > > > this thread.
> > > 
> > > I mean releasing 2.22.0 in May 2021. This is not about predictions, but 
> > > objectives. I think that instead of planning each small step on the fly 
> > > with no idea how the future looks like, we should settle an expected date 
> > > for the release and plan backwards accordingly. Sure, there could be 
> > > critical bugs that delay the actual release, but setting the expectation 
> > > enables more resources to focus on the release by the time it is due to 
> > > happen. In my opinion, this is the way we can avoid things like the 2.14 
> > > release that is documented in the CG.
> > > 
> > > So, an expected release in May 2021 would mean branching release/2.20 
> > > around January (?) and beta releases at a monthly cadence until the 
> > > release is out.
> > > 
> > > I'm curious about what others think of that. In fact it looks like you 
> > > already proposed something along these lines:
> > > https://www.mail-archive.com/lilypond-devel@gnu.org/msg72997.html
> > 
> > And it didn't get much support. Which is why I don't see what's
> > different today. Asking what it would take to branch is really the only
> > sensible thing I think we could possibly agree on.
> 
> As I see it, you're asking something nobody, apparently, can give you. We 
> need to create the process instead of finding it out: what do you think it 
> should take to branch?

For me, creating the branch is nothing more than saying "feature
development is over and the current set is worth making stable". Which
I'm arguing is already there with Python 3 and the possibility to use
Guile 2. See my very initial message.

On the administrative side, I think
 * there should be another reformatting for all C++ and Scheme
 * docs should be updated, including authors.itexi
Everything else can and should be picked as needed.

> Branching means collectively commiting to creating a stable release a handful 
> of months later, otherwise we get to the situation that David described in 
> his last reply: the stable branch comes so far from master that features need 
> to be ported to it; clearly, that's not a desirable workflow.

I don't understand why you would want to backport features? IMO that's
got nothing to do with how far the stable branch diverges.

> Whatever the option, we will need people to manage the release (yes, I could 
> possibly help next summer ... though I'm afraid I'd be NOT_SMART).

If it's just missing people to do the work, I obviously volunteer and
am willing to cherry-pick fixes from master as needed.

> So, I think the question is essentially wether we try to plan the release now 
> or just wait for the essential features we'd like in it to be implemented, 
> e.g., full Guile 2 support. Personally, I think it's better to plan it so 
> hopefully developers will naturally organize their respective works 
> accordingly. In this perspective, if full Guile 2 support is not implemented 
> by the deadline, it just waits for another release. But that's just my 
> opinion.

From my experience in the LLVM project, there is no such thing as
"naturally stabilizing code". Either you create a branch and pick fixes
or you have a strict policy that allows only fixes to master before
branching. That's basically the model GCC is using, and I don't think
it fits the community.

Jonas


signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

2020-08-24 Thread Jean Abou Samra

Hi,

> Le 24 août 2020 à 13:46, Jonas Hahnfeld  a écrit :
> 
> Am Montag, den 24.08.2020, 12:56 +0200 schrieb Jean Abou Samra:
>> Hi,
>> 
>>> Le 24 août 2020 à 08:30, Jonas Hahnfeld  a écrit :
>>> 
>>> Am Sonntag, den 23.08.2020, 23:44 +0200 schrieb Jean Abou Samra:
 Maybe we could try to release 2.20.1 with Python 3?
>>> 
>>> That would mean porting 50+ commits which sounds like a bad idea and
>>> even gets worse because of the reformatting in master. The latter
>>> implies that any bug fix made in master will result in merge conflicts.
>>> Plus I don't understand the proposal if you're at the same time saying
>>> that the scripts are fragile. By that logic, why would we backport such
>>> extensive changes and claim they're stable?
>> 
>> Right, I was oblique: the scripts are fragile at present, so branching 
>> release/2.22 now is no good in my opinion, but hopefully we can stabilize 
>> them faster than we stabilize LilyPond as a whole, and have that in 2.20.1 
>> or 2.20.2.
>> 
>> Can you explain why porting 50 commits from master to 2.20 is a bad idea?
> 
> I think it's a bad idea because it goes against my basic understanding
> that only bug fixes should be ported a stable branch.

This situation is special anyway: we did a release that is from the start 
outdated with respect to Python support.

All I'm saying is that porting this to 2.20 would be better than releasing 2.22 
in a hurry, for the reasons that David and I mentioned. So...

> Here's the total
> number of commits in stable/2.20 since branching:
> [...]
>> If we port all Python related-commits (including the reformatting), there 
>> won't be any merge conflicts, or am I being dense somehow?
> 
> [...]
> Being the individual with the most commits in there during the cycle, I
> strongly advise against taking all of this for a minor stable release.

... so since that appears unpossible, I guess we'd just let distros drop 
LilyPond.


>> I do understand that having LilyPond 2.20.0 support exclusively Python 2 but 
>> 2.20.1 be Python 3 only feels weird. However, I value the interest of the 
>> average user more than that of packagers. Neither Python 3 nor Guile 2 is a 
>> breaking change from the user's point of view. If having LilyPond 2.20.1 (or 
>> 2.20.2) support Python 3 is not feasible, I think we should just let 
>> distributions drop LilyPond (see below). I'm not happy about it, but this 
>> is, in my opinion, preferable to making a stable release, in a hurry, that 
>> will contain more bugs and few user-facing changes.
>> 
 Why Guile 2? If it's still imperfect and slower, we don't want to make it 
 the default in the binaries at lilypond.org, do we? So how will the 
 situation be different from 2.20? Sorry, I must be missing something 
 obvious here... I don't understand you.
>>> 
>>> At least in my book, it's not about changing the default but at least
>>> making it possible for distributions to compile with Guile 2.x instead
>>> of just throwing LilyPond out.
>> 
>> Does this mean that we'll receive bug reports that won't be reproducible by 
>> others because they'll actually be related to Guile 2? In my opinion, the 
>> distributions just throwing out LilyPond is better.
>> 
>> Additionally, the sh installers are recommended by the official website over 
>> distribution package managers.




>>> I don't think we'll get testing from distributions until we declare a
>>> way to stabilize.
>> 
>> We're speaking from different points of view here: in my book, our main 
>> source of testing is our development and beta releases brought to users 
>> through installers. I mean that LilyPond 2.22 should introduce full support 
>> for Guile 2 with byte compilation, probably dropping Guile 1 support too, 
>> and we get Guile 2 testing from those who try out the betas.
> 
> I seriously doubt you'll get that for 2.22 next year.

That date was a shot in the dark intended at starting the discussion.

See also below.

 Am Sonntag, den 23.08.2020, 14:59 +0200 schrieb David Kastrup:
>> I'd like to ask what it would take in principle to branch stable/2.22
>> and what others think about this.
> 
> I don't see that this is a good point of time.
> There has been an influx of badly tested changes to the build system and
> directory setup and the web pages that has to stabilise and result in a
> workable version of LilyPond. I don't see the point in branching a
> "stable" branch if there is so much in a destabilised state: you'd have
> to cherry-pick loads of stuff from the unstable branch as it comes in.
 
 [Jonas] I fully agree 
 
 ... and so do I (for what my opinion's worth, really) ...
 
 [Jonas] and I should have been more clear that I don't expect the branch 
 to happen in the next week. The point was to find out what it would take 
 because just waiting for some unspoken condition to become true is not 
 exactly going to happen without some effort.

Re: branching stable/2.22?

2020-08-24 Thread David Kastrup

Jonas Hahnfeld  writes:

> Am Montag, den 24.08.2020, 12:56 +0200 schrieb Jean Abou Samra:
>> 
>> Right, I was oblique: the scripts are fragile at present, so
>> branching release/2.22 now is no good in my opinion, but hopefully
>> we can stabilize them faster than we stabilize LilyPond as a whole,
>> and have that in 2.20.1 or 2.20.2.
>> 
>> Can you explain why porting 50 commits from master to 2.20 is a bad idea?
>
> I think it's a bad idea because it goes against my basic understanding
> that only bug fixes should be ported a stable branch. Here's the total
> number of commits in stable/2.20 since branching:
> $ git log --oneline release/2.20.0-1...release/2.19.65-1 | wc -l
> 588
>
> Sounds high at first, but most of that was translations, ie
> $ git log --oneline release/2.20.0-1...release/2.19.65-1 -- $(ls |
> grep -vE "Documentation|po") | wc -l
> 268
> If only focusing on actual code changes:
> $ git log --oneline release/2.20.0-1...release/2.19.65-1 -- flower/
> lily/ ly/ python/ scm/ scripts/ | wc -l
> 198
> with the majority in "core" LilyPond:
> $ git log --oneline release/2.20.0-1...release/2.19.65-1 -- lily/ ly/ scm/ | 
> wc -l
> 148

It's worth stressing that the 2.20 branch persisted much longer before
the 2.20 release than planned.  So there were also some feature
cherry-picks in order to avoid the situation that the 2.20 release would
be "outdated from the start" with regard to some "must-have" features
that would be expected to be common in suggested code on the mailing
lists.

So 2.20's history in some way reflects how to muddle through in a
situation that became a lot different from planning.  It's not really a
template for how things should work.

-- 
David Kastrup

Re: branching stable/2.22?

2020-08-24 Thread Jonas Hahnfeld

Am Montag, den 24.08.2020, 12:56 +0200 schrieb Jean Abou Samra:
> Hi,
> 
> > Le 24 août 2020 à 08:30, Jonas Hahnfeld  a écrit :
> > 
> > Am Sonntag, den 23.08.2020, 23:44 +0200 schrieb Jean Abou Samra:
> > > Hi,
> > > 
> > > (Sorry about the strange reply style.)
> > > 
> > > > On Sun, Aug 23, 2020 at 1:58 PM Jonas Hahnfeld  wrote:
> > > > I'd like to ask what it would take in principle to branch stable/2.22
> > > > and what others think about this.
> > > > 
> > > > Personally I'm convinced that creating the branch and only picking bug
> > > > fixes from master is the only guaranteed way to stabilize. Now you
> > > > might say that there were only few unstable releases (AFAICT there was
> > > > 2.19.65 before branching stable/2.20). However, I think there are
> > > > already quite some user-facing changes and also the switch to Python 3.
> > > > With Python 2.x being EOL since January, it's only a matter of time
> > > > until Linux distributions and BSDs want to drop that, and it
> > > > would be
> > > > unfortunate if that put a (temporary) end to providing LilyPond for
> > > > their users. If we had release candidates or even a stable version
> > > > until then, it would definitely help.
> > > 
> > > Maybe we could try to release 2.20.1 with Python 3?
> > 
> > That would mean porting 50+ commits which sounds like a bad idea and
> > even gets worse because of the reformatting in master. The latter
> > implies that any bug fix made in master will result in merge conflicts.
> > Plus I don't understand the proposal if you're at the same time saying
> > that the scripts are fragile. By that logic, why would we backport such
> > extensive changes and claim they're stable?
> 
> Right, I was oblique: the scripts are fragile at present, so branching 
> release/2.22 now is no good in my opinion, but hopefully we can stabilize 
> them faster than we stabilize LilyPond as a whole, and have that in 2.20.1 or 
> 2.20.2.
> 
> Can you explain why porting 50 commits from master to 2.20 is a bad idea?

I think it's a bad idea because it goes against my basic understanding
that only bug fixes should be ported a stable branch. Here's the total
number of commits in stable/2.20 since branching:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 | wc -l
588

Sounds high at first, but most of that was translations, ie
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- $(ls | grep -vE 
"Documentation|po") | wc -l
268
If only focusing on actual code changes:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- flower/ lily/ ly/ 
python/ scm/ scripts/ | wc -l
198
with the majority in "core" LilyPond:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- lily/ ly/ scm/ | wc 
-l
148

> If we port all Python related-commits (including the reformatting), there 
> won't be any merge conflicts, or am I being dense somehow?

Consider:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- python/ | wc -l
22
vs
$ git log --oneline HEAD...release/2.19.65-1 -- python/ | wc -l
115

and 

$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- scripts/*.py | wc -l
5
vs
$ git log --oneline HEAD...release/2.19.65-1 -- scripts/*.py | wc -l
67

Being the individual with the most commits in there during the cycle, I
strongly advise against taking all of this for a minor stable release.


> I do understand that having LilyPond 2.20.0 support exclusively Python 2 but 
> 2.20.1 be Python 3 only feels weird. However, I value the interest of the 
> average user more than that of packagers. Neither Python 3 nor Guile 2 is a 
> breaking change from the user's point of view. If having LilyPond 2.20.1 (or 
> 2.20.2) support Python 3 is not feasible, I think we should just let 
> distributions drop LilyPond (see below). I'm not happy about it, but this is, 
> in my opinion, preferable to making a stable release, in a hurry, that will 
> contain more bugs and few user-facing changes.
> 
> > > > The same can of course happen with Guile, but that situation has been
> > > > going on for years. Furthermore, it's at least possible to compile and
> > > > use current master with Guile 2.x, even if slower. In my understanding,
> > > > things are getting better but properly integrating byte compilation is
> > > > still a good amount of work that nobody could give a deadline on.
> > > > 
> > > > WDYT?
> > > 
> > > [Han-Wen] I think that both Python 3 support and usable (if imperfect) 
> > > GUILE 2 support is a strong argument for releasing a new stable version 
> > > soon.
> > > 
> > > Why Guile 2? If it's still imperfect and slower, we don't want to make it 
> > > the default in the binaries at lilypond.org, do we? So how will the 
> > > situation be different from 2.20? Sorry, I must be missing something 
> > > obvious here... I don't understand you.
> > 
> > At least in my book, it's not about changing the default but at least
> > making it possible for distributions to compile with Guile 2.x instead
> > of just throwing

Re: branching stable/2.22?

2020-08-24 Thread Jean Abou Samra

Hi,

> Le 24 août 2020 à 08:30, Jonas Hahnfeld  a écrit :
> 
> Am Sonntag, den 23.08.2020, 23:44 +0200 schrieb Jean Abou Samra:
>> Hi,
>> 
>> (Sorry about the strange reply style.)
>> 
>>> On Sun, Aug 23, 2020 at 1:58 PM Jonas Hahnfeld  wrote:
>>> I'd like to ask what it would take in principle to branch stable/2.22
>>> and what others think about this.
>>> 
>>> Personally I'm convinced that creating the branch and only picking bug
>>> fixes from master is the only guaranteed way to stabilize. Now you
>>> might say that there were only few unstable releases (AFAICT there was
>>> 2.19.65 before branching stable/2.20). However, I think there are
>>> already quite some user-facing changes and also the switch to Python 3.
>>> With Python 2.x being EOL since January, it's only a matter of time
>>> until Linux distributions and BSDs want to drop that, and it would be
>>> unfortunate if that put a (temporary) end to providing LilyPond for
>>> their users. If we had release candidates or even a stable version
>>> until then, it would definitely help.
>> 
>> Maybe we could try to release 2.20.1 with Python 3?
> 
> That would mean porting 50+ commits which sounds like a bad idea and
> even gets worse because of the reformatting in master. The latter
> implies that any bug fix made in master will result in merge conflicts.
> Plus I don't understand the proposal if you're at the same time saying
> that the scripts are fragile. By that logic, why would we backport such
> extensive changes and claim they're stable?

Right, I was oblique: the scripts are fragile at present, so branching 
release/2.22 now is no good in my opinion, but hopefully we can stabilize them 
faster than we stabilize LilyPond as a whole, and have that in 2.20.1 or 2.20.2.

Can you explain why porting 50 commits from master to 2.20 is a bad idea? If we 
port all Python related-commits (including the reformatting), there won't be 
any merge conflicts, or am I being dense somehow?

I do understand that having LilyPond 2.20.0 support exclusively Python 2 but 
2.20.1 be Python 3 only feels weird. However, I value the interest of the 
average user more than that of packagers. Neither Python 3 nor Guile 2 is a 
breaking change from the user's point of view. If having LilyPond 2.20.1 (or 
2.20.2) support Python 3 is not feasible, I think we should just let 
distributions drop LilyPond (see below). I'm not happy about it, but this is, 
in my opinion, preferable to making a stable release, in a hurry, that will 
contain more bugs and few user-facing changes.

>>> The same can of course happen with Guile, but that situation has been
>>> going on for years. Furthermore, it's at least possible to compile and
>>> use current master with Guile 2.x, even if slower. In my understanding,
>>> things are getting better but properly integrating byte compilation is
>>> still a good amount of work that nobody could give a deadline on.
>>> 
>>> WDYT?
>> 
>> [Han-Wen] I think that both Python 3 support and usable (if imperfect) GUILE 
>> 2 support is a strong argument for releasing a new stable version soon.
>> 
>> Why Guile 2? If it's still imperfect and slower, we don't want to make it 
>> the default in the binaries at lilypond.org, do we? So how will the 
>> situation be different from 2.20? Sorry, I must be missing something obvious 
>> here... I don't understand you.
> 
> At least in my book, it's not about changing the default but at least
> making it possible for distributions to compile with Guile 2.x instead
> of just throwing LilyPond out.

Does this mean that we'll receive bug reports that won't be reproducible by 
others because they'll actually be related to Guile 2? In my opinion, the 
distributions just throwing out LilyPond is better.

Additionally, the sh installers are recommended by the official website over 
distribution package managers.

>> At any rate, Python 3 support is great, but the scripts are fragile at the 
>> moment. This is clear from the tracker and the bug-lilypond list. Our Python 
>> scripts still need fixes in the way we distribute them, plus 
>> encoding-related issues (I'm planning on tackling the latter point in a 
>> short period at the end of August, but who knows what that will reveal).
> 
> Could you please point to concrete issues? The distribution of scripts
> hasn't changed, so it basically still works the same it did for years
> (decades?).

I'm writing this with a browser that is too old to view gitlab.com. There is at 
least
https://www.mail-archive.com/bug-lilypond@gnu.org/msg43376.html
which looks serious at first sight.

>> [Han-Wen] I've been working on the build system (obviously), with in the 
>> back of my mind having a build that is no longer recursive, but that work 
>> could be paused for a bit while we prepare for releasing a 2.22. Is there a 
>> list of problems in the current master that have to be resolved?
>> 
>> Problems are basically popping every week (e.g., info installation, 
>> translation tools,

Re: branching stable/2.22?

2020-08-24 Thread Jonas Hahnfeld

Am Sonntag, den 23.08.2020, 23:44 +0200 schrieb Jean Abou Samra:
> Hi,
> 
> (Sorry about the strange reply style.)
> 
> On Sun, Aug 23, 2020 at 1:58 PM Jonas Hahnfeld  wrote:
> > I'd like to ask what it would take in principle to branch stable/2.22
> > and what others think about this.
> > 
> > Personally I'm convinced that creating the branch and only picking bug
> > fixes from master is the only guaranteed way to stabilize. Now you
> > might say that there were only few unstable releases (AFAICT there was
> > 2.19.65 before branching stable/2.20). However, I think there are
> > already quite some user-facing changes and also the switch to Python 3.
> > With Python 2.x being EOL since January, it's only a matter of time
> > until Linux distributions and BSDs want to drop that, and it would be
> > unfortunate if that put a (temporary) end to providing LilyPond for
> > their users. If we had release candidates or even a stable version
> > until then, it would definitely help.
> 
> Maybe we could try to release 2.20.1 with Python 3?

That would mean porting 50+ commits which sounds like a bad idea and
even gets worse because of the reformatting in master. The latter
implies that any bug fix made in master will result in merge conflicts.
Plus I don't understand the proposal if you're at the same time saying
that the scripts are fragile. By that logic, why would we backport such
extensive changes and claim they're stable?

> > The same can of course happen with Guile, but that situation has been
> > going on for years. Furthermore, it's at least possible to compile and
> > use current master with Guile 2.x, even if slower. In my understanding,
> > things are getting better but properly integrating byte compilation is
> > still a good amount of work that nobody could give a deadline on.
> > 
> > WDYT?
> 
> [Han-Wen] I think that both Python 3 support and usable (if imperfect) GUILE 
> 2 support is a strong argument for releasing a new stable version soon.
> 
> Why Guile 2? If it's still imperfect and slower, we don't want to make it the 
> default in the binaries at lilypond.org, do we? So how will the situation be 
> different from 2.20? Sorry, I must be missing something obvious here... I 
> don't understand you.

At least in my book, it's not about changing the default but at least
making it possible for distributions to compile with Guile 2.x instead
of just throwing LilyPond out.


> At any rate, Python 3 support is great, but the scripts are fragile at the 
> moment. This is clear from the tracker and the bug-lilypond list. Our Python 
> scripts still need fixes in the way we distribute them, plus encoding-related 
> issues (I'm planning on tackling the latter point in a short period at the 
> end of August, but who knows what that will reveal).

Could you please point to concrete issues? The distribution of scripts
hasn't changed, so it basically still works the same it did for years
(decades?).

> [Han-Wen] I've been working on the build system (obviously), with in the back 
> of my mind having a build that is no longer recursive, but that work could be 
> paused for a bit while we prepare for releasing a 2.22. Is there a list of 
> problems in the current master that have to be resolved?
> 
> Problems are basically popping every week (e.g., info installation, 
> translation tools, etc.). You're fixing them every week, which is really 
> great, but before creating a release branch that is devoted to stability, I 
> think we need a couple months to see what new problems appear, don't we?
> 
> We have unstable releases to publish new features and get testing. In my 
> opinion, stable releases should really focus on stability, there's no need to 
> rush because of Python 3 and Guile 2.

I don't think we'll get testing from distributions until we declare a
way to stabilize.

> At least four areas are currently under flux: Python scripts, the build 
> system, Guile 2 support, and fonts (Owen's project), and I don't see that 
> master is coming any close to stability. I think we are better with focusing 
> on these areas as long as they still require substantial attention, so as to 
> get a stable and nice LilyPond 2.22 in a timely manner. You derecursify the 
> build, David completes his stack of rather extensive purportive changes, Owen 
> merges the GSoC branch, etc., and only then it'll be time to care about 
> LilyPond 2.22.

And that's my point: If we just sit and wait, the next stable release
might happen after all major distributions threw LilyPond out - simply
because it cannot be built in a world of Python 3.

> Am Sonntag, den 23.08.2020, 14:59 +0200 schrieb David Kastrup:
> > > I'd like to ask what it would take in principle to branch stable/2.22
> > > and what others think about this.
> > I don't see that this is a good point of time.
> > There has been an influx of badly tested changes to the build system and
> > directory setup and the web pages that has to stabilise and result in a
> >

Re: branching stable/2.22?

2020-08-23 Thread Dan Eble

On Aug 23, 2020, at 17:44, Jean Abou Samra  wrote:
> 
> At least four areas are currently under flux: Python scripts, the build 
> system, Guile 2 support, and fonts (Owen's project), and I don't see that 
> master is coming any close to stability. I think we are better with focusing 
> on these areas as long as they still require substantial attention, so as to 
> get a stable and nice LilyPond 2.22 in a timely manner. You derecursify the 
> build, David completes his stack of rather extensive purportive changes, Owen 
> merges the GSoC branch, etc., and only then it'll be time to care about 
> LilyPond 2.22.

... Dan submits his work on Simultaneous Alternatives, of which Unfoldable 
Lyrics are a special case ...
— 
Dan

Re: branching stable/2.22?

2020-08-23 Thread Jean Abou Samra

Hi,

(Sorry about the strange reply style.)

On Sun, Aug 23, 2020 at 1:58 PM Jonas Hahnfeld  wrote:
> I'd like to ask what it would take in principle to branch stable/2.22
> and what others think about this.
>
> Personally I'm convinced that creating the branch and only picking bug
> fixes from master is the only guaranteed way to stabilize. Now you
> might say that there were only few unstable releases (AFAICT there was
> 2.19.65 before branching stable/2.20). However, I think there are
> already quite some user-facing changes and also the switch to Python 3.
> With Python 2.x being EOL since January, it's only a matter of time
> until Linux distributions and BSDs want to drop that, and it would be
> unfortunate if that put a (temporary) end to providing LilyPond for
> their users. If we had release candidates or even a stable version
> until then, it would definitely help.

Maybe we could try to release 2.20.1 with Python 3?

> The same can of course happen with Guile, but that situation has been
> going on for years. Furthermore, it's at least possible to compile and
> use current master with Guile 2.x, even if slower. In my understanding,
> things are getting better but properly integrating byte compilation is
> still a good amount of work that nobody could give a deadline on.
>
> WDYT?

[Han-Wen] I think that both Python 3 support and usable (if imperfect) GUILE 2 
support is a strong argument for releasing a new stable version soon.

Why Guile 2? If it's still imperfect and slower, we don't want to make it the 
default in the binaries at lilypond.org, do we? So how will the situation be 
different from 2.20? Sorry, I must be missing something obvious here... I don't 
understand you.

At any rate, Python 3 support is great, but the scripts are fragile at the 
moment. This is clear from the tracker and the bug-lilypond list. Our Python 
scripts still need fixes in the way we distribute them, plus encoding-related 
issues (I'm planning on tackling the latter point in a short period at the end 
of August, but who knows what that will reveal).

[Han-Wen] I've been working on the build system (obviously), with in the back 
of my mind having a build that is no longer recursive, but that work could be 
paused for a bit while we prepare for releasing a 2.22. Is there a list of 
problems in the current master that have to be resolved?

Problems are basically popping every week (e.g., info installation, translation 
tools, etc.). You're fixing them every week, which is really great, but before 
creating a release branch that is devoted to stability, I think we need a 
couple months to see what new problems appear, don't we?

We have unstable releases to publish new features and get testing. In my 
opinion, stable releases should really focus on stability, there's no need to 
rush because of Python 3 and Guile 2.

At least four areas are currently under flux: Python scripts, the build system, 
Guile 2 support, and fonts (Owen's project), and I don't see that master is 
coming any close to stability. I think we are better with focusing on these 
areas as long as they still require substantial attention, so as to get a 
stable and nice LilyPond 2.22 in a timely manner. You derecursify the build, 
David completes his stack of rather extensive purportive changes, Owen merges 
the GSoC branch, etc., and only then it'll be time to care about LilyPond 2.22.

Am Sonntag, den 23.08.2020, 14:59 +0200 schrieb David Kastrup:
> > I'd like to ask what it would take in principle to branch stable/2.22
> > and what others think about this.
> I don't see that this is a good point of time.
> There has been an influx of badly tested changes to the build system and
> directory setup and the web pages that has to stabilise and result in a
> workable version of LilyPond. I don't see the point in branching a
> "stable" branch if there is so much in a destabilised state: you'd have
 > to cherry-pick loads of stuff from the unstable branch as it comes in.

[Jonas] I fully agree 

... and so do I (for what my opinion's worth, really) ...

[Jonas] and I should have been more clear that I don't expect the branch to 
happen in the next week. The point was to find out what it would take because 
just waiting for some unspoken condition to become true is not exactly going to 
happen without some effort.

What about scheduling the release?

While I do know that "Grass doesn't grow faster when you pull on it.", I would 
definitely like having a defined point in time where the stable release is to 
happen, so that everyone can focus on bug fixes before it happens. Sure, we 
aren't going to get agreement in a second about the date (even if not more 
precise than a month), but to me, having this talk now is preferable so as to 
give LilyPond development a tempo. To say it with other words, we've got a 
score to play; arguing about the tempo is better than starting the piece with 
different tempi.

As sort of a shot in the dark, how about planning

Re: branching stable/2.22?

2020-08-23 Thread Han-Wen Nienhuys

On Sun, Aug 23, 2020 at 1:58 PM Jonas Hahnfeld  wrote:
> I'd like to ask what it would take in principle to branch stable/2.22
> and what others think about this.
>
> Personally I'm convinced that creating the branch and only picking bug
> fixes from master is the only guaranteed way to stabilize. Now you
> might say that there were only few unstable releases (AFAICT there was
> 2.19.65 before branching stable/2.20). However, I think there are
> already quite some user-facing changes and also the switch to Python 3.
> With Python 2.x being EOL since January, it's only a matter of time
> until Linux distributions and BSDs want to drop that, and it would be
> unfortunate if that put a (temporary) end to providing LilyPond for
> their users. If we had release candidates or even a stable version
> until then, it would definitely help.
> The same can of course happen with Guile, but that situation has been
> going on for years. Furthermore, it's at least possible to compile and
> use current master with Guile 2.x, even if slower. In my understanding,
> things are getting better but properly integrating byte compilation is
> still a good amount of work that nobody could give a deadline on.
>
> WDYT?

I think that both Python 3 support and usable (if imperfect) GUILE 2
support is a strong argument for releasing a new stable version soon.

I've been working on the build system (obviously), with in the back of
my mind having a build that is no longer recursive, but that work
could be paused for a bit while we prepare for releasing a 2.22.

Is there a list of problems in the current master that have to be resolved?

We could also consider a freeze for some time period (eg. 1-2 months),
to allow the master branch to stabilize, before we cut 2.22.

--
Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen

Re: branching stable/2.22?

2020-08-23 Thread Jonas Hahnfeld

Am Sonntag, den 23.08.2020, 14:59 +0200 schrieb David Kastrup:
> Jonas Hahnfeld  writes:
> > Hi all,
> > 
> > I'd like to ask what it would take in principle to branch stable/2.22
> > and what others think about this.
> 
> I don't see that this is a good point of time.
> 
> There has been an influx of badly tested changes to the build system and
> directory setup and the web pages that has to stabilise and result in a
> workable version of LilyPond.  I don't see the point in branching a
> "stable" branch if there is so much in a destabilised state: you'd have
> to cherry-pick loads of stuff from the unstable branch as it comes in.

I fully agree and I should have been more clear that I don't expect the
branch to happen in the next week. The point was to find out what it
would take because just waiting for some unspoken condition to become
true is not exactly going to happen without some effort.

> > Personally I'm convinced that creating the branch and only picking bug
> > fixes from master is the only guaranteed way to stabilize.
> 
> Taking a look at Documentation/en/changes.tely, there has not exactly
> been a deluge of new features.
> 
> > Now you might say that there were only few unstable releases (AFAICT
> > there was 2.19.65 before branching stable/2.20). However, I think
> > there are already quite some user-facing changes and also the switch
> > to Python 3.  With Python 2.x being EOL since January, it's only a
> > matter of time until Linux distributions and BSDs want to drop that,
> > and it would be unfortunate if that put a (temporary) end to providing
> > LilyPond for their users. If we had release candidates or even a
> > stable version until then, it would definitely help.
> 
> At the current point of time, I see far too much focus on destabilising
> the code base that I consider a single-person(?) project of making a
> reasonably stable branch while the unstable branch is getting pounded on
> something that would work well.

Right, which makes discussion and a direction even more important.

> > The same can of course happen with Guile, but that situation has been
> > going on for years. Furthermore, it's at least possible to compile and
> > use current master with Guile 2.x, even if slower. In my
> > understanding, things are getting better but properly integrating byte
> > compilation is still a good amount of work that nobody could give a
> > deadline on.
> > 
> > WDYT?
> 
> While I see little point in making the quality of Guile-2+ support a
> showstopper, I don't see that the current pace of revamping the code
> base makes branching off a "stable" branch at this point of time a
> sensible proposition.  There just would be too much paralleled work in
> trying to get "stable" a sensible moniker.  The stable branch tends to
> see only rather superficial testing, and a large divergence from master
> would make its stability more a matter of wishful thinking than release
> engineering.


signature.asc
Description: This is a digitally signed message part

Re: branching stable/2.22?

2020-08-23 Thread David Kastrup

Jonas Hahnfeld  writes:

> Hi all,
>
> I'd like to ask what it would take in principle to branch stable/2.22
> and what others think about this.

I don't see that this is a good point of time.

There has been an influx of badly tested changes to the build system and
directory setup and the web pages that has to stabilise and result in a
workable version of LilyPond.  I don't see the point in branching a
"stable" branch if there is so much in a destabilised state: you'd have
to cherry-pick loads of stuff from the unstable branch as it comes in.

> Personally I'm convinced that creating the branch and only picking bug
> fixes from master is the only guaranteed way to stabilize.

Taking a look at Documentation/en/changes.tely, there has not exactly
been a deluge of new features.

> Now you might say that there were only few unstable releases (AFAICT
> there was 2.19.65 before branching stable/2.20). However, I think
> there are already quite some user-facing changes and also the switch
> to Python 3.  With Python 2.x being EOL since January, it's only a
> matter of time until Linux distributions and BSDs want to drop that,
> and it would be unfortunate if that put a (temporary) end to providing
> LilyPond for their users. If we had release candidates or even a
> stable version until then, it would definitely help.

At the current point of time, I see far too much focus on destabilising
the code base that I consider a single-person(?) project of making a
reasonably stable branch while the unstable branch is getting pounded on
something that would work well.

> The same can of course happen with Guile, but that situation has been
> going on for years. Furthermore, it's at least possible to compile and
> use current master with Guile 2.x, even if slower. In my
> understanding, things are getting better but properly integrating byte
> compilation is still a good amount of work that nobody could give a
> deadline on.
>
> WDYT?

While I see little point in making the quality of Guile-2+ support a
showstopper, I don't see that the current pace of revamping the code
base makes branching off a "stable" branch at this point of time a
sensible proposition.  There just would be too much paralleled work in
trying to get "stable" a sensible moniker.  The stable branch tends to
see only rather superficial testing, and a large divergence from master
would make its stability more a matter of wishful thinking than release
engineering.

-- 
David Kastrup

Re: branching

2013-12-11 Thread David Kastrup

Mike Solomon m...@mikesolomon.org writes:

 I had coffee with a developer a year or so ago who told me that he
 dropped out of the project because of commutation problems with David.
 Last night I wrote to him to share some of these frustrations and he
 wrote back: “as long as David is leading up the team, it’s a lost
 cause, nothing has changed and his way of acting is too problematic,
 independently of his technical excellence.”  I tend to conceive of
 problems in terms of one, two, or many.  The problem here has reached
 the “many level and I think we need to find a solution that goes
 beyond the individual.  List-etiquette policies, branching policies -
 I’m open to trying anything.

[...]

 What I’d like to see is a situation where David can blow off steam
 however he needs to, he doesn’t feel like people are ignoring him,

You are aware that the above are mutually exclusive?  If people consider
any negative feedback as blowing off steam, that exactly implies
suggesting to ignore it.

If you take a look at the communications with you that escalated, the
escalation happened _exactly_ because you did not take seriously what
I said.  So I stated my point more strongly.

 and LilyPond can be a dynamic, multiplayer environment without people
 getting offended and leaving the project.

We are currently taking a look at how to create regions of LilyPond that
can be worked on independently and without affecting the overall quality
of LilyPond.  If we manage to do this successfully, we'll be able to
create a community work area where the quality of the individual project
does not impact anybody but the active users of such a project.

I think that a buffer area between core or nothing will broaden the
base of people casually working on or with LilyPond and exchanging their
experiences.  It will also help with people who say feature xxx has to
be in LilyPond in order to let me work with application yyy: they or
their tool provider can just drop appropriate files into appropriate
parts of the user- or tool- designated search paths.

If we can get to a consensus that it's best for the project if I'm
thrown out, of course that is perfectly possible.  I would suggest the
end of 2014 as a suitable date, simply because I received forward-going
contributions that I'd prefer to spend as they were intended.  But if
people wish for an earlier date, I'd just pay back a corresponding
amount.

Now regardless of whether I continue participation with the LilyPond
project or not, I definitely think that the code base needs to be
changed to accommodate more independent work.

While tools like git facilitate work based on separate branches and
merges and rebasing, this is essentially negated by a code base where
significant changes will tend to happen all over the place.  This causes
merge conflicts, or even worse, produces inconsistent code since
conflicting changes not touching the same text lines are merged.

So while the fundamental problem with participating with LilyPond can be
circumscribed as David does not react gracefully when people step on
his and LilyPond's toes, it does seem to make sense to work towards
more generally available toe room anyway.

In our current developer base, I don't see much more than a theoretical
interest in modularity and usability: many of the developers have become
settled in the status quo which works for them.  Most contributions are
rather focused about creating new functionality rather than making
existing functionality more accessible or even consolidate what we
already have.

There are changes like

commit 38a4081efa4a8ee2f5da780ca0ed2991627afc46
Author: David Kastrup d...@gnu.org
Date:   Sun Sep 30 02:21:00 2012 +0200

Issue 2869: Regularize lyrics lexer mode

That makes lyrics mode rather similar to markup mode regarding how
words are formed.  {} are never considered part of words unless
enclosed in quotes.  Unquoted words do not contain whitespace, braces,
quotes, backslashes, numbers or Scheme expressions.  In addition, they
cannot start with * . = and | since that would mess with duration,
assignment and barcheck syntax.  This removes some remaining
TeX-oriented cruft in the lexer.  The set of word-non-starters might
need revisiting, but at least the regtests seem to pass.

where a similar change in syntax happened for markups in 2004 if I read
this correctly:

commit 3d04206a83222e89d99ddf1a0766b6b74f158967
Author: Nicolas Sceaux nicolas.sce...@free.fr
Date:   Sun Nov 28 17:50:32 2004 +

* lily/parser.yy: get rid off   in markups by treating { } as
real lists.

* lily/lexer.ll: remove   from markup lexer mode.

This change was cited as a typical nuisance for newcomers on a list
counted off the head of one user recently (he was surprised that it was
actually fixed by now, now meaning 2.17.4).  There are a number of other
long-standing problems and awkwardnesses that I occasionally stumble
upon: why haven't things like

Re: branching

On Dec 11, 2013, at 11:36 AM, David Kastrup d...@gnu.org wrote:

 Mike Solomon m...@mikesolomon.org writes:
 
 I had coffee with a developer a year or so ago who told me that he
 dropped out of the project because of commutation problems with David.
 Last night I wrote to him to share some of these frustrations and he
 wrote back: “as long as David is leading up the team, it’s a lost
 cause, nothing has changed and his way of acting is too problematic,
 independently of his technical excellence.”  I tend to conceive of
 problems in terms of one, two, or many.  The problem here has reached
 the “many level and I think we need to find a solution that goes
 beyond the individual.  List-etiquette policies, branching policies -
 I’m open to trying anything.
 
 [...]
 
 What I’d like to see is a situation where David can blow off steam
 however he needs to, he doesn’t feel like people are ignoring him,
 
 You are aware that the above are mutually exclusive?  If people consider
 any negative feedback as blowing off steam, that exactly implies
 suggesting to ignore it.
 
 If you take a look at the communications with you that escalated, the
 escalation happened _exactly_ because you did not take seriously what
 I said.  So I stated my point more strongly.

Your e-mail merits a thorough response, but I should state immediately a 
corollary to this.

I get negative feedback all the time from the community (we all do).  What I 
mean is that there is the component not to ignore (the content) and the 
component to ignore (any abrasiveness used in the transmission of the content). 
 Blowing off steam refers to the latter.  That said, the more abrasive 
communication gets, the more difficult it is to determine its content.

I always take what you say seriously - as seriously as any other developer.  I 
certainly don’t claim to always get it, and sometimes you may need to restate 
things because I don’t get them.  This restating may be frustrating for you, 
but it does not ever necessitate communication that would make people want to 
leave the project.

More to come after I read through your e-mail.

Cheers,
MS
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

On Dec 11, 2013, at 11:36 AM, David Kastrup d...@gnu.org wrote:

 Mike Solomon m...@mikesolomon.org writes:
 
 
 We are currently taking a look at how to create regions of LilyPond that
 can be worked on independently and without affecting the overall quality
 of LilyPond.  If we manage to do this successfully, we'll be able to
 create a community work area where the quality of the individual project
 does not impact anybody but the active users of such a project.

I think this is an excellent goal.

 
 I think that a buffer area between core or nothing will broaden the
 base of people casually working on or with LilyPond and exchanging their
 experiences.  It will also help with people who say feature xxx has to
 be in LilyPond in order to let me work with application yyy: they or
 their tool provider can just drop appropriate files into appropriate
 parts of the user- or tool- designated search paths.

I agree.

 
 If we can get to a consensus that it's best for the project if I'm
 thrown out, of course that is perfectly possible.  I would suggest the
 end of 2014 as a suitable date, simply because I received forward-going
 contributions that I'd prefer to spend as they were intended.  But if
 people wish for an earlier date, I'd just pay back a corresponding
 amount.

This seems like a rather extreme, all-or-nothing solution.

What I’d like is people reacting gracefully when they feel that their toes are 
stepped on.

There was one time when I started with the project where I blew up at Han Wen 
for pushing a major change with minimal review on a large scale beam collision 
project that I was working on.  I flipped out - it was 80ish hours of my work 
down the toilet.  I was not at all considerate in what I said and Han Wen 
dropped out of the project for a while.  I’m not sure if the two were linked, 
but I felt _terrible_ about this - I realized that this is _never_ appropriate 
or useful for LilyPond, or for anything, to have communication like this.

 
 Now regardless of whether I continue participation with the LilyPond
 project or not, I definitely think that the code base needs to be
 changed to accommodate more independent work.

I completely agree.

 
 While tools like git facilitate work based on separate branches and
 merges and rebasing, this is essentially negated by a code base where
 significant changes will tend to happen all over the place.  This causes
 merge conflicts, or even worse, produces inconsistent code since
 conflicting changes not touching the same text lines are merged.
 
 So while the fundamental problem with participating with LilyPond can be
 circumscribed as David does not react gracefully when people step on
 his and LilyPond's toes, it does seem to make sense to work towards
 more generally available toe room anyway.

Stepping on LilyPond’s toes is in the eye of the developer - what you consider 
stepping on LilyPond’s toes may not be for someone else and vice-versa.  All of 
us are LilyPond’s toes.

I agree that it makes sens to work towards more toe room.  I don’t think this 
alone will make LilyPond viable in the long run.

 
 In our current developer base, I don't see much more than a theoretical
 interest in modularity and usability: many of the developers have become
 settled in the status quo which works for them.  Most contributions are
 rather focused about creating new functionality rather than making
 existing functionality more accessible or even consolidate what we
 already have.
 
 There are changes like
 
 commit 38a4081efa4a8ee2f5da780ca0ed2991627afc46
 Author: David Kastrup d...@gnu.org
 Date:   Sun Sep 30 02:21:00 2012 +0200
 
Issue 2869: Regularize lyrics lexer mode
 
That makes lyrics mode rather similar to markup mode regarding how
words are formed.  {} are never considered part of words unless
enclosed in quotes.  Unquoted words do not contain whitespace, braces,
quotes, backslashes, numbers or Scheme expressions.  In addition, they
cannot start with * . = and | since that would mess with duration,
assignment and barcheck syntax.  This removes some remaining
TeX-oriented cruft in the lexer.  The set of word-non-starters might
need revisiting, but at least the regtests seem to pass.
 
 where a similar change in syntax happened for markups in 2004 if I read
 this correctly:
 
 commit 3d04206a83222e89d99ddf1a0766b6b74f158967
 Author: Nicolas Sceaux nicolas.sce...@free.fr
 Date:   Sun Nov 28 17:50:32 2004 +
 
* lily/parser.yy: get rid off   in markups by treating { } as
real lists.
 
* lily/lexer.ll: remove   from markup lexer mode.
 
 This change was cited as a typical nuisance for newcomers on a list
 counted off the head of one user recently (he was surprised that it was
 actually fixed by now, now meaning 2.17.4).  There are a number of other
 long-standing problems and awkwardnesses that I occasionally stumble
 upon: why haven't things like the \tuplet command not been provided
 years ago?

Re: branching

2013-12-11 Thread Trevor Daniels


Mike Solomon wrote Wednesday, December 11, 2013 10:22 AM

 On Dec 11, 2013, at 11:36 AM, David Kastrup d...@gnu.org wrote:

 As opposed to me, Graham excelled at organizing
 and maintaining community efforts like this which makes his leaving an
 even larger loss.

 His leaving is a huge loss, and as he has stated several times, one big 
 reason is the lack of sense of community.  devel-list quarrels and the 
 thaws that come from them have real impacts on LilyPond - I can’t 
 emphasize enough that we need to stop marginalizing this issue and 
 tackle it head on.  Modularity, while perhaps a good long term solution, 
 is a long ways away.  How are we going to deal with this in 2014?

Actually, I think there is a great sense of community within the LilyPond
project.  Why else would people comment so favourably on the responses
they receive on the various mailing lists; why else would people on the
bug squad, Phil on releases, James on managing reviews, etc devote their
time to mundane tasks?  Why else would people, even those no longer
active in LilyPond development, continue to read the lists?  No, we are
a great community and it is greatly enjoyable to belong to it!  

Also we are extremely fortunate to have so many brilliant developers
willing to spend their time augmenting, fixing and improving our baby.
That differences arise from time-to-time is inevitable - from the nature of
this project our community includes people covering a wide range of
temperament.  Artists, artisans and engineers all figure prominently.  They
have different viewpoints and approaches, but this is an advantage not
a problem - providing we can find a way to live and work together without
stress levels getting too high.

 How are we going to deal with this in 2014?

I would recommend Mike and David both consider privately what they
might have done differently to avoid this situation arising.  Not what the
other should have done, but what they themselves might have done.
If they'd like to share those thoughts with the list that would be good, but
that is not essential.  But they must share them with each other and try
to culture a mutual respect and understanding for the other's position.

Trevor

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching


On Dec 11, 2013, at 2:18 PM, Trevor Daniels t.dani...@treda.co.uk wrote:

 
 Mike Solomon wrote Wednesday, December 11, 2013 10:22 AM
 
 On Dec 11, 2013, at 11:36 AM, David Kastrup d...@gnu.org wrote:
 
 As opposed to me, Graham excelled at organizing
 and maintaining community efforts like this which makes his leaving an
 even larger loss.
 
 His leaving is a huge loss, and as he has stated several times, one big 
 reason is the lack of sense of community.  devel-list quarrels and the 
 thaws that come from them have real impacts on LilyPond - I can’t 
 emphasize enough that we need to stop marginalizing this issue and 
 tackle it head on.  Modularity, while perhaps a good long term solution, 
 is a long ways away.  How are we going to deal with this in 2014?
 
 Actually, I think there is a great sense of community within the LilyPond
 project.  Why else would people comment so favourably on the responses
 they receive on the various mailing lists; why else would people on the
 bug squad, Phil on releases, James on managing reviews, etc devote their
 time to mundane tasks?  Why else would people, even those no longer
 active in LilyPond development, continue to read the lists?  No, we are
 a great community and it is greatly enjoyable to belong to it!  
 

I agree that this part of the community is great.  Reading the communication 
between these people is a pleasure - it is professional, courteous and fun.

 Also we are extremely fortunate to have so many brilliant developers
 willing to spend their time augmenting, fixing and improving our baby.
 That differences arise from time-to-time is inevitable - from the nature of
 this project our community includes people covering a wide range of
 temperament.  Artists, artisans and engineers all figure prominently.  They
 have different viewpoints and approaches, but this is an advantage not
 a problem - providing we can find a way to live and work together without
 stress levels getting too high.

I agree.

 
 How are we going to deal with this in 2014?
 
 I would recommend Mike and David both consider privately what they
 might have done differently to avoid this situation arising.  Not what the
 other should have done, but what they themselves might have done.
 If they'd like to share those thoughts with the list that would be good, but
 that is not essential.  But they must share them with each other and try
 to culture a mutual respect and understanding for the other's position.

David and I have already been back and forth privately a lot about these issues.

I value David’s criticisms and have learned a lot from his insistance on 
commenting, code quality, and design.  To address issues that David has brought 
up, I have read cover to cover two books on programming, one of which is on 
code refactoring and design.  I think these have led to tangible improvements 
in my work on LilyPond.  I look forward to learning more.

It’d be a great goal in 2014 to see people who left the project because of 
communication difficulties come back because of a sincere effort on the part of 
everyone to communicate with mutual respect, irrespective of what the 
differences are in their positions.

Cheers,
MS
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2013-12-11 Thread David Kastrup

Mike Solomon m...@mikesolomon.org writes:

 On Dec 11, 2013, at 11:36 AM, David Kastrup d...@gnu.org wrote:

[...]

 But that’s no good - we have to find a solution.  Modularity, while
 perhaps a good long term solution, is a long ways away.  How are we
 going to deal with this in 2014?

By making headway and not be defeatist about it?  is a long ways away
is terribly close to somebody else's problem and let's think about
this the year after next year.

 And I see no-one in the current developer base who would work in that
 direction.

 One of my major projects - eliminating calls to translate_axis, was
 moving exactly towards this (https://codereview.appspot.com/7185044/).

It's called Caches the interior skylines of vertical axis groups and
systems. and there is no obvious reason why cached skylines would not
be translatable.  Of course, there is the non-obvious reason that a
translation might lead to a violation of skyline invariants due to
numerical effects, but that's nothing that can't be caught in-place.

In other words: there are several different issues that are conflated
here in a single patch.  There is no inherent reason why not being able
to use translate_axis will lead to more modular code.  It seems actually
more likely that having to track offsets separately is going to make
stuff more complex.

More modular does _not_ mean eliminate functions doing a common task
because of restructuring data in a manner incompatible with using a
common function on it.  Most graphic subsystems deal with this kind of
thing by including a current transform matrix in their data sets so
that transforms do not need to be performed until the final operation
(actually, since PostScript also has the concept of a current transform,
it's easy to pass this job to the backend).  Of course, many skyline
operators require a common coordinate system for the involved skylines,
so there are points where one needs to normalize.

 The less we’re calling translate_axis (the Defense Against the Dark
 Arts function of LilyPond) in the backend, the more we can isolate
 functionality to certain places and the less surprises there are.

I don't see the connection.

The net effect for the user, according to the Changes file, appears to
be that outside-staff-priority stops working unless special callbacks or
special commands (\tupletOutsideStaffPriority) are used, regardless of
whether the grobs have the side-position-interface or not.

As a result, it is not all that surprising that there were several grobs
overlooked.

Does this have anything to do with caching the interior skylines of
vertical axis groups and systems?  Not apparently.  Does it have
anything to do with Removes the translate_axis call from
axis-group-interface outside-staff positioning? Not obviously.  There
is no explanation in the review or patches for that.

I've taken a look at the current patch:
URL:https://codereview.appspot.com/7185044/#msg53 rather suggests that
there is considerable work needed to get this into commit quality.

There is also no explanation of the great plan behind this anywhere, and
since this is a whole lot of stuff muddled in one review, it appears
unlikely that the work will be split into a sequence of commits that
give some overall direction (it would be quite untypical for you to
split the material in a review, however involved, into separate
commits).

You say that there is plenty of followup work.

 I’ve for the time being dropped it until we can work out these
 community problems.

The respective comments are in
URL:http://code.google.com/p/lilypond/issues/detail?id=3134#c64 and
the following.  They have nothing whatsoever to do with community
problems.  What they _have_ to do with is release timing.

Yes, the time frame I estimated for getting 2.18 was too optimistic.
However, if this code would have been pushed at that stage and/or I
would have been required to review it until it was of stable release
quality, the time for getting every kink ironed out would have been much
much larger.

You stated that the bug fixes of previous patches were mostly done by
Keith since you were removing yourself from development due to bad
communication of mine.  That does not really fit the time frames: your
respective announcement came after most of the fixes.

What figured in in your lack of time and/or concentration at that time,
as it very well should, is you getting married, and working a whole lot
with your ensemble to boot.

The actually sad thing about the whole thing is that your withdrawal
from active development (based on the feedback on a different issue)
gave 2.18 and me as its warden the breathing room that was required for
it to get into releasable state.

So while you paint it as a loss for LilyPond, the _direct_ consequence
of your withdrawal regarding those patches was a net win for getting the
release done.  The indirect consequences, namely you stopping also fixes
necessary for getting to a release and, as you claim, others

Re: branching


On Dec 11, 2013, at 3:56 PM, David Kastrup d...@gnu.org wrote:

 Mike Solomon m...@mikesolomon.org writes:
 
 On Dec 11, 2013, at 11:36 AM, David Kastrup d...@gnu.org wrote:
 
 [...]
 
 But that’s no good - we have to find a solution.  Modularity, while
 perhaps a good long term solution, is a long ways away.  How are we
 going to deal with this in 2014?
 
 By making headway and not be defeatist about it?  is a long ways away
 is terribly close to somebody else's problem and let's think about
 this the year after next year.
 
 And I see no-one in the current developer base who would work in that
 direction.
 
 One of my major projects - eliminating calls to translate_axis, was
 moving exactly towards this (https://codereview.appspot.com/7185044/).
 
 It's called Caches the interior skylines of vertical axis groups and
 systems. and there is no obvious reason why cached skylines would not
 be translatable.  Of course, there is the non-obvious reason that a
 translation might lead to a violation of skyline invariants due to
 numerical effects, but that's nothing that can't be caught in-place.
 

The title for this is pretty terrible, I agree.  It was in sketch phase a few 
months ago before I stopped working on it.

 In other words: there are several different issues that are conflated
 here in a single patch.  There is no inherent reason why not being able
 to use translate_axis will lead to more modular code.  It seems actually
 more likely that having to track offsets separately is going to make
 stuff more complex.

In most of LilyPond, grob properties are calculated via callbacks, which 
themselves trigger the calculations of other properties they need a cascading 
sort of way.  This means that we can swap out the default function 
Script.Y-offset with another callback or a number.

When translate_axis is used in the code, it means that while looking up a 
property of grob A, the offsets of B, C, and D will be set internally.  This 
means that if a user sets the Y-offset of grob B to 2 and it is actually 4 
because translate_axis has been called somewhere deep in the code, it is 
counterintuitive how this happened and how to change it.  Furthermore, if the 
user overrides the callback that triggered translate axis, customary grob 
movements won’t happen anymore and the user won’t know why.  In this way, 
translate_axis impedes modularity: it makes us less able to plug in different 
callback functions for different grobs and get expected results.

My goal is to put all offset calculations in the callback function so that if I 
set TextScript.Y-offset to #0.1, I am positive that its Y offset with respect 
to its parent will be #0.1.  This is currently not the case.


 
 The less we’re calling translate_axis (the Defense Against the Dark
 Arts function of LilyPond) in the backend, the more we can isolate
 functionality to certain places and the less surprises there are.
 
 I don't see the connection.
 
 The net effect for the user, according to the Changes file, appears to
 be that outside-staff-priority stops working unless special callbacks or
 special commands (\tupletOutsideStaffPriority) are used, regardless of
 whether the grobs have the side-position-interface or not.
 
 As a result, it is not all that surprising that there were several grobs
 overlooked.
 
 Does this have anything to do with caching the interior skylines of
 vertical axis groups and systems?  Not apparently.  Does it have
 anything to do with Removes the translate_axis call from
 axis-group-interface outside-staff positioning? Not obviously.  There
 is no explanation in the review or patches for that.
 
 I've taken a look at the current patch:
 URL:https://codereview.appspot.com/7185044/#msg53 rather suggests that
 there is considerable work needed to get this into commit quality.
 

There is a lot of work ahead of it, and if I get back to it, I’ll be splitting 
it up into smaller commits.

 There is also no explanation of the great plan behind this anywhere,

The great plan is:
Remove as many calls to translate_axis as possible.
Create pure versions of the offset callbacks, which now calculate actual 
offsets without translate_axis tacked on.
Now that these pure versions are used, pure estimates of staff height will be 
more accurate because they’ll finally take into consideration outside-staff 
grobs.
That means that spacing stubs for cross staff grobs that need to user pure 
staff height calculations will be closer to the actual spacing.
This means that scores with lots of cross-staff grobs will have better vertical 
spacing.  Currently, vertical spacing of these scores is too airy because there 
is no good way to estimate outside-staff grobs, so LilyPond overestimates.

 and
 since this is a whole lot of stuff muddled in one review, it appears
 unlikely that the work will be split into a sequence of commits that
 give some overall direction (it would be quite untypical for you to
 split the material in a review, however

Re: branching

2013-12-10 Thread Graham Percival

On Tue, Dec 10, 2013 at 02:42:45PM +0200, Mike Solomon wrote:
 On the website, we would offer _all_ of these development
 branches, including the one built off of staging, as GUB builds.

A few years ago, we were asked to cut our downloads down to 5 GB.
I deleted a bunch of devel stuff and got it down to 7 or 8, but
we're probably back to 15 GB now.  I don't think we should fill up
the server with that many builds, nor do I think we should ask
James to do all that building.

 We would also contain a tracker showing number of downloads and
 encouraging users to download the branches that have been
 downloaded the _least_.

I don't like the idea of treating users as guinea pigs.  If
somebody wants to test out devel stuff, it's not unreasonable for
them to compile it themselves (inside lilydev if necessary).

- Graham

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching


On Dec 10, 2013, at 2:58 PM, Graham Percival gra...@percival-music.ca wrote:

 On Tue, Dec 10, 2013 at 02:42:45PM +0200, Mike Solomon wrote:
 On the website, we would offer _all_ of these development
 branches, including the one built off of staging, as GUB builds.
 
 A few years ago, we were asked to cut our downloads down to 5 GB.
 I deleted a bunch of devel stuff and got it down to 7 or 8, but
 we're probably back to 15 GB now.  I don't think we should fill up
 the server with that many builds, nor do I think we should ask
 James to do all that building.

I agree about the not asking James part - this should be split up (different 
gubs making different builds).
As for the filling the server part, this seems like something that can be 
solved with alternate hosting sites for the download links.

 
 We would also contain a tracker showing number of downloads and
 encouraging users to download the branches that have been
 downloaded the _least_.
 
 I don't like the idea of treating users as guinea pigs.  If
 somebody wants to test out devel stuff, it's not unreasonable for
 them to compile it themselves (inside lilydev if necessary).

I don’t either, but I do like the idea of asking them to be guinea pigs.
To this end, I’ll send an e-mai to the users list about this.

Cheers,
MS
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2013-12-10 Thread David Kastrup

Mike Solomon m...@mikesolomon.org writes:

 Hey all,

 As 2.18 draws near, I’d like to work with everyone to establish a
 system of branching for LilyPond development.  After several rounds of
 discussing things with David K, this emerged as the best way to avoid
 arguments about integrating work into the development branch that have
 led several contributors, including myself, to significantly reduce
 time on the project. [1]

It will take me considerable time to address this more thoroughly, but
in the mean time I want to have on record that what Mike writes here
under this emerged as the best way _not_ in _any_ way summarizing our
discussion.  Instead it is his _personal_ idea how to address topics we
covered.  Writing something like this emerged as the best way when we
did not even talk about it wrongly insinuates that it presents a common
understanding of how to address problems covered in the discussion.

Feel free to quote passages or complete mails from me in the course of
that discussion: there is nothing in there that I would not have equally
said in public.

-- 
David Kastrup

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching


On Dec 10, 2013, at 4:23 PM, David Kastrup d...@gnu.org wrote:

 Mike Solomon m...@mikesolomon.org writes:
 
 Hey all,
 
 As 2.18 draws near, I’d like to work with everyone to establish a
 system of branching for LilyPond development.  After several rounds of
 discussing things with David K, this emerged as the best way to avoid
 arguments about integrating work into the development branch that have
 led several contributors, including myself, to significantly reduce
 time on the project. [1]
 
 It will take me considerable time to address this more thoroughly, but
 in the mean time I want to have on record that what Mike writes here
 under this emerged as the best way _not_ in _any_ way summarizing our
 discussion.  Instead it is his _personal_ idea how to address topics we
 covered.  Writing something like this emerged as the best way when we
 did not even talk about it wrongly insinuates that it presents a common
 understanding of how to address problems covered in the discussion.
 

Sorry, I in no way mean to apply that this is a proposition on David’s behalf.  
This is a proposition from me that it is a response to our discussion, not a 
summary of it.

 Feel free to quote passages or complete mails from me in the course of
 that discussion: there is nothing in there that I would not have equally
 said in public.


(quotes from David)

The basic idea behind that is not to make confrontations nicer but
reduce the necessity for them by establishing playing fields with
different authorities.  So that people can get work done without
endangering the release, and I can get releases done without pissing
people off as a prerequisite.

We won't be able to completely separate the two, but both our current
code base and our current development model are quite bad for getting
this under control.

…

As I said: there are certainly decisions where a vote does make sense:
mostly when there is a choice between alternatives.  For go ahead or
not kind of decisions, the answer should likely be go ahead in a
sandbox, and with enough exposure of the sandbox to testers and end
users, we'll be better equipped to make an informed decision.

That's how it tends to work with the Linux kernel, but both our code
base as well as our infrastructure is not really diverse enough to make
this easy.  It _would_ be easier in a Scheme universe, or with loadable
C++ modules.  But either case requires that enough of the internals are
implemented through exchangeable interfaces that swapping out key parts
for user-written code can be done without disrupting too many unrelated
parts in the process.

…

My proposal is a first-pass at starting a dialogue about this that people can 
respond to - I expect some or all of it to be rejected, but the important thing 
is to start talking about it now.

Cheers,
MS
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2013-12-10 Thread Graham Percival

On Tue, Dec 10, 2013 at 04:30:32PM +0200, Mike Solomon wrote:
 
 (quotes from David)
 
 The basic idea behind that is not to make confrontations nicer but
 reduce the necessity for them by establishing playing fields with
 different authorities.  So that people can get work done without
 endangering the release, and I can get releases done without pissing
 people off as a prerequisite.

I agree with this idea, but I feel like I'm missing something.
How would regular git branches be insufficient for this?  Is it
simply the lack of GUB for branches?

I mean, is the problem a technical or social one?  My gut feeling
is that this is a social issue, so any technical wish-lists are a
red herring.  That said, I admit that I haven't even skimmed the
patch countdowns in the past few months, let alone read any
arguments on -devel.

- Graham

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2013-12-10 Thread David Kastrup

Graham Percival gra...@percival-music.ca writes:

 On Tue, Dec 10, 2013 at 04:30:32PM +0200, Mike Solomon wrote:
 
 (quotes from David)
 
 The basic idea behind that is not to make confrontations nicer but
 reduce the necessity for them by establishing playing fields with
 different authorities.  So that people can get work done without
 endangering the release, and I can get releases done without pissing
 people off as a prerequisite.

 I agree with this idea, but I feel like I'm missing something.
 How would regular git branches be insufficient for this?  Is it
 simply the lack of GUB for branches?

 I mean, is the problem a technical or social one?  My gut feeling
 is that this is a social issue, so any technical wish-lists are a
 red herring.

We won't make social issues go away by creating policies.  Basically we
have limited possibilities to change how amicably people react when they
are stepping on each others' toes.

Creating more room is a technical measure.  It will not change the kind
of social interaction we perceive as problematic, but it will still
serve to reduce the opportunities for it to occur and cause problems.

 That said, I admit that I haven't even skimmed the patch countdowns in
 the past few months, let alone read any arguments on -devel.

Mike frequently referenced a Rietveld review as an example in his last
mails to the list.

-- 
David Kastrup

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2013-12-10 Thread Carl Sorensen



On 12/10/13 5:42 AM, Mike Solomon m...@mikesolomon.org wrote:

Hey all,

As 2.18 draws near, I¹d like to work with everyone to establish a system
of branching for LilyPond development.  After several rounds of
discussing things with David K, this emerged as the best way to avoid
arguments about integrating work into the development branch that have
led several contributors, including myself, to significantly reduce time
on the project. 

I can see how the proposed mechanism avoids arguments about work that is
going into individual developers branches, but I don't see how this avoids
arguments about what goes into the development branch.

As far as I can see, this proposal supports the creation of multiple forks
of the lilypond development branch, one for each of the privileged
developers, plus one for the accepted features.  And then lilypond.org
should support all of those forks.

This seems like a nightmare to me.  It is good for somebody who wants to
get their features out the the user base.  But this just makes the
decisions about what to include in the development branch more emotionally
charged.  I'll present a hypothetical exchange between two caricatures:
the creative developer who is only concerned about adding a really neat
feature, and doesn't care how it affects the code base; and the consistent
developer who is only concerned about the consistency of the code base,
and would rather have no new features added than have a new feature added
that requires contortions in the syntax, the parser/lexer, or the code
base.  Note that both of these are caricatures, drawn for the purpose of
highlighting the conflict, not for the purpose of illustrating the
behavior of any real developer.

Creative developer: See how many people like this feature?  It's so
positive that we absolutely have to include it.

Consistent developer: See how badly this would destroy the current code
base?  It's just an ugly hack.  If we accept features like this, our code
base will soon consist of nothing but ugly hacks.  I don't care how many
people like the feature; it's disastrous as currently implemented, so I
won't accept it.

And now, in addition to the different points of view of these developers,
we have the added pressure of users clamoring for a feature.

Personally, I want *both* the creative developer and the consistent
developer to be satisfied.  I want ugly hacks in the codebase to be
decreasing, not increasing.  I want new features that make the engraving
better.  If we can't implement new features without ugly hacks, I don't
think they should be added to the codebase.  But I don't want stasis in
the codebase to prevent the addition of new features.

Developers already have branches on the main repository.  So this proposal
is not really to add branches.

The real issue in the proposal is about how to get users using these
branches.  I have a very hard time thinking that it's in the best interest
of lilypond as a project to have multiple official development branches.
Instead, I think that developers who create a branch that provides some
new functionality should invite users to test the branch (which probably
means that the developer needs to get GUB running on his or her branch and
then make the binaries available).


[1]  I feel that this reduction in commit diversity is the biggest
obstacle to LilyPond¹s future evolution, which is why I¹m calling on
everyone to make a concerted effort to think this through.


I have made a concerted effort to think this through. I believe that a
reduction in commit diversity is a serious problem.  I think the community
was greatly weakened when Mike felt that it was no longer worth putting up
with the hassles to make his contributions.

However, I think that a fractured development branch (up to 6 different
branches) would be an even bigger obstacle to future evolution.  I think
it would lead to a balkanization of LilyPond.

Of course, given my participation in development over the last couple of
years, my input may not be worth much.

Thanks,

Carl S.


___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching


On Dec 10, 2013, at 9:32 PM, Carl Sorensen c_soren...@byu.edu wrote:

 
 
 On 12/10/13 5:42 AM, Mike Solomon m...@mikesolomon.org wrote:
 
 Hey all,
 
 As 2.18 draws near, I¹d like to work with everyone to establish a system
 of branching for LilyPond development.  After several rounds of
 discussing things with David K, this emerged as the best way to avoid
 arguments about integrating work into the development branch that have
 led several contributors, including myself, to significantly reduce time
 on the project. 
 
 This seems like a nightmare to me.  It is good for somebody who wants to
 get their features out the the user base.  But this just makes the
 decisions about what to include in the development branch more emotionally
 charged.

It seems like everyone agrees, so this is definitely not the way to go.

 I'll present a hypothetical exchange between two caricatures:
 the creative developer who is only concerned about adding a really neat
 feature, and doesn't care how it affects the code base; and the consistent
 developer who is only concerned about the consistency of the code base,
 and would rather have no new features added than have a new feature added
 that requires contortions in the syntax, the parser/lexer, or the code
 base.

This is a fantastic hypothetical exchange.

 
 
 [1]  I feel that this reduction in commit diversity is the biggest
 obstacle to LilyPond¹s future evolution, which is why I¹m calling on
 everyone to make a concerted effort to think this through.
 
 
 I have made a concerted effort to think this through. I believe that a
 reduction in commit diversity is a serious problem.  I think the community
 was greatly weakened when Mike felt that it was no longer worth putting up
 with the hassles to make his contributions.
 
 However, I think that a fractured development branch (up to 6 different
 branches) would be an even bigger obstacle to future evolution.  I think
 it would lead to a balkanization of LilyPond.
 
 Of course, given my participation in development over the last couple of
 years, my input may not be worth much.

After seeing people’s responses, I agree.  It is obvious from everyone’s 
reflections (especially yours, which is very thorough) that if branches become 
an institutionalized feature of development, they will hurt LilyPond.  I 
rescind my idea.

The only hassle for me, which I did not run up against when I started with the 
project, is David’s way of communicating.  I’m not claiming this is all on him, 
but I’m also pretty sure that I’m not the only one who has peaced out because 
of this.  I am looking for ways for this to no longer be an issue.  I was 
hoping that branches would go a way towards making this happen for myself and 
hopefully other developers, but it’s clear that this is not a good idea.

In my two day jobs, director of the ensemble 101 and developer for the Guido 
project, I work with two (very different) teams of people on projects that 
require creativity, consistency, and tons of communication.  Neither of them 
has any of this friction resulting from communication issues, both of them 
enjoy a diversity in major contributions, and both are evolving rapidly and 
stably in several interesting ways at the same time.  I truly hope that 
LilyPond can be like that.

Thank you for taking the time to think this over and for your response.

Cheers,
MS
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2013-12-10 Thread Carl Peterson

On Tue, Dec 10, 2013 at 3:21 PM, Mike Solomon m...@mikesolomon.org wrote:


 The only hassle for me, which I did not run up against when I started with
 the project, is David’s way of communicating.  I’m not claiming this is all
 on him, but I’m also pretty sure that I’m not the only one who has peaced
 out because of this.  I am looking for ways for this to no longer be an
 issue.  I was hoping that branches would go a way towards making this
 happen for myself and hopefully other developers, but it’s clear that this
 is not a good idea.

 In my two day jobs, director of the ensemble 101 and developer for the
 Guido project, I work with two (very different) teams of people on projects
 that require creativity, consistency, and tons of communication.  Neither
 of them has any of this friction resulting from communication issues, both
 of them enjoy a diversity in major contributions, and both are evolving
 rapidly and stably in several interesting ways at the same time.  I truly
 hope that LilyPond can be like that.


I don't know how you communicate with your other two teams, but the simple
fact is that email is a terrible method of communication, when it comes to
the things that you appear to be seeking. An amused or straightforward
comment can across as harsh or sarcastic when visual and aural cues are
absent (citing the studies that show that 90% of communication is
nonverbal, i.e., not connected to the actual words). Some people's manners
of speech translate into text-only communication better than others', and
some don't translate at all. I had a boss a couple of years back who could
be very friendly and personable face-to-face, but unless she was obvious
happy about something, always came across as stern and upset with the way
things were done. It happens. But you may already be well aware of all this.

It is regrettable that you would let such things interfere with your
contributions to LilyPond. Ultimately, it is about the project, not the
people. Perhaps counter-intuitively, the answer to the problem you perceive
is not to reduce participation, but to increase participation. In my own
case, my interactions with David had the effect of getting me more involved
in the behind the scenes workings of the code. Why? So that eventually,
David won't be able to criticize me for not being willing to get my hands
dirty. I haven't made a commit yet, but that's probably a matter of weeks
or days (whenever I get git-cl set up on my dev machine). In the meantime,
instead of complaining about this feature or that feature, or going Oh,
poor, pitiful me, someone give me a code snippet to do x, I've tried to
dig into things to make them work. Now the thing I'm trying to figure out
is how to make what I'm doing usable for others who do the same things so
that LilyPond is an easier environment to use. Will I ever get to where I'm
wrangling the underlying C++ code? Probably not. But I'm working on what I
can.

Cheers,
Carl P.
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2013-12-10 Thread David Nalesnik

Hi,

On Tue, Dec 10, 2013 at 2:46 PM, Carl Peterson carlopeter...@gmail.comwrote:

 On Tue, Dec 10, 2013 at 3:21 PM, Mike Solomon m...@mikesolomon.orgwrote:


 The only hassle for me, which I did not run up against when I started
 with the project, is David’s way of communicating.  I’m not claiming this
 is all on him, but I’m also pretty sure that I’m not the only one who has
 peaced out because of this.  I am looking for ways for this to no longer be
 an issue.  I was hoping that branches would go a way towards making this
 happen for myself and hopefully other developers, but it’s clear that this
 is not a good idea.

 In my two day jobs, director of the ensemble 101 and developer for the
 Guido project, I work with two (very different) teams of people on projects
 that require creativity, consistency, and tons of communication.  Neither
 of them has any of this friction resulting from communication issues, both
 of them enjoy a diversity in major contributions, and both are evolving
 rapidly and stably in several interesting ways at the same time.  I truly
 hope that LilyPond can be like that.


 I don't know how you communicate with your other two teams, but the simple
 fact is that email is a terrible method of communication, when it comes to
 the things that you appear to be seeking. An amused or straightforward
 comment can across as harsh or sarcastic when visual and aural cues are
 absent (citing the studies that show that 90% of communication is
 nonverbal, i.e., not connected to the actual words). Some people's manners
 of speech translate into text-only communication better than others', and
 some don't translate at all. I had a boss a couple of years back who could
 be very friendly and personable face-to-face, but unless she was obvious
 happy about something, always came across as stern and upset with the way
 things were done. It happens. But you may already be well aware of all this.


I always feel a bit silly writing emoticons and exclamation points, but
they are nice to see(!)


 It is regrettable that you would let such things interfere with your
 contributions to LilyPond.


Exactly.  Both you and David are invaluable to this project.  I watched the
paralysis set in, the deadlock, and wondered a bit about the future of the
project.  I think there has to be some compromise in this
Apollonian/Dionysian test of wills (to throw in a little pretentiousness).


 Ultimately, it is about the project, not the people. Perhaps
 counter-intuitively, the answer to the problem you perceive is not to
 reduce participation, but to increase participation. In my own case, my
 interactions with David had the effect of getting me more involved in the
 behind the scenes workings of the code. Why? So that eventually, David
 won't be able to criticize me for not being willing to get my hands dirty.


Well, I ordinarily have a bit of a thin skin, and I remember reading
somewhere on the lists that you have to have some nerve to contribute.  My
personal response to the possibility of brutally honest criticism--which is
a necessary thing if this project isn't going to go to hell--is to make
sure I've got everything as polished as I can make it before I make it
public.  And accepting that I've got a lot to learn,  (This is about me,
and is in no way directed at you, Mike.)

However, when the standstill happens, something has to be done.  I'm sorry
that I don't have any solution to offer, other than to try to work together
for the sake of LilyPond,  (Emoticons!  Kidding.)

--David
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

On Dec 11, 2013, at 12:03 AM, David Nalesnik david.nales...@gmail.com wrote:

Hi,

On Tue, Dec 10, 2013 at 2:46 PM, Carl Peterson carlopeter...@gmail.com
wrote:
On Tue, Dec 10, 2013 at 3:21 PM, Mike Solomon m...@mikesolomon.org wrote:

The only hassle for me, which I did not run up against when I started with
the project, is David’s way of communicating. I’m not claiming this is all
on him, but I’m also pretty sure that I’m not the only one who has peaced out
because of this. I am looking for ways for this to no longer be an issue. I
was hoping that branches would go a way towards making this happen for myself
and hopefully other developers, but it’s clear that this is not a good idea.

In my two day jobs, director of the ensemble 101 and developer for the Guido
project, I work with two (very different) teams of people on projects that
require creativity, consistency, and tons of communication. Neither of them
has any of this friction resulting from communication issues, both of them
enjoy a diversity in major contributions, and both are evolving rapidly and
stably in several interesting ways at the same time. I truly hope that
LilyPond can be like that.

I don't know how you communicate with your other two teams,

Face-to-face communication for one, e-mail for another.

I always feel a bit silly writing emoticons and exclamation points, but they
are nice to see(!)

It is regrettable that you would let such things interfere with your
contributions to LilyPond.

Exactly. Both you and David are invaluable to this project. I watched the
paralysis set in, the deadlock, and wondered a bit about the future of the
project. I think there has to be some compromise in this
Apollonian/Dionysian test of wills (to throw in a little pretentiousness).

See:
http://lilypond.1069038.n5.nabble.com/Allows-minimum-length-to-work-for-end-of-line-spanners-issue-7453046-td141952.html#a142870
as one of several examples. There is truth in anything David says, meaning
that I (like him (and most of us on this list)) have caused bugs that I did not
find or fix before someone else. How, does this warrant this communication
style? This chain of e-mails was the single determinant that ruled LilyPond
out of a government funded, multi-national European typesetting project I’m
organizing in which the team will need to extend aspects of the software. I
imagined the score of man hours that goes into all the projects I do and how
important team morale is over the long term. I don’t want anyone on my team to
lose time feeling bad from e-mail confrontations - it’s not worth it for many
reasons.

Ultimately, it is about the project, not the people. Perhaps
counter-intuitively, the answer to the problem you perceive is not to reduce
participation, but to increase participation. In my own case, my interactions
with David had the effect of getting me more involved in the behind the
scenes workings of the code. Why? So that eventually, David won't be able to
criticize me for not being willing to get my hands dirty.

Well, I ordinarily have a bit of a thin skin, and I remember reading
somewhere on the lists that you have to have some nerve to contribute. My
personal response to the possibility of brutally honest criticism--which is a
necessary thing if this project isn't going to go to hell--is to make sure
I've got everything as polished as I can make it before I make it public.
And accepting that I've got a lot to learn, (This is about me, and is in no
way directed at you, Mike.)

Of course I have a lot to learn, too.

I release things in various states of polished-ness to the list, often because
its their unpolished-ness that I need comments or opinions on. This in and of
itself is valuable information. Imagining a hypothetical scenario where I sent
a patch that someone wants to see restructured into several commits before they
can review it. One developer responds:

It’s difficult for me to understand what you’re doing - please split it up.

and another responds:

Please act like a member of the community, start taking other people into
consideration and split up your patches.

The information is the same, but (A) makes me want to stay in the project a lot
more than (B). A lot of (B) causes me to lose interest. How does one proceed?
Should one gloss over the way (B) is written and get (A) out of it? Should
one respond to (B) on its own terms? How does one let these things not
interfere with one’s contributions?

I currently have a boss who is an industry-expert in typesetting that decided
not to get involved with LilyPond development 3 or 4 years ago because of feuds
he saw on the devel list. This is not good. By old developers not
contributing as much and new ones not wanting to join, our community has
dwindled down to one where 40% of commits are made by one person, and this is
including documentation, meaning that

Re: branching

2013-12-10 Thread Werner LEMBERG


 One thing that is important to restate is that I realize that,
 unlike my bad-but-getting-better commenting style, _all_ of this is
 out of David’s control in that it stems from a medical issue.  Which
 means that the communication problems won’t go away - they’re going
 to be a fixture of LilyPond development so long as he’s involved.
 As a group of humans, it is _everyone’s_ responsibility to find
 creative ways to get past handicaps because _no one_ wants to have
 them.  What I’d like to see is a situation where David can blow off
 steam however he needs to, he doesn’t feel like people are ignoring
 him, and LilyPond can be a dynamic, multiplayer environment without
 people getting offended and leaving the project.

I fully agree.  I got into the habit of contacting apparently offended
people off-list to explain the very reasons why communication on this
list can be tedious sometimes.

And please remember the most important thing: There is *never* an
intention, as far as I can see, to offend people.  The more factual
the discussion, the better it works.


Werner
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-05-09 Thread Joseph Wakeling

On 04/29/2011 07:15 PM, Carl Sorensen wrote:
 This might work, but fails to meet the major criterion of the proposed
 branching scheme.  The proposal is to make 2.14 stable.

Yes, that's why my proposal was to apply every BUGFIX to 2.14 first, not
every patch. :-)

(Of course, by bugfix, I mean fix for a bug in 2.14.  Bugs in dev
only should get fixed in dev only.)

 Actually, I think your final comment is backwards.  Every patch is
 applicable to dev, but only some are applicable to 2.14.

Backwards for patches in general; not for bugfixes.

The idea is that bugfixes get applied to 2.14 first and then merged into
dev (minimal cherrypicking here); while other patches (new features
etc.) get applied to dev and never go near 2.14.

dev still gets everything, 2.14 gets just what it needs, and in the best
case scenario, you should _never have to cherrypick_, just merge from
2.14 to dev every time there is a bugfix.

Is the idea clearer now?

By the way, sorry for not following up on this sooner.  Busy time in the
office. :-(

Thanks  best wishes,

-- Joe

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-29 Thread Joseph Wakeling

On 04/15/2011 07:05 PM, Carl Sorensen wrote:
 Just to be sure I understand correctly, the only things I will cherry-pick
 into stable/2.14 will be bugfixes for critical bugs.

Just as a remark, I wonder if you may find it easier to adopt an
alternative workflow:

   -- bugfix gets applied first in 2.14 branch

   -- bugfix gets merged into master dev branch

 since that probably reduces the amount of cherry-picking that's
needed -- I expect that probably almost every patch applied to 2.14 is
also applicable to dev, but not vice versa.

Best wishes,

-- Joe

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-29 Thread Carl Sorensen




On 4/29/11 9:55 AM, Joseph Wakeling joseph.wakel...@webdrake.net wrote:

 On 04/15/2011 07:05 PM, Carl Sorensen wrote:
 Just to be sure I understand correctly, the only things I will cherry-pick
 into stable/2.14 will be bugfixes for critical bugs.
 
 Just as a remark, I wonder if you may find it easier to adopt an
 alternative workflow:
 
-- bugfix gets applied first in 2.14 branch
 
-- bugfix gets merged into master dev branch
 
  since that probably reduces the amount of cherry-picking that's
 needed -- I expect that probably almost every patch applied to 2.14 is
 also applicable to dev, but not vice versa.


This might work, but fails to meet the major criterion of the proposed
branching scheme.  The proposal is to make 2.14 stable.

Actually, I think your final comment is backwards.  Every patch is
applicable to dev, but only some are applicable to 2.14.

Thanks,

Carl


___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching isn't working

2011-04-29 Thread Keith OHara

Graham Percival graham at percival-music.ca writes:
 On Thu, Apr 28, 2011 at 02:45:07PM +0100, Graham Percival wrote:
  We can't just change VERSION in git master, because that's used to
  generate the website,
 
 oh wait, I just remembered that we have a separate MINOR_VERSION
 from VERSION_DEVEL.   

I encourage you to implement your branching plan ... somehow.

I don't know exactly how, but it seems there is plenty of possibilities if you 
are willing to shepherd the build.  Given that the branch labeled 'stable/2.14' 
will generate the next few builds, with version numbers 2.13.61 etc., I had 
imagined it might generate the website as well for a while.


___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching isn't working

2011-04-28 Thread Graham Percival

On Thu, Apr 28, 2011 at 02:45:07PM +0100, Graham Percival wrote:
 We can't just change VERSION in git master, because that's used to
 generate the website,

oh wait, I just remembered that we have a separate MINOR_VERSION
from VERSION_DEVEL.  ha, a badly designed almost-duplicate of
information might be useful after all!

ok, this is no longer a valid reason for dumping the branching
idea -- master could have a VERSION (from MAJOR_ MINOR_
PATCH_LEVEL) of 2.15.0, but a VERSION_DEVEL of 2.13.60 or .61.

That said, I still believe that:
 This doesn't seem to be working.

Cheers,
- Graham

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-17 Thread Graham Percival

On Sat, Apr 16, 2011 at 01:04:16PM -0600, Carl Sorensen wrote:
 
 On 4/16/11 12:50 PM, Han-Wen Nienhuys hanw...@gmail.com wrote:
 
  My proposal is that ready-to-release still is not strict enough for
  stable, so 2.14 version should be coming from something which moves
  slower than the master branch.
 
 So does this mean we need three branches?
 
 stable  (currently 2.12.x)
 stable beta (currently stable/2.14)
 development (currently 2.13.x)

Well, we already have more than three branches.  In your scheme,
they'd be stable/2.12, stable/2.14, and master.

 It seems to me that we ought to have three releases available:
 
 stable -- demonstrated to work properly
 stable beta -- all syntax from stable works, passes regtests, but not
 demonstrated to be critical regression free.  Should be acceptable for
 general release, but not for production work.
 development -- passes regtests.  Recommended for developers and adventurous
 users, but possibility exists for syntax changes.
 
 Does this seem feasible?

I'm not going to make three releases, because it's too confusing
for users.  We'll have two releases: stable and unstable.

At the moment, we only have unstable releases happening.  They
happen from stable/2.14.  Anybody wanting to play with git master
right now needs to compile from source.

Once 2.14 is out, we'll have stable releases happening from
stable/2.14 and unstable releases happening from git master.

Cheers,
- Graham

___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-16 Thread Trevor Daniels



Carl Sorensen wrote Friday, April 15, 2011 8:40 PM

How do we define the difference between stable development and 
unstable
development?  It seems to me that stable development means we 
pass the
regtests -- we've been doing that for at least a couple of years. 
But we
still end up with regressions when users test the code beyond what 
the

developers have tested.

As long as we have a policy that any regression will delay a 
stable release
for two weeks, it seems to me that we *must* stop adding features 
in order

to get a stable release.  We can't prevent unintended regressions.


+1

OTOH, if the standard for ready-to-release were make check 
succeeds and
no segfaults have been identified, then we've been ready to 
release a

stable version for a long time now.


There is an alternative.  Look back through git history
and choose a point earlier than any recent iffy development
and branch out from there.  Cherrypick more recent good
bugfixes and apply them.  Also cherrypick any more recent
developments which have not caused user problems.  With care
that will give a good stable release, and permit the
master branch to continue with development unimpeded.  This
process can be repeated at any time to enable stable releases
to be issued more frequently.

Trevor



___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-16 Thread Xavier Scheuer

On 16 April 2011 01:11, m...@apollinemike.com m...@apollinemike.com wrote:

 I agree - it is for this reason that I feel the only two solutions are to:

 (1) Freeze pushes to the master branch save ones that are explicitly 
 authorized by Graham.
 (2) Keep working at our normal pace, but split off stable versions to 
 which only bug fixes will be applied.

Translations and some doc updates could also benefit of (2), isn't it?

Right now it happens that some doc are not fully translated for a stable
release.  Then we should wait for a new stable release to have the new
translations available, but right now stable release does not appear
often (more than 15 month between 2.12.3 and upcoming-2.14).

Cheers,
Xavier

-- 
Xavier Scheuer x.sche...@gmail.com

___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-16 Thread Graham Percival

On Fri, Apr 15, 2011 at 09:30:06PM +0200, Valentin Villenave wrote:
 I'm not entirely sure everything can be implemented using small steps
 (e.g. spacing code overhaul, as has happened in the past).

Umm, the spacing code overhaul *did* start off on a separate
branch.  IIRC Joe worked on it there for 3 or 4 months before it
was merged.

 As I suggested this summer, the new Fedora branching model looks
 pretty interesting to me:
 https://fedoraproject.org/wiki/Releases/Branched

I skimmed the article, but I didn't see anything interesting.  Are
you talking about freezes?  Freezes are nothing new.

 Of course, it does require some resources. But (as always) I do trust
 Carl to be exactly the person we need :-)

Yes, Carl is excellent.  I'm certain that he will do a fine job of
cherry-picking patches, taking 2-5 hours a week of his time.
That's time that he's not writing code, review patches, helping
new contributors, or doing other things like his actual day job or
spending time with his family.

I don't feel at all good about dumping this task on him.  I
seriously think that most people are underestimating the amount of
work it will be.
Hopefully I'm wrong.  I still haven't spent the time to understand
the git branching/merging stuff, so maybe it will be a breeze.

- Graham


___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-16 Thread Han-Wen Nienhuys

On Fri, Apr 15, 2011 at 11:29 AM, Graham Percival
gra...@percival-music.ca wrote:

 The beauty of branching off is that nobody needs to hold off anything.
  You just continue to put stuff in master (2.15.0), and cherry-pick
 whatever needs to go to 2.14.

 I am pessimistic about this, but let's ask for volunteers.  Who
 wants to cherry-pick stuff?


 The reason that I'm pessimistic is that we racked up a huge amount
 of technical debt (i.e. bugs) during 2.11 and the early phase of
 2.13.  I'm concerned that if we don't have regular releases, the
 unstable branch is going to accumilate bugs.

 I am also too tired to fight over it right now, but I also think
 that this is the wrong model of branching.  There's basically two
 ways:
 1. keep master in a ready-to-release mode at all times; any
 serious bug gets reverted or fixed ASAP.  Unstable development
 happens on separate branches, which are merged to master when
 they're ready.

We are doing this, with the ready-to-release criterion being the
regtest passing.  I propose we stick to this schema.

My proposal is that ready-to-release still is not strict enough for
stable, so 2.14 version should be coming from something which moves
slower than the master branch.

-- 
Han-Wen Nienhuys - han...@xs4all.nl - http://www.xs4all.nl/~hanwen

___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-16 Thread Han-Wen Nienhuys

On Fri, Apr 15, 2011 at 8:23 PM, Graham Percival
gra...@percival-music.ca wrote:

 (2)   Keep working at our normal pace, but split off stable
 versions to which only bug fixes will be applied.

 (3) any big change -- say, any patch that takes longer than 6
 hours to write -- is done on a separate branch, and only merged
 with master once we have good reason to believe that it's
 finished.

This won't work.  The reason we keep finding regressions is that we
release packaged binaries of 2.13 to the general public.  These
releases widens the amount of testers (ie. users).  Any development
happening in a side branch will not get the same exposition.

I recommend using developer branches only for work that is known to
break the regtest if it is merged.

-- 
Han-Wen Nienhuys - han...@xs4all.nl - http://www.xs4all.nl/~hanwen

___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-16 Thread Carl Sorensen




On 4/16/11 12:50 PM, Han-Wen Nienhuys hanw...@gmail.com wrote:

 On Fri, Apr 15, 2011 at 11:29 AM, Graham Percival
 gra...@percival-music.ca wrote:
 
 The beauty of branching off is that nobody needs to hold off anything.
  You just continue to put stuff in master (2.15.0), and cherry-pick
 whatever needs to go to 2.14.
 
 I am pessimistic about this, but let's ask for volunteers.  Who
 wants to cherry-pick stuff?
 
 
 The reason that I'm pessimistic is that we racked up a huge amount
 of technical debt (i.e. bugs) during 2.11 and the early phase of
 2.13.  I'm concerned that if we don't have regular releases, the
 unstable branch is going to accumilate bugs.
 
 I am also too tired to fight over it right now, but I also think
 that this is the wrong model of branching.  There's basically two
 ways:
 1. keep master in a ready-to-release mode at all times; any
 serious bug gets reverted or fixed ASAP.  Unstable development
 happens on separate branches, which are merged to master when
 they're ready.
 
 We are doing this, with the ready-to-release criterion being the
 regtest passing.  I propose we stick to this schema.
 
 My proposal is that ready-to-release still is not strict enough for
 stable, so 2.14 version should be coming from something which moves
 slower than the master branch.

So does this mean we need three branches?

stable  (currently 2.12.x)
stable beta (currently stable/2.14)
development (currently 2.13.x)

stable beta and development should pass regtests.

stable beta will be ready-to-release when there are no critical bugs left.

After some  publishing of stable beta with no new critical issues, stable
beta will become stable.  At this point, stable beta will be the release
candidate for the next stable version.

Any patches that don't eliminate old syntax and don't break regtests can be
cherry-picked onto stable beta.

It seems to me that we ought to have three releases available:

stable -- demonstrated to work properly
stable beta -- all syntax from stable works, passes regtests, but not
demonstrated to be critical regression free.  Should be acceptable for
general release, but not for production work.
development -- passes regtests.  Recommended for developers and adventurous
users, but possibility exists for syntax changes.

Does this seem feasible?

Carl

 
 --
 Han-Wen Nienhuys - han...@xs4all.nl - http://www.xs4all.nl/~hanwen
 
 ___
 lilypond-devel mailing list
 lilypond-devel@gnu.org
 http://lists.gnu.org/mailman/listinfo/lilypond-devel


___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

On 4/15/11 8:29 AM, Graham Percival gra...@percival-music.ca wrote:

 On Fri, Apr 15, 2011 at 11:08:08AM -0300, Han-Wen Nienhuys wrote:
 On Fri, Apr 15, 2011 at 7:30 AM, m...@apollinemike.com
 m...@apollinemike.com wrote:
 this issue was exposed due to a (seemingly innocuous) one-line change
 by Mike.  Can I ask that you branch off the 2.14 branch so the release
 candidate does not get disturbed by other one-liners with unintended
 effects?  If you don't branch off a stable branch, 2.14 will never get
 finished.
 
 The beauty of branching off is that nobody needs to hold off anything.
  You just continue to put stuff in master (2.15.0), and cherry-pick
 whatever needs to go to 2.14.
 

 
 But I suppose we can come back to that debate later.  For now, who
 is volunteering to handle the cherry-picking?

I will cherry pick.

Thanks,

Carl


___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-15 Thread Graham Percival

On Fri, Apr 15, 2011 at 08:33:20AM -0600, Carl Sorensen wrote:
 On 4/15/11 8:29 AM, Graham Percival gra...@percival-music.ca wrote:
 
  But I suppose we can come back to that debate later.  For now, who
  is volunteering to handle the cherry-picking?
 
 I will cherry pick.

Great!  I'll make another unstable release tomorrow and merge it
into stable/2.14, and thereafter you can cherry-pick at will.
I'll make all future 2.13 releases from stable/2.14.

Cheers,
- Graham

___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching




On 4/15/11 9:23 AM, Graham Percival gra...@percival-music.ca wrote:

 On Fri, Apr 15, 2011 at 08:33:20AM -0600, Carl Sorensen wrote:
 On 4/15/11 8:29 AM, Graham Percival gra...@percival-music.ca wrote:
 
 But I suppose we can come back to that debate later.  For now, who
 is volunteering to handle the cherry-picking?
 
 I will cherry pick.
 
 Great!  I'll make another unstable release tomorrow and merge it
 into stable/2.14, and thereafter you can cherry-pick at will.
 I'll make all future 2.13 releases from stable/2.14.

Just to be sure I understand correctly, the only things I will cherry-pick
into stable/2.14 will be bugfixes for critical bugs.

Let me know if this isn't right.

Thanks,

Carl


___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

On 4/15/11 9:23 AM, Graham Percival gra...@percival-music.ca wrote:

 On Fri, Apr 15, 2011 at 08:33:20AM -0600, Carl Sorensen wrote:
 On 4/15/11 8:29 AM, Graham Percival gra...@percival-music.ca wrote:
 
 But I suppose we can come back to that debate later.  For now, who
 is volunteering to handle the cherry-picking?
 
 I will cherry pick.
 
 Great!  I'll make another unstable release tomorrow and merge it
 into stable/2.14, and thereafter you can cherry-pick at will.
 I'll make all future 2.13 releases from stable/2.14.

How shall we report issues with the unstable development version?  I think
that's the major problem I see with branching stable.  Should a bug report
be filed with a tag unstable, and a commit instead of a version?

Thanks,

Carl
 


___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-15 Thread Graham Percival

On Fri, Apr 15, 2011 at 11:19:12AM -0600, Carl Sorensen wrote:
 On 4/15/11 9:23 AM, Graham Percival gra...@percival-music.ca wrote:
 
  Great!  I'll make another unstable release tomorrow and merge it
  into stable/2.14, and thereafter you can cherry-pick at will.
  I'll make all future 2.13 releases from stable/2.14.
 
 How shall we report issues with the unstable development version?

We won't.  The Bug Squad will ignore (well, reply to say that
they're not handling them) any bug reports about unstable.  They
will only deal with reports about official GUB releases.

 I think that's the major problem I see with branching stable.
 Should a bug report be filed with a tag unstable, and a commit
 instead of a version?

If a developer wants to add an issue to the tracker, then sure, he
can include the commit revision rather than a version in the
issue.

Cheers,
- Graham

___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-15 Thread Matthias Kilian

[random notes from soneone who is *not* actively hacking on LilyPond,
so feel free to ignore it ;-)]

On Fri, Apr 15, 2011 at 03:29:52PM +0100, Graham Percival wrote:
 The reason that I'm pessimistic is that we racked up a huge amount
 of technical debt (i.e. bugs) during 2.11 and the early phase of
 2.13.  I'm concerned that if we don't have regular releases, the
 unstable branch is going to accumilate bugs.

Well, I think I've written this a few months ago: (stable) releases
*must* happen more often (at least one or two times a year) if you
want to be able to cope with stable and unstable.

 I am also too tired to fight over it right now, but I also think
 that this is the wrong model of branching.  There's basically two
 ways:
 1. keep master in a ready-to-release mode at all times; any
 serious bug gets reverted or fixed ASAP.  Unstable development
 happens on separate branches, which are merged to master when
 they're ready.
 2. toss whatever garbage you want onto master, and make a stable
 branch where a bunch of poor suckers cleans up the branch until
 it's ready for a release.

1. is the way to go. Sure, it would put some pressure on people
working on big changes, which kind of sucks (it could even slow
down implementing cool new stuff). On the other hand, it enforces
smaller steps towards new features, which is good (easier to track
down regressions, easier to *understand* what's going on).

For maintaining a stable branch and cherry picking: the more stable
and master diverge, and the bigger the changes in master are, the
more difficult merges will be.

I'm note sure wether something like an unstable branch make sense
at all, because it may encourage to do big changes diverging from
master over a period too long to be mergable without problems later.

Just my 2p.

Ciao,
Kili

___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching

2011-04-15 Thread Valentin Villenave

On Fri, Apr 15, 2011 at 8:32 PM, Matthias Kilian k...@outback.escape.de wrote:
 1. is the way to go. Sure, it would put some pressure on people
 working on big changes, which kind of sucks (it could even slow
 down implementing cool new stuff). On the other hand, it enforces
 smaller steps towards new features, which is good (easier to track
 down regressions, easier to *understand* what's going on).

I'm not entirely sure everything can be implemented using small steps
(e.g. spacing code overhaul, as has happened in the past).

As I suggested this summer, the new Fedora branching model looks
pretty interesting to me:
https://fedoraproject.org/wiki/Releases/Branched
(then again, developing a whole distribution is certainly very
different from developing a relatively specialized project such as
Lily).

Of course, it does require some resources. But (as always) I do trust
Carl to be exactly the person we need :-)

Cheers,
Valentin.

___
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: branching