PATCHES - Countdown for May 25th

2020-05-24 Thread James

Hello,

Here is the current patch countdown list. The next countdown will be on 
May 27th.


A list of all merge requests can be found here:
https://gitlab.com/lilypond/lilypond/-/merge_requests?sort=label_priority


 Push:

!65 musicxml2ly: support multiple  elements - Valentin 
Villenave

https://gitlab.com/lilypond/lilypond/-/merge_requests/65

!63 Fix #967: add a --svg command line option. - Valentin Villenave
https://gitlab.com/lilypond/lilypond/-/merge_requests/63

!48 skyline: minor performance tweaks - Han-Wen Nienhuys
https://gitlab.com/lilypond/lilypond/-/merge_requests/48


 Countdown:

!79 Rational: trade set_infinite () for infinity () - Dan Eble
https://gitlab.com/lilypond/lilypond/-/merge_requests/79

!78 Resolve "With `title = ""` SVG output differs from PDF output" - 
Valentin Villenave

https://gitlab.com/lilypond/lilypond/-/merge_requests/78

!77 Remove old .css link before installing new one - Han-Wen Nienhuys
https://gitlab.com/lilypond/lilypond/-/merge_requests/77

!75 Clarify Moment math in the C++ code - Dan Eble
https://gitlab.com/lilypond/lilypond/-/merge_requests/75

!74 Issue 5262: LM: incorrect staff-staff-spacing example (regression) - 
Kevin Barry

https://gitlab.com/lilypond/lilypond/-/merge_requests/74

!73 Handle round-filled-box with infinity in arguments - Han-Wen Nienhuys
https://gitlab.com/lilypond/lilypond/-/merge_requests/73

!70 random cleanups in the output framework - Han-Wen Nienhuys
https://gitlab.com/lilypond/lilypond/-/merge_requests/70

!68 Resolve "Keys won't display as text in saxophone diagram" - Valentin 
Villenave

https://gitlab.com/lilypond/lilypond/-/merge_requests/68

!60 Resolve "Use templates for lots of conversions to SCM and back" - 
David Kastrup

https://gitlab.com/lilypond/lilypond/-/merge_requests/60


 Review:

!82 Enable debug checks in CI - Jonas Hahnfeld
https://gitlab.com/lilypond/lilypond/-/merge_requests/82

!81 Add CFF font name cache to get_postscript_name () - Masamichi Hosoda
https://gitlab.com/lilypond/lilypond/-/merge_requests/81

!76 cleanups related to break-visibility handling - Han-Wen Nienhuys
https://gitlab.com/lilypond/lilypond/-/merge_requests/76

!72 Move skylines_from_stencil out of the Stencil class - Han-Wen Nienhuys
https://gitlab.com/lilypond/lilypond/-/merge_requests/72

!69 ps-to-png: use GhostScript's DownScaleFactor for antialiasing - 
Han-Wen Nienhuys

https://gitlab.com/lilypond/lilypond/-/merge_requests/69

!53 Allow CSS-style colors anywhere - Valentin Villenave
https://gitlab.com/lilypond/lilypond/-/merge_requests/53

!47 Stop smobifying Transform - Han-Wen Nienhuys
https://gitlab.com/lilypond/lilypond/-/merge_requests/47


 New:

!86 Use exist_ok=True instead of checking os.path.isdir() first - 
Han-Wen Nienhuys

https://gitlab.com/lilypond/lilypond/-/merge_requests/86

!84 WIP: rewrite doc build - Han-Wen Nienhuys
https://gitlab.com/lilypond/lilypond/-/merge_requests/84

!83 doc build cleanups - Han-Wen Nienhuys
https://gitlab.com/lilypond/lilypond/-/merge_requests/83

!71 Fix input/regression/skyline-grob-rotation.ly - Han-Wen Nienhuys
https://gitlab.com/lilypond/lilypond/-/merge_requests/71


 Waiting:

No patches in Waiting at this time.





***

Regards,

James



GitLab permissions

2020-05-24 Thread Jean Abou Samra

Hi,

Could someone please add me to the LilyPond group on GitLab?

Since I'm trying to work on MusicXML, it would be nice if I could add 
the corresponding label to the appropriate issues, and perhaps 
self-assign a few of them. Also, there are some old issues that look 
fixed but are still open, so I'd like to close those.


As to the level of permissions, there should likely be a policy for what 
to grant to new contributors, but as far as I am concerned , I'd only 
need Reporter (at least, that's what I think after reading 
https://docs.gitlab.com/ee/user/permissions.html).


Best,

Jean Abou Samra




Re: new procedure with GitLab CI

2020-05-24 Thread Jonas Hahnfeld
Am Sonntag, den 24.05.2020, 16:28 +0200 schrieb David Kastrup:
> Jonas Hahnfeld  writes:
> > Am Sonntag, den 24.05.2020, 13:19 +0100 schrieb James Lowe:
> > > So, and you didn't answer this specific question, if I set the label to 
> > > 'review' before the pipeline runs will make doc still run?
> > 
> > Sorry: Yes, CI pipelines will run irrespective of the labels.
> 
> One note for submissions: one can do a push using
> 
> git push -o ci.skip
> 
> and as far as I understand, no CI will happen.  "skip" does sound like
> no CI would even be required for letting the patch move on, so it will
> likely depend on the discretion of the submitter to not follow up with
> an actual merge.

You can also put "[skip ci]" in the commit message. However I think
this only applies to pipelines on commits. Pipelines for merge requests
could maybe still run, I haven't tested this.
Eventually GitLab requires that there is a passing pipeline before a
merge. If you cancel the pipeline, I verified GitLab is not fooled. I
hope it's the same for skips, but again haven't tested.

If you remember, could you try it with one of your next MRs?

Jonas


signature.asc
Description: This is a digitally signed message part


Re: new procedure with GitLab CI

2020-05-24 Thread Jonas Hahnfeld
Am Sonntag, den 24.05.2020, 14:37 +0100 schrieb James Lowe:
> > But what if a patch in countdown is rebased (without a diff) and is
> > currently running? Should it be put back to Patch::new and afterwards
> > skip the labels until Patch::countdown? That sounds very confusing to
> > me...
> 
> I guess that depends on why it was rebased - rhetorical question. But 
> shouldn't any rebase be re-tested regardless of a diff or not from 
> previous MR after all 'something' has changed else it wouldn't/shouldn't 
> be rebased right?

If you remember, we were hitting this for patches in Patch::push. We
should run the automated tests on it because this might be the commits
hitting master eventually. However your testing is applying the (same)
diff to a moving target branch (master). That doesn't provide much
value in this case.

Jonas


signature.asc
Description: This is a digitally signed message part


Re: new procedure with GitLab CI

2020-05-24 Thread David Kastrup
Jonas Hahnfeld  writes:

> Am Sonntag, den 24.05.2020, 13:19 +0100 schrieb James Lowe:
>
>> So, and you didn't answer this specific question, if I set the label to 
>> 'review' before the pipeline runs will make doc still run?
>
> Sorry: Yes, CI pipelines will run irrespective of the labels.

One note for submissions: one can do a push using

git push -o ci.skip

and as far as I understand, no CI will happen.  "skip" does sound like
no CI would even be required for letting the patch move on, so it will
likely depend on the discretion of the submitter to not follow up with
an actual merge.

-- 
David Kastrup



Re: new procedure with GitLab CI

2020-05-24 Thread James Lowe

On 24/05/2020 14:11, Jonas Hahnfeld wrote:

Am Sonntag, den 24.05.2020, 13:19 +0100 schrieb James Lowe:

On 24/05/2020 12:09, Jonas Hahnfeld wrote:

You should see it at the individual MR, can you check
https://gitlab.com/lilypond/lilypond/-/merge_requests/82
?

I am still not certain. Sorry to be a pain here.

!83 and !84 show me no pipeline has been run yet the patches are on the
list to test.

The reason is that the branch in !83 is based on a commit 6 days ago,
before we had CI. This needs a rebase before it's tested.

However, the scripts are working as intended:
---
  $ ./countdown.py --testing
No patches for testing right now!
---

That is even though the MR is in Patch::new. See also below.


!81 has just now changed from 'unable to get pipeline information' to
looking like it has passed

Yes (thanks Hosoda-san!)


So, and you didn't answer this specific question, if I set the label to
'review' before the pipeline runs will make doc still run?

Sorry: Yes, CI pipelines will run irrespective of the labels.
OK. I will try to spot any anomalies on each countdown although I guess 
if something get through to 'push' status and doc has still not been 
tested, then I guess, assuming that the patch breaks make doc, CI will 
fall over.



and if it does and it fails will the label change so it doesn't stay in the 
county
down.

I will take care of this until I've automated it.

OK.




I know we're still finding our feet here, but I need a relatively 'fire
and forget' method that doesn't have me hunting around the very busy
interface (I have a harder time compared to Allura/Rietveld following
discussions nested in amongst a lot of other noise I don't care about).

See above, the --testing flag should do.

Yes, sorry I keep forgetting this flag.



Perhaps we need another 'patch' label, i.e. Patch::new = Patch merged
but not doc tested rather than what it is at the moment. Then they can
be listed but I don't test them for make check. Then we have a
Patch::passed kind of label that I do test and then it moves to 'review'
etc.

Dan also proposed this. My argument against this is that GitLab will
test every commit, irrespective of the label (see above). In particular
it will also run the tests on rebase without a diff change where we
have decided _not_ to change the label.

OK

On failure, the mode is pretty clear: It should go back to needs_work.

Yes and no more CI is done to it and it comes off my lists.

But what if a patch in countdown is rebased (without a diff) and is
currently running? Should it be put back to Patch::new and afterwards
skip the labels until Patch::countdown? That sounds very confusing to
me...


I guess that depends on why it was rebased - rhetorical question. But 
shouldn't any rebase be re-tested regardless of a diff or not from 
previous MR after all 'something' has changed else it wouldn't/shouldn't 
be rebased right?


Or are we going to get in to a case where a patch keeps getting pushed 
back because things in front force the rebase each time (if you see what 
I mean?).


As long as we're automating the make doc now, and I don't need to do it 
anymore thereby shaving ~75% off of each patch test now, it still won't 
matter too much to me doing countdown and retesting a 
rebased-but-no-diff patch, but I guess it might annoy the dev.


We'll get there.

James





Re: new procedure with GitLab CI

2020-05-24 Thread Jonas Hahnfeld
Am Sonntag, den 24.05.2020, 13:19 +0100 schrieb James Lowe:
> On 24/05/2020 12:09, Jonas Hahnfeld wrote:
> > You should see it at the individual MR, can you check
> > https://gitlab.com/lilypond/lilypond/-/merge_requests/82
> > ?
> 
> I am still not certain. Sorry to be a pain here.
> 
> !83 and !84 show me no pipeline has been run yet the patches are on the 
> list to test.

The reason is that the branch in !83 is based on a commit 6 days ago,
before we had CI. This needs a rebase before it's tested.

However, the scripts are working as intended:
---
 $ ./countdown.py --testing
No patches for testing right now!
---

That is even though the MR is in Patch::new. See also below.

> !81 has just now changed from 'unable to get pipeline information' to 
> looking like it has passed

Yes (thanks Hosoda-san!)

> So, and you didn't answer this specific question, if I set the label to 
> 'review' before the pipeline runs will make doc still run?

Sorry: Yes, CI pipelines will run irrespective of the labels.

> and if it does and it fails will the label change so it doesn't stay in the 
> county 
> down.

I will take care of this until I've automated it.


> I know we're still finding our feet here, but I need a relatively 'fire 
> and forget' method that doesn't have me hunting around the very busy 
> interface (I have a harder time compared to Allura/Rietveld following 
> discussions nested in amongst a lot of other noise I don't care about).

See above, the --testing flag should do.

> Perhaps we need another 'patch' label, i.e. Patch::new = Patch merged 
> but not doc tested rather than what it is at the moment. Then they can 
> be listed but I don't test them for make check. Then we have a 
> Patch::passed kind of label that I do test and then it moves to 'review' 
> etc.

Dan also proposed this. My argument against this is that GitLab will
test every commit, irrespective of the label (see above). In particular
it will also run the tests on rebase without a diff change where we
have decided _not_ to change the label.
On failure, the mode is pretty clear: It should go back to needs_work.
But what if a patch in countdown is rebased (without a diff) and is
currently running? Should it be put back to Patch::new and afterwards
skip the labels until Patch::countdown? That sounds very confusing to
me...

Jonas

> 
> That way I know for certain which patches to test and everyone else can 
> see if their patch is in the pipeline or not.
> 
> James
> 
> 


signature.asc
Description: This is a digitally signed message part


Re: new procedure with GitLab CI

2020-05-24 Thread James Lowe

On 24/05/2020 12:09, Jonas Hahnfeld wrote:

Am Sonntag, den 24.05.2020, 11:41 +0100 schrieb James Lowe:

OK. Using Masamichi's MR as an example (nothing personal Hosoda-san!) I
saw that his MR !81 came up via countdown.py - I am using the latest
update of this BTW - so looking at this MR via the web I could not tell
if this had done its make doc or not.

A few mins later I saw your email/update in the ticket to Hosoda-san and
and I can also see a new commit/MR update in the same thread and that
!81 still appears in the countdown.py list.

So has this done a make doc or not? I am still unsure at this point.

I can't tell you for sure either. It could be that the fork at
https://gitlab.com/trueroad/lilypond has Pipelines enabled, but only
visible to project members. Hosoda-san, could you check this in your
project? The drop-down is at Settings > General > Visibility ... >
Pipelines. I'm guessing here, could be something else...
Because the jobs appear to have run, see the green check at the commit
view: https://gitlab.com/lilypond/lilypond/-/merge_requests/81/commits
That's also what the API tells, but we don't see the logs 


See: https://gitlab.com/lilypond/lilypond/pipelines (as of 11:25BST anyway)

It has (had) no mention of !81 and I cannot see anything in the thread
that says it has passed make doc.

I think this is because pipelines for merge requests run in the context
of the forked project, see
https://docs.gitlab.com/ee/ci/merge_request_pipelines/index.html#important-notes-about-merge-requests-from-forked-projects

FWIW the same happens for my merge requests, see the page at
https://gitlab.com/hahnjo/lilypond/pipelines


So now I am unsure what to do, because if I run my new set of tests I
*won't* be doing make doc, and if I change this MR to Patch::review and
it hasn't had a make doc, then it will go through the countdown not
tested properly right?


Q1. What cases will I get a false positive with countdown.py if a MR has
not had a make doc done - this seems, at the moment to be user-error prone.

Q2. How can I be sure (at least for the first few dozen or so patch
tests) that the MR really has done the make doc via CI? (i.e. without
the script) should I see the MR in that URL I listed above?

You should see it at the individual MR, can you check
https://gitlab.com/lilypond/lilypond/-/merge_requests/82
?


I am still not certain. Sorry to be a pain here.

!83 and !84 show me no pipeline has been run yet the patches are on the 
list to test.


!81 has just now changed from 'unable to get pipeline information' to 
looking like it has passed


So, and you didn't answer this specific question, if I set the label to 
'review' before the pipeline runs will make doc still run? and if it 
does and it fails will the label change so it doesn't stay in the county 
down.


I know we're still finding our feet here, but I need a relatively 'fire 
and forget' method that doesn't have me hunting around the very busy 
interface (I have a harder time compared to Allura/Rietveld following 
discussions nested in amongst a lot of other noise I don't care about).


Perhaps we need another 'patch' label, i.e. Patch::new = Patch merged 
but not doc tested rather than what it is at the moment. Then they can 
be listed but I don't test them for make check. Then we have a 
Patch::passed kind of label that I do test and then it moves to 'review' 
etc.


That way I know for certain which patches to test and everyone else can 
see if their patch is in the pipeline or not.


James




Re: new procedure with GitLab CI

2020-05-24 Thread Masamichi Hosoda
> I can't tell you for sure either. It could be that the fork at 
> https://gitlab.com/trueroad/lilypond has Pipelines enabled, but only
> visible to project members. Hosoda-san, could you check this in your
> project? The drop-down is at Settings > General > Visibility ... >
> Pipelines. I'm guessing here, could be something else...

I've changed the Pipelines settings
from `Only Project Members' to `Everyone With Access'.



Re: new procedure with GitLab CI

2020-05-24 Thread Jonas Hahnfeld
Am Sonntag, den 24.05.2020, 11:41 +0100 schrieb James Lowe:
> OK. Using Masamichi's MR as an example (nothing personal Hosoda-san!) I 
> saw that his MR !81 came up via countdown.py - I am using the latest 
> update of this BTW - so looking at this MR via the web I could not tell 
> if this had done its make doc or not.
> 
> A few mins later I saw your email/update in the ticket to Hosoda-san and 
> and I can also see a new commit/MR update in the same thread and that 
> !81 still appears in the countdown.py list.
> 
> So has this done a make doc or not? I am still unsure at this point.

I can't tell you for sure either. It could be that the fork at 
https://gitlab.com/trueroad/lilypond has Pipelines enabled, but only
visible to project members. Hosoda-san, could you check this in your
project? The drop-down is at Settings > General > Visibility ... >
Pipelines. I'm guessing here, could be something else...
Because the jobs appear to have run, see the green check at the commit
view: https://gitlab.com/lilypond/lilypond/-/merge_requests/81/commits
That's also what the API tells, but we don't see the logs 

> See: https://gitlab.com/lilypond/lilypond/pipelines (as of 11:25BST anyway)
> 
> It has (had) no mention of !81 and I cannot see anything in the thread 
> that says it has passed make doc.

I think this is because pipelines for merge requests run in the context
of the forked project, see
https://docs.gitlab.com/ee/ci/merge_request_pipelines/index.html#important-notes-about-merge-requests-from-forked-projects

FWIW the same happens for my merge requests, see the page at 
https://gitlab.com/hahnjo/lilypond/pipelines

> So now I am unsure what to do, because if I run my new set of tests I 
> *won't* be doing make doc, and if I change this MR to Patch::review and 
> it hasn't had a make doc, then it will go through the countdown not 
> tested properly right?
> 
> 
> Q1. What cases will I get a false positive with countdown.py if a MR has 
> not had a make doc done - this seems, at the moment to be user-error prone.
> 
> Q2. How can I be sure (at least for the first few dozen or so patch 
> tests) that the MR really has done the make doc via CI? (i.e. without 
> the script) should I see the MR in that URL I listed above?

You should see it at the individual MR, can you check
https://gitlab.com/lilypond/lilypond/-/merge_requests/82
?

> Q3. How does user know that his patch has failed the make doc (and that 
> I won't have even tested it) I assume this is because the patch will get 
> set back to 'needs_work' (and they'll get notified)?

The user gets an email that the pipeline failed. Setting to
'needs_work' is currently manual (me), but I'll make sure to also put a
comment in the thread.

Jonas


signature.asc
Description: This is a digitally signed message part


Re: helping with testing resources

2020-05-24 Thread David Kastrup
Jonas Hahnfeld  writes:

> Am Sonntag, den 24.05.2020, 10:18 +0200 schrieb Jonas Hahnfeld:
>> Am Sonntag, den 24.05.2020, 00:21 +0200 schrieb David Kastrup:
>> > I think configure options should likely point to Guile-1.8 (for now) and
>> > use --enable-checking and (to save CI minutes) --enable-gs-api .
>> > 
>> > Reasonable?
>> 
>> Not only reasonable, but already happening in part: The Docker image
>> only has guile-1.8 installed, so no flags needed. --enable-gs-api is
>> passed in the description file .gitlab-ci.yml, see
>> https://gitlab.com/lilypond/lilypond/-/blob/03b399a20e/.gitlab-ci.yml#L10
>> 
>> I missed --enable-checking so far, depends on how "expensive run-time
>> checks" translates to numbers.
>
> I see no measurable impact on my Laptop:
> https://gitlab.com/lilypond/lilypond/-/merge_requests/82

I don't think they are really all that expensive.  They do create some
additional stats.

-- 
David Kastrup



Re: helping with testing resources

2020-05-24 Thread Jonas Hahnfeld
Am Samstag, den 23.05.2020, 21:30 +0200 schrieb Valentin Villenave:
> On 5/23/20, Jonas Hahnfeld  wrote:
> > If you have spare hardware and / or want to help with CI testing, this
> > is easy to setup with GitLab. First you'll need their runner and I'll
> > defer to the excellent documentation: 
> > https://docs.gitlab.com/runner/
> > 
> > As testing uses Docker, those sections also apply to you.
> 
> Thanks for setting that up; I noticed you’re willing to run it on your
> laptop; what possibilities are there for contributing to the “swarm”
> part-time?  The doc appears to be tilted towards servers and mostly
> refers to the runner as a system service (as far as I’ve seen); I can
> run it at night (Europe time) but I can’t afford to run it while I’m
> needing the CPU cycles for my own make & make doc…
> 
> So I just wanted to know about your own setup, and the possible
> downsides of having too many intermittent runners.

I'm intending to run it as a system service that I start when I don't
need my laptop. Using a bit of extra configuration, it will shutdown
gracefully without killing the currently running job:
https://docs.gitlab.com/runner/configuration/init.html#overriding-systemd

I'm currently researching how GitLab schedules jobs. Unfortunately it
seems to be first-come-first-serve, so no priority for currently online
specific runners. But every runner, if intermittent or not, has a
chance of getting a job assigned.

Jonas


signature.asc
Description: This is a digitally signed message part


Re: lilypond grammar in the contributor guide

2020-05-24 Thread David Kastrup
Han-Wen Nienhuys  writes:

> thanks, I'll take this as an OK to drop grammar from the manual.

I am actually unhappier about seeing the mechanism for generating it
disappear rather than the grammar itself.  But for LilyPond, it really
does no longer provide a reasonable value.

-- 
David Kastrup



Re: new procedure with GitLab CI

2020-05-24 Thread James Lowe

On 24/05/2020 08:59, Jonas Hahnfeld wrote:

Hi James,

Am Samstag, den 23.05.2020, 19:08 +0100 schrieb James Lowe:

Jonas,

On 23/05/2020 18:59, Jonas Hahnfeld wrote:

Hi all,

I've now made the necessary settings, merged the changes in
https://gitlab.com/lilypond/lilypond/-/merge_requests/57, changed all
existing merge requests to target 'master', and deleted 'staging'.
I've not rebased the existing merge requests and there's no need to do
so if it's already in one of the later stages (you'll be forced on
submission). However remember that James won't get MRs with Patch::new
for manual regression testing unless it has passed automated tests.

I just noticed that I forgot to merge the change to countdown.py:
https://gitlab.com/lilypond/infrastructure/-/merge_requests/5

The renamed flag --testing now only shows patches in Patch::new that
have passed CI testing. There's no (automatic) infrastructure yet to
reset failed patch to Patch::needs_work, I'll play bot until this
becomes a reality.


So what do I continue to/stop doing manually?

Sorry if it isn't that clear for me, but is this replacing my testing or
just patchy-staging merge?

GitLab CI is replacing patchy-staging entirely. With respect to your
process, it does not (yet) run 'make check' for regression testing.
This would need to continue happening in a manual fashion as before.
However you don't need to run 'make doc' and you don't need to test
patches that failed automatic testing (see above).

Jonas


OK. Using Masamichi's MR as an example (nothing personal Hosoda-san!) I 
saw that his MR !81 came up via countdown.py - I am using the latest 
update of this BTW - so looking at this MR via the web I could not tell 
if this had done its make doc or not.


A few mins later I saw your email/update in the ticket to Hosoda-san and 
and I can also see a new commit/MR update in the same thread and that 
!81 still appears in the countdown.py list.


So has this done a make doc or not? I am still unsure at this point.

See:https://gitlab.com/lilypond/lilypond/pipelines (as of 11:25BST anyway)

It has (had) no mention of !81 and I cannot see anything in the thread 
that says it has passed make doc.


So now I am unsure what to do, because if I run my new set of tests I 
*won't* be doing make doc, and if I change this MR to Patch::review and 
it hasn't had a make doc, then it will go through the countdown not 
tested properly right?



Q1. What cases will I get a false positive with countdown.py if a MR has 
not had a make doc done - this seems, at the moment to be user-error prone.


Q2. How can I be sure (at least for the first few dozen or so patch 
tests) that the MR really has done the make doc via CI? (i.e. without 
the script) should I see the MR in that URL I listed above?


Q3. How does user know that his patch has failed the make doc (and that 
I won't have even tested it) I assume this is because the patch will get 
set back to 'needs_work' (and they'll get notified)?


Thanks as always Jonas




Re: helping with testing resources

2020-05-24 Thread Jonas Hahnfeld
Am Sonntag, den 24.05.2020, 10:18 +0200 schrieb Jonas Hahnfeld:
> Am Sonntag, den 24.05.2020, 00:21 +0200 schrieb David Kastrup:
> > I think configure options should likely point to Guile-1.8 (for now) and
> > use --enable-checking and (to save CI minutes) --enable-gs-api .
> > 
> > Reasonable?
> 
> Not only reasonable, but already happening in part: The Docker image
> only has guile-1.8 installed, so no flags needed. --enable-gs-api is
> passed in the description file .gitlab-ci.yml, see
> https://gitlab.com/lilypond/lilypond/-/blob/03b399a20e/.gitlab-ci.yml#L10
> 
> I missed --enable-checking so far, depends on how "expensive run-time
> checks" translates to numbers.

I see no measurable impact on my Laptop:
https://gitlab.com/lilypond/lilypond/-/merge_requests/82


signature.asc
Description: This is a digitally signed message part


Re: helping with testing resources

2020-05-24 Thread Jonas Hahnfeld
Am Sonntag, den 24.05.2020, 00:21 +0200 schrieb David Kastrup:
> Jonas Hahnfeld  writes:
> > (see my previous message(s) for context on GitLab CI)
> > 
> > If you have spare hardware and / or want to help with CI testing, this
> > is easy to setup with GitLab. First you'll need their runner and I'll
> > defer to the excellent documentation: 
> > https://docs.gitlab.com/runner/
> > 
> > As testing uses Docker, those sections also apply to you.
> > 
> > Once ready for registration, please send me a note off-list so that I
> > can share the secret token. Because outside people could use this to
> > install rogue runners, I don't want to post this publicly. Go through
> > the registration steps (URL: 
> > https://gitlab.com/
> > ) and choose the Docker
> > executor.
> > Before starting the runner, consider editing the configuration file
> > manually. There's extensive documentation on what to customize, for
> > LilyPond the following is interesting:
> >   environment = [ "MAKE_FLAGS=-j4 CPU_COUNT=4" ]
> > (or whatever value you like).
> > 
> > If none of our resources pick up the job, GitLab will run it on their
> > shared runners. I hope we'll stay below the maximum of 50,000 minutes
> > per month, but I have to figure out how to track this. I think our
> > runners are prioritized, but this should become clear soon.
> 
> I think configure options should likely point to Guile-1.8 (for now) and
> use --enable-checking and (to save CI minutes) --enable-gs-api .
> 
> Reasonable?

Not only reasonable, but already happening in part: The Docker image
only has guile-1.8 installed, so no flags needed. --enable-gs-api is
passed in the description file .gitlab-ci.yml, see
https://gitlab.com/lilypond/lilypond/-/blob/03b399a20e/.gitlab-ci.yml#L10

I missed --enable-checking so far, depends on how "expensive run-time
checks" translates to numbers.


signature.asc
Description: This is a digitally signed message part


Re: new procedure with GitLab CI

2020-05-24 Thread Jonas Hahnfeld
Hi James,

Am Samstag, den 23.05.2020, 19:08 +0100 schrieb James Lowe:
> Jonas,
> 
> On 23/05/2020 18:59, Jonas Hahnfeld wrote:
> > Hi all,
> > 
> > I've now made the necessary settings, merged the changes in
> > https://gitlab.com/lilypond/lilypond/-/merge_requests/57, changed all
> > existing merge requests to target 'master', and deleted 'staging'.
> > I've not rebased the existing merge requests and there's no need to do
> > so if it's already in one of the later stages (you'll be forced on
> > submission). However remember that James won't get MRs with Patch::new
> > for manual regression testing unless it has passed automated tests.

I just noticed that I forgot to merge the change to countdown.py:
https://gitlab.com/lilypond/infrastructure/-/merge_requests/5

The renamed flag --testing now only shows patches in Patch::new that
have passed CI testing. There's no (automatic) infrastructure yet to
reset failed patch to Patch::needs_work, I'll play bot until this
becomes a reality.

> So what do I continue to/stop doing manually?
> 
> Sorry if it isn't that clear for me, but is this replacing my testing or 
> just patchy-staging merge?

GitLab CI is replacing patchy-staging entirely. With respect to your
process, it does not (yet) run 'make check' for regression testing.
This would need to continue happening in a manual fashion as before.
However you don't need to run 'make doc' and you don't need to test
patches that failed automatic testing (see above).

Jonas


signature.asc
Description: This is a digitally signed message part


Re: lilypond grammar in the contributor guide

2020-05-24 Thread Han-Wen Nienhuys
thanks, I'll take this as an OK to drop grammar from the manual.

On Sun, May 24, 2020 at 12:19 AM David Kastrup  wrote:
>
> David Kastrup  writes:
>
> > Han-Wen Nienhuys  writes:
> >
> >> We have a dump of the bison grammar in the contributor guide (see
> >> http://lilypond.org/doc/v2.19/Documentation/contributor/lilypond-grammar).
> >>
> >> Is there any value in keeping this? It complicates the generation, as
> >> it is a cross-directory dependency.
> >
> > Much of LilyPond's language has been offloaded to music functions and
> > the parsing of music function arguments uses synthetic tokens and to a
> > good degree is directed not as much from the rules but the underlying
> > actions.
> >
> > As an end user tool, it
>
> by which I mean the printed Bison grammar
>
> > reflects far too little of what the input
> > language of LilyPond is about.  And it does not contain enough to work
> > with when placed, say, in the CG.  While one might want to think about
> > whether the responsible scripts could in any useful manner be
> > contributed to Bison
>
> by which I mean the GNU project "Bison"
>
> > (after all, Texinfo is the official GNU
> > documentation language), for LilyPond itself it does no longer make much
> > sense in my opinion.
> >
> > It
>
> (the printed Bison grammar)
>
> > allows interpreting the output of -ddebug-parser of a binary
>
> by which I mean a LilyPond binary
>
> > corresponding to the version of the NR.  But the complexity of
> > LilyPond's grammar is such that I would not expect somebody not working
> > with a full checkout-out source to be likely in a capacity of
> > interpreting the respective traces of Bison.
>
> Sorry for writing too much between the lines.
>
> --
> David Kastrup



-- 
Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen