Re: GUB failing

[Dan Eble]

On Jul 12, 2020, at 11:09, Jonas Hahnfeld  wrote:
> 
> Dan, ok to do the following? (I'd push directly to release/unstable…)
...
> -  auto operator * () const->decltype (from_scm (dereference_scm ()))
> +  // (Explicit 'this' in trailing decltype needed because of bug in GCC, see
> +  // https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57543)
> +  auto operator * () const->decltype (from_scm (this->dereference_scm ()))

LGTM
— 
Dan

Re: outlet v. context

[Dan Eble]

On Jul 7, 2020, at 19:20, Carl Sorensen  wrote:
> 
> The only time I would like a member function of a class to have a verb name 
> is when it causes a side effect.
> 
> When the job of a function is to return a result that can be used elsewhere, 
> to me the goal is a result, and hence, it is a noun.  flag() refers to an 
> object -- a flag.  get_flag focuses on the action, not the result.  And most 
> often, it's the result that I want -- which is a noun.

I'll flag() your message and consider it later this evening.

Should we also consider the Scheme language?
git grep 'ly:get' scm
— 
Dan

Re: outlet v. context

[Dan Eble]

On Jul 4, 2020, at 11:27, Dan Eble  wrote:
> 
> Ugh.  I'm working on the outlet/context change now (involving lots of 
> rebasing of my work in progress).  I'll probably be unwilling to do more than 
> that immediately, but regardless, I should let others weigh in on get_foo() 
> v. foo() first.

I prefer get_foo().  I tend to interpret function names as verbs and "get_" 
clarifies when they are not.  Context can help, but I'd rather have the "get_".

group()get_group()
start()get_start()
error()get_error()
rank() get_rank()
bound()get_bound()
output()   get_output()
rest() get_rest()
note() get_note()
flag() get_flag()
position() get_position()
control_points()   get_control_points()
list() get_list()
— 
Dan

Re: outlet v. context

[Dan Eble]

On Jul 4, 2020, at 11:15, David Kastrup  wrote:
> 
> While you are at it: I don't see that there is lots of sense in having
> get_context () for some classes and context () for others.  Or
> get_parent_context () for some and get_daddy_context () for others.
> Generally the get_ prefix seems a bit spurious: given that we use the
> naming convention field_ for data members, calling the read accessor
> field () seems like it should always be workable.

Ugh.  I'm working on the outlet/context change now (involving lots of rebasing 
of my work in progress).  I'll probably be unwilling to do more than that 
immediately, but regardless, I should let others weigh in on get_foo() v. foo() 
first.
— 
Dan

outlet v. context

[Dan Eble]

virtual Context *get_outlet () const;
virtual void set_context (Context *);
. . .
void init_context (Music *, Context *);
. . .
void substitute_outlet (Context *from, Context *to);

This is driving me nuts.  Is anyone attached to the term "outlet"?  If so, 
please suggest how I might modify my thinking to avoid typing "get_context" 
when I should have typed "get_outlet".
— 
Dan

Re: question about configure for reg tests

[Dan Eble]

On Jul 4, 2020, at 07:09, Jonas Hahnfeld  wrote:
> 
>> Should I be using the same options as the CI image is? (and if so, can 
>> you tell me what they are in case I have missed any)?
> 
> As said above, I don't think it makes a difference. If you want to, Dan
> recently added "autogen.sh --ci" which gets you exactly those flags 😉

Jonas' response came in while I was halfway through composing mine.  I agree 
with all of it.
— 
Dan

Re: [RFC] Use GitLab Milestones

[Dan Eble]

On Jun 29, 2020, at 07:49, Jonas Hahnfeld  wrote:
> 
> I got the list down to five corner cases, all involving multiple
> commits related to a single issue.
...
> I think it would be sufficient to have the final release that included
> all commits, but wanted to check back if everybody agrees on that.

Sounds reasonable to me.
— 
Dan

Run CI pipeline without a merge request

[Dan Eble]

Is there a way to run a CI pipeline on a branch without creating a merge 
request first?  It seems that some people are outsourcing their development 
testing to GitLab, which I don't mind per se, but when there is an MR attached 
to the branch, the pushes generate email noise.

GitLab provides instructions[1] that sound simple.  When I tried following them 
with an old branch I just rebased, I got the message

The form contains the following error:
• No stages / jobs for this pipeline.

Maybe there are environment variables unknown to me which I am supposed to set.

Regards,
— 
Dan

[1] https://docs.gitlab.com/ee/ci/pipelines/#run-a-pipeline-manually

Re: [RFC] Use GitLab Milestones

[Dan Eble]

On Jun 23, 2020, at 04:40, Jonas Hahnfeld  wrote:
>> 
> Pretty much that: You can only have one label from the same scope, and
> assigning a second automatically removes the old (cf. Patch::*). I
> actually agree that multiple Type's might be useful. If others are in
> favor as well, we can just rename the labels.

My default position is to avoid restrictions when there isn't a good reason for 
them.  There are some types in the current set that I can't imagine using 
together – for example (Enhancement|Maintainability) with 
(Crash|Defect|Regression) – but unless that endangers the efficiency of 
someone's workflow, I don't think we should spend time compartmentalizing them.

Patch::* obviously need to remain scoped because they name states in a state 
machine.
— 
Dan

Re: LilyPond | Pipeline #156017999 has failed for dev/hanwen/doc-build | 47b263b1 in !84

[Dan Eble]

On Jun 18, 2020, at 03:05, Jonas Hahnfeld  wrote:
> 
> Am Mittwoch, den 17.06.2020, 18:11 -0400 schrieb Dan Eble:
>> On Jun 14, 2020, at 07:04, Han-Wen Nienhuys  wrote:
>>> Hey Dan,
>>> 
>>> does your CI runner have limitations on sending data back to Gitlab by any 
>>> chance? It looks like this hung while uploading the failure logs
...
>> It would be nice to advance this MR since it seems to take about 25% off the 
>> time required for the doc stage (on my runner).
> 
> Where do you get that 25% from, with no version of the MR passing
> automated CI testing?  Sure, the jobs currently finish earlier than
> other doc builds - simply because they fail along the way.

I misunderstood what was happening.  I thought the main part of the build was 
OK and the runner was failing while uploading results.  Thanks for the 
correction.
— 
Dan

Re: -Werror

[Dan Eble]

On Jun 17, 2020, at 17:18, Han-Wen Nienhuys  wrote:
> 
> My proposal is to use -Werror only in CI, so we can keep code free of 
> warnings.
> 

I'd go along with that if I could have a single switch or environment variable 
to configure my local build with all the options that will be enabled in CI, 
without having to go look up what they are or keep up to date when they change.
— 
Dan

Re: LilyPond | Pipeline #156017999 has failed for dev/hanwen/doc-build | 47b263b1 in !84

[Dan Eble]

On Jun 14, 2020, at 07:04, Han-Wen Nienhuys  wrote:
> 
> Hey Dan,
> 
> does your CI runner have limitations on sending data back to Gitlab by any 
> chance? It looks like this hung while uploading the failure logs 
> 
> -- Forwarded message -
> From: GitLab mailto:git...@mg.gitlab.com>>
> Date: Sun, Jun 14, 2020 at 12:49 PM
> Subject: LilyPond | Pipeline #156017999 has failed for dev/hanwen/doc-build | 
> 47b263b1 in !84

Would you like me to take my runner offline for a bit and trigger a rebuild of 
this pipeline to see whether it will succeed on another runner?  It would be 
nice to advance this MR since it seems to take about 25% off the time required 
for the doc stage (on my runner).
— 
Dan

-Werror

[Dan Eble]

Han-Wen proposed building with -Werror in a merge request.  What do you think?

I like -Werror, but I've only ever used it where there were very few (or one) 
supported build environments, all of which were covered in CI.  A dimension of 
the CI coverage was optimization level, which can change what the compiler 
discovers.

We don't have that here, so we might regret turning warnings into errors all at 
once.  How about we turn particular warnings into errors, starting with 
narrowing conversions and printf formatting, and wait to see if anyone reports 
problems?  (Am I being too conservative?)
— 
Dan

Re: LilyPond | Pipeline #156017999 has failed for dev/hanwen/doc-build | 47b263b1 in !84

[Dan Eble]

I haven't put any limits on it.  (I don't know if that's possible.)  There have 
been a couple of times in the last few months when something happened with my 
cable connection and rebooting my modem fixed it, but apart from your build, 
everything looks fine today.
— 
Dan

> On Jun 14, 2020, at 07:04, Han-Wen Nienhuys  wrote:
> 
> Hey Dan,
> 
> does your CI runner have limitations on sending data back to Gitlab by any 
> chance? It looks like this hung while uploading the failure logs 
> 
> -- Forwarded message -
> From: GitLab mailto:git...@mg.gitlab.com>>
> Date: Sun, Jun 14, 2020 at 12:49 PM
> Subject: LilyPond | Pipeline #156017999 has failed for dev/hanwen/doc-build | 
> 47b263b1 in !84
> To: 
> 
> 
>  
>   Your pipeline has failed. 
>  
> Project   LilyPond  / LilyPond 
> 
> Branch
>   dev/hanwen/doc-build  
>  
> Commit
>   47b263b1  
> in
>  !84   
> Move all doc building logic to Documentation/GN...
> Commit Author 
>   Han-Wen Nienhuys   
>  
> Pipeline #156017999  
> triggered by  
>Han-Wen Nienhuys   
> had 1 failed build.
> Logs may contain sensitive data. Please consider before forwarding this email.
>   documentation
> doc:ubuntu-16.04 
>  
>  dist/fonts/type1/public/amsfonts/cm/cmb10.pfb> nts/type1/public/amsfonts/cm/cmbx12.pfb> pe1/public/amsfonts/cm/cmcsc10.pfb> ublic/amsfonts/cm/cmr10.pfb> msfonts/cm/cmr7.pfb> cm/cmr9.pfb> 0.pfb>  
> b> r/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmti10.pfb> e/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmtt10.pfb> ive/texmf-dist/fonts/type1/public/amsfonts/cm/cmtt12.pfb> xmf-dist/fonts/type1/public/amsfonts/cm/cmtt9.pfb> t/fonts/type1/public/amsfonts/latxfont/lcircle1.pfb>
> Output written on learning.pdf (187 pages, 5139645 bytes).
> Transcript written on learning.log.
> /usr/bin/texi2dvi: pdfetex exited with bad status, quitting.
> make[2]: *** [out-www/cs/learning.pdf] Error 1
> make[2]: *** Deleting file 'out-www/cs/learning.pdf'
> /builds/lilypond/lilypond/build/.././Documentation/GNUmakefile:145: recipe 
> for target 'out-www/cs/learning.pdf' failed
> make[2]: *** Waiting for unfinished jobs
> /builds/lilypond/lilypond/build/../stepmake/stepmake/generic-targets.make:161:
>  recipe for target 'WWW-1' failed
> make[1]: *** [WWW-1] Error 2
> /builds/lilypond/lilypond/build/../stepmake/stepmake/generic-targets.make:180:
>  recipe for target 'doc-stage-1' failed
> make: *** [doc-stage-1] Error 2
> Running after_script
> Uploading artifacts for failed job
> Uploading artifacts...
> ERROR: Job failed: execution took longer than 1h0m0s seconds
> 
> You're receiving this email because of your account on gitlab.com 
> . Manage all notifications 
>  · Help 
> 
> 
> -- 
> Han-Wen Nienhuys - hanw...@gmail.com  - 
> http://www.xs4all.nl/~hanwen 

Re: Parallelizing the CI doc build

[Dan Eble]

On Jun 6, 2020, at 16:03, Jonas Hahnfeld  wrote:
> Am Samstag, den 06.06.2020, 15:47 -0400 schrieb Dan Eble:
>> On Jun 6, 2020, at 15:26, Dan Eble  wrote:
>>> split the doc stage of the build into multiple stages that can be run in 
>>> parallel on different runners
>> 
>> For example, assigning each runner one or more languages to build.
> 
> In my understanding, invocations of lilypond-book for different
> languages share compiled snippets. By splitting into multiple jobs,
> you'd compile them repeatedly, probably at least offsetting the gain
> from parallelism.

I tried it.

60% building one language
40% building the remaining ten languages

So running with two equally capable runners would add 60% to the total resource 
use and save 20% of real time.  That's pretty unappealing on its own, and we 
don't actually have matched runners.  It still finishes soonest when the 
fastest runner takes the whole pipeline.
— 
Dan

Re: Parallelizing the CI doc build

[Dan Eble]

On Jun 6, 2020, at 15:26, Dan Eble  wrote:
> 
> split the doc stage of the build into multiple stages that can be run in 
> parallel on different runners

For example, assigning each runner one or more languages to build.
— 
Dan

Parallelizing the CI doc build

[Dan Eble]

Would we still be validating LilyPond properly if we split the doc stage of the 
build into multiple stages that can be run in parallel on different runners, or 
is it fundamentally important that we test "make doc"?
— 
Dan

Re: helping with testing resources

[Dan Eble]

On May 24, 2020, at 06:51, Jonas Hahnfeld  wrote:
> 
> I'm currently researching how GitLab schedules jobs. Unfortunately it
> seems to be first-come-first-serve, so no priority for currently online
> specific runners. But every runner, if intermittent or not, has a
> chance of getting a job assigned.

This has been disappointing.  I've seen a slower runner taking a job while a 
faster one sits idle.  We could try increasing the check_interval option[1] on 
the slower runners so that the faster ones are more likely to get jobs.
—
Dan

[1] https://docs.gitlab.com/runner/configuration/advanced-configuration.html

Re: new procedure with GitLab CI

[Dan Eble]

On May 27, 2020, at 21:03, David Kastrup  wrote:
> 
> Dan Eble  writes:
>> I wonder how the rest of you feel about having another developer click
>> the buttons to rebase and merge your MRs?
> 
> If you refer to me doing that on Han-Wen's merge request, he actually

I had no idea.  I just noticed that there are still some patches on "Push" and 
no pipelines running.  At such a time, I could start one, but I wouldn't want 
to be a pest.

> I don't think it would be a good idea to start a pipeline on somebody
> else's merge request without having an indication that they were going
> to follow through with "Patch_push": Patch_push indicates no objections
> from foreign review, but the responsibility for starting the final
> pipeline/merge should really lie with the original submitter.

What if we (by convention) reserve Approval as a way for the submitter to 
indicate this, so that anyone who sees an Approved MR with the Patch::push 
label is free to trigger the merge?
— 
Dan

Re: new procedure with GitLab CI

[Dan Eble]

On May 27, 2020, at 07:16, David Kastrup  wrote:
> 
> Now that we have the first "please get in line" merge that isn't
> actually to any degree unusual, I get the suspicion that my previous
> alternative proposal of pushing to a CI-less staging branch that then
> uses CI to get to master will eventually become a reality.

I wasn't much bothered by this round of merging.

I wonder how the rest of you feel about having another developer click the 
buttons to rebase and merge your MRs?  Maybe we could adopt a convention that 
if you would object to that, you configure your MR so that you are the only one 
who is allowed to merge; and if you don't do that, then anyone who comes by at 
a time no pipelines are running would be free to start one.
— 
Dan

Re: new procedure with GitLab CI

[Dan Eble]

On May 23, 2020, at 13:59, Jonas Hahnfeld  wrote:
> 
> Hi all,
> 
> I've now made the necessary settings, merged the changes in 
> https://gitlab.com/lilypond/lilypond/-/merge_requests/57, changed all
> existing merge requests to target 'master', and deleted 'staging'.

This helped me update the upstream for all my local branches.  (YMMV)

git branch -vv | awk '/\[origin\/staging:/ { print $1 }' | xargs -n1 git branch 
--set-upstream-to origin/master

— 
Dan

Re: LilyPond GSoC logistics

[Dan Eble]

> On May 20, 2020, at 03:17, Valentin Villenave  wrote:
> 
> On 5/20/20, Arthur Reutenauer  wrote:
>>  “How often would you like me to make contact?” :-)
>>  https://www.lexico.com/definition/touch_base
> 
> I urge you not to look up that phrase on Urban Dictionary… ;-)

s/that phrase/anything

— 
Dan

Re: [RFC] Enabling GitLab CI

[Dan Eble]

On May 17, 2020, at 15:27, Jonas Hahnfeld  wrote:
> 
> before merging. As we only allow fast-forward merges, this means each
> MR has received testing in the form it hits master. This would
> effectively replace the current setup of pushing to staging.

I'm for this.
—
Dan

Re: [RFC] Enabling GitLab CI

[Dan Eble]

On May 17, 2020, at 15:27, Jonas Hahnfeld  wrote:
> if we want to get faster builds, we can always add our own machines.
> That is a matter of installing Docker and the runner (packages provided
> by GitLab). Configuration is as simple as running one command and
> pasting the URL as well as a token.

How much routine network traffic will the runner generate?  Obviously, it will 
need to pull the Docker image when it is updated, and that will happen 
occasionally, but I'm more curious about the building & testing itself.  Will 
there be large logs or build artifacts sent back to GitLab or otherwise served 
to people browsing the project?

I might be willing to plug in a cheap, headless computer to crank through 
patches night and day, but probably not if it will upload GBs of results.
— 
Dan

Re: Obstacles for using GitLab CI

[Dan Eble]

On May 13, 2020, at 17:13, David Kastrup  wrote:
>>> At the current point of time, our pipeline does not tend to be all that
>>> full I think.  We are not at Linux kernel levels of participation...
>> 
>> No, you're probably right. It's only a bit more bothersome if you have
>> multiple changes to submit from the same countdown.
> 
> The real problem starts when people don't manage to get their patch in
> because the pipeline is always busy.  But if we are there, the free CI
> plan will most certainly not do the trick anymore.

One thing is clear: we need to make sure we don't abuse James' generosity, so 
I'm willing to try whatever system you think will be the fairest all around, 
even if it might be a bit "bothersome" as an individual submitter from time to 
time.
— 
Dan

GitLab review email

[Dan Eble]

Rietveld used to send email to lilypond-devel for all comments by default, 
though one could disable that when commenting.  Are we satisfied with GitLab's 
not doing that?
— 
Dan

Re: GitLab repo breaks make website?

[Dan Eble]

On May 11, 2020, at 19:42, Caio Barros  wrote:
> 
> Then I noticed that on the "old" repo there was GNUmakefile.in
> and GNUmakefile. The second one is not present on the "new" repo, so i did
...
> 
> Am I missing something?

Run autogen.sh and/or configure?
— 
Dan

Re: migration to GitLab

[Dan Eble]

On May 10, 2020, at 18:51, Joram Noeck  wrote:
> 
> as far as I understand (from experience with other projects), you can
> clone anonymously (and fetch without authentication then).
> If you want to be able to push, you need to authenticate and then you
> also need to authenticate for fetch or pull.
> A way around that is to use the ssh urls and use ssh keys. In
> combination with the ssh-agent, you can then enter your ssh key password
> only once and push and pull afterwards.

Thanks for the reply.  git configuration might be my problem, but I doubt it.  
I've been using https to fetch and ssh to push to gnu.org repo for months 
(using the pushInsteadOf option).  Now I'm trying to fetch from the new 
lilypond repo with https and it prompts me for credentials.  I wonder if the 
new repo is configured not to allow anonymous fetching.

$ git remote -v
. . .
origin  https://gitlab.com/lilypond/lilypond.git (fetch)
origin  g...@gitlab.com:lilypond/lilypond.git (push)
$ git fetch origin
Username for 'https://gitlab.com': ^C
— 
Dan

Re: migration to GitLab

[Dan Eble]

On May 10, 2020, at 15:36, Jonas Hahnfeld  wrote:
> 
> All right, I populated the repository

When I try to fetch from gitlab with https, it prompts me to authenticate.  
It's inconvenient.  Is it expected?  
— 
Dan

invoke-mf2pt1.sh: 7: python: not found

[Dan Eble]

I'm trying to upgrade the LilyDev Dockerfile to use Ubuntu 20.04 and I get this 
error when I try to build.  It's coming from here:

# realpath doesn't exist on OSX
realpath() {
  python -c "import os; print(os.path.realpath('$1'))"
}

Is there a Right Way to resolve this?  Should this call python3?  Should I 
change the Docker image, say by creating a symlink?  TIA.
— 
Dan

Re: Another patch

[Dan Eble]

On May 10, 2020, at 08:11, lilyp...@de-wolff.org wrote:
> 
> I did replace all implicit casts to an int by a inline function, checking if
> the value is valid, and then casting  to int.
> 
> Together with my previous patch now all but one compiler warnings are
> solved.

Jaap,

I love the fact that you are taking the initiative to work on these.  I haven't 
reviewed them thoroughly, but my first comment is a note of caution.  A couple 
of warnings, like the printf one, are just simple oversights; however, many of 
the warnings that remain in LilyPond are the tip of an iceberg.

There was a discussion on the developer list about simply casting to silence 
warnings, but more developers were in favor of leaving the warnings in until 
someone is motivated to fix them properly.

I commend you that your work is not just simply casting, but in at least some 
cases, it still doesn't appear to be going to the depth these issues deserve.  
For example, the number of tracks in a MIDI file is a 16-bit number[1], so 
clipping to INT_MAX isn't quite right.  (I've got an old branch where I've 
fixed the header generation to use uint16_t, but it's still missing code to 
warn properly when the number of tracks needs to be clipped.  I'll try to 
rebase that to save you the work.)

I encourage you to sort the simple and obvious fixes (like the printf one) into 
one commit, and the other cases into their own commits.  It will limit the 
consequences if we later have to revert a change because of a subtle bug that 
escaped regression testing.

Regards,
— 
Dan

[1] 
http://www.music.mcgill.ca/~ich/classes/mumt306/StandardMIDIfileformat.html#BM2_1

Re: migrating to GitLab

[Dan Eble]

On May 9, 2020, at 15:13, David Kastrup  wrote:
> Carl Sorensen  writes:
> 
>> ->CS At any rate, I think that we should have appropriate CG
>> instructions at the time we make the switch.  They don't have to be
>> perfect (the CG has a much lower editing bar than the NR), but they
>> need to be in place, IMO.
> 
> It's sort of a hen and egg problem: if we want to have all that before,
> it increases the workload for those preparing the migration and they
> have to guess.
> 
> I totally agree that the CG should reflect the new workflows.  But at
> the time we do the switch, those new workflows are still in flux.

I understand Carl's perspective, but I'm on the side of jumping in.  I don't 
expect that we'll be inundated with newbie contributors between now and the 
time we figure out what to put in the CG.

Maybe we could put in a minimal note referring to the project on GitLab and 
explaining that a more thorough CG revision is pending.
— 
Dan

Re: Fix #5964: MM Rests shouldn’t segfault when there’s no StaffSymbol. (issue 576090043 by v.villen...@gmail.com)

[Dan Eble]

On May 7, 2020, at 11:12, v.villen...@gmail.com wrote:
> No, I actually want oneline to be true . I did consider changing the
> variable name to no_multiple_lines, because that’s what it’s really
> about. I also considered adding a comment such as:
> // If there is no StaffSymbol, print MMrests on one (invisible) line.

That comment would be sufficient.

> https://codereview.appspot.com/576090043/
— 
Dan

Re: Almost, but not quite: C++ STL in LilyPond

[Dan Eble]

On May 5, 2020, at 18:03, David Kastrup  wrote:
> 
> If everything can be represented/mapped in similar manner, the Scheme
> garbage collector does not need any interaction with user-written code
> for doing its job.

I would not like to see the STL go, but if the getting rid of all the explicit 
GC code is the other side of it, it might be worth it.
— 
Dan

Re: Almost, but not quite: C++ STL in LilyPond

[Dan Eble]

On May 5, 2020, at 17:48, David Kastrup  wrote:
> 
> One thing here is that being under control of a wrapper, one can use
> unchecked unsmobs.

This is a good idea.  I considered this briefly, but I wanted to focus first on 
making it natural to deal robustly with potentially improper or heterogeneous 
lists coming from a user, and leave optimizing more restrictive cases for later.
— 
Dan

Re: Almost, but not quite: C++ STL in LilyPond

[Dan Eble]

On May 5, 2020, at 13:37, David Kastrup  wrote:
> 
> What I have ready-to-use is something that stores like an SCM value but
> behaves like a Smob pointer with regard to -> and * usage.

Oh.  I believe I have some of that too.  Excerpt:

// specialization for pointers
template 
class ly_scm_t
{
private:
  using traits = ly_scm_traits;

private:
  SCM scm_ = SCM_UNDEFINED;

public:
  ly_scm_t () = default;
  ly_scm_t (const ly_scm_t &) = default;
  ly_scm_t &operator = (const ly_scm_t &) = default;
  ~ly_scm_t () = default;

  explicit ly_scm_t (T *v) : scm_ (traits::to_scm (v)) {}

  explicit operator bool () const { return traits::has_value (scm_); }
  operator T *() const { return traits::to_value (scm_); }
  T *operator ->() { return operator T * (); }
  // TODO: operator *
};
— 
Dan

Re: Almost, but not quite: C++ STL in LilyPond

[Dan Eble]

On May 5, 2020, at 11:09, David Kastrup  wrote:
> I'd like to come up with an allocator/container programming model
> comparatively similar to the STL one so that one could mostly steal the
> implementations and "just" add the required Scheme awareness while
> removing of destruction/deallocation tracking.

It might be difficult for me to devote the time to this discussion that it 
deserves, but I want to say that I've been working on C++ wrappers that allow 
walking SCM lists (of immediate values or Smobs) with standard iterator syntax 
and semantics.  I suppose that's tangential to what you are talking about, but 
it would be disappointing if we did significant overlapping work.

Regards,
— 
Dan

Re: [trial] current status

[Dan Eble]

On May 3, 2020, at 07:00, Jonas Hahnfeld  wrote:
> 
> As the last two points involve some (short) scripts and I already have
> two more for the migration, I'd like to put them into a new repository;
> maybe https://gitlab.com/lilypond/infrastructure ? (I think they're
> sufficiently orthogonal to LilyPond itself that they deserve their own
> place to live.)

Sounds good.  A reason to put them with LilyPond would be if you wanted them to 
be branched and versioned with LilyPond, but I don't see that as important.
—
Dan

Sequential_iterator start-up issue

[Dan Eble]

RFC on Case 3 below.  Thanks.
— 
Dan

\version "2.20.0"

upper = f''1
lower = e'1

%% CASE 1: When the music in each simultaneous element begins with a
%% note, the upper staff is created first.
\score {
  <<
\upper
\lower
  >>
}

%% CASE 2: When there are sets preceding the notes in both branches,
%% the upper staff is still created first.
\score {
  <<
{
  \set Timing.vocalName = "2a"
  \set Timing.vocalName = "2b"
  \upper
}
{
  \set Timing.vocalName = "2"
  \lower
}
  >>
}

%% CASE 3: When only the first branch begins with a set, the staves
%% switch places.  Surprise!
%%
%% To fix this inconsistency, I suggest that as
%% Sequential_iterator::process() currently handles all the music up
%% to a given moment, Sequential_iterator::construct_children() should
%% handle all music at the initial moment--not just the first element.
%%
\score {
  <<
{
  \set Timing.vocalName = "3"
  \upper
}
\lower
  >>
}

Re: poll: switching our development platform

[Dan Eble]

On Apr 23, 2020, at 16:24, Jonas Hahnfeld  wrote:
>> I'd prefer file-names and thumbnails, unfolded by clicking.
>> 
>> Is this behaviour adjustable?
> 
> No, I'm merely linking the uploaded file (GitLab decides to display the
> image inline) and AFAICS there are no options to influence this
> behavior.

The HTML for an image link looks like this.



Notice alt="new-instrument-name-x-offset.png".

Can we add project-specific CSS rules?  Maybe we could derive something 
acceptable from this:

a.gfm.no-attachment-icon img::after {
content: attr(alt);
}
—
Dan

Re: \mask and \unmaskWithTag

[Dan Eble]

On Apr 23, 2020, at 19:25, Dan Eble  wrote:
> 
>(let ((masked-music (ly:music-property m 'void)))
> (if (and (ly:music? masked-music) (tags-keep-predicate tags))
   ^^
Oops.  This is bogus and the output of the score is correct in spite of it.  
What I intended was to check whether the masked music has the right tags.
— 
Dan

\mask and \unmaskWithTag

[Dan Eble]

Is this \mask function worth developing as a feature of LilyPond?  Is there 
already something that can achieve this that I am overlooking?

The use case will probably be clearer if you start by running this example 
through LilyPond than if you start by reading the code.

Regards,
—
Dan

\version "2.20.0"

mask =
#(define-music-function (tags music) (symbol-list-or-symbol? ly:music?)
  (_i "Tag @var{music} as if with @code{\\tag} @var{tags} @var{music},
but hide it from the interpreter until @code{\\unmaskWithTag} is used.")
  #{ \tag #tags #(make-music 'SequentialMusic 'void music) #} )

%% TODO (?) retain the tags on the unmasked music
unmaskWithTag =
#(define-music-function (tags music) (symbol? ly:music?)
  (_i "Expose any music within @var{music} that was hidden with
@code{\\mask}.  The @var{tags} are used as in @code{\\keepWithTags}.")
  (music-map
   (lambda (m)
(let ((masked-music (ly:music-property m 'void)))
 (if (and (ly:music? masked-music) (tags-keep-predicate tags))
  masked-music
  m)))
   music))

%%--

onceUnfolded =
#(define-music-function (m) (ly:music?)
  #{ \mask #'unfolded #m #})

untilUnfolded =
#(define-music-function (m) (ly:music?)
  #{ \tag #'folded #m #})

newUnfoldRepeats =
#(define-music-function (m) (ly:music?)
  #{ \unfoldRepeats \removeWithTag #'folded \unmaskWithTag #'unfolded #m #})

%%--

song = <<
  \new Staff {
\clef "treble"

%% This part is tagged to respond to \unfoldRepeats even though it
%% does not have the outer repeat that other parts do.
\onceUnfolded R1*5
\repeat volta 2 { % bar form
  r4 a'\untilUnfolded ^"tacet until D.C." 2 | 2 2 |
}
b'1 |
  }

  \new Staff {
\clef "bass"
\repeat volta 2 { % strophic
  \repeat volta 2 { % bar form
c1~ | 1
  }
  g,1 |
  \untilUnfolded \mark "D.C."
  \bar "||"
}
\bar "|."
  }
>>

\score {
  \song
  \layout { }
}

\score {
  \newUnfoldRepeats \song
  \layout { }
  \midi { }
}

Re: Issue 5919 Make InstrumentName.X-offset more robust (issue 553930044 by thomasmorle...@gmail.com)

[Dan Eble]

On Apr 18, 2020, at 14:38, nine.fierce.ball...@gmail.com wrote:
> 
> On 2020/04/18 18:14:33, thomasmorley651 wrote:
> 
> I ran make check and didn't see any differences.  Without test coverage,
> someone could easily break this again without knowing it.

Oops.  *I* wrote that; I just neglected to delete the automatic attribution 
line.
— 
Dan

Re: performance header content

[Dan Eble]

On Apr 15, 2020, at 19:41, Dan Eble  wrote:
> 
> What I have seen while testing this is that the header returned by 
> (ly:performance-header performance) contains only the items from the score 
> headers.

https://sourceforge.net/p/testlilyissues/issues/5917/
—
Dan

Re: Copy alist instead of deep copy on Grob clone (issue 561640045 by hanw...@gmail.com)

[Dan Eble]

On Apr 16, 2020, at 01:16, Werner LEMBERG  wrote:
> 
> Have you ever tried valgrind's `callgrind` tool for profiling (and
> using `kcachegrind` for displaying the results)?  While very slow it
> would avoid temperature issues and the like – no need to call it
> multiple times to get reliable values.

In case it helps, here are notes on the last time I used callgrind (long ago):

# Callgrind

Build lilypond with optimization enabled.

```cpp
#include “valgrind/callgrind.h”

Paper_book *
Book::process (Output_def *default_paper,
   Output_def *default_layout)
{

 CALLGRIND_TOGGLE_COLLECT;
 Paper_book *paper_book = process (default_paper,
   default_layout, 0);
 CALLGRIND_TOGGLE_COLLECT;
 return paper_book;
}
```

```shell
valgrind --tool=callgrind --collect-atstart=no out/bin/lilypond 
/media/sf_lilypond_git/input/regression/rest-dot-position.ly
callgrind_annotate callgrind.out.PID > whatever.txt
```

performance header content

[Dan Eble]

You're not going to be able to run this because you don't have the 
after-writing callback that it requires, but I hope that it is unnecessary to 
run this to answer my question.

What I have seen while testing this is that the header returned by 
(ly:performance-header performance) contains only the items from the score 
headers.  For example, it does not contain winkle.  Is that expected, and if 
so, is there a path I can follow from the performance to find winkle?

Thanks.
— 
Dan

\version "2.21.1"

\header {
  texidoc = "The @code{after-writing} callback ..."
  winkle = "winkle"
  punctA = ", "
}

\midi {
  after-writing = #(lambda (performance filename)
(let* ((header (ly:performance-header performance))
   (pavar (module-variable header 'punctA))
   (pbvar (module-variable header 'punctB))
   (wvar (module-variable header 'winkle))
   (pa (variable-ref pavar))
   (pb (variable-ref pbvar))
   (w (variable-ref wvar)))
 (ly:warning "~a: ~a~a~a~a" filename w pa w pb)))
}

#(ly:set-option 'warning-as-error #t)
#(ly:expect-warning "midi-after-writing.midi: winkle-winkle?")
#(ly:expect-warning "midi-after-writing-1.midi: winkle, winkle!")

\score {
  \header { punctA = "-" punctB = "?" }
  \midi { }
  { c''1 }
}

\score {
  \header { punctB = "!" }
  \midi { }
  { d''1 }
}

Re: poll: switching our development platform

[Dan Eble]

On Apr 15, 2020, at 15:44, David Kastrup  wrote:
> 
> Jonas Hahnfeld  writes:
> 
>> To conclude, I believe we should choose one of Gerrit and GitLab and
>> have a trial to see if the processes can be carried over. (If not, we
>> can still give the other platform a try.)
> 
> IIRC, Gerrit was not really tracking issues and thus would be a fill-in
> rather just for the Rietveld part of our current setup rather than also
> covering the SourceForge angle.  Do I remember wrong here?

You remember rightly.  From [1]:

> Disadvantages
...
> Gerrit lacks an issue tracker or integrated CI solution
> 
> Gerrit lacks "community" features, such as Wiki, @mentions, etc.
> 
> Gerrit as hosted by GerritForge requires a GitHub account. If this is
> an issue, I volunteer to manage a self-hosted Gerrit instance.
...
>> 1: https://lists.gnu.org/archive/html/lilypond-devel/2020-02/msg00408.html


MHO: Gerrit would be change without progress.  I'd rather continue using a set 
of tools that is flawed but familiar.

I don't object to switching to GitLab if it meets others' requirements.  (And I 
hope I don't end up being the tie-breaker that disappoints half the 
contributors.)
— 
Dan

Re: Abstract Grob property storage into Mutable_properties. (issue 561640043 by hanw...@gmail.com)

[Dan Eble]

On Apr 13, 2020, at 16:31, hanw...@gmail.com wrote:
> 
>> Does const serve a purpose here?  The iterator doesn't carry through
> with any
>> kind of enforcement.  The same question applies to try_retrieve() and
>> to_alist().
> 
> It allows one to iterate over properties in a const method.
> 
> What kind of enforcement are you looking for?

There's a tension between C++ and Scheme structures that I struggle with.  In 
C++, a const method usually doesn't change the externally observable state of 
its object, and *usually* avoids returning a non-const reference to a component 
part, so as not to open a way for the caller to mutate the object.

If this const method is not enforcing these things because SCM data structures 
have no constness, then what is the value in declaring it const?  You've said 
that you're declaring it const so that it can be called from a const method. 
Granting that the caller may be const for a good reason, the suggestion that a 
Mutable_properties object will not be changed by a caller that calls 
properties.to_alist() is misleading.

I think it would be better to declare methods as const where it agrees with 
reality, e.g. contains().  That way you can meaningfully pass a const Whatever& 
around and trust that the recipient can not change its state.  I would do this 
even if the consequence is that passing a const Whatever& is not useful at 
present.

In the case that concerns you--iterating the SCM list in a const method of a 
class that owns a Mutable_properties object—I think I would declare the 
Mutable_properties object as mutable.  That way, all the owner's methods can do 
as they please with the owned Mutable_properties, consistent with its nature as 
primarily a SCM structure.

In short, that way "const" means const, and "mutable" means mutable.
— 
Dan

Re: 2.21.0 released

[Dan Eble]

On Apr 9, 2020, at 14:42, Valentin Villenave  wrote:
> 
> Here’s to hoping we can get back on track with our once bi-monthly
> development release cycle!

Oh, you don't like software development on Biblical timescales?

At the end of every seven years thou shalt make a release
(Deuteronomy 15:1, KJV).
— 
Dan

Video generation

[Dan Eble]

Knut,

Regarding [1], I thought I would check whether you have anything more recent or 
less intrusive before I try it.

Thanks,
-- 
Dan

[1] "Video generation on linux systems: Note and rests change color"
https://lists.gnu.org/archive/html/lilypond-user/2017-07/msg00234.html

Re: Remove references to test-output-distance.ly. (issue 549780044 by hanw...@gmail.com)

[Dan Eble]

On Mar 30, 2020, at 17:34, Han-Wen Nienhuys  wrote:
> 
> On Sun, Mar 29, 2020 at 11:45 PM Dan Eble  wrote:
>> 
>> On Mar 29, 2020, at 17:39, Han-Wen Nienhuys  wrote:
>>>> test-output-distance was removed on the grounds that the self-test
>>>> serves the same purpose, but I don't see how it does.
>>> 
>>> Could you elaborate? What failure scenario are you worried about?
>> 
>> My question is, how does the self-test "verify that the regression tests 
>> have, in fact, run"?  I don't see how it could do more than verify that 
>> itself has run.
> 
> How does test-output-distance verify that the regression tests have run?
> 
> The verification doesn't come from the test file. It comes from the
> fact that someone is looking at the test output.

(Rhetorically) How much verification comes from looking at the output since you 
removed the test?

> The self-test gives us assurance that the test result index.html file
> is making sense.

Yes, and that does not overlap with the purpose of test-output-distance.

> You can verify that the tests were run by checking
> that the index.html file is there.

I think the only thing that the presence of index.html implies is that 
output-distance.py was run.

> For further context, I am interested in automating our test setup
> further: I want to have CI that can post feedback ("test failed!") on
> code reviews automatically. test-output-distance is a test that always
> fails, which makes that more complicated.

This is what the description of the commit that removed test-output-distance 
ought to have been.

I've been trying to point out that something of value was lost.  This last 
statement of yours finally gives something to weigh that value against.
-- 
Dan

Re: Remove references to test-output-distance.ly. (issue 549780044 by hanw...@gmail.com)

[Dan Eble]

On Mar 29, 2020, at 17:39, Han-Wen Nienhuys  wrote:
>> test-output-distance was removed on the grounds that the self-test
>> serves the same purpose, but I don't see how it does.
> 
> Could you elaborate? What failure scenario are you worried about?

My question is, how does the self-test "verify that the regression tests have, 
in fact, run"?  I don't see how it could do more than verify that itself has 
run.
— 
Dan

Re: Anything missing for 2.21.0?

[Dan Eble]

On Mar 29, 2020, at 15:31, David Kastrup  wrote:

> I was actually only planning to touch convertrules.py .  Sigh.

Sure, don't let it bother you.
— 
Dan

Re: Anything missing for 2.21.0?

[Dan Eble]

On Mar 29, 2020, at 14:46, David Kastrup  wrote:
> 
> Dan Eble  writes:
> 
>> On Mar 29, 2020, at 11:54, David Kastrup  wrote:
>>> 
>>> Anybody can think of a holdup?
>> 
>> convert-ly is broken:
...
> Oh wow, how did that happen?
> 
> Oh, and now I remember that there was a convert-ly thing about some
> spacing properties that I also needed to get done.  More important for
> 2.20.1 than 2.21.0 probably, but I need to get it done either way.

The log levels in the --help text are also wrong.  If you try to use NONE or 
WARNING, it complains,

convert-ly: error: Unknown or invalid loglevel 'NONE'

It does accept WARN rather than WARNING, and in that mode it doesn't print 
progress messages, though I haven't verified that it actually prints warnings.

Obviously, this isn't a show-stopper, and should probably be fixed in 2.20 if 
there are any more patches released, but if you're touching the script anyway...
— 
Dan

Re: Anything missing for 2.21.0?

[Dan Eble]

On Mar 29, 2020, at 11:54, David Kastrup  wrote:
> 
> Anybody can think of a holdup?

convert-ly is broken:

find . -name '*.ly' -print0 | xargs -0 convert-ly -d -e -l WARN
Traceback (most recent call last):
 File "/home/user/lilypond-build/out/bin/convert-ly", line 236, in do_conversion
   newstr = x[1] (str)
 File 
"/home/user/lilypond-build/out/lib/lilypond/current/python/convertrules.py", 
line 3980, in conv
   str = re.sub (r"\\(compress|expand)FullBarRests)", r"\\\1EmptyMeasures",
   
   HERE?
— 
Dan

Re: Running fixcc.py

[Dan Eble]

On Mar 23, 2020, at 18:41, David Kastrup  wrote:
> 
> Dan Eble  writes:
> 
>> I'll submit that for review along with some tweaks to the clang-format
>> configuration.  I still have some testing to do.
> 
> Thanks, and sorry for causing that workload out of not really thinking
> through things.

Think nothing of it.  I prefer Ubuntu's latest package over a 7-year-old source 
tarball from SourceForge.

https://codereview.appspot.com/551640043/
— 
Dan

Re: Running fixcc.py

[Dan Eble]

On Mar 23, 2020, at 17:10, David Kastrup  wrote:
> 
> Dan Eble  writes:

>> As far as I'm concerned, we could just declare 3.1 to be the new
>> preferred version.  I'm not sure whether that would inconvenience
>> anyone else, though.
> 
> It's the current version, so eventually it might have become a candidate
> anyway.  If we do that, fixcc should probably be changed such that 3.1
> is the version accepted without --sloppy .

I'll submit that for review along with some tweaks to the clang-format 
configuration.  I still have some testing to do.
— 
Dan

Re: make builds everything

[Dan Eble]

On Mar 23, 2020, at 10:06, Jean-Charles Malahieude  wrote:
> 
> Le 21/03/2020 à 18:11, Malte Meyn a écrit :
>> Hi list,
>> first of all, I’d like thank those who made the make output less verbose, 
>> this makes errors much easier to find. 
> 
> It is, unfortunately, sometimes too little verbose: say I've omitted a 
> closing 
...
> How would it be possible to access the Nirvana, or to reinstate some 
> verbosity in this case, with:

make VERBOSE=1 ...
(Verbosity, not Nirvana.)

Regards,
— 
Dan

Re: Running fixcc.py

[Dan Eble]

On Mar 22, 2020, at 17:04, David Kastrup  wrote:
> 
> Dan Eble  writes:
> 
>> Did you use astyle 2.04 or another version?  I built 2.04 from source
...
> 3.1.  I am afraid that I may have updated my system since the review,
...
> So where should we go from here?

As far as I'm concerned, we could just declare 3.1 to be the new preferred 
version.  I'm not sure whether that would inconvenience anyone else, though.
— 
Dan

Re: Running fixcc.py

[Dan Eble]

On Mar 21, 2020, at 11:26, David Kastrup  wrote:
> it got a bit lost in other things, but I think I would want to run
> fixcc.py right now, reformatting the C++ stuff (it doesn't help with the
> conventions for template arguments but there are comparatively few of
> those).

Did you use astyle 2.04 or another version?  I built 2.04 from source and ran 
it on master and I see a handful of differences in 3 files.
— 
Dan

Re: Running fixcc.py

[Dan Eble]

On Mar 21, 2020, at 11:26, David Kastrup  wrote:
> it got a bit lost in other things, but I think I would want to run
> fixcc.py right now, reformatting the C++ stuff (it doesn't help with the
> conventions for template arguments but there are comparatively few of
> those).

OK here.  Also, thanks for suggesting an approach to rebasing.
— 
Dan

Re: Remove @ substitutions from python/*.py (issue 549740043 by hanw...@gmail.com)

[Dan Eble]

On Mar 21, 2020, at 08:25, hanw...@gmail.com wrote:
> 
> For context: many projects that I contribute to keep harping about
> trailing whitespace

I recently started using the ws-butler package to limit whitespace changes to 
lines that also have substantive changes.  It has helped me.

https://github.com/lewang/ws-butler
— 
Dan

Re: Fix output-distance tests (issue 569540043 by hanw...@gmail.com)

[Dan Eble]

On Mar 15, 2020, at 12:21, David Kastrup  wrote:
> 
> Han-Wen Nienhuys  writes:
>> suggests that OSX actually uses bash.
> 
> Paywalled site.

The current macOS (10.15.3, "Catalina") sets zsh as the default shell for new 
users.  bash is still provided.

$ zsh --version
zsh 5.7.1 (x86_64-apple-darwin19.0)
$ /bin/sh --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin19)
Copyright (C) 2007 Free Software Foundation, Inc.
— 
Dan

Re: Tidy check on Ubuntu Xenial

[Dan Eble]

On Mar 15, 2020, at 10:26, Han-Wen Nienhuys  wrote:
> 
> On Sun, Mar 15, 2020 at 2:33 AM Dan Eble  wrote:
>> 
>> On Mar 14, 2020, at 19:39, Han-Wen Nienhuys  wrote:
>>> 
>>> On Sat, Mar 14, 2020 at 9:16 PM Dan Eble  wrote:
>>>> I assume the warnings are incorrect, otherwise you would be asking for 
>>>> help to fix them rather than asking whether checking the HTML is valuable. 
>>>>  Is that assumption correct?
>>> 
>>> The version on Xenial generates these warnings, while the version on
>>> Fedora doesn't, which is pretty confusing.
>> 
>> Certainly, but the question is whether they are legitimate.
> 
> Nobody has complained about HTML compliance of the regtests, so I
> think this is fine either way.

Another dodge.

>> How many grains of rice make a heap?  Line up all the small things in a 
>> makefile that "you could just do" and it isn't long before you can't just do 
>> them anymore.
> 
> Line up all the things you could just add to the Makefile, and you'll
> end up with large Makefiles that are hard to understand and maintain.
> This is the place where we are today, and I'd rather move away from
> that.

You haven't justified it in my eyes, but have it your way.
— 
Dan

Re: Tidy check on Ubuntu Xenial

[Dan Eble]

On Mar 14, 2020, at 19:39, Han-Wen Nienhuys  wrote:
> 
> On Sat, Mar 14, 2020 at 9:16 PM Dan Eble  wrote:
>> I assume the warnings are incorrect, otherwise you would be asking for help 
>> to fix them rather than asking whether checking the HTML is valuable.  Is 
>> that assumption correct?
> 
> The version on Xenial generates these warnings, while the version on
> Fedora doesn't, which is pretty confusing.

Certainly, but the question is whether they are legitimate.

>  You could just do
> 
>  for  f in $(find -name '*.html') ; do  tidy -q -o/dev/null $f ; done
> 
> to verify that it's working as expected when working on output-distance.

How many grains of rice make a heap?  Line up all the small things in a 
makefile that "you could just do" and it isn't long before you can't just do 
them anymore.
— 
Dan

Re: Tidy check on Ubuntu Xenial

[Dan Eble]

On Mar 14, 2020, at 15:26, Han-Wen Nienhuys  wrote:
> 
> In my docker install of Ubuntu Xenial, I see the following failure,
> apparently coming from tidy.
> 
> creating 
> /lilypond/out/test-results/input/regression/out-test-baseline/test-output-distance.png
> creating 
> /lilypond/out/test-results/input/regression/out-test/test-output-distance.png
> Validating 
> /lilypond/out/test-results/input/regression/out-test/test-output-distance.details.html
> line 6 column 1 - Warning:  proprietary attribute "charset"
> line 6 column 1 - Warning:  lacks "content" attribute
> line 9 column 1 - Warning:  lacks "summary" attribute
> make: *** [local-check] Error 1
> 
> $ docker run --rm -it lilypond-base-ubuntu dpkg -l tidy
> ..
> ii  tidy20091223cvs-1.5amd64
> HTML syntax checker and reformatter
> 
> 
> For now, I can work around this by removing tidy, but do we really
> need this dependency? It also looks like it introduces some extra
> makefile complexity.

Tidy is specified as OPTIONAL in configure.ac.  The reason it is there is that 
I found it valuable when I was working on output-distance.py several months 
ago, but you should be able to omit it from your Docker image and everything 
should still work.  You just won't be checking the generated HTML.

I assume the warnings are incorrect, otherwise you would be asking for help to 
fix them rather than asking whether checking the HTML is valuable.  Is that 
assumption correct?

If these are false positives coming from the version of tidy provided with 
Xenial, then I think it would make sense for the configure script to avoid 
using that version of tidy.  The level of makefile complexity involved is minor.
— 
Dan

Re: PATCHES - Countdown for March 13th

[Dan Eble]

On Mar 13, 2020, at 09:03, pkx1...@posteo.net wrote:
> 
> 5703 Run scripts/auxiliar/fixcc.py - David Kastrup
> https://sourceforge.net/p/testlilyissues/issues/5703
> http://codereview.appspot.com/549480043

What is the plan for this?
— 
Dan

Re: Address output-distance problems: (issue 563730043 by hanw...@gmail.com)

[Dan Eble]

On Mar 13, 2020, at 04:43, Kevin Barry  wrote:
> 
> The direction of this statement is correct, but the magnitude is not.  The 
> kernel is still provided by the host.   Getting a crash report can be 
> frustrating when the guest's behavior hinges on /proc features that the host 
> OS has configured appropriately for the host, not the guest.  Configurable 
> security restrictions can make the debugging experience different from one 
> installation to another.  Et cetera.
> 
> Yes it's true that containers are not completely safe from host 
> configurations, but I didn't think talking about the 1% would help this 
> discussion. If you think it makes pursuing this idea a waste of time then 
> fair enough.

I think you misread me.  I was agreeing that relying on containers greatly 
reduces "works for me" problems among the group using them; but I thought that 
in this forum, a good number of people would take "eliminate" literally and 
would therefore be thankful for some clarification.

Regards,
— 
Dan

Re: Address output-distance problems: (issue 563730043 by hanw...@gmail.com)

[Dan Eble]

On Mar 12, 2020, at 08:36, Kevin Barry  wrote:
> 
>> Would docker give us this 'proverbial canary' or would it turn into
>> 'worksforme' when someone tried to build their own version of LP  on a
>> vanilla base of Linux?
>> 
> Docker would eliminate 'worksforme' type issues yes.

The direction of this statement is correct, but the magnitude is not.  The 
kernel is still provided by the host.   Getting a crash report can be 
frustrating when the guest's behavior hinges on /proc features that the host OS 
has configured appropriately for the host, not the guest.  Configurable 
security restrictions can make the debugging experience different from one 
installation to another.  Et cetera.
— 
Dan

Re: Added transition lines for lyrics (issue 565750043 by davidgrant...@gmail.com)

[Dan Eble]

On Mar 10, 2020, at 04:13, Han-Wen Nienhuys  wrote:
> I never said that blue is a kind of red.

True.  What you said was this:

> Wouldn't this be much simpler if you'd implement
> the transition as a layout tweak to a hyphen? You'd get something like:
> 
> vowelTransition = \once \override LyricVoice.LyricHyphen.style =
> #'transition
> 
> a \vowelTranstion — b

Which I interpreted as suggesting that a vowel transition is a kind of hyphen.

Regards,
— 
Dan

Re: Added transition lines for lyrics (issue 565750043 by davidgrant...@gmail.com)

[Dan Eble]

On Mar 9, 2020, at 04:42, Han-Wen Nienhuys  wrote:
> 
> On Mon, Mar 9, 2020 at 12:44 AM Dan Eble  wrote:
>> I agree that lots of duplication is something that should be avoided, but so 
>> is the conflation of style and meaning.  A lyric hyphen separates syllables; 
>> this arrow thing means something more.
> 
> Can you have a hyphen and a transition between two syllables at the
> same time? If not, that suggests that they are two variations of
> essentially the same thing.

I would agree that a hyphen and a "transition line" are mutually exclusive ways 
of demarcating syllables; but that doesn't make a "transition line" a kind of 
hyphen any more than it makes a staccato mark a kind of legato mark or blue a 
kind of red.
— 
Dan

Re: Added transition lines for lyrics (issue 565750043 by davidgrant...@gmail.com)

[Dan Eble]

On Mar 8, 2020, at 11:21, hanw...@gmail.com wrote:
> 
> I think it would be better to do this as something that doesn't require
> special syntax at first, ie. some identifier. 
> 
> We can always add special syntax later, if this becomes a very popular
> feature.

+1.  In the meantime, the minority of users who want an abbreviation can define 
one, right?

> Also, it looks like the transition-engraver is almost a literal copy of
> the hyphen engraver. Wouldn't this be much simpler if you'd implement
> the transition as a layout tweak to a hyphen? You'd get something like:
> 
>  vowelTransition = \once \override LyricVoice.LyricHyphen.style =
> #'transition
> 
>  a \vowelTranstion — b

I agree that lots of duplication is something that should be avoided, but so is 
the conflation of style and meaning.  A lyric hyphen separates syllables; this 
arrow thing means something more.

It reminds me of the abuse of rehearsal marks for several categories of 
instructions.  You build up a body of work full expedient little tricks; then 
one day you want to move all your "D.C." marks, but not your rehearsal marks, 
to the other side of the staff in certain scores; and then you learn what a 
mess it really was from the beginning.

I haven't looked at the code, but maybe factoring out shared functions would 
help reduce duplication.
— 
Dan

Re: PATCHES - Countdown for March 5th

[Dan Eble]

On Mar 5, 2020, at 01:38, pkx1...@posteo.net wrote:
> 
> Push:
...
> 5809 output-distance: avoid calling strip() on None - Han-Wen Nienhuys
> https://sourceforge.net/p/testlilyissues/issues/5809
> http://codereview.appspot.com/549640043
...
> Countdown:
...
> 5803 output-distance: treat non-existent files as empty string - Han-Wen 
> Nienhuys
> https://sourceforge.net/p/testlilyissues/issues/5803
> http://codereview.appspot.com/583590043

Duplicates?
— 
Dan

Re: converting tabs to spaces in .mf files

[Dan Eble]

On Feb 25, 2020, at 05:29, Werner LEMBERG  wrote:
> 
> What do you think about converting all tabs in the .mf files to
> spaces?  If you agree, I would apply this to the staging.

I don't usually work in that domain, so SGTM.
— 
Dan

Re: Allow parallelism in input/regression/lilypond-book/ (issue 547680043 by hanw...@gmail.com)

[Dan Eble]

On Feb 23, 2020, at 09:11, David Kastrup  wrote:
> 
>>  "Sharing Job Slots with GNU make"
>>  https://www.gnu.org/software/make/manual/html_node/Job-Slots.html
> 
> But that still doesn't solve the problem that the database approach of
> lilypond-book does not work for running multiple lilypond-book jobs in
> parallel.

Maybe I haven't devoted enough time to understand this situation well.  
Basically, my thought was that whatever currently uses CPU_COUNT could 
communicate with the job server instead so that one wouldn't need to repeat -jN 
CPU_COUNT=N.
— 
Dan

Re: Allow parallelism in input/regression/lilypond-book/ (issue 547680043 by hanw...@gmail.com)

[Dan Eble]

On Feb 23, 2020, at 09:04, Han-Wen Nienhuys  wrote:
> 
>> What would you recommend to a developer who doesn't want to run more than J 
>> = M + N - 1 concurrent jobs due to lilypond development?  What values of M 
>> and N would serve best?
> 
> Normally M=N= #cpus should be OK. A bit of extra parallelism doesn't
> hurt, especially if you have hyperthreaded CPUs.

While that qualifies as guidance, I'm not sure you've heard me.  Extra 
parallelism is unwelcome by definition in this case.

Also, don't forget that although the parallelism is (sadly only) governed by 
the number of jobs, each job uses resources other than CPU time that the 
developer might be concerned about.

Regards,
— 
Dan

Re: Allow parallelism in input/regression/lilypond-book/ (issue 547680043 by hanw...@gmail.com)

[Dan Eble]

On Feb 23, 2020, at 06:08, Han-Wen Nienhuys  wrote:
> I think we should do both: the lilypond runs in lp-book should be
> protected by some sort of lock, and we should use both CPU_COUNT=M and
> -jN.
> 
> then worst case, you have M lilypond processes and N-1 other jobs.

What would you recommend to a developer who doesn't want to run more than J = M 
+ N - 1 concurrent jobs due to lilypond development?  What values of M and N 
would serve best?

On Sun, Feb 23, 2020 at 12:00 PM  wrote:
> So wouldn't it appear that the way to exploit parallelism with
> lilypond-book, short of writing its own jobserver, is to use CPU_COUNT
> like we did before?

Making lilypond-book a client of the GNU make job server sounds like an option.

  "Sharing Job Slots with GNU make"
  https://www.gnu.org/software/make/manual/html_node/Job-Slots.html

Examples (good or bad? I haven't looked):

  "Add GNU make jobserver client support"
  https://github.com/ninja-build/ninja/pull/1140

  "Add a GNU make jobserver implementation to Cargo"
  https://github.com/rust-lang/cargo/pull/4110

— 
Dan

Re: Naming question for get_property, set_property

[Dan Eble]

On Feb 10, 2020, at 20:48, David Kastrup  wrote:
> 
> Templating on a string constant is, unfortunately, not a thing at least
> in C++11 (don't know whether they managed since then).  Or one could go
> that route rather than GCC-specific in-expression use of a static
> initializer.

Well, constexpr enables some new and interesting things, though I imagine that 
having to interact with Guile nixes them.

Here, someone has figured out how to compute a CRC of a literal string at 
compile time:
https://stackoverflow.com/a/9842857
— 
Dan

Re: Naming question for get_property, set_property

[Dan Eble]

On Feb 10, 2020, at 21:35, David Kastrup  wrote:
> property_set (grob, "stencil", SCM_BOOL_F);
> 
> and
> 
> property_get (grob, "color")

These are fine.
— 
Dan

Re: Naming question for get_property, set_property

[Dan Eble]

On Feb 10, 2020, at 17:47, David Kastrup  wrote:
> It will look a bit redundant either way with
> 
> grob->Get (Grob, "color");
> or
> grob->grob_set ("stencil", SCM_BOOL_F);

"Yuck" either way.  Removing "property" to shorten the name is not a good 
course of action.

My brainstorming without knowing your reasons does not seem likely to help, but 
what the hey ...

grob->set ("stencil", SCM_BOOL_F);

grob->set (Grob_property ("stencil"), SCM_BOOL_F);

grob->properties["stencil"] = SCM_BOOL_F; // Too much to ask?
— 
Dan

Re: Grow heap aggressively during music interpretation (issue 561390043 by hanw...@gmail.com)

[Dan Eble]

On Feb 9, 2020, at 04:25, jonas.hahnf...@gmail.com wrote:
> 
> Anyway I think adding a comment here and in smobs.cc how this is
> initialized (refer to static lifetime in C/C++) would likely clarify the
> intent.
 
Initialization is an area where the legacy of the language is complicated (more 
connotative adjectives come to mind), and mutable static data is not in daily 
use, and failure to initialize is a well-known class of error, so a "what it 
does" comment (that people usually don't like) would help.
— 
Dan

Re: RFC: docker for CI

[Dan Eble]

On Feb 7, 2020, at 16:23, Han-Wen Nienhuys  wrote:
> 
> More work , and I'm lazy :)

No problem!  Jonas is probably bored now that there's nothing left to port to 
Python 3.
— 
Dan
I aim to communicate with empathy.  Have I failed?  ¯\_(ツ)_/¯

Re: RFC: docker for CI

[Dan Eble]

On Feb 7, 2020, at 15:22, Kevin Barry  wrote:
> P.S. I think I have seen a dockerfile for creating a build environment
> for LilyPond somewhere.

https://github.com/fedelibre/LilyDev/tree/master/docker
I'm using it.  I'm not sure who else is.
— 
Dan

Re: RFC: docker for CI

[Dan Eble]

On Feb 7, 2020, at 15:21, Han-Wen Nienhuys  wrote:
>>>   * use a headless browser to take a image snapshot of the top of regtest
>>>  result page.
>>> 
>> Sounds convoluted.  Why not attach the difference images directly?
>  
> Those are potentially 1372 images to attach if you made a change with global 
> impact.

Why not attach the N images with the greatest differences directly?

More generally, I'd want a digest of the results (not all of which are visual) 
that is as useful as possible for the size we are willing to post to the 
review.  We control output-distance.py, so we could generate something new that 
fits this case.

>> Are full logs and test results retained, or does a developer need to 
>> reproduce the test locally to get them?
> 
> You'd retain the full logs and results as part of the docker image. Currently 
> my checkout is about 1.8G of data, and a lilypond docker image itself would 
> be close  to that too.

This approach is new to me.  I'm used to CI systems that are configured to 
archive particular files from the workspace (e.g., the regtest output tree, the 
final docs) and full build log for a limited time (days to weeks).  I think it 
balances the types of things you can investigate without reproducing the build 
yourself against retaining a huge amount of data.

Can you expand on the purpose of saving the full Docker image--which is not 
just the LilyPond workspace but the OS too, correct?  Are you thinking that 
someone would prefer to download it and debug in a container rather than 
reproduce the build in their usual development environment?
— 
Dan

Re: RFC: docker for CI

[Dan Eble]

On Feb 7, 2020, at 07:21, Han-Wen Nienhuys  wrote:
>  * use a headless browser to take a image snapshot of the top of regtest
> result page.

Sounds convoluted.  Why not attach the difference images directly?

> On success, the driver uploads the image snapshot to code review.
> 
> On failure, the driver uploads the last bit of the log-file to code review.

Are full logs and test results retained, or does a developer need to reproduce 
the test locally to get them?
— 
Dan

Re: RFC: docker for CI

[Dan Eble]

On Feb 7, 2020, at 07:21, Han-Wen Nienhuys  wrote:
> 
>  * runs (make ; make test-baseline)

If this says "(make ;" because you think that "make test-baseline" requires a 
prior make, I think it is incorrect.  (If "make test-baseline" doesn't work on 
its own, it should be fixed.)

If this says "(make ;" because you want to build everything else that "make 
test-baseline" might not build, then it's fine, though maybe it should be "make 
&& make test-baseline" or two separate steps, depending on how errors are 
detected and reported.
— 
Dan

Re: RFC: docker for CI

[Dan Eble]

On Feb 7, 2020, at 07:21, Han-Wen Nienhuys  wrote:
> 
> contains a copy of ccache

ccache is interesting.  It speeds up recompiling files after a make clean, 
which is great if you often have to make clean because your makefile is broken, 
or if you often reconfigure your build options (e.g. debug v. release), but 
doesn't improve regular incremental builds.  There are some documented pitfalls 
that I can't remember off the top of my head.

At work in 2019, I added ccache to a Docker-based build at the request of one 
of the developers.  It worked fine when hosted on PCs but failed on the CI 
servers with symptoms that did not make obvious sense and which I couldn't 
justify spending time to investigate.  

That's my experience with ccache and Docker.  Cool, but watch out.
— 
Dan

Re: event queue with thread for c++

[Dan Eble]

On Feb 7, 2020, at 13:26, Han-Wen Nienhuys  wrote:
> 
> To do https://codereview.appspot.com/561390043/ properly, I have to expand 
> the heap when GC notifies us that a collection took place. Unfortunately, 
> libgc notifications are done with the garbage collector lock held. So I'll 
> need schedule a call to GC_expand_heap on a different thread. 
> 
> Would you know what is the best way to do this in C++ nowadays? 

I wish I could give you a ready-made solution, but the multi-threaded systems 
I've worked on in the past decade were designed so that most of the development 
team could concentrate on single-threaded problems.  Only a couple of people 
needed to understand the load-balancing/scheduling code, and I was not one of 
them.

I can point to the C++11 thread support and atomic operations libraries:

https://en.cppreference.com/w/cpp/thread
https://en.cppreference.com/w/cpp/atomic

Anything else I might say would be speculative.  I'd probably start with the 
std::thread class.
— 
Dan

My availability

[Dan Eble]

Dear LilyPond developers,

I'm facing the bittersweet experience of becoming an employee again on Monday.  
I'm not sure what the ramp-up will require or what the new cadence of my life 
will be, but I will certainly have less time to devote to LilyPond.  Hopefully, 
it will be a reversion to the mean and not more!  I thought I should let you 
know so that when my engagement diminishes subito you don't think there's 
something wrong.

Regards,
— 
Dan

Re: [RFC] Commit message format

[Dan Eble]

On Feb 7, 2020, at 10:39, Han-Wen Nienhuys  wrote:
> 
> There are a couple of downsides to this format:
> * The number takes up space in the
>git log --format=short

upside: the number appears in git log —format=short

> * The number is meaningless without the site that hosts the tracker

exaggerated, but I see your point

>Link to code review
>Link to issue
> 
> By embedding the links, we offer something clickable to whomever is
> browsing the commit message.

-1, maybe.  Will the commit messages be rewritten when we migrate to a 
different issue tracker or the provider decides to reorganize the URL paths?  A 
stale link from which I must extract an ID would annoy me more than a bare ID, 
I expect.

(I might have put links in a commit message or two, but please do as I say, not 
as I do.)

Also, I've worked with issue trackers that scan commit messages for IDs in a 
specific format (e.g. "#1234" or "[1234]") and turn them into hyperlinks from 
the ticket to the related commits in a repository view.  It's probably best to 
postpone dictating a citation format until we know the constraints of the tools 
we move to (hoping we do).  We might even be able to configure patterns for 
recognizing our older citations.

> Finally, I want to encourage everyone to write Why something was
> changed rather What. One can deduce what changed by looking at the
> commit message.  It is much harder to fathom Why some change was
> necessary.

+1.  This is good practice for code comments too.  I probably appreciate "why 
it was not done the obvious way" the most.
— 
Dan

Re: Collisions on OttavaBracket in the SVG output (BUG)

[Dan Eble]

On Feb 6, 2020, at 18:35, Paolo Prete  wrote:
> 
> (I re-post this here, because it's a bug and maybe you have an easy fix for
> it)

Paolo,

I'm not sure where you're re-posting from, but the expected forum for bug 
reports is https://lists.gnu.org/mailman/listinfo/bug-lilypond .  The audience 
is not the same as the development list, so I would recommend posting directly 
to the bug list if you need to report any bugs in the future.  I've forwarded 
this message for you.

Cheers.

> Hello,
> 
> please consider this snippet:
> 
> %%%
> {
> \override Staff.TextSpanner.bound-details.left.text = \markup { \small
> \bold "Rit." }
> \once \override Slur.outside-staff-priority = 100
> \once \override DynamicText.outside-staff-priority = 200
> \once \override TupletBracket.outside-staff-priority = 300
> \once \override Staff.OttavaBracket.outside-staff-priority = 400
> \once \override Staff.TextSpanner.outside-staff-priority = 500
> \once \override TextScript.outside-staff-priority = 600
> 
> \override TupletBracket.bracket-visibility = ##t
> \override TupletBracket.direction = #UP
> \tuplet 5/4 { \ottava #1 c'''8^>^\mf[(^\markup{ legato }\startTextSpan  e''
> f''' g''' e]} c4\stopTextSpan \ottava #0 c'')
> }
> %%%
> 
> On 2.19.83 the SVG output produces a collision on the OttavaBracket, which
> is not placed at the right outside-staff-priority order. The PDF output is
> ok. Is there a workaround/fix for this?
> 
> thanks

Re: Splitting Staves

[Dan Eble]

On Feb 6, 2020, at 09:32, Kieren MacMillan  
wrote:
> 
>> So what I want to work on is some sort of ”Splittable Staff“.

This terminology looks at the use case from a point of view that might not be 
very consistent with LilyPond.  Let's think carefully about what we want to 
call this, because names have a way of coloring the problem-solving process.

LilyPond already has some features that can move a voice between staves 
(\change and \autoChange), and is able to hide staves without music.  We should 
entertain "moving voices" as an alternative idea to "splitting staves" so that 
we are more likely to find the best solution.  Maybe there are more ways of 
looking at it that we should also consider.

(I don't mean for that to sound the slightest bit discouraging.  I think it's 
great that this is being discussed.)

> I love the concept… but I‘m dubious that it can be done fully and correctly 
> without a proper and reliable partcombiner as well.

+1, however something less than "fully and correctly" might work for a good 
number of people and be implemented sooner.
— 
Dan

Re: Add Code of Conduct (issue 575620043 by janek.lilyp...@gmail.com)

[Dan Eble]

On Feb 5, 2020, at 20:26, Thomas Morley  wrote:

> So to repeat myself, everyone should take his post literal, not offending!
> 
> I'd love to see a community bearing different personalities, even
> personalities with problematic conversation skills.
> For me it's like strange english from a non-native speaker (like me).
> It's sometimes difficult and/or tedious to understand but mostly worth
> the attention.
> 
> Well, long mail for a non-native speaker, and I still have the feeling
> I didn't express myself very well.
> Though, I did the best I could.
> 
> Thanks,
>  Harm

I also want to put in a kind word for David K. and point out that Harm's 
attempt to de-escalate this conversation demonstrates that the ideas of 
mentorship and teamwork that have been circulating with regard to technical 
matters are also applicable to interpersonal matters.
— 
Dan

Re: [RFC] switch to GitLab / gitlab.com

[Dan Eble]

On Feb 5, 2020, at 10:09, Jonas Hahnfeld  wrote:
> 
> required to synchronize the review and the associated issue. I propose
> to start using GitLab hosted on gitlab.com [4] for all of this:
> Repository, Issues, and Merge Requests (MR) for reviews. It was
> evaluated 'C' in 2015 [5] and should be an 'acceptable hosting for a
> GNU package' [6].

Fine with me.  I don't expect to donate my time to make _any_ move happen, but 
I'd accept working with these tools to get my patches into LilyPond.  It could 
hardly be worse than the current combination.
— 
Dan

Re: Code of Conduct

[Dan Eble]

On Feb 5, 2020, at 05:45, Han-Wen Nienhuys  wrote:
> 
> Having a CoC gives us a set of guidelines, a process and a set of
> corrective actions to take to help keep things nice.

I prefer the implicit good-neighbour agreement we have now.
—
Dan

Re: Grow heap aggressively during music interpretation (issue 561390043 by hanw...@gmail.com)

[Dan Eble]

On Feb 5, 2020, at 07:59, jonas.hahnf...@gmail.com wrote:
> 
> and it seems stable, but I have yet to build a theory why OS thread
> migration actually improves performance…

Educated guess:  The core gets too hot.  The OS is not allowed to move the 
thread to a cooler core, so it throttles the clock rate instead.
— 
Dan

Re: development process

[Dan Eble]

On Feb 5, 2020, at 06:57, David Kastrup  wrote:
> 
> Our current process is awkward technically, not because of the roles its
> human players assume.

+1(000)

For a long time, the LilyPond project has needed these:

  1. more and better automation to reduce the load on the (amazingly
 perseverant) patch tester and lower the bar for others to help
  2. better-integrated bug tracking, code review, and source control
  3. a cross-platform build that is more maintainable, faster, and
 more frequently run

And these are the proposals I've been seeing:

  1. reduced opportunity for review; reduced emphasis on consensus
  2. new roles
  3. a code of conduct
  4. a straw man or two (coming from a lack of familiarity with the
 status quo)
  5. better-integrated bug tracking, code review, and source control

There is some common ground here, but it also comes across in part as trying to 
fix what's not broken.  Perhaps if you would first improve the tools and 
testing, people's confidence in them would grow and they would then be more 
willing to reduce the level of scrutiny on patches.
-- 
Dan

Re: [testlilyissues:issues] #5719 Tie formatting maintenance

[Dan Eble]

> On Feb 4, 2020, at 16:14, Dan Eble  wrote:
> 
> Begin forwarded message:
>> 
>> Subject: [testlilyissues:issues] #5719 Tie formatting maintenance
> ...
>> commit 937e413c0f399b2ec44785b7ca29d61cf7b24cff
>> Author: Dan Eble <nine.fierce.ball...@gmail.com>
>> Date:   Fri Jan 31 13:48:35 2020 -0500
> 
> The changes for Issue 5719 remove a header file.  If you want to avoid 
> running make clean to reset all dependencies, try removing just these:
> 
>out/lily/tie*
>out/lily/semi-tie*

Oops: those should be lily/out/…
— 
Dan

Fwd: [testlilyissues:issues] #5719 Tie formatting maintenance

[Dan Eble]

Begin forwarded message:
> 
> Subject: [testlilyissues:issues] #5719 Tie formatting maintenance
...
> commit 937e413c0f399b2ec44785b7ca29d61cf7b24cff
> Author: Dan Eble <nine.fierce.ball...@gmail.com>
> Date:   Fri Jan 31 13:48:35 2020 -0500

The changes for Issue 5719 remove a header file.  If you want to avoid running 
make clean to reset all dependencies, try removing just these:

out/lily/tie*
out/lily/semi-tie*

I think that will be sufficient, but if you hit errors, remove the 
corresponding files.
— 
Dan

Re: Inline assembler fallback for _FPU_SETCW() missing in MINGW libraries (issue 577450043 by thomasmorle...@gmail.com)

[Dan Eble]

On Feb 4, 2020, at 11:02, Jonas Hahnfeld  wrote:
>> That would be my impulse as well.  It is not like this code appears to
>> have notable drawbacks for the unafflicted platforms.
> 
> Except for very funny overflows and negative signs if the value is too
> large to fit into I64 ;-P
> 
> unsigned long long a = 0xC000;
> signed long long b = a;
> printf("%d\n", b);
> -> -1073741824

That's already an ingredient of this recipe.

   class Rational
   {
 ...
 U64 num_, den_;
 ...
 I64 numerator () const { return sign_ * num_; }

Yum!
— 
Dan