Re: shiftDurations and tempo

[David Kastrup]

Jean Abou Samra  writes:

> When you say "tackle the resulting parser conflicts", that means
> "add precedences to solve them by making interpretation choices",
> not "solve them in a way that always does what the user might intend",
> right? As I understand it, there is a shift/reduce conflict on
>
> \tempo    [here]   # ...
>
> where the single lookahead token # is not enough
> to distinguish between \tempo   = 
> and \tempo  #  . The only
> way to "solve" while not breaking existing use of the second
> possibility would be to reduce on
>
> \tempo  [here] #
>
> and thus fail on
>
> \tempo  # = 
>
> as that would be interpreted as
>
> \tempo 
> # =  % don't know what to do with this
>
> Does that sound right? I'm interested in understanding the parser
> better, but it's a bit foggy for me at the moment.

It's a bit foggy to the wizards.  It's quite opaque for most.

Here is Bison's take on the conflicts:

lily/parser.yy: warning: shift/reduce conflict on token SCM_TOKEN 
[-Wcounterexamples]
  First example: "#{" $@1 "\\tempo" text • SCM_TOKEN '=' tempo_range
  Shift derivation
start_symbol
↳ 3: "#{" $@1 embedded_lilypond
  ↳ 51: identifier_init_nonumber
↳ 81: music_assign
  ↳ 190: simple_music
 ↳ 201: event_chord
↳ 408: tempo_event
   ↳ 175: "\\tempo" text 
embedded_scm_bare '=' tempo_range
 ↳ 22: 
• SCM_TOKEN
  Second example: "#{" $@1 "\\tempo" text • SCM_TOKEN music_list "end of input"
  Reduce derivation
$accept
↳ 0: start_symbol   
 "end of input"
 ↳ 3: "#{" $@1 embedded_lilypond
   ↳ 55: music_embedded 
music_embeddedmusic_list
 ↳ 184: music   
↳ 186: music_embedded_backup
↳ 180: music_assign 
   ↳ 189: embedded_scm
   ↳ 190: simple_music  
  ↳ 40: embedded_scm_bare
  ↳ 201: event_chord
↳ 22: SCM_TOKEN
 ↳ 408: tempo_event
↳ 174: 
"\\tempo" text •
lily/parser.yy: warning: shift/reduce conflict on token SCM_IDENTIFIER 
[-Wcounterexamples]
  First example: "#{" $@1 "\\tempo" text • SCM_IDENTIFIER '=' tempo_range
  Shift derivation
start_symbol
↳ 3: "#{" $@1 embedded_lilypond
  ↳ 51: identifier_init_nonumber
↳ 81: music_assign
  ↳ 190: simple_music
 ↳ 201: event_chord
↳ 408: tempo_event
   ↳ 175: "\\tempo" text 
embedded_scm_bare  '=' tempo_range
 ↳ 23: 
• SCM_IDENTIFIER
  Second example: "#{" $@1 "\\tempo" text • SCM_IDENTIFIER music_list "end of 
input"
  Reduce derivation
$accept
↳ 0: start_symbol   
"end of input"
 ↳ 3: "#{" $@1 embedded_lilypond
   ↳ 55: music_embedded 
music_embedded   music_list
 ↳ 184: music   
↳ 186: music_embedded_backup
↳ 180: music_assign 
   ↳ 189: embedded_scm
   ↳ 190: simple_music  
  ↳ 40: embedded_scm_bare
  ↳ 201: event_chord
↳ 23: SCM_IDENTIFIER
 ↳ 408: tempo_event
↳ 174: 
"\\tempo" text •




Which actually does not look all that different from your analysis.

-- 
David Kastrup

Re: shiftDurations and tempo

[Simon Albrecht]

Hi David,

On 27/06/2022 21:07, David Kastrup wrote:

Before looking for a possible following `=`
LilyPond has to have made a decision what type the token/expression
coming before that has.  It cannot look at the type of a #... expression
before deciding to look for `=` but it can (and will) look at the type
of a $... expression.

Now syntax errors are comparatively unhelpful, so one could try to
tentatively accept an #... = ... expression anyway and then barf if the
expression type does not support it.  The result would likely not be
worse for the user.



thanks for your consideration. I don’t think I’m qualified to speak on 
that… a decision might depend on how strongly a user is expected to know 
about the # vs. $ issue.


Best, Simon

Re: shiftDurations and tempo

[Simon Albrecht]

Hi Jean,

On 27/06/2022 20:37, Jean Abou Samra wrote:



Le 27/06/2022 à 20:10, Simon Albrecht a écrit :

Hello everyone,

I’m trying to encode a piece such that I can switch whether or not 
durations will be shifted (note values halved/doubled). Unfortunately 
\tempo isn’t affected by \shiftDurations (that would be a sensible 
feature request, right?),


Go open an issue.



I did: 



so I tried this:
%%%
\version "2.23.9"

durationShiftOne = #-1
{
  \tempo #(ly:make-duration

[…]


Can I make that work some other way?



\version "2.23.9"

durationShiftOne = #-1
{
  \tempo $(ly:make-duration (+ 3 durationShiftOne)) = 152
  \shiftDurations \durationShiftOne #0 { 8 }
}


You just need to use $ instead of #.



I keep forgetting to try that. I’ll try understanding the difference 
better as well. Thanks for the help!


Best, Simon

Re: shiftDurations and tempo

[Jean Abou Samra]

Le 28/06/2022 à 00:15, David Kastrup a écrit :

David Kastrup  writes:


Jean Abou Samra  writes:


Le 27/06/2022 à 20:10, Simon Albrecht a écrit :

Hello everyone,

I’m trying to encode a piece such that I can switch whether or not
durations will be shifted (note values
halved/doubled). Unfortunately \tempo isn’t affected by
\shiftDurations (that would be a sensible feature request, right?),

Go open an issue.



so I tried this:
%%%
\version "2.23.9"

durationShiftOne = #-1
{
   \tempo #(ly:make-duration (+ 3 durationShiftOne)) = 152
   \shiftDurations \durationShiftOne #0 { 8 }
}


but unfortunately Lily won’t accept that Scheme expression instead
of the normal duration.

Can I make that work some other way?

Should the parser be able to take that Scheme expression there?


\version "2.23.9"

durationShiftOne = #-1
{
   \tempo $(ly:make-duration (+ 3 durationShiftOne)) = 152
   \shiftDurations \durationShiftOne #0 { 8 }
}


You just need to use $ instead of #. See these pages about the difference:

https://lilypond.org/doc/v2.23/Documentation/extending/lilypond-scheme-syntax.html
https://extending-lilypond.readthedocs.io/en/latest/lily-and-scheme.html#hash-vs-dollar

The problem is that \tempo has multiple syntaxes (\tempo ,
\tempo  =  and \tempo   = ),
which makes it impossible to accept # here because early evaluation
is needed to disambiguate, I think. I'm no expert of the parser
though.

It's sort of like that.  Before looking for a possible following `=`
LilyPond has to have made a decision what type the token/expression
coming before that has.  It cannot look at the type of a #... expression
before deciding to look for `=` but it can (and will) look at the type
of a $... expression.

Now syntax errors are comparatively unhelpful, so one could try to
tentatively accept an #... = ... expression anyway and then barf if the
expression type does not support it.  The result would likely not be
worse for the user.

To wit:
It's debatable whether this partial ability to specify the duration as
#... but not in the long form including text is worth the confusion.
Anyone who wants to tackle the resulting parser conflicts is welcome to
try: Bison appears to do a pretty good job these days proposing
productions that may trigger the problem when giving the right options.




When you say "tackle the resulting parser conflicts", that means
"add precedences to solve them by making interpretation choices",
not "solve them in a way that always does what the user might intend",
right? As I understand it, there is a shift/reduce conflict on

\tempo    [here]   # ...

where the single lookahead token # is not enough
to distinguish between \tempo   = 
and \tempo  #  . The only
way to "solve" while not breaking existing use of the second
possibility would be to reduce on

\tempo  [here] #

and thus fail on

\tempo  # = 

as that would be interpreted as

\tempo 
# =  % don't know what to do with this

Does that sound right? I'm interested in understanding the parser
better, but it's a bit foggy for me at the moment.

Thanks,
Jean

Re: shiftDurations and tempo

[David Kastrup]

David Kastrup  writes:

> Jean Abou Samra  writes:
>
>> Le 27/06/2022 à 20:10, Simon Albrecht a écrit :
>>> Hello everyone,
>>>
>>> I’m trying to encode a piece such that I can switch whether or not
>>> durations will be shifted (note values
>>> halved/doubled). Unfortunately \tempo isn’t affected by
>>> \shiftDurations (that would be a sensible feature request, right?),
>>
>> Go open an issue.
>>
>>
>>> so I tried this:
>>> %%%
>>> \version "2.23.9"
>>>
>>> durationShiftOne = #-1
>>> {
>>>   \tempo #(ly:make-duration (+ 3 durationShiftOne)) = 152
>>>   \shiftDurations \durationShiftOne #0 { 8 }
>>> }
>>> 
>>>
>>> but unfortunately Lily won’t accept that Scheme expression instead
>>> of the normal duration.
>>>
>>> Can I make that work some other way?
>>>
>>> Should the parser be able to take that Scheme expression there?
>>
>>
>> \version "2.23.9"
>>
>> durationShiftOne = #-1
>> {
>>   \tempo $(ly:make-duration (+ 3 durationShiftOne)) = 152
>>   \shiftDurations \durationShiftOne #0 { 8 }
>> }
>>
>>
>> You just need to use $ instead of #. See these pages about the difference:
>>
>> https://lilypond.org/doc/v2.23/Documentation/extending/lilypond-scheme-syntax.html
>> https://extending-lilypond.readthedocs.io/en/latest/lily-and-scheme.html#hash-vs-dollar
>>
>> The problem is that \tempo has multiple syntaxes (\tempo ,
>> \tempo  =  and \tempo   = ),
>> which makes it impossible to accept # here because early evaluation
>> is needed to disambiguate, I think. I'm no expert of the parser
>> though.
>
> It's sort of like that.  Before looking for a possible following `=`
> LilyPond has to have made a decision what type the token/expression
> coming before that has.  It cannot look at the type of a #... expression
> before deciding to look for `=` but it can (and will) look at the type
> of a $... expression.
>
> Now syntax errors are comparatively unhelpful, so one could try to
> tentatively accept an #... = ... expression anyway and then barf if the
> expression type does not support it.  The result would likely not be
> worse for the user.

To wit:

>From 56869db10837eac331e89d074efa5952116ce3d6 Mon Sep 17 00:00:00 2001
From: David Kastrup 
Date: Tue, 28 Jun 2022 00:06:17 +0200
Subject: [PATCH] Allow \tempo # = ... for tempo specifications

What doesn't work is specifying \tempo "text" # = ...
because its implementation would lead to parser conflicts.
---
 lily/parser.yy | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/lily/parser.yy b/lily/parser.yy
index 9207e479c8..7fff2fd019 100644
--- a/lily/parser.yy
+++ b/lily/parser.yy
@@ -1403,6 +1403,16 @@ tempo_event:
 	TEMPO steno_duration '=' tempo_range	{
 		$$ = MAKE_SYNTAX (tempo, @$, SCM_EOL, $2, $4);
 	}
+	| TEMPO embedded_scm_bare '=' tempo_range {
+		if (unsmob ($2))
+		{
+			$$ = MAKE_SYNTAX (tempo, @$, SCM_EOL, $2, $4);
+		} else {
+			parser->parser_error (@2, _("duration expected"));
+			$$ = MAKE_SYNTAX (tempo, @$,
+	  SCM_EOL, Duration (2, 0).smobbed_copy (), $4);
+		}
+	}
 	| TEMPO text steno_duration '=' tempo_range	{
 		$$ = MAKE_SYNTAX (tempo, @$, $2, $3, $5);
 	}
-- 
2.34.1


It's debatable whether this partial ability to specify the duration as
#... but not in the long form including text is worth the confusion.
Anyone who wants to tackle the resulting parser conflicts is welcome to
try: Bison appears to do a pretty good job these days proposing
productions that may trigger the problem when giving the right options.

-- 
David Kastrup

Re: shiftDurations and tempo

[David Kastrup]

Jean Abou Samra  writes:

> Le 27/06/2022 à 20:10, Simon Albrecht a écrit :
>> Hello everyone,
>>
>> I’m trying to encode a piece such that I can switch whether or not
>> durations will be shifted (note values
>> halved/doubled). Unfortunately \tempo isn’t affected by
>> \shiftDurations (that would be a sensible feature request, right?),
>
> Go open an issue.
>
>
>> so I tried this:
>> %%%
>> \version "2.23.9"
>>
>> durationShiftOne = #-1
>> {
>>   \tempo #(ly:make-duration (+ 3 durationShiftOne)) = 152
>>   \shiftDurations \durationShiftOne #0 { 8 }
>> }
>> 
>>
>> but unfortunately Lily won’t accept that Scheme expression instead
>> of the normal duration.
>>
>> Can I make that work some other way?
>>
>> Should the parser be able to take that Scheme expression there?
>
>
> \version "2.23.9"
>
> durationShiftOne = #-1
> {
>   \tempo $(ly:make-duration (+ 3 durationShiftOne)) = 152
>   \shiftDurations \durationShiftOne #0 { 8 }
> }
>
>
> You just need to use $ instead of #. See these pages about the difference:
>
> https://lilypond.org/doc/v2.23/Documentation/extending/lilypond-scheme-syntax.html
> https://extending-lilypond.readthedocs.io/en/latest/lily-and-scheme.html#hash-vs-dollar
>
> The problem is that \tempo has multiple syntaxes (\tempo ,
> \tempo  =  and \tempo   = ),
> which makes it impossible to accept # here because early evaluation
> is needed to disambiguate, I think. I'm no expert of the parser
> though.

It's sort of like that.  Before looking for a possible following `=`
LilyPond has to have made a decision what type the token/expression
coming before that has.  It cannot look at the type of a #... expression
before deciding to look for `=` but it can (and will) look at the type
of a $... expression.

Now syntax errors are comparatively unhelpful, so one could try to
tentatively accept an #... = ... expression anyway and then barf if the
expression type does not support it.  The result would likely not be
worse for the user.

-- 
David Kastrup

Re: shiftDurations and tempo

[Jean Abou Samra]

Le 27/06/2022 à 20:10, Simon Albrecht a écrit :

Hello everyone,

I’m trying to encode a piece such that I can switch whether or not 
durations will be shifted (note values halved/doubled). Unfortunately 
\tempo isn’t affected by \shiftDurations (that would be a sensible 
feature request, right?),


Go open an issue.



so I tried this:
%%%
\version "2.23.9"

durationShiftOne = #-1
{
  \tempo #(ly:make-duration (+ 3 durationShiftOne)) = 152
  \shiftDurations \durationShiftOne #0 { 8 }
}


but unfortunately Lily won’t accept that Scheme expression instead of 
the normal duration.


Can I make that work some other way?

Should the parser be able to take that Scheme expression there?



\version "2.23.9"

durationShiftOne = #-1
{
  \tempo $(ly:make-duration (+ 3 durationShiftOne)) = 152
  \shiftDurations \durationShiftOne #0 { 8 }
}


You just need to use $ instead of #. See these pages about the difference:

https://lilypond.org/doc/v2.23/Documentation/extending/lilypond-scheme-syntax.html
https://extending-lilypond.readthedocs.io/en/latest/lily-and-scheme.html#hash-vs-dollar

The problem is that \tempo has multiple syntaxes (\tempo ,
\tempo  =  and \tempo   = ),
which makes it impossible to accept # here because early evaluation
is needed to disambiguate, I think. I'm no expert of the parser
though.

Best,
Jean

shiftDurations and tempo

[Simon Albrecht]

Hello everyone,

I’m trying to encode a piece such that I can switch whether or not 
durations will be shifted (note values halved/doubled). Unfortunately 
\tempo isn’t affected by \shiftDurations (that would be a sensible 
feature request, right?), so I tried this:


%%%
\version "2.23.9"

durationShiftOne = #-1
{
  \tempo #(ly:make-duration (+ 3 durationShiftOne)) = 152
  \shiftDurations \durationShiftOne #0 { 8 }
}


but unfortunately Lily won’t accept that Scheme expression instead of 
the normal duration.


Can I make that work some other way?

Should the parser be able to take that Scheme expression there?

Best, Simon