Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

Okay, thank you


22 Şubat 2021 Pazartesi tarihinde Chet Ramey  yazdı:

> On 2/22/21 12:59 AM, Oğuz wrote:
>
>>
>>
>> 21 Şubat 2021 Pazar tarihinde Chet Ramey > chet.ra...@case.edu>> yazdı:
>>
>> On 2/21/21 3:05 PM, Oğuz wrote:
>>
>> With the latest push to devel bash now accepts `if ! [[ ... ]]
>> then
>> :; fi' too, but not `:() ! [[ ... ]]'. Is `! [[ ... ]]' (or `time
>> [[ ... ]]') a valid compound command or not?
>>
>>
>> Technically not; they're pipelines.
>>
>>
>> So, if the last component of a pipeline is a compound command, a
>> separator between the pipeline and `then', `fi', `do', `done', or `esac' is
>> optional,
>>
>
> It's really a reserved word that terminates a command that makes the
> closing `esac' optional.
>
>
> but a pipeline alone can not be a function body no matter what.
>>
>
> Correct. The POSIX grammar makes this fairly clear, its issues with
> `rule 4' aside.
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>


-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/22/21 12:59 AM, Oğuz wrote:



21 Şubat 2021 Pazar tarihinde Chet Ramey > yazdı:


On 2/21/21 3:05 PM, Oğuz wrote:

With the latest push to devel bash now accepts `if ! [[ ... ]] then
:; fi' too, but not `:() ! [[ ... ]]'. Is `! [[ ... ]]' (or `time
[[ ... ]]') a valid compound command or not?


Technically not; they're pipelines.


So, if the last component of a pipeline is a compound command, a separator 
between the pipeline and `then', `fi', `do', `done', or `esac' is optional,


It's really a reserved word that terminates a command that makes the
closing `esac' optional.



but a pipeline alone can not be a function body no matter what.


Correct. The POSIX grammar makes this fairly clear, its issues with
`rule 4' aside.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

21 Şubat 2021 Pazar tarihinde Chet Ramey  yazdı:

> On 2/21/21 3:05 PM, Oğuz wrote:
>
>> With the latest push to devel bash now accepts `if ! [[ ... ]] then :;
>> fi' too, but not `:() ! [[ ... ]]'. Is `! [[ ... ]]' (or `time [[ ... ]]')
>> a valid compound command or not?
>>
>
> Technically not; they're pipelines.


So, if the last component of a pipeline is a compound command, a separator
between the pipeline and `then', `fi', `do', `done', or `esac' is optional,
but a pipeline alone can not be a function body no matter what. Right?


> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>


-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/21/21 3:05 PM, Oğuz wrote:
With the latest push to devel bash now accepts `if ! [[ ... ]] then :; fi' 
too, but not `:() ! [[ ... ]]'. Is `! [[ ... ]]' (or `time [[ ... ]]') a 
valid compound command or not?


Technically not; they're pipelines.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

With the latest push to devel bash now accepts `if ! [[ ... ]] then :; fi'
too, but not `:() ! [[ ... ]]'. Is `! [[ ... ]]' (or `time [[ ... ]]') a
valid compound command or not?

-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/17/21 8:07 PM, Koichi Murase wrote:


Now, they are fixed by Chet in the latest devel branch. Chet, I'm
sorry that it seems I have pushed you, and thank you very much for
your hard work.


If I get interested in something, and I have the time, I'll work on it even
if there's no practical benefit.


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/17/21 6:40 PM, Robert Elz wrote:


It isn't all that hard, but isn't trivial
either, and amounts to quite a bit of code
(and parsing all the bash extensions the
way it does would make it harder and bigger)


Why would I write an entirely new parser when I already have one that works
just fine?


ps:  I am sure Chet will confirm that I am not
shy about pointing out bash bugs/defects that
shoukd be fixed when I come accross one.


Can confirm.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Lawrence Velázquez]

> On Feb 17, 2021, at 10:27 PM, Dale R. Worley  wrote:
> 
> Lawrence Velázquez  writes:
 `;;' is optional for the last case item.
>>> 
>>> The manual page (for my version) says it's required. If, in some
>>> certain circumstances, it works without, that's nice.
>> 
>> It's also required by POSIX.
> 
> Ah, now that's different.  Has the current man page been updated to
> match?

No.

https://www.gnu.org/software/bash/manual/html_node/Conditional-Constructs.html#index-case

Bash has permitted omitting the final ;; since the beginning (more
or less). I assume the documentation omits that detail for a reason
-- perhaps to discourage its use.

vq

Re: syntax error while parsing a case command within `$(...)'

[Dale R. Worley]

Lawrence Velázquez  writes:
>>> `;;' is optional for the last case item.
>> 
>> The manual page (for my version) says it's required. If, in some
>> certain circumstances, it works without, that's nice.
>
> It's also required by POSIX.

Ah, now that's different.  Has the current man page been updated to
match?

Dale

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月18日(木) 7:43 Robert Elz :
> From scratch, no, but I maintain one.
>
> With all the details
>
> It isn't all that hard, but isn't trivial
> either, and amounts to quite a bit of code
> (and parsing all the bash extensions the
> way it does would make it harder and bigger)
>
> Like everything, it becomes a cost/benefit
> tradeoff, are the costs worth the benefits?

Yeah, right. I was so sleepy last night (in my timezone) that I have
written things that just came to my mind without thinking deeply,
sorry. The fixing cost actually depends on many factors such as how it
is implemented currently, how busy the maintainer is, ...

> That one is not for me (or you) to answer.

Now, they are fixed by Chet in the latest devel branch. Chet, I'm
sorry that it seems I have pushed you, and thank you very much for
your hard work.

--
Koichi

Re: syntax error while parsing a case command within `$(...)'

[Robert Elz]

Date:Thu, 18 Feb 2021 01:34:59 +0800
From:Koichi Murase 
Message-ID:  


  | ever written a shell parser?

From scratch, no, but I maintain one.

With all the details

It isn't all that hard, but isn't trivial
either, and amounts to quite a bit of code
(and parsing all the bash extensions the
way it does would make it harder and bigger)

Like everything, it becomes a cost/benefit
tradeoff, are the costs worth the benefits?

That one is not for me (or you) to answer.

kre

ps:  I am sure Chet will confirm that I am not
shy about pointing out bash bugs/defects that
shoukd be fixed when I come accross one.

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

 2021年2月18日(木) 0:56 Robert Elz :
> For a long time [...]
>
> [...] And it all has to be recursive,
> as $() cmdsubs can occur in the middle of words in the middle of other 
> cndsubs.

Thank you for the historical story.

> Chet can explain more if he feels it is warranted, but I believe that even
> today, bash uses a very heuristic type of almost pretend to be a parser
> to deal with these things

Yes, that is done in `parse_comsub' in `parse.y' which I mentioned in
some earlier reply in this tree.

> None of the (current anyway) problems represent code that anyone trying
> to solve a real problem (as distinct from stressing the parser) would
> ever write, so one can understand a certain reluctance to spending a lot
> of time fixing things that only even happen in stress tests.

I actually doubt that it takes so much time to fix them... Have you
ever written a shell parser? Some time ago I have written a simple
shell parser mimicking the Bash behavior without thinking the details,
but it correctly parses all the command substitutions reported here.

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/17/21 11:56 AM, Robert Elz wrote:


Chet can explain more if he feels it is warranted, but I believe that even
today, bash uses a very heuristic type of almost pretend to be a parser
to deal with these things - not really parsing them fully (not until they're
to be executed, where the issues all go away, as at that point, nothing
else is being parsed) but attempting to handle most cases so things work
for real code.


I'd add that the reason I did this back in 2008 was that it was just
about impossible to have a re-entrant bison parser (one that calls
yyparse() recursively) and completely impossible to do portably with
yacc. The situation with bison may have improved.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/17/21 9:53 AM, Oğuz wrote:
16 Şubat 2021 Salı tarihinde Chet Ramey > yazdı:


This is just academic. No sane person would write these without some kind
of delimiter.


Yeah, `parse_comsub' really needs some work anyway.


Don't worry.


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Robert Elz]

Date:Wed, 17 Feb 2021 23:39:07 +0800
From:Koichi Murase 
Message-ID:  


Andreas answered the technical part of your message, and I agree
with that.

  | Anyway, even if the POSIX specifies some inconsistent grammar, I feel
  | that cannot be the reason that Bash behaves in a way even more
  | inconsistent than what POSIX specifies.

For a long time it was believed (incorrectly) that () counting & matching
was sufficient to find the end of a $() command substitution - just as the
next unescaped ` finds the end of a `` cmdsub.   Parsing these things is
a bit tricky, as they occur in the middle of a word, which is nominally a 
lexical token (an indivisible unit to the grammar) - and so needs either
fairly horrid tricks, or a parser generator which allows suspending its
current parse and running a new one, so the end of the cmdsub can be
correctly found (for the $() case).   Because that's so hard, and because
() counting was assumed to work, some shells (I believe including bash
initially, though I have never examined its code) used that technique.

That doesn't work however, some kind of parser is required, not just character
scanning for parentheses (and quoting).And it all has to be recursive,
as $() cmdsubs can occur in the middle of words in the middle of other cndsubs.

Chet can explain more if he feels it is warranted, but I believe that even
today, bash uses a very heuristic type of almost pretend to be a parser
to deal with these things - not really parsing them fully (not until they're
to be executed, where the issues all go away, as at that point, nothing
else is being parsed) but attempting to handle most cases so things work
for real code.

This is why commands that parse file and work properly outside cmdsubs
don't always parse correctly inside (while parsing the command that
contains them) - they're being done by two completely different parsers,
one that is correct, and works, and the other than is just mostly OK.

None of the (current anyway) problems represent code that anyone trying
to solve a real problem (as distinct from stressing the parser) would
ever write, so one can understand a certain reluctance to spending a lot
of time fixing things that only even happen in stress tests.

Really, who ever writes a real case statement with no patterns in it?

kre

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

17 Şubat 2021 Çarşamba tarihinde Koichi Murase 
yazdı:

> 2021年2月17日(水) 1:52 Oğuz :
> > They resemble `[ ... ]' (I know it's a simple command, but still), maybe
> that's why. I think it'd seem more inconsistent to ordinary user if
> >
> >   if [[ x ]] then [[ x ]] fi
> >
> > worked but
> >
> >   if [ x ] then [ x ] fi
> >
> > didn't.
>
> ``Ordinary users'' may consider it being inconsistent, but what would
> you think of this?


I wouldn't mind. I always put a semicolon before `then' anyway.


>
> Actually, as you have written, [ ... ] is a simple command, so there
> are already many differences that ``ordinary users'' would think
> inconsistent. For example, the word splitting and pathname expansion
> rules are different. For another example, `tempenv=1 [[ -v tempenv ]]'
> is disallowed while `tempenv=1 [ -v tempenv ]' is allowed. and more...
> Letting [[ ... ]] behave differently from either of [ ... ] and ( ...
> ) means that we create the third category of command which
> grammatically behaves like neither the simple command nor the compound
> command.


Yes, this makes sense. You obviously put more thought into this than I did,
my earlier post was just the first thing I could think of that is against
what you suggested.


-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Andreas Schwab]

On Feb 17 2021, Koichi Murase wrote:

> Yes, that's what I wanted to claim by the above example, that is, XCU
> 2.4 isn't the (most) essential point for the reason that `fi esac' or
> `fi fi` is a valid construct. The above example is just a
> counter-example against explaining `fi esac` by solely XCU 2.4.

XCU 2.4 only explains when words are to be recognized as reserved words
(the lexer part of parsing).  Only because of these rules the `fi esac'
example contains two consecutive reserved words.  The grammar depends on
having these rules in place, and then applies more rules on the
placement of tokens.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月17日(水) 1:52 Oğuz :
> They resemble `[ ... ]' (I know it's a simple command, but still), maybe 
> that's why. I think it'd seem more inconsistent to ordinary user if
>
>   if [[ x ]] then [[ x ]] fi
>
> worked but
>
>   if [ x ] then [ x ] fi
>
> didn't.

``Ordinary users'' may consider it being inconsistent, but what would
you think of this?

Actually, as you have written, [ ... ] is a simple command, so there
are already many differences that ``ordinary users'' would think
inconsistent. For example, the word splitting and pathname expansion
rules are different. For another example, `tempenv=1 [[ -v tempenv ]]'
is disallowed while `tempenv=1 [ -v tempenv ]' is allowed. and more...
Letting [[ ... ]] behave differently from either of [ ... ] and ( ...
) means that we create the third category of command which
grammatically behaves like neither the simple command nor the compound
command. I actually feel it is more consistent to allow `if [[ ... ]]
then' as far as we accept `if ( ... ) then'. That would be the only
neat thing under the POSIX constraints.

--
Koichi

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月17日(水) 21:56 Robert Elz :
>   | For example, this alone doesn't explain why
>   |
>   | $ if :; then echo A; fi if :; then echo A; fi
>   |
>   | (i.e., the combination "fi if") is a syntax error.
>
> That one is quite a different issue,

Yes, that's what I wanted to claim by the above example, that is, XCU
2.4 isn't the (most) essential point for the reason that `fi esac' or
`fi fi` is a valid construct. The above example is just a
counter-example against explaining `fi esac` by solely XCU 2.4. I used
`fi if' as an example because I did know that `fi if' isn't allowed by
the grammar, so I have referred to the grammar in the next paragraph
in my previous reply. Instead, as you have explained in your next
paragraph, the essential reason why `fi esac' can appear in a valid
command is that the grammatical structure allows the combination of
`fi esac' as a result of some embedded compound command in another.

> ps: many of these strange rules all originate in the original Bourne
> shell's recursive descent parser,

I was also thinking these apparent rules are strange before carefully
reading the BNF (in yacc style) specified in XCU 2.10, but now I
somehow feel these rules are understandable (but still not so natural
of course). I admit that it is confusing to most users, but at least,
I wouldn't say it's ``insane'' now.

> [...] and consequently, POSIX, which documents
> what actually exists, specifies it as well.

Thank you for the explanation of the historical backgrounds. That's
actually exactly what I guessed when I looked at the BNF.

Anyway, even if the POSIX specifies some inconsistent grammar, I feel
that cannot be the reason that Bash behaves in a way even more
inconsistent than what POSIX specifies.

--
Koichi

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

16 Şubat 2021 Salı tarihinde Chet Ramey  yazdı:
>
> This is just academic. No sane person would write these without some kind
> of delimiter.
>

Yeah, `parse_comsub' really needs some work anyway.

  $ ./bash -c '$(case x in x|esac) foo;; esac)'
  ./bash: -c: line 1: syntax error near unexpected token `;;'
  ./bash: -c: line 1: `$(case x in x|esac) foo;; esac)'


-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Robert Elz]

Date:Tue, 16 Feb 2021 18:19:48 +0800
From:Koichi Murase 
Message-ID:  


  | For example, this alone doesn't explain why
  |
  | $ if :; then echo A; fi if :; then echo A; fi
  |
  | (i.e., the combination "fi if") is a syntax error.

That one is quite a different issue, if you look at the grammar
you'll find there's no way to have 2 consecutive commands (no matter
what kind of commands they are) without some kind of separator
between them.   The second "if" there is a total red herring, it makes
no difference what word is there (except one of the legal operators in
that location) - it is going to be a syntax error.

The other examples given in earlier messages have one command
embedded in another, there's no requirement for separators in those
(and they are separators, not terminators).

Your "case x in (x) if :; then echo; fi esac" is the same.

kre

ps: many of these strange rules all originate in the original Bourne
shell's recursive descent parser, which was crammed into a very limited
space, and so didn't always enforce sanity - it made sure that what
needed to work worked, but cared much less about prohibiting things
that are useless, and could easily have been rejected (like case statements
with no patterns) had it been considered worthwhile to spend the code
space needed to make that happen.Since then everyone copies so as
to maintain backward compatibility ("some idiot might have written code
like that, we cannot reject it"), and consequently, POSIX, which documents
what actually exists, specifies it as well.

Re: syntax error while parsing a case command within `$(...)'

[Lawrence Velázquez]

> On Feb 16, 2021, at 10:42 PM, Dale R. Worley  wrote:
> 
>> Oğuz  writes:
>> 
>> `;;' is optional for the last case item.
> 
> The manual page (for my version) says it's required. If, in some
> certain circumstances, it works without, that's nice.


It's also required by POSIX.

> But there's no commitment that it will work now, or in future
> releases.

The commitment is bash's claim to POSIX compliance. Outside of
command substitutions, the referenced construct currently works,
as POSIX requires. I expect this is by design. If it ceases to be
recognized in the future, then bash will have intentionally become
less compliant with POSIX, for no good reason.

>> `case x in esac' (without the linebreak) works fine outside the
>> command substitution.
> 
> The manual page (for my version) says that "esac" will be recognized
> in positions where a simple command may appear.

The 5.1 man page doesn't say that, or I can't find it, but it doesn't
matter because a simple command can't go immediately after that "in".

> If, in some other circumstances, it works, that's nice.


Recognition of "case x in esac" is also required by POSIX.

> But there's no commitment that it will work now, or in future
> releases.

See above.

> Now, if you want to advocate that it *should* always work, go ahead.
> But that's a feature request, not a bug report.

TO: Chet
SUBJECT: Feature request

Please do not make bash less POSIX-compliant for no good reason!

thx
vq

Re: syntax error while parsing a case command within `$(...)'

[Dale R. Worley]

Oğuz  writes:
>> Before we worry about what to change, I want to note that the original
>> example is syntactically incorrect.  The example is
>>
>>   $ bash -c ': $(case x in x) esac)'
>>
>> But the manual page makes it clear that each case must be ended with
>> ";;".
>
> `;;' is optional for the last case item.

The manual page (for my version) says it's required.  If, in some
certain circumstances, it works without, that's nice.  But there's no
commitment that it will work now, or in future releases.

> `case x in esac' (without the linebreak) works fine outside the command
> substitution.

The manual page (for my version) says that "esac" will be recognized in
positions where a simple command may appear.  If, in some other
circumstances, it works, that's nice.  But there's no commitment that it
will work now, or in future releases.

Now, if you want to advocate that it *should* always work, go ahead.
But that's a feature request, not a bug report.

Dale

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

On Tue, Feb 16, 2021 at 5:34 PM Koichi Murase 
wrote:

> Maybe Bash could insist that [[ ... ]] and ((...)) are not part of the
> standard so that it can behave inconsistently.  But it seems to me
> that there is really no reason to behave inconsistently here.
>

They resemble `[ ... ]' (I know it's a simple command, but still), maybe
that's why. I think it'd seem more inconsistent to ordinary user if

  if [[ x ]] then [[ x ]] fi

worked but

  if [ x ] then [ x ] fi

didn't.

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

> XCU 2.10.2/(1) Shell Grammar Rules [Command Name]
> > When the TOKEN is exactly a reserved word, the token identifier for
> > that reserved word shall result. Otherwise, the token WORD shall be
> > returned. Also, if the parser is in any state where only a reserved
> > word could be the next correct token, proceed as above.
>
> [...]
>
> The above special rule means that
>
>   $ if [[ str ]] then [[ str ]] fi
>   $ if ((expr)) then ((expr)) fi
>
> should also be acceped as valid constructs.  Other shells parses them,
> but Bash fails:
>
> [...]
>
> Maybe Bash could insist that [[ ... ]] and ((...)) are not part of the
> standard so that it can behave inconsistently.  But it seems to me
> that there is really no reason to behave inconsistently here.

This is actually an easy fix as attached. I guess this change can be
safely applied because originally these constructs were a syntax
error, so no shell scripts rely on the current behavior.


0001-accept-reserved-words-after-arithmetic-condtional-co.patch
Description: Binary data

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/16/21 12:29 AM, Oğuz wrote:
16 Şubat 2021 Salı tarihinde Koichi Murase > yazdı:


bash -c ': $(case x in esac)'
bash -c ': $(case x in (x) if :; then echo a; fi esac)'
bash -c ': $(case x in (x) a() { echo a; } esac)'
bash -c ': $(case x in (x) for ((i=0;i<10;i++)) { echo a; } esac)'
bash -c ': $(case x in (x) while false; do echo a; done esac)'
bash -c ': $(case x in (x) case y in (y) echo a;; esac esac)'


I'm lost. Are these even supposed to work outside command substitution? Why?


The same principle that you articulated earlier: the last command-list
doesn't need to be delimited by a DSEMI as long as the esac is recognized
as a separate token, and then it goes on from there to the conditions
under which reserved words are recognized.

This is just academic. No sane person would write these without some kind
of delimiter.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月16日(火) 21:41 Oğuz :
>> > XRAT Shell Grammar (
>> > https://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xcu_chap02.html#tag_23_02_10
>> > ) explains that giving almost the same examples as we posted in
>> > this thread.
>>
>> Thank you, interesting! So, if I understand correctly, the following
>> construct which Bash allows is finally disallowed by POSIX?
>>
>> $ if (false) then (echo x) else (echo y) fi
>
> No, it is allowed.

OK, I have again checked the page. I haven't read the latter half of
the section.  The special rule is

XCU 2.10.2/(1) Shell Grammar Rules [Command Name]
> When the TOKEN is exactly a reserved word, the token identifier for
> that reserved word shall result. Otherwise, the token WORD shall be
> returned. Also, if the parser is in any state where only a reserved
> word could be the next correct token, proceed as above.

It seems like XCU 2.4 is telling lie..., or at least some important
information is missing.  Is XCU 2.4 just informative texts?  Instead,
is XCU 2.10 solely the actual normative requirements for the shell
grammar?

The above special rule means that

  $ if [[ str ]] then [[ str ]] fi
  $ if ((expr)) then ((expr)) fi

should also be acceped as valid constructs.  Other shells parses them,
but Bash fails:

  $ bash -c 'if [[ str ]] then [[ str ]] fi'; echo $?
  bash: -c: line 0: syntax error near unexpected token `then'
  bash: -c: line 0: `if [[ str ]] then [[ str ]] fi'
  1
  $ zsh -c 'if [[ str ]] then [[ str ]] fi'; echo $?
  0
  $ ksh -c 'if [[ str ]] then [[ str ]] fi'; echo $?
  0
  $ yash -c 'if [[ str ]] then [[ str ]] fi'; echo $?
  0

  $ bash -c 'if ((expr)) then ((expr)) fi'; echo $?
  bash: -c: line 0: syntax error near unexpected token `then'
  bash: -c: line 0: `if ((expr)) then ((expr)) fi'
  1
  $ zsh -c 'if ((expr)) then ((expr)) fi'; echo $?
  0
  $ ksh -c 'if ((expr)) then ((expr)) fi'; echo $?
  0

  Note: yash doesn't support arithmetic commands.

Maybe Bash could insist that [[ ... ]] and ((...)) are not part of the
standard so that it can behave inconsistently.  But it seems to me
that there is really no reason to behave inconsistently here.

--
Koichi

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

16 Şubat 2021 Salı tarihinde Koichi Murase  yazdı:

> 2021年2月16日(火) 19:24 Oğuz :
> > XRAT Shell Grammar (  https://pubs.opengroup.org/
> onlinepubs/9699919799/xrat/V4_xcu_chap02.html#tag_23_02_10 ) explains
> that giving almost the same examples as we posted in this thread.
>
> Thank you, interesting! So, if I understand correctly, the following
> construct which Bash allows is finally disallowed by POSIX?
>
> $ if (false) then (echo x) else (echo y) fi


No, it is allowed.


>
> To summarize, the following cases are required by the POSIX standard
> to be correctly parsed?
>
> $ bash -c ': $(case x in esac)'
> $ bash -c ': $(case x in (x) if :; then echo a; fi esac)'
> $ bash -c ': $(case x in (x) a() { echo a; } esac)'
> $ bash -c ': $(case x in (x) { echo a; } esac)'
> $ bash -c ': $(case x in (x) while false; do echo a; done esac)'
> $ bash -c ': $(case x in (x) case y in (y) echo a;; esac esac)'


Yes.


>
> and this case
>
> $ bash -c ': $(case x in (x) (echo a) esac)'
>
> could be a Bash extension.
>

No.


-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月16日(火) 19:24 Oğuz :
> XRAT Shell Grammar (  
> https://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xcu_chap02.html#tag_23_02_10
>  ) explains that giving almost the same examples as we posted in this thread.

Thank you, interesting! So, if I understand correctly, the following
construct which Bash allows is finally disallowed by POSIX?

$ if (false) then (echo x) else (echo y) fi

To summarize, the following cases are required by the POSIX standard
to be correctly parsed?

$ bash -c ': $(case x in esac)'
$ bash -c ': $(case x in (x) if :; then echo a; fi esac)'
$ bash -c ': $(case x in (x) a() { echo a; } esac)'
$ bash -c ': $(case x in (x) { echo a; } esac)'
$ bash -c ': $(case x in (x) while false; do echo a; done esac)'
$ bash -c ': $(case x in (x) case y in (y) echo a;; esac esac)'

and this case

$ bash -c ': $(case x in (x) (echo a) esac)'

could be a Bash extension. But actually, it totally depends on what is
"the matching closing parenthesis" in the description of the command
substitution by the POSIX:

XCU 2.6.3 Command Substitution
> With the $(command) form, all characters following the open
> parenthesis to the matching closing parenthesis constitute the
> command.

I guess the rationale of the current Bash behavior is that "the
matching closing parenthesis" in the context of the command
substitution is actually not strictly defined in the standard.

--
Koichi

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月16日(火) 19:29 Chris Elvidge :
> > I think that section just describes the necessary condition that the
> > word is recognized as the reserved keyword: "This recognition shall
> > *only* occur...". It doesn't explain whether the reserved keyword can
> > be really used to construct the AST following the shell grammar. For
> > example, this alone doesn't explain why
> >
> > $ if :; then echo A; fi if :; then echo A; fi
> >
> > (i.e., the combination "fi if") is a syntax error.
>
> It seems to me that you're missing a semicolon between 'fi' and 'if' to
> delineate the two constructs.

Yes, that's an example of a syntax error that I made by intentionally
removing the semicolon. If we just look at POSIX XCU 2.4 which Andreas
pointed to justify "fi fi", a similar combination "fi if" without
semicolon in between appears to be also justified, but that's false.
Thus there should be also other essential rules. That's what I wanted
to say.

By the way, in the case of nested compounds such as `case x in (x) if
:; then echo; fi esac' or `{ { echo; } }', the fact was actually
somewhat opposite. According to the formal grammar by BNF (without
caring XCU 2.4), actually, semicolonless commands like

$ case x in (x) if : then echo fi esac

or

$ { { echo } }

are valid. However, it is ambiguous and confusing, so this kind of
construct is afterward disallowed by the grammatical "Constraints"
specified in XCU 2.4.

--
Koichi

Re: syntax error while parsing a case command within `$(...)'

[Chris Elvidge]

On 16/02/2021 10:19 am, Koichi Murase wrote:

2021年2月16日(火) 16:28 Andreas Schwab :

See 2.4 Reserved Words.

 This recognition shall only occur when none of the characters is
 quoted and when the word is used as:


I think that section just describes the necessary condition that the
word is recognized as the reserved keyword: "This recognition shall
*only* occur...". It doesn't explain whether the reserved keyword can
be really used to construct the AST following the shell grammar. For
example, this alone doesn't explain why

$ if :; then echo A; fi if :; then echo A; fi

(i.e., the combination "fi if") is a syntax error.


It seems to me that you're missing a semicolon between 'fi' and 'if' to 
delineate the two constructs.




Hmm, it seems we need to combine the Yacc-BNF notation at the end of
Sec. 2.10 to understand this? For example,

$ case x in (x) if :; then echo; fi esac

can be derived as

case_clause
-> Case WORD linebreak in linebreak case_list Esac
-> Case 'x' ε in ε case_list Esac
-> Case 'x' ε in ε case_item_ns Esac
-> Case 'x' ε in ε '(' pattern ')' compound_list Esac
-> Case 'x' ε in ε '(' 'x' ')' linebreak term Esac
-> Case 'x' ε in ε '(' 'x' ')' ε and_or Esac
-> Case 'x' ε in ε '(' 'x' ')' ε pipeline Esac
-> Case 'x' ε in ε '(' 'x' ')' ε pipe_sequence Esac
-> Case 'x' ε in ε '(' 'x' ')' ε command Esac
-> Case 'x' ε in ε '(' 'x' ')' ε compound_command Esac
-> Case 'x' ε in ε '(' 'x' ')' ε if_clause Esac
-> ...

where ε is an empty string. OK, now I understood this behavior is
actually required by the POSIX standard. Can we find any textual
explanation on this rule? Or maybe this behavior is intuitive enough
for those who understand the shell grammar so that they don't see the
necessity of an additional explanation...

Thank you for the comment!

--
Koichi





--
Chris Elvidge
England

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

16 Şubat 2021 Salı tarihinde Koichi Murase  yazdı:

> OK, now I understood this behavior is
> actually required by the POSIX standard. Can we find any textual
> explanation on this rule? Or maybe this behavior is intuitive enough
> for those who understand the shell grammar so that they don't see the
> necessity of an additional explanation...
>

XRAT Shell Grammar (
https://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xcu_chap02.html#tag_23_02_10
) explains that giving almost the same examples as we posted in this thread.


-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月16日(火) 16:28 Andreas Schwab :
> See 2.4 Reserved Words.
>
> This recognition shall only occur when none of the characters is
> quoted and when the word is used as:

I think that section just describes the necessary condition that the
word is recognized as the reserved keyword: "This recognition shall
*only* occur...". It doesn't explain whether the reserved keyword can
be really used to construct the AST following the shell grammar. For
example, this alone doesn't explain why

$ if :; then echo A; fi if :; then echo A; fi

(i.e., the combination "fi if") is a syntax error.

Hmm, it seems we need to combine the Yacc-BNF notation at the end of
Sec. 2.10 to understand this? For example,

$ case x in (x) if :; then echo; fi esac

can be derived as

case_clause
-> Case WORD linebreak in linebreak case_list Esac
-> Case 'x' ε in ε case_list Esac
-> Case 'x' ε in ε case_item_ns Esac
-> Case 'x' ε in ε '(' pattern ')' compound_list Esac
-> Case 'x' ε in ε '(' 'x' ')' linebreak term Esac
-> Case 'x' ε in ε '(' 'x' ')' ε and_or Esac
-> Case 'x' ε in ε '(' 'x' ')' ε pipeline Esac
-> Case 'x' ε in ε '(' 'x' ')' ε pipe_sequence Esac
-> Case 'x' ε in ε '(' 'x' ')' ε command Esac
-> Case 'x' ε in ε '(' 'x' ')' ε compound_command Esac
-> Case 'x' ε in ε '(' 'x' ')' ε if_clause Esac
-> ...

where ε is an empty string. OK, now I understood this behavior is
actually required by the POSIX standard. Can we find any textual
explanation on this rule? Or maybe this behavior is intuitive enough
for those who understand the shell grammar so that they don't see the
necessity of an additional explanation...

Thank you for the comment!

--
Koichi

Re: syntax error while parsing a case command within `$(...)'

[Andreas Schwab]

On Feb 16 2021, Koichi Murase wrote:

> Because the ending keywords---}, fi, done, and esac---are allowed to
> be chained, which I think is undocumented though.

See 2.4 Reserved Words.

This recognition shall only occur when none of the characters is
quoted and when the word is used as:

* The first word of a command
* The first word following one of the reserved words other than
  case, for, or in
* The third word in a case command (only in is valid in this case)
* The third word in a for command (only in and do are valid in this
  case)

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月16日(火) 13:56 Oğuz :
> $ if (:) then (:) fi
> $
>
> I had no idea.

Oh, I forgot about `(list)'. I should also add this to the list:

$ bash -c ': $(case x in (x) (echo a) esac)'

--
Koichi

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

On Tue, Feb 16, 2021 at 8:34 AM Koichi Murase 
wrote:

> Because the ending keywords---}, fi, done, and esac---are allowed to
> be chained, which I think is undocumented though.
>
> $ { { { { { echo "Something like this is allowed"; } } } } }
>

Huh. So a compound list consisting of a single compound command and no
separator is still a valid compound list.

$ if (:) then (:) fi
$

I had no idea.


>
> --
> Koichi
>

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月16日(火) 13:29 Oğuz :
> 16 Şubat 2021 Salı tarihinde Koichi Murase  yazdı:
>> bash -c ': $(case x in esac)'
>> bash -c ': $(case x in (x) if :; then echo a; fi esac)'
>> bash -c ': $(case x in (x) a() { echo a; } esac)'
>> bash -c ': $(case x in (x) for ((i=0;i<10;i++)) { echo a; } esac)'
>> bash -c ': $(case x in (x) while false; do echo a; done esac)'
>> bash -c ': $(case x in (x) case y in (y) echo a;; esac esac)'
>
> I'm lost. Are these even supposed to work outside command substitution? Why?

Because the ending keywords---}, fi, done, and esac---are allowed to
be chained, which I think is undocumented though.

$ { { { { { echo "Something like this is allowed"; } } } } }

--
Koichi

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

16 Şubat 2021 Salı tarihinde Koichi Murase  yazdı:
>
> bash -c ': $(case x in esac)'
> bash -c ': $(case x in (x) if :; then echo a; fi esac)'
> bash -c ': $(case x in (x) a() { echo a; } esac)'
> bash -c ': $(case x in (x) for ((i=0;i<10;i++)) { echo a; } esac)'
> bash -c ': $(case x in (x) while false; do echo a; done esac)'
> bash -c ': $(case x in (x) case y in (y) echo a;; esac esac)'
>

I'm lost. Are these even supposed to work outside command substitution? Why?


-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/15/21 5:39 PM, Koichi Murase wrote:

2021年2月16日(火) 5:12 Chet Ramey :

It is unexpected, though the syntax is obscure. I'll fix it.


If this is to be fixed, maybe also the following cases could be
handled better (if the corresponding fixes are not too complicated).


At some point, maybe. They're going to be extremely low priority.


bash -c ': $(case x in esac)'
bash -c ': $(case x in (x) if :; then echo a; fi esac)'
bash -c ': $(case x in (x) a() { echo a; } esac)'
bash -c ': $(case x in (x) for ((i=0;i<10;i++)) { echo a; } esac)'


We've officially hit the point of diminishing returns.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月16日(火) 5:12 Chet Ramey :
> It is unexpected, though the syntax is obscure. I'll fix it.

If this is to be fixed, maybe also the following cases could be
handled better (if the corresponding fixes are not too complicated).

bash -c ': $(case x in esac)'
bash -c ': $(case x in (x) if :; then echo a; fi esac)'
bash -c ': $(case x in (x) a() { echo a; } esac)'
bash -c ': $(case x in (x) for ((i=0;i<10;i++)) { echo a; } esac)'

I know that those are also undocumented (AFAIK) and obscure cases, but
they are all accepted outside the command substitution. Also, the
following similar cases are currently handled without syntax errors as
expected.

bash -c ': $(case x in (x) while false; do echo a; done esac)'
bash -c ': $(case x in (x) case y in (y) echo a;; esac esac)'

--
Koichi

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/15/21 4:35 PM, Lawrence Velázquez wrote:

On Feb 15, 2021, at 10:01 AM, Chet Ramey  wrote:

The bash command substitution parser handles the majority of these
cases (heh)


lol I chuckled, well done


I'll be here all week.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Lawrence Velázquez]

> On Feb 15, 2021, at 10:01 AM, Chet Ramey  wrote:
> 
> The bash command substitution parser handles the majority of these
> cases (heh)

lol I chuckled, well done

vq

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/13/21 3:21 PM, Oğuz wrote:

   $ bash -c ': $(case x in x) esac)'
   bash: -c: line 1: unexpected EOF while looking for matching `)'
   bash: -c: line 2: syntax error: unexpected end of file

`case x in x) esac' is a complete case command, a syntax error is
unexpected here.


It is unexpected, though the syntax is obscure. I'll fix it.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/14/21 6:02 AM, Robert Elz wrote:


   | I guess you are using Bash for so many years,

Yes, since Paul Fox created and maintained it (version 1).   It allowed
me to escape from csh.


(It's Brian Fox.)

How long ago, and do you have any old source versions lying around? I'm
always trying to get old versions of bash. The oldest I have is bash-1.02.
Regrettably, neither Brian nor I saved copies of the really old ones.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Chet Ramey]

On 2/13/21 5:21 PM, Robert Elz wrote:



   |   $ bash -c ': $(case x in x) esac)'

This is a well known bash deficiency.  When parsing command substitutions
it (approximately) simply counts (unquoted) parentheses to find the end.
Anything with a valid closing ')' but with no opening '(' confuses it.


The situation's not that dire. The bash command substitution parser handles
the majority of these cases (heh) without a problem, including the ones in
the POSIX test suite.

It's only the really obscure cases like this one (esac after right paren
ending a pattern list) that it doesn't handle. I don't expect that to be a
problem to fix.


This problem has been known for a LONG time, without it being fixed.


This particular example has never been reported. I infer from that that
it's not a widely used syntax variant.


Fixing it would take major work in the parser, so the chances of it
happening are not all that great.


Yes, that's the problem. Yacc-based parsers are not easily re-entrant, and
it takes major work to make them so. Bison is supposed to be able to
generate `pure' parsers, but I haven't looked at that for a long time. (I
tried it once, years ago, but there were problems with making it work.)


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: syntax error while parsing a case command within `$(...)'

[Andreas Schwab]

On Feb 14 2021, Eli Schwartz wrote:

> On 2/14/21 3:35 PM, Andreas Schwab wrote:
>> On Feb 14 2021, Eli Schwartz wrote:
>> 
>>> Just a running trend that esac does not get recognized without a
>>> separator.
>> 
>> $ bash -c 'case x in x) esac'
>
> The thread title does mention $(...) as the relevant context up for
> discussion. Apologies if my terseness made you think I've moved off of
> the thread subject; I did not do so.

It's still a counterexample.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: syntax error while parsing a case command within `$(...)'

[Eli Schwartz]

On 2/14/21 3:35 PM, Andreas Schwab wrote:
> On Feb 14 2021, Eli Schwartz wrote:
> 
>> Just a running trend that esac does not get recognized without a
>> separator.
> 
> $ bash -c 'case x in x) esac'

The thread title does mention $(...) as the relevant context up for
discussion. Apologies if my terseness made you think I've moved off of
the thread subject; I did not do so.

-- 
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User



OpenPGP_signature
Description: OpenPGP digital signature

Re: syntax error while parsing a case command within `$(...)'

[Andreas Schwab]

On Feb 14 2021, Eli Schwartz wrote:

> Just a running trend that esac does not get recognized without a
> separator.

$ bash -c 'case x in x) esac'

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: syntax error while parsing a case command within `$(...)'

[Lawrence Velázquez]

> On Feb 14, 2021, at 3:00 PM, Oğuz  wrote:
> 
> 14 Şubat 2021 Pazar tarihinde Dale R. Worley  yazdı:
> 
>> Before we worry about what to change, I want to note that the original
>> example is syntactically incorrect.  The example is
>> 
>>  $ bash -c ': $(case x in x) esac)'
>> 
>> But the manual page makes it clear that each case must be ended with
>> ";;".
> 
> 
> `;;' is optional for the last case item.

For reference:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_04_05

For comparison:

% ksh -c ': $(case x in x) esac)'; echo $?
0
% zsh -c ': $(case x in x) esac)'; echo $?
0
% dash -c ': $(case x in x) esac)'; echo $?
0
% yash -c ': $(case x in x) esac)'; echo $?
0
% bash -c ': $(case x in x) esac)'; echo $?
bash: -c: line 1: unexpected EOF while looking for matching `)'
bash: -c: line 2: syntax error: unexpected end of file
2

vq

Re: syntax error while parsing a case command within `$(...)'

[Eli Schwartz]

On 2/14/21 2:43 PM, Dale R. Worley wrote:
> Before we worry about what to change, I want to note that the original
> example is syntactically incorrect.  The example is
> 
>   $ bash -c ': $(case x in x) esac)'
> 
> But the manual page makes it clear that each case must be ended with
> ";;".
> 
>case word in [ [(] pattern [ | pattern ] ... ) list ;; ] ... esac
> 
> Now, I haven't investigated what cleverness Bash uses, but all the cases
> I've tested that conform to the case syntax are handled correctly inside
> this $(...):
> 
> $ bash -c ': $( case x in x) : ;; esac )'
> $ bash -c ': $( case x in x) true ;; esac )'
> $ bash -c ': $( case x in (x) true ;; esac )'
> 
> It even works with the degenerate case where there are no coices, though
> writing it is hard because "esac" is a keyword:
> 
> $ bash -c ': $( case x in 
> more> esac )'
> 
> This is with an old version, 4.2.53(1).

$ bash -c ': $( case x in x)
esac ); echo $?'
0


Doesn't seem like a special power of your degenerate case. Just a
running trend that esac does not get recognized without a separator.

-- 
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User



OpenPGP_signature
Description: OpenPGP digital signature

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

14 Şubat 2021 Pazar tarihinde Dale R. Worley  yazdı:

> Before we worry about what to change, I want to note that the original
> example is syntactically incorrect.  The example is
>
>   $ bash -c ': $(case x in x) esac)'
>
> But the manual page makes it clear that each case must be ended with
> ";;".


`;;' is optional for the last case item.


>
>case word in [ [(] pattern [ | pattern ] ... ) list ;; ] ... esac
>
> Now, I haven't investigated what cleverness Bash uses, but all the cases
> I've tested that conform to the case syntax are handled correctly inside
> this $(...):
>
> $ bash -c ': $( case x in x) : ;; esac )'
> $ bash -c ': $( case x in x) true ;; esac )'
> $ bash -c ': $( case x in (x) true ;; esac )'
>
> It even works with the degenerate case where there are no coices, though
> writing it is hard because "esac" is a keyword:
>
> $ bash -c ': $( case x in
> more> esac )'


`case x in esac' (without the linebreak) works fine outside the command
substitution.


> This is with an old version, 4.2.53(1).
>
> Dale
>


-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Dale R. Worley]

Before we worry about what to change, I want to note that the original
example is syntactically incorrect.  The example is

  $ bash -c ': $(case x in x) esac)'

But the manual page makes it clear that each case must be ended with
";;".

   case word in [ [(] pattern [ | pattern ] ... ) list ;; ] ... esac

Now, I haven't investigated what cleverness Bash uses, but all the cases
I've tested that conform to the case syntax are handled correctly inside
this $(...):

$ bash -c ': $( case x in x) : ;; esac )'
$ bash -c ': $( case x in x) true ;; esac )'
$ bash -c ': $( case x in (x) true ;; esac )'

It even works with the degenerate case where there are no coices, though
writing it is hard because "esac" is a keyword:

$ bash -c ': $( case x in 
more> esac )'

This is with an old version, 4.2.53(1).

Dale

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

14 Şubat 2021 Pazar tarihinde Robert Elz  yazdı:
>
> That's a step up on outright rejecting them, which they do from time
> to time.   I don't much care, anyone who really cares about receiving
> e-mail shouldn't be using gmail.
>

What else am I gonna use? It's free


-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Robert Elz]

Date:Sun, 14 Feb 2021 13:14:29 +
From:Stephane Chazelas 
Message-ID:  <20210214131429.yxr5egs7zs6fe...@chazelas.org>

  | ITYM Brian Fox.

Yes, probably ... my memory is pathetic, and that was all a
long time ago.

kre

Re: syntax error while parsing a case command within `$(...)'

[Stephane Chazelas]

2021-02-14 18:02:52 +0700, Robert Elz:
[...]
>   | I guess you are using Bash for so many years,
> 
> Yes, since Paul Fox created and maintained it (version 1).   It allowed
> me to escape from csh.

ITYM Brian Fox. Maybe the confusion comes from zsh's Paul
Falstad.

-- 
Stephane

Re: syntax error while parsing a case command within `$(...)'

[Robert Elz]

Date:Sun, 14 Feb 2021 17:00:06 +0800
From:Koichi Murase 
Message-ID:  



  | I guess you are using Bash for so many years,

Yes, since Paul Fox created and maintained it (version 1).   It allowed
me to escape from csh.

  | but it was fixed in Bash 4.0 (2009).

That is good to know.   I (these days, and really forever) only use
bash as an interactive shell, not for scripts, and don't tend to use
constructs like the ones in question interactively.

And:

oguzismailuy...@gmail.com said:
  | By the way, Gmail's marking your emails as spam again.

That's a step up on outright rejecting them, which they do from time
to time.   I don't much care, anyone who really cares about receiving
e-mail shouldn't be using gmail.

kre

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月14日(日) 17:16 Oğuz :
>> This is interesting. I think this is just an oversight. A quick fix
>> like the attached patch should be fine. [ Note: The patch seems to
>> work fine, but I haven't carefully read the code, so I'm not sure if
>> this fix is really correct. ]
>
> It seems to be on the right track but incomplete:
>
>$ ./bash -c '$(case x in x) ;; x) esac)'
>   ./bash: -c: line 1: unexpected EOF while looking for matching `)'
>   ./bash: -c: line 2: syntax error: unexpected end of file
>

Yeah, let's wait for Chet's fix. I haven't read the entire code of
this long function `parse_comsub ()' so cannot properly fix it soon. I
think the problem has been pinpointed by the patch, so I expect it can
be right fixed by Chet.

--
Koichi

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

14 Şubat 2021 Pazar tarihinde Koichi Murase  yazdı:

> 2021年2月14日(日) 15:41 Andreas Kusalananda Kähäri :
> > And using (x) doesn't help:
> >
> > % bash -c ': $(case x in (x) esac)'
> > bash: -c: line 1: unexpected EOF while looking for matching `)'
> > bash: -c: line 2: syntax error: unexpected end of file
>
> This is interesting. I think this is just an oversight. A quick fix
> like the attached patch should be fine. [ Note: The patch seems to
> work fine, but I haven't carefully read the code, so I'm not sure if
> this fix is really correct. ]
>

It seems to be on the right track but incomplete:

   $ ./bash -c '$(case x in x) ;; x) esac)'
  ./bash: -c: line 1: unexpected EOF while looking for matching `)'
  ./bash: -c: line 2: syntax error: unexpected end of file


-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Koichi Murase]

2021年2月14日(日) 15:41 Andreas Kusalananda Kähäri :
> And using (x) doesn't help:
>
> % bash -c ': $(case x in (x) esac)'
> bash: -c: line 1: unexpected EOF while looking for matching `)'
> bash: -c: line 2: syntax error: unexpected end of file

This is interesting. I think this is just an oversight. A quick fix
like the attached patch should be fine. [ Note: The patch seems to
work fine, but I haven't carefully read the code, so I'm not sure if
this fix is really correct. ]

2021年2月14日(日) 6:23 Robert Elz :
> [...]
>
> This problem has been known for a LONG time, without it being fixed.

I guess you are using Bash for so many years, but it was fixed in Bash
4.0 (2009). You can see the result of the following command:

$ bash-4.0 -c ': $(case x in x) :; esac)'

2021年2月14日(日) 6:23 Robert Elz :
> Fixing it would take major work in the parser, so the chances of it
> happening are not all that great.

The case reported here appears to be just an oversight in the fix of
Bash 4.0, so I guess an additional fix should be relatively easy.

--
Koichi


0001-fix-parsing-of-case-x-in-x-esac.patch
Description: Binary data

Re: syntax error while parsing a case command within `$(...)'

[Andreas Kusalananda Kähäri]

On Sun, Feb 14, 2021 at 12:02:02AM +0100, Andreas Schwab wrote:
> On Feb 14 2021, Robert Elz wrote:
> 
> > Date:Sat, 13 Feb 2021 23:21:36 +0300
> > From:=?UTF-8?B?T8SfdXo=?= 
> > Message-ID:  
> > 
> >
> >   |   $ bash -c ': $(case x in x) esac)'
> >
> > This is a well known bash deficiency.  When parsing command substitutions
> > it (approximately) simply counts (unquoted) parentheses to find the end.
> > Anything with a valid closing ')' but with no opening '(' confuses it.
> 
> Bash already handles that properly, when the compound-list is non-empty.
> 
> $ bash -c ': $(case x in x) :; esac)'
> 
> Andreas.

And using (x) doesn't help:

% bash -c ': $(case x in (x) esac)'
bash: -c: line 1: unexpected EOF while looking for matching `)'
bash: -c: line 2: syntax error: unexpected end of file


-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: syntax error while parsing a case command within `$(...)'

[Lawrence Velázquez]

> On Feb 13, 2021, at 5:21 PM, Robert Elz  wrote:
> 
> This problem has been known for a LONG time, without it being fixed.

I played around with shbot on Freenode for a bit. "LONG time" is right:

 bash50 -c ': $(case x in x) esac)'
  bash50: -c: line 0: unexpected EOF while looking for matching `)'
  bash50: -c: line 1: syntax error: unexpected end of file

 bash44 -c ': $(case x in x) esac)'
  bash44: -c: line 0: unexpected EOF while looking for matching `)'
  bash44: -c: line 1: syntax error: unexpected end of file

 bash32 -c ': $(case x in x) esac)'
  bash32: -c: line 0: syntax error near unexpected token `)'
  bash32: -c: line 0: `: $(case x in x) esac)'

 bash2 -c ': $(case x in x) esac)'
  bash2: -c: line 1: syntax error near unexpected token `)'
  bash2: -c: line 1: `: $(case x in x) esac)'

 bash1 -c ': $(case x in x) esac)'
  Segmentation fault

--
vq

Re: syntax error while parsing a case command within `$(...)'

[Oğuz]

14 Şubat 2021 Pazar tarihinde Robert Elz  yazdı:

> Date:Sat, 13 Feb 2021 23:21:36 +0300
> From:=?UTF-8?B?T8SfdXo=?= 
> Message-ID:   j...@mail.gmail.com>
>
>   |   $ bash -c ': $(case x in x) esac)'
>
> This is a well known bash deficiency.  When parsing command substitutions
> it (approximately) simply counts (unquoted) parentheses to find the end.
> Anything with a valid closing ')' but with no opening '(' confuses it.
>
> Your example is one, perhaps the most common (and is why the optional
> leading '(' in case patterns was added to the syntax - so it is possible
> to survive with shells that work this way, but ')' in a here doc in the
> command substitution is another).
>
> This problem has been known for a LONG time, without it being fixed.
> Fixing it would take major work in the parser, so the chances of it
> happening are not all that great.


I didn't know that, but I don't think fixing this case only would take
major work.

By the way, Gmail's marking your emails as spam again.


>
> kre
>
>

-- 
Oğuz

Re: syntax error while parsing a case command within `$(...)'

[Andreas Schwab]

On Feb 14 2021, Robert Elz wrote:

> Date:Sat, 13 Feb 2021 23:21:36 +0300
> From:=?UTF-8?B?T8SfdXo=?= 
> Message-ID:  
> 
>
>   |   $ bash -c ': $(case x in x) esac)'
>
> This is a well known bash deficiency.  When parsing command substitutions
> it (approximately) simply counts (unquoted) parentheses to find the end.
> Anything with a valid closing ')' but with no opening '(' confuses it.

Bash already handles that properly, when the compound-list is non-empty.

$ bash -c ': $(case x in x) :; esac)'

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: syntax error while parsing a case command within `$(...)'

[Robert Elz]

Date:Sat, 13 Feb 2021 23:21:36 +0300
From:=?UTF-8?B?T8SfdXo=?= 
Message-ID:  


  |   $ bash -c ': $(case x in x) esac)'

This is a well known bash deficiency.  When parsing command substitutions
it (approximately) simply counts (unquoted) parentheses to find the end.
Anything with a valid closing ')' but with no opening '(' confuses it.

Your example is one, perhaps the most common (and is why the optional
leading '(' in case patterns was added to the syntax - so it is possible
to survive with shells that work this way, but ')' in a here doc in the
command substitution is another).

This problem has been known for a LONG time, without it being fixed.
Fixing it would take major work in the parser, so the chances of it
happening are not all that great.

kre

Re: syntax error while parsing a case command within `$(...)'

[Andreas Kusalananda Kähäri]

On Sat, Feb 13, 2021 at 03:41:30PM -0500, Lawrence Velázquez wrote:
> > On Sat, Feb 13, 2021, 21:34 Alex fxmbsw7 Ratchev  wrote:
> > 
> > you didnt end the case, wrong syntax
> > echo $( case x in y) printf 1 ;; x) printf 2 ;; esac )
> 
> $ case x in x) esac
> $ echo $?
> 0
> 
> > On Feb 13, 2021, at 3:36 PM, Alex fxmbsw7 Ratchev  wrote:
> > 
> > you have to specify something for ) even when its just a * wildcard
> 
> Oğuz did specify a pattern. It's the second 'x'.
> 
> vq

.. but he never had to:

% bash -c 'case x in esac; echo $?'
0


-- 
Andreas (Kusalananda) Kähäri
SciLifeLab, NBIS, ICM
Uppsala University, Sweden

.

Re: syntax error while parsing a case command within `$(...)'

[Lawrence Velázquez]

> On Sat, Feb 13, 2021, 21:34 Alex fxmbsw7 Ratchev  wrote:
> 
> you didnt end the case, wrong syntax
> echo $( case x in y) printf 1 ;; x) printf 2 ;; esac )

$ case x in x) esac
$ echo $?
0

> On Feb 13, 2021, at 3:36 PM, Alex fxmbsw7 Ratchev  wrote:
> 
> you have to specify something for ) even when its just a * wildcard

Oğuz did specify a pattern. It's the second 'x'.

vq

Re: syntax error while parsing a case command within `$(...)'

[Alex fxmbsw7 Ratchev]

you have to specify something for ) even when its just a * wildcard

On Sat, Feb 13, 2021, 21:34 Alex fxmbsw7 Ratchev  wrote:

> you didnt end the case, wrong syntax
> echo $( case x in y) printf 1 ;; x) printf 2 ;; esac )
>
> On Sat, Feb 13, 2021, 21:21 Oğuz  wrote:
>
>>   $ bash -c ': $(case x in x) esac)'
>>   bash: -c: line 1: unexpected EOF while looking for matching `)'
>>   bash: -c: line 2: syntax error: unexpected end of file
>>
>> `case x in x) esac' is a complete case command, a syntax error is
>> unexpected here.
>>
>>
>> --
>> Oğuz
>>
>

Re: syntax error while parsing a case command within `$(...)'

[Alex fxmbsw7 Ratchev]

you didnt end the case, wrong syntax
echo $( case x in y) printf 1 ;; x) printf 2 ;; esac )

On Sat, Feb 13, 2021, 21:21 Oğuz  wrote:

>   $ bash -c ': $(case x in x) esac)'
>   bash: -c: line 1: unexpected EOF while looking for matching `)'
>   bash: -c: line 2: syntax error: unexpected end of file
>
> `case x in x) esac' is a complete case command, a syntax error is
> unexpected here.
>
>
> --
> Oğuz
>