Re: Word splitting for $@ in variable assignment

[Oğuz]

25 Haziran 2021 Cuma tarihinde Nora Platiel  yazdı:
>
> To me, "$@" expanding to multiple words would mean that:
>
> $ var="$@" foo
>
> for $# > 0, behaves the same as:
>
> $ var="$1" "${@:2}" foo
>

But it wouldn't make any sense. `foo' would be subjected to alias
substitution even though it isn't known for sure to be a command name. How
would that work?


-- 
Oğuz

Re: Word splitting for $@ in variable assignment

[Greg Wooledge]

On Fri, Jun 25, 2021 at 01:47:01AM +0200, Nora Platiel wrote:
> To me, "$@" expanding to multiple words would mean that:
> 
> $ var="$@" foo
> 
> for $# > 0, behaves the same as:
> 
> $ var="$1" "${@:2}" foo
> 
> which is obviously not the case.

"$@" expands like "$1 "$2" ... when used in most contexts.  For example,

foo --bar "$@"

passes the script's arguments along to foo, exactly as received, with
the extra --bar argument in front of them.

Likewise,

args=("$@")

stores the script's arguments as the elements of an indexed array.  Each
argument is retained as a separate word, becoming one element of the
new array.

var="$@" is simply a special case.  It's an assignment *first*, and it's
parsed as such.

The assignment-ness of the command overrides everything else.  It
completely changes how the "$@" expansion occurs.  Now, instead of
expanding like "$1" "$2" ... it expands like "$1 $2 ...".  It's
completely unique.  "$@" does not act like that in any other context.

Bash is ALL about these special cases.  If you don't like it, don't write
code that uses it.  In any sensible programming language, var="$@"
would have been an error.  In bash, it's not.  But that doesn't mean
you have to *write* it.

Re: Word splitting for $@ in variable assignment

[Nora Platiel]

I was also confused by the same statement.

On 2021-06-24 10:20 Chet Ramey wrote:
> > But 3.4 Shell Parameters is a bit confusing: "Word splitting is not
> > performed, with the exception of "$@" as explained below."
>
> This means that "$@" expands to multiple words, even though double quotes
> would usually inhibit that.

To me, "$@" expanding to multiple words would mean that:

$ var="$@" foo

for $# > 0, behaves the same as:

$ var="$1" "${@:2}" foo

which is obviously not the case.

I don't know how this is implemented, maybe they are actually split into 
multiple words, and such words somehow remain in the context of the assignment 
and are rejoined later.
But the user doesn't see that. The user just see that on the RHS of an 
assignment, "$@" produces a single string (by joining arguments), and such 
string is not split in any way.

I second the proposal of removing the part: `with the exception of "$@" as 
explained below', because I think it serves no useful purpose and adds 
confusion.

Regards,
NP

Re: Latest push for command substitutions

[Chet Ramey]

On 6/24/21 5:44 AM, Koichi Murase wrote:

Maybe this is not caused by the change of this time, but another
related problem. When the nested command substitutions are incomplete,
an error message of the (false) syntax error is printed before PS2.
The command is still executed correctly after closing the command
substitutions.


This doesn't have anything to do with the parser, per se, or this change.
It's an interactive-only error similar to

https://lists.gnu.org/archive/html/bug-bash/2021-02/msg00037.html

caused by the history code trying to determine which delimiter to use when
adding this line to the history list.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Latest push for command substitutions

[Chet Ramey]

On 6/24/21 5:18 AM, Koichi Murase wrote:

Please run it through any tests or scripts you have so we can shake out any
bugs early.


I haven't thoroughly tested it, but I found with a quick trial that
the parser state seems to remain broken in the new command line after
a syntax error:


Thanks for the report. This is a new class of error, I suppose is the best
way to put it, introduced with this new implementation. In the previous ad-
hoc implementation, the "parse" would succeed until word expansion, then
fail when the string was run through the parser again to find the closing
`)'.

There's already code in there to handle this case, it just needs different
return values to handle the error differently. There are a couple of ways
to deal with this interactive-only error, so I'll pick one.

Chet


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: 'wait' in command group in pipeline fails to recognize child process

[Chet Ramey]

On 6/23/21 6:17 PM, Martin Jambon wrote:

On 6/23/21 6:24 AM, Chet Ramey wrote:

On 6/22/21 9:54 PM, Martin Jambon wrote:


In the posix definition, a subshell
- is not necessarily implemented as a separate process
- belongs to a unique shell
- is not a shell


Why is it not "a shell?"


Because '$$' doesn't match its process ID.


OK. That's a rather unique definition, but it's a definition.

I'm not sure why I have to fight over this. It's clearly my 
misunderstanding. That's why I suggest clarifications in the documentation, 
if you're interested in creating a better experience for users like me.


I don't think we're fighting here. We're discussing whether or not we need
to invent new nomenclature to clarify an obscure point, or whether a
clearer explanation of the concept of a subshell will suffice.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Word splitting for $@ in variable assignment

[Chet Ramey]

On 6/24/21 4:09 AM, Ilkka Virta wrote:


But 3.4 Shell Parameters is a bit confusing: "Word splitting is not
performed, with the exception of "$@" as explained below." 


This means that "$@" expands to multiple words, even though double quotes
would usually inhibit that.


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: function def crash

[Phi Debian]

Yes figured the hard way :)
Thanx

On Thu, Jun 24, 2021 at 1:51 PM Greg Wooledge  wrote:

> On Thu, Jun 24, 2021 at 09:48:59AM +0200, Phi Debian wrote:
> > $ function echo { \echo the function version; \echo "$@"; }
>
> For the record, this isn't how you write a function that wraps a builtin.
> Use the "builtin" command instead.
>
> echo() {
>   printf 'echo() wrapper invoked.\n'
>   builtin echo "$@"
> }
>
> Or, if you aren't sure whether the command you're wrapping is a builtin
> or a program, use "command":
>
> echo() {
>   printf 'echo() wrapper invoked.\n'
>   command echo "$@"
> }
>
> The backslash hack only stops aliases from being used.  It doesn't stop
> your function from calling itself recursively.
>
>

Re: function def crash

[Greg Wooledge]

On Thu, Jun 24, 2021 at 09:48:59AM +0200, Phi Debian wrote:
> $ function echo { \echo the function version; \echo "$@"; }

For the record, this isn't how you write a function that wraps a builtin.
Use the "builtin" command instead.

echo() {
  printf 'echo() wrapper invoked.\n'
  builtin echo "$@"
}

Or, if you aren't sure whether the command you're wrapping is a builtin
or a program, use "command":

echo() {
  printf 'echo() wrapper invoked.\n'
  command echo "$@"
}

The backslash hack only stops aliases from being used.  It doesn't stop
your function from calling itself recursively.

Re: Word splitting for $@ in variable assignment

[Greg Wooledge]

On Thu, Jun 24, 2021 at 05:52:47PM +1000, Alvin Seville wrote:
>  Hello! I want to understand why the following code doesn't produce any
> error:
> set -- "a b" "c"
> a="$@"

It doesn't produce an error because the designers of the shell tried
to make it "user-friendly" by having it guess what you really want,
whenever you do silly things.  Sometimes it guesses correctly.  Sometimes
not.

In this case, you've written code that makes no sense to an experienced
programmer, because on the surface you're assigning a vector (list, array)
value to a scalar (string) variable.

But the shell decided it would try to "help" you by assuming you really
meant a="$*".  Except, it's not actually that simple, and it just gets
more and more ugly the deeper you dive into it.

> ? I expected smth like: main.sh: line 2: b: command not found due to word
> splitting according to documentation
> .
> What am I missing?

Word splitting does not occur on the right hand side of a string variable
assignment.  That's why you can do this:

a='two words'
b=$a

This is well-defined behavior, completely portable across every Bourne-type
shell ever written.

The difference between this and your case (a="$@") is that "$@" doesn't
actually involve word splitting.  It's like... the opposite of word
splitting.  Word retention, maybe?  It doesn't have a name.

But it doesn't make sense for a retained list of words to be assigned to
a string variable.  There has to be a single string that gets assigned.
So the shell tries to come up with a result that it thinks will make
sense.

What it ends up doing is taking the list of words generated by "$@" and
concatenating them all into a single string with spaces between them.
It does NOT use the contents of IFS -- it simply chooses "space" as
the separator.

unicorn:~$ bash
unicorn:~$ IFS=x
unicorn:~$ set -- "a b" c
unicorn:~$ a="$*"; echo "$a"
a bxc
unicorn:~$ a="$@"; echo "$a"
a b c

This surprising leniency is one of the reasons there are so many terrible
shell scripts (and shell script writers) in the world.  The shell lets you
do many ridiculous and even dangerous things, with no warnings of any kind.

Re: function def crash

[Phi Debian]

Arghh! I knew it was a pilot error :)

unset PROMPT_COMMAND fix it :)
my PROMPT_COMMAND  reference a function that do 'echo'  and I should use
"command echo" in the echo function not \echo.




On Thu, Jun 24, 2021 at 10:51 AM Phi Debian  wrote:

> I discovered that
>
> $ type -ta word
>
> Would tell me what I wanted i.e all the defs of word :)
>
>
>
> On Thu, Jun 24, 2021 at 9:48 AM Phi Debian  wrote:
>
>> Hi All,
>>
>> Dunno if this is a bug or pilot error, I bumped into this.
>>
>>
>> $ type echo
>> echo is a shell builtin
>>
>> $ function echo { \echo the function version; \echo "$@"; }
>>
>> After this I got different behavior on 'same' OS version, same BASH
>> version, meaning my environment must interact but can't tell to what extend
>> at the moment.
>>
>> On 2 systems I got bash 'crash', probaly in recursion runaway, during the
>> function definition.
>> On 1 system the definition is OK, then the run for the function crash,
>> again on what's look a recursion run away.
>>
>> I tried this tiny test case on debian 4.19.98-1 (2020-01-26) (little old)
>> and GNU bash, version 5.0.3(1)-release (i686-pc-linux-gnu) (little old too)
>> and I got the definition crash behavior.
>>
>> I bumped into this as I tried to implement something that would tell me
>> all the 'command' definition of a word0 of a shell command, so having the
>> same identifier for a program basename, an alias and a function, I wanted
>> to make something (a function?) that would tell me
>>
>> $ word0 echo
>> echo is an alias
>> echo is a function
>> echo is a program (a.out...)
>>
>>
>> I choosed randomly 'echo' as a test case, I could have used ls, etc
>> only echo seems to choke.
>>
>> Cheers,
>> Phi
>>
>

Re: Latest push for command substitutions

[Koichi Murase]

Maybe this is not caused by the change of this time, but another
related problem. When the nested command substitutions are incomplete,
an error message of the (false) syntax error is printed before PS2.
The command is still executed correctly after closing the command
substitutions.

$ LANG=C ./bash --norc
$ echo $(echo $(echo A
bash: command substitution: line 3: unexpected EOF while looking for
matching `)'
> )
> echo B
> )
A B
$

A non-nested command substitution works as expected.

$ echo $(echo A
> )
A
$

Actually, Bash 5.1 seems to have behaved in an even stranger way. It
shows a new PS1 after the error message, but we can still continue to
input the remaining part of the previous command and run it correctly:

$ LANG=C bash-5.1 --norc
$ echo $(echo $(echo A
bash-5.1: unexpected EOF while looking for matching `)'
$ )
> echo B
> )
A B
$

The behavior of Bash 5.0 is the expected one:

$ LANG=C bash-5.0 --norc
$ echo $(echo $(echo A
> )
> echo B
> )
A B

--
Koichi

2021年6月23日(水) 5:41 Chet Ramey :
>
> The latest devel branch push has a rewrite of parse_comsub() to recursively
> call the parser (yyparse()). I was finally able to get all the state
> working correctly, though it will probably only work with bison.
>
> Please run it through any tests or scripts you have so we can shake out any
> bugs early.
>
> Chet
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
>  ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>

Re: Latest push for command substitutions

[Alex fxmbsw7 Ratchev]

lol what the weird behavior

On Thu, Jun 24, 2021, 11:19 Koichi Murase  wrote:

> > Please run it through any tests or scripts you have so we can shake out
> any
> > bugs early.
>
> I haven't thoroughly tested it, but I found with a quick trial that
> the parser state seems to remain broken in the new command line after
> a syntax error:
>
> $ ./bash --norc
> $ printf '[%s]\n' $(if)
> bash: syntax error near unexpected token `)'
> $ echo abcdef
> [echo]
> bash: abcdef: command not found
> $
>
> --
> Koichi
>
>

Re: Latest push for command substitutions

[Koichi Murase]

> Please run it through any tests or scripts you have so we can shake out any
> bugs early.

I haven't thoroughly tested it, but I found with a quick trial that
the parser state seems to remain broken in the new command line after
a syntax error:

$ ./bash --norc
$ printf '[%s]\n' $(if)
bash: syntax error near unexpected token `)'
$ echo abcdef
[echo]
bash: abcdef: command not found
$

--
Koichi

Re: function def crash

[Phi Debian]

I discovered that

$ type -ta word

Would tell me what I wanted i.e all the defs of word :)



On Thu, Jun 24, 2021 at 9:48 AM Phi Debian  wrote:

> Hi All,
>
> Dunno if this is a bug or pilot error, I bumped into this.
>
>
> $ type echo
> echo is a shell builtin
>
> $ function echo { \echo the function version; \echo "$@"; }
>
> After this I got different behavior on 'same' OS version, same BASH
> version, meaning my environment must interact but can't tell to what extend
> at the moment.
>
> On 2 systems I got bash 'crash', probaly in recursion runaway, during the
> function definition.
> On 1 system the definition is OK, then the run for the function crash,
> again on what's look a recursion run away.
>
> I tried this tiny test case on debian 4.19.98-1 (2020-01-26) (little old)
> and GNU bash, version 5.0.3(1)-release (i686-pc-linux-gnu) (little old too)
> and I got the definition crash behavior.
>
> I bumped into this as I tried to implement something that would tell me
> all the 'command' definition of a word0 of a shell command, so having the
> same identifier for a program basename, an alias and a function, I wanted
> to make something (a function?) that would tell me
>
> $ word0 echo
> echo is an alias
> echo is a function
> echo is a program (a.out...)
>
>
> I choosed randomly 'echo' as a test case, I could have used ls, etc
> only echo seems to choke.
>
> Cheers,
> Phi
>

Re: Word splitting for $@ in variable assignment

[Ilkka Virta]

On Thu, Jun 24, 2021 at 10:53 AM Alvin Seville 
wrote:

>  Hello! I want to understand why the following code doesn't produce any
> error:
> set -- "a b" "c"
> a="$@"
> ? I expected smth like: main.sh: line 2: b: command not found due to word
> splitting according to documentation
> <
> https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Shell-Parameters
> >.
> What am I missing?
>

It's a bit oddly put in the manual. It doesn't actually get word-split as
such, it just gives the individual positional parameters joined with spaces
to a single string.

3.4.2 Special Parameters explains that: "$@: In contexts where word
splitting is not performed, this expands to a single word with each
positional parameter separated by a space."

But 3.4 Shell Parameters is a bit confusing: "Word splitting is not
performed, with the exception of "$@" as explained below." I'm not sure if
this is meant to refer to that sentence 3.4.2, but it's a bit confusing
since that one refers to cases without word splitting. It would probably be
clearer to not call it an exception and just refer to Special Parameters
for how $@ behaves when not word-split.

Anyway, it may be better to use a="$*" as it's treated more consistently
between different shells. a="$@" uses the first character of IFS as the
separator in some shells (Zsh, Busybox, Debian/Ubuntu's Dash), and a space
always in others (Bash, Ksh), while a="$*" consistently uses the first
character of IFS. With the default IFS, the result is of course the same.

Word splitting for $@ in variable assignment

[Alvin Seville]

 Hello! I want to understand why the following code doesn't produce any
error:
set -- "a b" "c"
a="$@"
? I expected smth like: main.sh: line 2: b: command not found due to word
splitting according to documentation
.
What am I missing?

-- 
alvinseville7cf@Alvins-MacBook-Pro ~ $* echo *-e "Best regards, \e[33m$(
whoami)\e[0m."* && exit *$SUCCESS_EC

function def crash

[Phi Debian]

Hi All,

Dunno if this is a bug or pilot error, I bumped into this.


$ type echo
echo is a shell builtin

$ function echo { \echo the function version; \echo "$@"; }

After this I got different behavior on 'same' OS version, same BASH
version, meaning my environment must interact but can't tell to what extend
at the moment.

On 2 systems I got bash 'crash', probaly in recursion runaway, during the
function definition.
On 1 system the definition is OK, then the run for the function crash,
again on what's look a recursion run away.

I tried this tiny test case on debian 4.19.98-1 (2020-01-26) (little old)
and GNU bash, version 5.0.3(1)-release (i686-pc-linux-gnu) (little old too)
and I got the definition crash behavior.

I bumped into this as I tried to implement something that would tell me all
the 'command' definition of a word0 of a shell command, so having the same
identifier for a program basename, an alias and a function, I wanted to
make something (a function?) that would tell me

$ word0 echo
echo is an alias
echo is a function
echo is a program (a.out...)


I choosed randomly 'echo' as a test case, I could have used ls, etc
only echo seems to choke.

Cheers,
Phi