man page: minor problem regarding ${parameter@operator}

[Clark Wang]

See:

   ${parameter@operator}
  Parameter transformation.  The expansion is either a
transforma-
  tion of the value of parameter or  information  about
parameter
  itself,  depending on the value of operator.  Each operator
is a
  single letter:
  [...]
  If parameter is @ or *, the operation is applied to  each
posi-
  tional  parameter  in  turn,  and the expansion is the
resultant
  list.  If parameter is an array variable subscripted with  @
or
  *,  the case modification operation is applied to each member
of
  the array in turn, and the expansion is the resultant list.

In the last sentence, "the case modification operation" should be "the
operation".

Why are long functions slower?

[Bernhard M. Wiedemann]

Hi,

today I was debugging performance issues with a 200KB bash script [1]
with bash-4.3 and 4.4
and it seems that much of it came from a function call that took 0.1
seconds (and it was done in a loop for 37000 files)
even though it basically just consisted of an
  if [[ 0 != 0 ]] ; then


I also wrote this simpler reproducer:

function func1
{
[[ 0 != 0 ]] || return
echo neverdone
}

function func2
{
[[ 0 != 0 ]] || return
echo neverdone
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
}

time for i in $(seq 3) ; do
func1 someparameters
done
time for i in $(seq 3) ; do
func2 someparameters
done


showing significant difference in execution time with the longer
function taking 2x to 4x as much time.
Even though the length is comparable, it is not nearly as slow as 0.1s
per call of the original file.

1) could it be that the overall file size or complexity influences the
time it takes for a function to be parsed and executed?

2) is there a way to avoid the slowdown from length of such functions?
I tried
readonly func2
declare -fr func2

but it did not make a difference


Thanks in advance for your insights
Bernhard M.


[1]
https://github.com/g23guy/supportutils/blob/1e89b672d61ac6da5d8cf4a164b529693eab0cd9/bin/supportconfig#L304

Re: glibc [BZ #22145]: {p,ty}fds and mount namespaces

[Chet Ramey]

On 10/9/17 10:37 AM, Christian Brauner wrote:

> A common scenario where this happens is with /dev/console in containers.
> Usually container runtimes/managers will call openpty() on a ptmx device in 
> the
> host's mount namespace to safely allocate a {p,t}ty master-slave pair since 
> they
> can't trust the container's devpts mount after the container's init binary has
> started (potentially malicious fuse mounts and what not).  The slave {p,t}ty 
> fd
> will then usually be sent to the container and bind-mounted over the 
> container's
> /dev/console which in this scenario is simply a regular file. This is 
> especially
> common with unprivileged containers where mknod() syscalls are not possible. 
> In
> this scenario ttyname{_r}() will correctly report that /dev/console does in 
> fact
> refer to a {p,t}ty device whose path exists in the current mount namespace but
> whose origin is a devpts mount in a different mount namespace. Bash however
> seems to not like this at all and fails to initialize job control correctly. 
> In
> case you have lxc available this is simply reproducible by creating an
> unprivileged container and calling lxc-execute -n  -- bash.  
> If
> you could look into this and whether that makes sense to you it'd be greatly
> appreciated.

Bash doesn't try to open /dev/console. It will, however, try to open
/dev/tty and, if that fails, call ttyname() to get the pathname of a
terminal device to pass to open(). The idea is that if you're started
without a controlling terminal, the first terminal device you open becomes
your controlling terminal. However, if that fails, job control will
eventually be disabled -- you can't have job control without a controlling
terminal.

Under the circumstances described in the original bug report, bash attempts
to use stderr as its controlling terminal (having already called isatty and
been told that it's a terminal), discovers that it cannot set the process
group on that, and disables job control. If you can't set the process group
on what you think is your controlling terminal, you're not going to be able
to do job control, period.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Array not defined when being set inline

[Chet Ramey]

On 10/10/17 9:06 AM, Dan Douglas wrote:

> Bash parses the array assignment as though it were valid while reading
> words, assignments, and redirects of the command. 

Because at the time the parser reads the assignment, it has to assume
that it's a candidate for compound assignment.  It's only when it
reads a non-assignment word that it becomes a temporary assignment
preceding a simple command.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Array not defined when being set inline

[Greg Wooledge]

On Tue, Oct 10, 2017 at 09:05:55AM -0400, shawn wilson wrote:
> Nope, I knew how to correct it (as I showed in the last example), I
> just figured the other two should work.

"Work" how?  You placed a variable in the TEMPORARY execution ENVIRONMENT
of a command.  You can tell it's an environment variable by the "-x"
in the output of declare.  And since it's an environment variable, it
is therefore a string variable, because you can't export arrays.

And it doesn't persist because it's a TEMPORARY variable.  It's just
there for the duration of that one command.

What are you trying to do?

Re: Array not defined when being set inline

[Dan Douglas]

On 10/10/2017 07:00 AM, shawn wilson wrote:
> I guess that's the right way to describe what I'm seeing:
> 
> [swilson@localhost ~]$ unset f; f=(aaa bbb ccc) declare -p f
> declare -x f="(aaa bbb ccc)"
> [swilson@localhost ~]$ unset f; f=("aaa" "bbb" "ccc") declare -p f
> declare -x f="(aaa bbb ccc)"
> [swilson@localhost ~]$ unset f; f=(aaa bbb ccc)
> [swilson@localhost ~]$ declare -p f
> declare -a f='([0]="aaa" [1]="bbb" [2]="ccc")'
> 
> Is this known? What exactly is going on / what does this mean?
> 

Bash parses the array assignment as though it were valid while reading
words, assignments, and redirects of the command. After processing
expansions, the saved assignment text is re-interpreted as a string
assignment, presumably because at this point bash no longer cares about
the quoting of metacharacters so they're just treated as literal.

Array assignments aren't valid preceding commands of course because
the environment fundamentally stores strings / key-value pairs of
binary blobs.

This should probably be an error, but this is what bash has always done,
and this hasn't really caused any problems other than occasionally
tricking people into thinking arrays are somehow exportable.



signature.asc
Description: OpenPGP digital signature

Re: Array not defined when being set inline

[shawn wilson]

On Tue, Oct 10, 2017 at 8:21 AM, Greg Wooledge  wrote:
> On Tue, Oct 10, 2017 at 08:00:58AM -0400, shawn wilson wrote:
>> I guess that's the right way to describe what I'm seeing:
>>
>> [swilson@localhost ~]$ unset f; f=(aaa bbb ccc) declare -p f
>> declare -x f="(aaa bbb ccc)"
>
> You placed a string variable in the temporary execution environment of
> the declare command.  If you wanted an actual array variable that would
> persist past this command, you need a semicolon or newline after the
> assignment, and before the declare command.
>

Nope, I knew how to correct it (as I showed in the last example), I
just figured the other two should work.

>
> See also: http://mywiki.wooledge.org/BashFAQ/104

Thanks for that (and the link it provides to BashParser - I'm going to
have to digest that).

Is there a way to detect that the parser has already processed a
temporary variable and maybe change the exit status or something?

Array not defined when being set inline

[shawn wilson]

I guess that's the right way to describe what I'm seeing:

[swilson@localhost ~]$ unset f; f=(aaa bbb ccc) declare -p f
declare -x f="(aaa bbb ccc)"
[swilson@localhost ~]$ unset f; f=("aaa" "bbb" "ccc") declare -p f
declare -x f="(aaa bbb ccc)"
[swilson@localhost ~]$ unset f; f=(aaa bbb ccc)
[swilson@localhost ~]$ declare -p f
declare -a f='([0]="aaa" [1]="bbb" [2]="ccc")'

Is this known? What exactly is going on / what does this mean?