Re: Potentially misleading documentation of SECONDS variable

[felix]

The variable $SECOND won't intend to be exact to the nanoseconds!
Not even to 1/100th... This variable is intended to show current
time of execution, at SECOND resolution.

This variable is simply incremented each time SYSTEM CLOCK increment
his SECOND part of current time.

Try this little command line many times:

~$ secTest() {
  bash -c 'start=${EPOCHREALTIME};
   while !((SECONDS));do
sleep .02;done;
   end=${EPOCHREALTIME};
   dur=0$(( ${end/.}-${start/.} ));
   printf "Second duration: %.5f\n" ${dur::-6}.${dur: -6}';
   }

~$ secTest 
Second duration: 0.33484
~$ secTest 
Second duration: 0.04215

You won't ever see an answer higher than 0.9 seconds!
... But you may notice some seconds incremented at 0.1 seconds!

Le Wed, Aug 07, 2024 at 09:10:40AM -0500, G. Branden Robinson a écrit :
> At the risk of splitting hairs, I would cast the added phrase as:
> 
> "at a resolution of one second".

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: improving '{...}' in bash?

[felix]

This will juste require embeding second braces:

Le Tue, Jul 23, 2024 at 07:50:24AM +0200, Harald Dunkel a écrit :
>   % echo x{1..3,5}x
>   x1..3x x5x
> 
> I would have expected "x1x" and "x1x x2x x3x x5x".
Try:

$ echo x{{1..3},5}x
x1x x2x x3x x5x


-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: pwd and prompt don't update after deleting current working directory

[felix]

Le Tue, Jul 16, 2024 at 09:08:14AM -0400, Chet Ramey a écrit :
> On 7/16/24 3:47 AM, David Hedlund wrote:
> > > 
> > > pwd -P >/dev/null 2>&1 || cd ..
> > > 
> > Do you think that it would be appropriate to submit this feature request
> > to the developers of the rm command instead.
> 
> You can try, but I would not expect them to implement it.

Unfortunately, this could be not enough.

~$ mkdir -p /tmp/testdir/subdir/scndsubdir
~$ cd $_
/tmp/testdir/subdir/scndsubdir$ rm -fR /tmp/testdir
/tmp/testdir/subdir/scndsubdir$ pwd
/tmp/testdir/subdir/scndsubdir
/tmp/testdir/subdir/scndsubdir$ cd ..
cd: error retrieving current directory: getcwd: cannot access parent 
directories: No such file or directory

For this, you could use PROMPT_COMMAND feature:

validCurrentDir() { 
local __vCD_pwd=$PWD;
while ! [[ -d $__vCD_pwd ]]; do
__vCD_pwd=${__vCD_pwd%/*};
done;
cd $__vCD_pwd
}

~$ PROMPT_COMMAND=validCurrentDir

~$ mkdir -p /tmp/testdir/subdir/scndsubdir
~$ cd $_
/tmp/testdir/subdir/scndsubdir$ rm -fR /tmp/testdir
/tmp$ 



-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: yet another $(case ... parse bug

[felix]

Le Thu, May 23, 2024 at 08:19:49PM +0300, Oğuz a écrit :
> 
> $ bash -c 'for (( $(case x in x) esac);; )); do :; done'
>From there, I've tested:

  bash-5.3-alpha$ uname=1
  bash-5.3-alpha$ echo $(( uname - $(echo 1) ))
  0
  bash-5.3-alpha$ echo $(( uname - $(case x in x) echo 1;exit;;esac;echo 0) ))
  uname: extra operand '-'
  Try 'uname --help' for more information.

Which could be avoided by using double quotes:

  bash-5.3-alpha$ echo $(( uname - "$(case x in x) echo 1;exit;;esac;echo 0)" ))
  0

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: [Help-bash] difference of $? and ${PIPESTATUS[0]}

[felix]

Le Mon, Apr 22, 2024 at 10:56:03AM +0300, Oğuz a écrit :
> 
> Me neither, but the current behavior is useful.
I agree!

Anyway, reading man page, `$?` have to be equivalent to `${PIPESTATUS[-1]}`!
which is not always the case.

> I've never seen anything like that in a real shell script.
I have to confess my neither, upto this post on SO, more than ten years later!

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: [Help-bash] difference of $? and ${PIPESTATUS[0]}

[felix]

Le Mon, Apr 22, 2024 at 07:44:48AM +0100, Kerin Millar a écrit :
> On Mon, 22 Apr 2024, at 7:13 AM, felix wrote:
> > ...
> >   if ls /wrong/path | wc | cat - /wrong/path | sed 'w/wrong/path' 
> > >/dev/null ; then
> >   echo Don't print this'
> >   fi ; echo ${?@Q} ${PIPESTATUS[@]@A}  $(( $? ${PIPESTATUS[@]/#/+} ))
> >
> >   ls: cannot access '/wrong/path': No such file or directory
> >   cat: /wrong/path: No such file or directory
> >   sed: couldn't open file /wrong/path: No such file or directory
> >   '0' declare -a PIPESTATUS=([0]="2" [1]="0" [2]="1" [3]="4") 7
> >
> > Where $PIPESTATUS[0]=>2 and $?=>0 !!
> > ...
> > If so, "$?" have to be equivalent to "${PIPESTATUS[0]}", I think.
> 
> No. That would only be true in the event that the pipeline comprises a single 
> command. The present documentation is correct.

I was wrong: the last is `${PIPESTATUS[-1]}' -> '${PIPESTATUS[3]}' in this case,
anyway
 $PIPESTATUS[3]=>4 and $?=>0 !!

> It's worth reading the section of the manual that concerns "Pipelines".
Reading this, if I could understand why
  false;echo ${PIPESTATUS[0]} $?
  1 1
I'm still don't be able to explain this:
  if false;then echo Don't print that's;fi; echo ${PIPESTATUS[0]} $?
  1 0

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: [Help-bash] difference of $? and ${PIPESTATUS[0]}

[felix]

Hi,

Comming on this very old thread:

On Wed, 4 Dec 2013 14:40:11 -0500, Greg Wooledge wrote:
> 
> The most obvious difference is that $? is shorter.
> 
> $? is also POSIX standard (older than POSIX in fact), so it works in sh
> scripts as well.  PIPESTATUS is a Bash extension.
> 
> Finally, note that if you execute a pipeline, $? will contain the exit
> status of the last command in the pipeline, not the first command,
> which is what ${PIPESTATUS[0]} would contain.  (If you execute a simple
> command instead of a pipeline, then they would both have the same value.)

Some asked on StackOverflow.com:
  Why does a Bash while loop result in PIPESTATUS "1" instead of "0"?
  https://stackoverflow.com/q/78351657/1765658

Then after some tests:

  if ls /wrong/path | wc | cat - /wrong/path | sed 'w/wrong/path' >/dev/null ; 
then
  echo Don't print this'
  fi ; echo ${?@Q} ${PIPESTATUS[@]@A}  $(( $? ${PIPESTATUS[@]/#/+} ))

  ls: cannot access '/wrong/path': No such file or directory
  cat: /wrong/path: No such file or directory
  sed: couldn't open file /wrong/path: No such file or directory
  '0' declare -a PIPESTATUS=([0]="2" [1]="0" [2]="1" [3]="4") 7

Where $PIPESTATUS[0]=>2 and $?=>0 !!

I could explain that '$?' is result of bash's if...then...fi group command
executed correctly and PIPESTATUS hold result of "most-recently-executed
foreground pipeline", but man page say:

PIPESTATUS
 An  array  variable (see Arrays below) containing a list of exit
 status values from the processes in  the  most-recently-executed
 foreground pipeline (which may contain only a single command).

 ?   Expands  to  the exit status of the most recently executed fore‐
 ground pipeline.

If so, "$?" have to be equivalent to "${PIPESTATUS[0]}", I think.

I suggest that man page should be modified to replace "foreground pipeline"
by "command" under "?" paragraph.


-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Examples of concurrent coproc usage?

[felix]

Some other way of thinking:

> On 3/14/24 5:58 AM, Carl Edquist wrote:
> > ...
> > But if you try...
> > 
> >  $ coproc WC { wc; }
> >  $ coproc CAT { cat; }
> >  $ exec {WC[1]}>&-
> >  $ read -u ${WC[0]} X
> > 
> >  # HANGS

To prevent  `exec {WC[1]}>&-` to close both FD, just because subpid is
ended, I open a *backup FD*:

  coTest() { 
local X WC CAT bakWcOut wcPid
coproc WC { exec wc; }
wcPid=$!
coproc CAT { exec cat; }
echo foo exec bar >&${WC[1]}  # Not really usefull for demo
exec {bakWcOut}>&${WC[0]}
exec {WC[1]}>&-
wait $wcPid
read -u ${bakWcOut} X
echo $X
exec {bkWc}>&-
exec {CAT[1]}>&-
wait
ls -l /dev/fd/
  }

This function's output look good (on my Debian's bash 5.2.15(1)-release):

  $ coTest 
  [1] 1606
  bash: warning: execute_coproc: coproc [1606:WC] still exists
  [2] 1607
  [1]-  Donecoproc WC { exec wc; }
  1 3 13
  [2]+  Donecoproc CAT { exec cat; }
  total 0
  lrwx-- 1 user user 64 Apr 17 11:12 0 -> /dev/pts/3
  lrwx-- 1 user user 64 Apr 17 11:12 1 -> /dev/pts/3
  lrwx-- 1 user user 64 Apr 17 11:12 2 -> /dev/pts/3
  lr-x-- 1 user user 64 Apr 17 11:12 3 -> /proc/1608/fd

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: printf %n is not documented

[felix]

My two cents:

Le Tue, Dec 27, 2022 at 03:44:29PM -0500, Chet Ramey a écrit :
> 
> So maybe the better thing to do is to list the set of valid format
> specifiers from the standard printf(3) set.
> 
I'm not a C programmer, so I didn't hear about `printf %n` before this thread.
Re-reading all `man {1,3,3pl,3avr,...} printf' present on my host, I'v finally
found one short explanation, abount %n in printf(3), but with a lot of other
informations useless, regarding memory size and other. For sample, I'd never
used `%ld`, `%lld` nor`%qd` ( where `%q` is now binded to another bashism...)
(And a bug/remark about security flaw regarding memory misallocation when
using a format script from non validated user input, but if some flaws still
exists under bash, things are very differents)

The first use of this i've imaginated, was for differentiating char length
with byte length:

For this my previous version was: https://stackoverflow.com/a/31009961/1765658

  strU8DiffLen0() { local _cLen=${#1} LC_ALL=C;return $((${#1}-_cLen));}

  for string in Généralités Language "Yin Yang ☯";do
  strU8DiffLen0 "$string";
  printf " - %-*s is %2d chars length, but uses %2d bytes\n" \
   $((14+$?)) "'$string'" ${#string} $((${#string}+$?));
  done
   - 'Généralités'  is 11 chars length, but uses 14 bytes
   - 'Language' is  8 chars length, but uses  8 bytes
   - 'Yin Yang ☯'   is 10 chars length, but uses 12 bytes

My new version of this became:

   strU8DiffLen1() { local _bl;printf -v _ %s%n "$1" _bl;return $((_bl-${#1}));}

This version is near two time quicker!!

   time for i in {1..1};do strU8DiffLen0 'Généralité';ret=$?;done
   real0m0.260s
   user0m0.252s
   sys 0m0.005s

   time for i in {1..1};do strU8DiffLen1 'Généralité';ret=$?;done
   real0m0.148s
   user0m0.144s
   sys 0m0.000s

But I have to confess:

  printf -v _ %s%n "$string" targetVar

This syntax has someting counter-intuitive.

So yes I think personely that printf chapter in bash man page could be expanded.

Thanks for all, regards!

-- 
 Félix

Localised variables become exposed to environment when FUNCNEST reached

[felix]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2
uname output: Linux medium 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 
(2022-10-21) x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 5.2
Patch Level: 12
Release Status: release

Description:
Localised variable in a function could alter environ when FUNCNEST is 
reached

Repeat-By:
$ funcTest() { local FUNCNEST=1 LocalVar=Not\ Global;$FUNCNAME;}
$ echo $LocalVar

$ funcTest
bash: funcTest: maximum function nesting level exceeded (1)
$ echo $LocalVar
Not Global

( Once playing with this, don't miss to `unset FUNCNEST`! ;-)

Performances comparission between 5.1 and 5.2.

[felix]

Configuration Information:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2
uname output: Linux medium 5.10.0-12-amd64 #1 SMP Debian 5.10.103-1 
(2022-03-07) x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 5.2
Patch Level: 0
Release Status: rc2

Description:
Trying some script under 5.2 beta, rc1 and rc2, I was surprised by 
execution time.
In order to compare, I've tried to establish execution time of 
elementar operations, like:

 - {0..999}.`: {0..999}`
 - 3x{0..9} `: {0..9}{0..9}{0..9}`
 - readUt   `read -r ut idl https://f-hauri.ch/vrac/timedTest.sh.txt
https://f-hauri.ch/vrac/looptest.sh.txt

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Arbitrary command execution from test on a quoted string

[felix]

Unfortunely, this won't be useable with associative array, like:

   declare -A AssocVar='([Some string.]=foo)'
   test -v AssocVar['Some string.'] && echo yes || echo no
   yes
   isvar AssocVar['Some string.']  && echo yes || echo no
   no

But Lea's solution seem work:

   test "${AssocVar['Some string.']@Q}"  && echo yes || echo no
   yes

Even with empty variables:

   declare -A AssocVar='([Some string.]=)'
   test "${AssocVar['Some string.']@Q}"  && echo yes || echo no
   yes
   test "${AssocVar['Some other string?']@Q}"  && echo yes || echo no
   no

Le Fri, Oct 29, 2021 at 07:54:17AM -0400, Greg Wooledge a écrit :
> On Fri, Oct 29, 2021 at 07:37:13AM +0200, Léa Gris wrote:
> > A safe way to replace:
> > test -v "$USER_INPUT"
> > 
> > Would be:
> > test "${USER_INPUT@Q}"
> > 
> > But it is not backward-compatible with older bash versions.
> 
> test -v is fairly recent as well.  That was introduced in 4.2, and the @Q
> syntax in 4.4.
> 
> I would suggest a three-step validation:
> 
> isvar() {
>   [[ $1 = LC_ALL ]] && { test -v "$1"; return; }
> 
>   local LC_ALL=C
>   [[ $1 = [a-zA-Z_]*([a-zA-Z0-9_]) ]] || return 1
> 
>   test -v "$1"
> }

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: ?maybe? RFE?... read -h ?

[felix]

On Mon, Sep 06, 2021 at 08:11:20AM -0400, Greg Wooledge wrote:
> The real one looks like this:
> unicorn:~$ help realpath

> realpath: realpath [-csv] pathname [pathname...]

> Display pathname in canonical form.
> 
> Display the canonicalized version of each PATHNAME argument, resolving
> symbolic links.  The -c option checks whether or not each resolved name
> exists.  The -s option produces no output; the exit status determines the
> validity of each PATHNAME.  The -v option produces verbose output.  The
> exit status is 0 if each PATHNAME was resolved; non-zero otherwise.
> 
> Since -v is already taken, the proposed modification used -a instead,
> 
> I have no idea why the proposal tries to write the result into an
> array variable, though.  That makes no sense to me.

Because `realpath` accept more than one *pathname*. So if you pass many
path entries to realpath as arguments, all answer could be stored into
an array.

  realpath /bin/sh /usr/bin/X

  /bin/dash
  /usr/bin/Xorg


-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Why tail coreutil cannot work as its input is from tee

[felix]

Because your last pipe will pipe BOTH output of `echo` AND `head` with
delay due to fork, head output will comme after. try this:

  echo -e '9\n4\n3\n2\n8\n7\n3\n1' |tee >(head -1 >&2) |tail -1

or 

  echo -e '9\n4\n3\n2\n8\n7\n3\n1' |tee >(head -1 >&2) >(tail -1 >&2) >/dev/null

...

On Mon, Sep 06, 2021 at 07:38:03AM +, Budi wrote:
> How come tail coreutil cannot work as its input is from tee
> 
> $ echo -e '9\n4\n3\n2\n8\n7\n3\n1' |tail -1
> 1
> 
> But :
> 
> $ echo -e '9\n4\n3\n2\n8\n7\n3\n1' |tee >(head -1) |tail -1
> 9
> 
> Please help explain
> 

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: ?maybe? RFE?... read -h ?

[felix]

On Sun, Sep 05, 2021 at 11:54:05PM -0400, Lawrence Velázquez wrote:
> On Sun, Sep 5, 2021, at 11:11 PM, Dale R. Worley wrote:

> > > ... I was wondering, is there a way for bash to know where the 
> > > symlink points (without using an external program)?
> 
> The distribution ships with a "realpath" loadable builtin, FWIW.
> bash-5.1$ enable -f /opt/local/lib/bash/realpath realpath
> bash-5.1$ realpath /usr/bin/cc
> /usr/bin/clang

I think this answer the need, but using this to populate a variable implie
anyway a fork.

  realPathArray=$(realpath /bin/sh /usr/bin/X)

It would be nice if builtins intended to produce *answer* use more or less
common switch like `printf -v` behaviour.

  usage: realpath [-a array] [-csv] pathname [pathname...]

  options: -a NAME assign the output to shell array NAME rather than
   display it on the standard output
   -c  check whether or not each resolved path exists
   -s  no output, exit status determines whether path is valid
   -v  produce verbose output



-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Squiggly heredoc - new feature request

[felix]

Hi,

> ... ignoring both leading spaces and leading tabs. eg.

Something like (there must be nothing but tabulation before closing ``EOF'')?

  func(){
    sed 's/^ \{2\}//' <<- EOF
      blabla
      EOF
  }

Depending on how and why, there is a lot of *variants*:
 - leading spaces and/or/not leading tabs
 - trailing spaces and/or/not trailing tabs
 - keeping rest of line structure as is or replacing every number of
 spaces/tabs by one space
 - ...

Even merging lines like `xargs` does, there is a way I've used in some scripts,
  ( like https://f-hauri.ch/vrac/live-install.sh.txt ):

   myXargs() { sed -ne '/#/d;s/ \t//g;H;${x;s/\n/ /g;s/^ //;p}'; }

   This drop lines containing `#`, every spaces and merge all line as space
   separated words. I use this as:

   ExtraBootParams=$(myXargs <<-eobp
   boot=live
   config
   #   locales=de_CH
   locales=fr_CH
   keyboard-layouts=ch
   keyboard-variant=fr
   persistent=cryptsetup
   persistence-encryption=luks
   persistence
   eobp
   )
   PackageList=$(myXargs <<-eopl
   gnome
   gnome-core
   # gnome-full
   # debian-forensics
   debian-installer-launcher
   eopl
   )

On Mon, Aug 30, 2021 at 10:06:37PM +0200, Přemysl Šťastný wrote:
> 
> I think, it would be nice, if you implemented Squiggly heredoc, which solves
> this problem by ignoring both leading spaces and leading tabs. eg.
> 
> func()(
>   cat <<~ EOF
>     blabla
>   EOF
> )
> 


-- 
 Félix Hauri  --  http://www.f-hauri.ch

Issue with parameter expansion with array when index > 2^31

[felix]

Configuration Information:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2 -Wno-parentheses -Wno-format-security
uname output: Linux medium 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 
(2021-03-19) x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 5.1
Patch Level: 8
Release Status: release

Description:
parameter expansion don't work correctly with array when index become
bigger than 31 bit value

Repeat-By:
for ((i=(1<<31)-3;i<(1<<31)+4;i++)){ a[i]='foo bar';}
declare -p a
declare -a a=([2147483645]="foo bar" [2147483646 .. 3651]="foo bar")

for i in ${!a[@]};do echo $i ${a[i]% *};done
2147483645 foo
2147483646 foo
2147483647 foo
2147483648
2147483649
2147483650
2147483651

Fix:
Not a fix, but:
paste -d\  <(printf %s\\n ${!a[@]}) <(printf %s\\n "${a[@]% *}")
2147483645 foo
2147483646 foo
2147483647 foo
2147483648 foo
2147483649 foo
2147483650 foo
2147483651 foo

Re: gettext feature request

[felix]

On Tue, Jul 27, 2021 at 10:44:09PM +0200, Jean-Jacques Brucker wrote:
> 
> I like what you wrote 12 years ago (
> https://lists.gnu.org/archive/html/bug-bash/2009-02/msg00258.html ). IMHO,
> it could have change some people's mind about bash, and could have saved
> developer energy.
> 
> 
> > $'...' already exists, so you can't use that for localization.
> 

I've done a little job around *localization*, approx 8 years ago:
  bash localization won't work with multilines strings (with strong
 syntax or through `eval`)
  https://stackoverflow.com/a/14147336/1765658

There are some interesting *works around*!

My last function is not perfect, but work on most cases:

 f() {
local store=false OPTIND OPTARG OPTERR varname
while getopts 'd:v:' opt ;do
case $opt in
d ) local TEXTDOMAIN=$OPTARG ;;
v ) varname=$OPTARG ;;
esac
done
shift $((OPTIND-1))
eval 'local msg=$"'"${1//[\"\$\`]}"\"
shift
printf ${varname+-v} $varname "$msg" "$@"
 }
 
 f -d libc -v string "Permission denied"
 echo $string
 Keine Berechtigung

 f -d coreutils $'Written by %s, %s, %s,\nand %s.\n' Huey Dewey Louie Georges
 Geschrieben von Huey, Dewey, Louie
 und Georges.


-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: `&>' doesn't behave as expected in POSIX mode

[felix]

On Sun, Jun 20, 2021 at 10:18:59AM -0400, Eli Schwartz wrote:
> 
> *Neither* behavior is similar to e.g. dash's behavior, or for that
> matter ksh. Both print the uname output after registering it as a
> backgrounded job (redirection does not get applied to the command itself).

Quick comparission script (note my default bash version in 5.0.3(1) and my 
 current working directory is my last bash-5.1.8 built):
  $ echo $BASH_VERSION $PWD
  5.0.3(1)-release /tmp/bash/bash-5.1.8

  $ for KindOfShell in csh ksh dash 'busybox sh' {,./}'bash --posix' ; do
  printf "\n#< %s >#\n" ${KindOfShell%% *}
  LANG=C $KindOfShell -xc 'uname &>/dev/null ls -ld /t{mp,nt}'
  done

  #< csh >#
  [1] 587
  uname
  Linux
  ls -ld /tmp /tnt
  ls: cannot access '/tnt': No such file or directory
  [1]  + Done   uname

  #< ksh >#
  + ls -ld /tmp /tnt
  + 1> /dev/null
  + uname
  Linux
  ls: cannot access '/tnt': No such file or directory

  #< dash >#
  + ls -ld /t{mp,nt}
  + uname
  Linux
  ls: cannot access '/t{mp,nt}': No such file or directory

  #< busybox >#
  + uname ls -ld '/t{mp,nt}'

  #< bash >#
  + uname ls -ld /tmp /tnt

  #< ./bash >#
  + uname ls -ld /tmp /tnt


There is some *different* behaviours... (but against `/t{mp,nt}` too!)

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: [PATCH] Prefer non-gender specific pronouns

[felix]

I totally agree with Léa: Awesome patch, but...

Please stop this troll!

As children of Gaia for me, there's no matter if Greg is a woman or Léa a
 men... Are bash male or female!?

This forum is technical, not political!

I've already seen many human groups splited because of stupid consideration
who was not linked with the core interest of group.

My purpose: Keep doc readable, until altering them become required (by laws).

On Sun, Jun 06, 2021 at 02:08:09PM +0200, Léa Gris wrote:
> Le 06/06/2021 à 06:35, G. Branden Robinson écrivait :
> > Here you go, if you're inclined.  Minimally invasive and decidedly
> > non-revolutionary in terms on English lexicon.
> 
> Your careful patch not using custom grammar is admirable. Although I remain
> alarmed because this is a work to obey a demand from cancel culture tenants.
> 
> With all your clever carefulness in patching this. It remains a rewrite of
> history motivated by political reasons from a lobbying group of people
> spreading their damaging delusions everywhere.
> 
> 
> 
> -- 
> Léa Gris
> 
> 

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: [patch #10070] toggle invert flag when reading `!'

[felix]

On Mon, May 24, 2021 at 11:47:32AM -0400, Chet Ramey wrote:
> On 5/22/21 2:45 PM, Vincent Menegaux wrote:
> 
> > Details:
> > 
> > Previously, these commands:
> > 
> >[[ ! 1 -eq 1 ]]; echo $?
> >[[ ! ! 1 -eq 1 ]]; echo $?
> > 
> > would both result in `1', since parsing `!' set CMD_INVERT_RETURN
> > instead of toggling it.

I will try this as:

  $ [ 1 -eq 1 ]; echo $?
  0
  $ ! [ 1 -eq 1 ]; echo $?
  1
  $ ! ! [ 1 -eq 1 ]; echo $?
  0



-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Undocumented feature: Unnamed fifo '<(:)'

[felix]

On Fri, Apr 09, 2021 at 01:23:09AM +0200, Léa Gris wrote:
> > You could have a look:
> >https://f-hauri.ch/vrac/mandelbrot_backgndBc.sh.txt
> >https://f-hauri.ch/vrac/mandelbrot_backgndBc_4macOs.sh.txt

> Have a look at my POSIX/shell version that is even slightly faster:
> https://gist.github.com/leagris/59e1b7e72462024b278652696f375e71

Faster if run under dash or busybox, not using bash!

But, why did you 
   exec 8<>"$bcin";exec 9<>"$bcout"
then don't use `echo >&8` nor `read <&9`?
As you are re-adressing FS at every read/write, your script is still
something slower! Test and compare my re-modified version of your (dropped
trap exit and addressing FSs, operate few seconds quicker).
  https://f-hauri.ch/vrac/mandelbrot_posix.sh.txt

But there is less to do with [bash].

> There is no need for bash specific features, although coproc and other
> fancynesses can help, these lack portability.

Of course many bashisms could be avoided to stay POSIX portable.
In that way, commands like `coproc` are useless if not counter productive.
My meaning here was to improve coproc in order to by able to drive STDERR too.

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Undocumented feature: Unnamed fifo '<(:)'

[felix]

Comming back on this 10month old thread...

On Tue, Jun 30, 2020 at 12:30:53PM -0400, Chet Ramey wrote:
> On 6/28/20 9:49 AM, felix wrote:
> 
> > Bash Versions: 3.2.57(1)-release, 5.0.3(1)-release, 5.1.0(1)-alpha
> > 
> > In order to reduce forks and make some tasks a lot quicker, I run
> > forked filters as background tasks, with dedicated I/O fd.
> > 
> > For sample, to convert human datetime to UNIX SECONDS, instead of running
> > ...
> > 
> > become a lot quicker!

Found this bash mandelbrot sample at Stackoverflow:
  https://stackoverflow.com/a/63749612/1765658

This script take many hours to draw a mandelbrot.

Reducing number of fork to bc to only one, then same draw is done in less
than 10 minutes.

> But not guaranteed to work everywhere, especially those systems that use
> /dev/fd and pipes for this instead of FIFOs. (And lack `stdbuf'.)

Yes, to make this run under MacOS. I first dropped `stdbuf`, then because
of an ``old'' version (3.2.57(1)-release) of bash, there is the most
relevant part of my patch:
  -exec {bcout}<> <(:)
  -exec {bcin}> >(exec stdbuf -o0 bc -l >&$bcout)
  +fifo=/tmp/fifo-$RANDOM$RANDOM-$$
  +mkfifo $fifo
  +exec 10> >(exec bc -l >$fifo)
  +exec 11<$fifo
  +rm $fifo

You could have a look:
  https://f-hauri.ch/vrac/mandelbrot_backgndBc.sh.txt
  https://f-hauri.ch/vrac/mandelbrot_backgndBc_4macOs.sh.txt

> > There is maybe something to document or even create a new feature
> > about open2 and open3...

I was wrong about ``open2'', there is already ``coproc''.

But about ``open3'' permitting to pass all 3 STDIN STDOUT and STDERR,
there is maybe something to do!?

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: zsh style associative array assignment bug

[felix]

Quick array assignment from command...

> On Tue, Mar 30, 2021 at 12:42:46PM -0400, Eric Cook wrote:
> > eval 'tags=('"${*@Q}"\)

On Tue, Mar 30, 2021 at 01:16:14PM -0400, Greg Wooledge wrote:
> declare -A tags=()
> while IFS=\| read -r tag value; do
>   tags[$tag]=$value
> done < <(exiftool ...)

As bash read loop could be something slow, I use (when I'm quiet about data
origin) something like:

  declare -A tags="($(
  sed -e 's/^\([^|]*\)|\?\(.*\)/[\1]="\2"/' < <(
 exiftool ...)) )"

Then
  declare -p tags
  declare -A tags=([Title]="A2 - Northern Lights" [Artist]="AK420" 
   [Genre]="lofi" [Track]="" )

Note `|\?` in sed command will ensure EACH line would be ouput as `[LHS]="RHS"`
even if RHS is empty and pipe separator is missing.

But I have to confess: If not worst than using `eval`,
  I'm not absolutely sure about border effect and security issues.

Other sample, there is a quick `tcpstat` working on Linux:

declare -a fds=(/proc/[1-9]*/fd/*)
declare -a sockets="($(
sed < <(ls -l  "${fds[@]}" 2>&1
) -ne 's@^.*proc/\([0-9]\+\)/.*socket:.\([0-9]\+\).@[\2]+="\1 "@p'))"
while IFS=': ' read foo{,} port foo{,} mode foo{,,} node foo; do
[ "$mode" = "0A" ] && [ "${sockets[node]}" ] &&
while IFS= read -ru $lps line; do
printf '%8d %s\n' 0x$port "$line"
  done {lps}< <(ps h ${sockets[node]})
done < /proc/net/tcp

Where, when (not associative) array is declared as

declare -a sockets=( [12345]+="1234 " [123]+="234 " [12345]+="456 " )

they hold:

declare -p sockets
declare -a sockets=([123]="234 " [12345]="1234 456 ")

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: missing way to extract data out of data

[felix]

I think, memory footprint doesn't really matter when standard availlable ram
is lot away...

But *time required to ``fork''* could matter:

  QuickTest() {
local TIMEFORMAT='%R ( %U + %S )';
printf "%-10s: " "${1##*/}";
time for i in {1..1000};
do
res=$("$@");
done;
[ "$res" != "12" ] && echo "WARN: $1: '$res' != 12."
  }
  CacheHostName=$(&1;
done |
  sort -nk 3

After ~23 seconds, this print (on my host):

  dash  : 1.667 ( 1.206 + 0.519 )
  busybox   : 1.868 ( 1.264 + 0.665 )
  sed   : 2.150 ( 1.364 + 0.841 )
  bash  : 2.201 ( 1.520 + 0.719 )
  perl  : 2.608 ( 1.679 + 0.978 )
  awk   : 2.662 ( 1.666 + 1.041 )
  python: 9.881 ( 6.867 + 3.052 )

Of course, all this was called without modules, so this times represent
something like *minimal footprint when forked to*.

There is no more consideration of efficience when used, for sample, as
filter throught big volume of data, for making complex calculs, etc...

On Wed, Mar 24, 2021 at 10:52:16PM -0400, Dale R. Worley wrote:
> Andreas Schwab  writes:
> > $ size /usr/bin/perl /bin/bash
> >textdata bss dec hex filename
> > 2068661   27364 648 2096673  1ffe21 /usr/bin/perl
> > 1056850   22188   61040 1140078  11656e /bin/bash
> $ size /usr/bin/perl /bin/bash
>text  data bss dec hex filename
>8588   876   0946424f8 /usr/bin/perl
>  898672 36064   22840  957576   e9c88 /bin/bash
> 
> I do suspect that's because perl is using more loadable modules than
...

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: 'exec' produced internal code output instead of normal

[felix]

On Sat, Mar 13, 2021 at 01:05:32AM +0100, Alex fxmbsw7 Ratchev wrote:
> i have no example to write here as this was past long
Without sample, i'ts hard to represent your case!

> the story was, i was coding a file server daemon, with socat,
Wow!

> and i figured to use exec why not more exact more efficient

Something like:

-- In any terminal (first)
  $ coproc socat TCP4-LISTEN: STDIO
  $ while read -u $COPROC foo;do echo $foo; echo $RANDOM >&${COPROC[1]};done

-- In another terminal
  $ exec {netconn}<>/dev/tcp/localhost/
  $ echo >&$netconn foo ; read -u $netconn foo ; echo $foo
  24522
  $ echo >&$netconn bar ; read -u $netconn foo ; echo $foo
  29969

(while you will see in first terminal:
  foo
  bar
)

> but using it resulted sometimes output of code of the script in the output
> of the files
> removing exec made it work normally as supposed

Or are did you mean `EXEC`  of `socat`?

-- Don't run this if you don't understand what you do!  
  socat TCP4-LISTEN: EXEC:/bin/bash

??

-- 
 Félix

uname lead to embed code execution

[felix]

There it is:

$ declare -A map; key='foo$(uname >/dev/tty)bar'; map[$key]=
$ echo map["$key"]
map[foo$(uname >/dev/tty)bar]
$ echo ${map["$key"]}

$ unset map["$key"]
Linux

Or even:

$ declare -A map; key='foo$(read -t 3 /dev/tty bar: 
$foo)bar'; map[$key]=
...
$ unset map["$key"]
baz  # hitted in  3 seconds
bar: baz

-- 
 Félix Hauri

Re: is it normal that set -x unset commands dont display special chars in the content

[felix]

On Sun, Feb 28, 2021 at 10:32:13PM +, k...@plushkava.net wrote:
> Why not indeed. However, I think that I'm able to decipher this.
> 
> $ declare -A map; key=$'foo\34bar'; map[$key]=
> $ set -x
> $ : "$key"
> + : $'foo\034bar'
> $ unset "map[$key]"
> + unset 'map[foobar]'
> 
> For the unset command, xtrace elects to emit the (non-printable) FS
> character verbatim rather than employ the ANSI-style notation. Presumably,
> this led to the conclusion that bash doesn't "display em".

I agree:

$ declare -A map; key=$'foo\34bar'; map[$key]=
$ exec {BASH_XTRACEFD}> >(sed -ue 's/\o34/<<\\34>>/')
$ set -x
$ : "$key"
+ : $'foo\034bar'
$ unset map["$key"]
+ unset 'map[foo<<\34>>bar]'


-- 
 Félix Hauri   -  -   http://www.f-hauri.ch

Re: [PATCH] Fix blocking read timeouts at a small probability

[felix]

On Mon, Feb 15, 2021 at 09:41:29AM -0500, Chet Ramey wrote:
> Just for testing, not for any real use case.
> 
> > He also provided me with his new test case
> > to see the interaction with SIGCHLD. The failure of this test case
> > seems to have also involved SIGCHLD in the previous devel Bash. Now in
> > the current devel, it is improved but still sometimes blocks.

Originally, this was a (my) bug in a multi-tasked script**, where timeout was
(wrongly) computed from previous step. When my script locked for unknown
reason, I wrote test script in order to reveal this.

> If you need a timeout on the order of a single microsecond in a shell
> script, I urge you to reconsider your choices.
I agree, for this kind of tricks, Python (or even Perl) is more apropriated,
but I like to see how a poor system (with only few binary libraries) could
do unattended things...

...And it's fun to see how far we can go with bash, today.

  ** https://f-hauri.ch/vrac/multiping.sh.txt
-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: feature request: ${ regex s expander, and multiple expandrs inside one

[felix]

On Fri, Feb 05, 2021 at 09:29:49PM +0100, Alex fxmbsw7 Ratchev wrote:
> ..
> 
> var=mynewfoo
> printf ${var~~n[^f]+}
>  ->
>  myfoo
> 
> and also multiple expanders inside one statement would greatly speed up
> processment and also fulfill old greatly coding wanting possibilities

This could be done by:

  var=mynewfoo
  echo ${var/n*f/f}
  myfoo

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: V+=1 doesn't work if V is a reference to an integer array element

[felix]

This seem linked:

  $ f() {
local -a a=('' 'foo');
local -n b=a[1];
echo $b;
b+=\ bar;
echo $b;
declare -p a;
}

  $ f
  foo
  foo bar
  declare -a a=([0]="" [1]="foo bar")

Ok, fine!

  $ bash <(declare -f f | sed '1,2d; $s/.*/echo $BASH_VERSION/;
   s/local /declare /;')
  foo
  fooa[1] bar
  declare -a a=([0]="" [1]="fooa[1] bar")
  5.1.4(1)-release

There is something!
New try:

  $ f() {
local -ai a=(0 12);
local -n b=a[1];
echo $b;
b+=4;
echo $b;
declare -p a;
}

  $ f
  12
  16
  declare -ai a=([0]="0" [1]="16")

Seem good!

  $ bash <(declare -f f |
   sed '1,2d;$s/.*/echo $BASH_VERSION/;s/local /declare /;')
  12
  /dev/fd/63: line 4: a[1]4: syntax error in expression (error token is "4")
  12
  declare -ai a=([0]="0" [1]="12")
  5.1.4(1)-release

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Bug Using Brackets

[felix]

Hi,

On Wed, Jan 06, 2021 at 11:32:49PM +0800, Harry Lee wrote:

> root@server:~# bash -version
> GNU bash, version 5.0.3(1)-release (x86_64-pc-linux-gnu)
> Copyright (C) 2019 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later 
> 
> This is free software; you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> root@server:~# echo [0-1]
> 1
> root@server:~# echo [208174hkjdfsglh0-13984jf0]
> 1
I use near same version, but can't reproduce:

  $ echo $BASH_VERSION 
  5.0.3(1)-release

  $ echo [0-1]
  [0-1]

Please try this:

  $ LANG=C bash -norc -c 'echo [0-1]'


-- 
 Félix Hauri  --  http://www.f-hauri.ch

Strange behaviour using negative index in arrays

[felix]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2 -Wno-parentheses -Wno-format-security
uname output: Linux medium 4.19.0-9-amd64 #1 SMP Debian 4.19.118-2 (2020-04-29) 
x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 5.1
Patch Level: 4
Release Status: release

Description:
Assigning array with negative number not alway detected as an error

Trying to sort integers, I was using:

f() { local a i;for i;{ a[$i]=;echo $i;};a=(${!a[@]});echo $a;}

This could work fine with any positive numbers, but trying:

f 34 56 12 56 53 -12 -21
this return quietly `12` ?? (No error)

( Under 4.2.37(1)-release, I get: ``-bash: a[$i]: bad array subscript'' )

But with ``my'' 5.0.3(1)-release, as with new 5.1.4(1)-release, no error...

Further:

f() { local a i c;for i;{ a[$i]=$((c++));echo $i;};declare -p a ;
   a=(${!a[@]}); declare -p a; echo $a ;}

f 34 56 12 56 53 -12 -21
34
56
12
56
53
-12
-21
declare -a a=([12]="2" [34]="0" [36]="6" [45]="5" [53]="4" [56]="3")
declare -a a=([0]="12" [1]="34" [2]="36" [3]="45" [4]="53" [5]="56")
12

Where 5th value is 45 instead of -12 and 6th is 36 instead of -21
 45 + 12 = 57,  36 + 21 = 57 ... (highest submited value was 56!)??

f 34 56 12 1000 53 -12 -21
34
56
12
1000
53
-12
-21
declare -a a=([12]="2" [34]="0" [53]="4" [56]="1" [980]="6" [989]="5" 
[1000]="3")
declare -a a=([0]="12" [1]="34" [2]="53" [3]="56" [4]="980" [5]="989" 
[6]="1000")
12

Then now:
989 + 12 = 1001, 980 + 21 = 1001 ( and highest value was 1000 :)


... And happy new year!!

Re: New Feature Request

[felix]

I agree: python seem to be more apropriated language for complex operation.

Anyway, bash already offer a lot of features (like `coproc` and `read -t 0`)
usefull for IPC.

I wrote a little ``multiping`` bash script, as multithread demo, running many
parallels ping, reading all outputs and merging them in one line.

Sample: (Hitting `q` after ~4 seconds)

$ multiping.sh www.google.com www.archlinux.org www.f-hauri.ch
Started: PING www.google.com (172.217.168.68) 56(84) bytes of data.
Started: PING www.archlinux.org (95.217.163.246) 56(84) bytes of data.
Started: PING www.f-hauri.ch (62.220.134.117) 56(84) bytes of data.
   www.google.com  www.archlinux.orgwww.f-hauri.ch
11:00:10  1 12.61 46.10   1 9.27   
11:00:11  2 12.02 47.42 9.24   
11:00:12  3 12.73 47.63 9.22   
11:00:18  4 10.84 46.54 9.40   
www.google.com 4 / 4 -> 0%err. 10.822/12.017/12.661/0.734 ms
www.archlinux.org 4 / 4 -> 0%err. 46.466/47.125/47.632/0.493 ms
www.f-hauri.ch 4 / 4 -> 0%err. 9.219/9.282/9.404/0.120 ms

You could find them there:
  https://f-hauri.ch/vrac/multiping.sh.txt
  https://f-hauri.ch/vrac/multiping.sh

On Sun, Dec 27, 2020 at 08:26:49PM +0100, Léa Gris wrote:
> On 27/12/2020 at 19:30, Saint Michael wrote:
> > Yes, superglobal is great.
> 
> Maybe you should consider that Bash or shell is not the right tool for your
> needs.
> 
> If you need to manipulate complex objects, work with shared resources, Bash
> is a very bad choice. If you want to stay with scripting, as you already
> mentioned using Python; Python is a way better choice for dealing with the
> features and requirements you describes.

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: No expansions performed while declaring an associative array using a list of keys and values

[felix]

On Fri, Dec 11, 2020 at 03:08:04PM +0300, Oğuz wrote:
> I was trying the new features of bash 5.1 and came across this inconsistent
> behavior:
> 
> $ foo='1 2'
> $ declare -A bar=($foo 3)
> $ declare -p bar
> declare -A bar=(["\$foo"]="3" )
> $
> $ bar+=($foo 3)
> $ declare -p bar
> declare -A bar=(["\$foo"]="3" ["1 2"]="3" )
> 
> Is there a particular reason to avoid performing expansions in `declare -A
> bar=($foo 3)'?

When declaring ``associative array'', I alway use ``quoted'' syntax: 

$ foo='1 2'
$ declare -A bar='($foo 3)'
$ declare -p bar
declare -A bar=(["1 2"]="3" )

Not sure how to explain this, but I'm not surprised seeing this working so.

$ bar+=($foo 3)
$ declare -p bar
declare -A bar=(["1 2"]="3" )

Seem ok for me!

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: bash doesn't display user-typed characters; can interfere with COPY/PASTE

[felix]

On Tue, Dec 08, 2020 at 08:07:18PM +0700, pepa65 wrote:
> On 08/12/2020 19.55, Greg Wooledge wrote:
> > Some terminals, when fed a tab character, will preserve that knowledge
> > in memory; then, when you copy text from that part of the terminal
> > window using your mouse, the terminal will put a tab byte into the
> > selection/clipboard.
> 
> Interesting! Which terminal does that??

It seem that ``gnome-terminal'' and ``mate-terminal'' do so.

while ``xterm'', ``rxvt'' and ``konsole'' does'nt.

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: increment & decrement error when variable is 0

[felix]

On Mon, Nov 23, 2020 at 05:20:22PM -0500, Greg Wooledge wrote:
> On Mon, Nov 23, 2020 at 07:36:29PM +, Jetzer, Bill wrote:
>...
> Exercises 1 and 2 apply directly...

>From man bash:
   ((expression))
  The expression is evaluated according to the rules described be‐
  low under ARITHMETIC EVALUATION.  If the value of the expression
  is non-zero, the return status is 0; otherwise the return status
  is 1.  This is exactly equivalent to let "expression".

So little modified test/demo:

for ((i=-100; i<100; i++)) ;do
x=$i
((v = --x ,v)) || echo "err code $? on --x going from $i to $x -> $v"
x=$i
((v = x-- ,v)) || echo "err code $? on x-- going from $i to $x -> $v"
x=$i
((v = ++x ,v)) || echo "err code $? on ++x going from $i to $x -> $v"
x=$i
((v = x++ ,v)) || echo "err code $? on x++ going from $i to $x -> $v"
done

Will render near same output:

err code 1 on ++x going from -1 to 0 -> 0
err code 1 on x-- going from 0 to -1 -> 0
err code 1 on x++ going from 0 to 1 -> 0
err code 1 on --x going from 1 to 0 -> 0

Where *when the value of the expression is zero, the return status is 1*.

Difference between ((x--)) and ((--x)) is only usefull when *using* this
variable simultaneously while decrementing them (for assignment, testing,
or printing):

i=10;while ((i));do echo $i;((i--));done
and
i=10;while ((i));do echo $i;((--i));done
will do same result.
But
i=10;while ((i));do echo $((i--));done  # return 10 to 1
i=10;while ((i));do echo $((--i));done  # print 9 to 0
or
i=10;while ((i--));do echo $i;done   # print 9 to 0
i=10;while ((--i));do echo $i;done   # print 9 to 1 (exit at 0 before print)

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: [ping] declare -c still undocumented.

[felix]

On Fri, Nov 13, 2020 at 12:19:27AM +0100, Léa Gris wrote:
> 
> Happy 10 years 10 months anniversary to the issue:
> 
> 
> 
> > address@hidden:~$ declare -c moo=moo; echo $moo
> > Moo

  cap() { local -ca _cap=("$@");echo ${_cap[*]};}
  cap great éric.
  Great Éric.

Like this:
  https://lists.gnu.org/archive/html/bug-bash/2010-02/msg00081.html

  ${parameter:~pattern}  Toggle case of 1st character if matching optional 
pattern
  ${parameter:~~pattern} Toggle case of all characters regarding pattern:

  cap() { local -cl _cap=("$@");echo ${_cap[*]~};}
  cap great éric. 
  Great Éric.

  myvar="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"; echo 
${myvar~~[H-p]}
  ABCDEFGhijklmnopqrstuvwxyzABCDEFGHIJKLMNOPqrstuvwxyz


-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: bash: please provide an easy way to spawn a new process group from script

[felix]

On Sun, Nov 01, 2020 at 11:13:37AM +0100, clime wrote:
> On Sun, 1 Nov 2020 at 11:01, Oğuz  wrote:
> >
> > You can use the loadable builtin `setpgid' if you have to. Assuming 
> > BASH_LOADABLES_BUILTIN is set, this should work:
> >
> > enable -f setpgid{,}
> > { setpgid $BASHPID{,}; a | b; } &
> > setpgid $!{,}
> > kill -- -$!
> 
> Hello, it doesn't work for me:
I've not found any reference about
kill -- -$!
but try this:

clear;tty=$(tty) tty=${tty#*/dev/};{
{ sleep 2; echo a ;} |
{ wc -c ; sleep 2; echo b ;}
} & ps --tty $tty fw; kill -s INT -$! ; ps --tty $tty fw

Seem make the job.

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: read with very small timeout sometime hand on stdin

[felix]

On Thu, Oct 29, 2020 at 01:57:33PM +0100, felix wrote:
> > I can't reproduce this using the following stripped-down reproducer:

Sorry to insist, but... This is not easy because not constant.

For this, I wrote a little script tested on several host, then searching
for installation that was not mine... Tested at https://www.jdoodle.com/ia/3EO

  $ sed <<"eotest" >/tmp/test-timeout-race.sh 's/^//'
#!/bin/bash

declare -p BASH_VERSI{NFO,ON}
uptime
uname -a

po() {
for f in {1..1};do
read -t .01 v
rc=$?
[ $rc -ne 142 ] || [ "$v" ] &&
echo f:$f, v:$v, RC:$rc.
done < <(
for i in {1..1};do sleep 3;echo Timed;done
)
}

exec 2>&1
TIMEFORMAT="U:%U S:%S R:%R"
for test in {1..20};do
time po
done
eotest

If first test don't show strange** output, run test again.
 (** lines begining by `f:`)

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: read with very small timeout sometime hand on stdin

[felix]

On Wed, Oct 28, 2020 at 09:23:09AM -0400, Chet Ramey wrote:
> 
> I can't reproduce this using the following stripped-down reproducer:

> trap 'echo $f ; exit' SIGINT
> 
> for f in {1..1}; do
>   read -t .01 v
>   if [ $? -ne 142 ]; then
>   echo $f: $?
>   fi
> done
This is strange: $? seem to be alway 142! On my host:
  Debian 10.4 - Intel(R) Core(TM) i5-4570 CPU @ 3.20GHz - 32Gb ram

  $ TIMEFORMAT="U:%U S:%S R:%R"
  $ time for f in {1..1000};do read -t .01 v; [ $? -ne 142 ] &&
  echo f:$f, v:$v, RC:$?. ; done < <(
  for i in {1..1000};do sleep 3;echo;done)
  U:0.206 S:0.032 R:0.240

  $ time for f in {1..1000};do read -t .01 v; [ $? -ne 142 ] &&
  echo f:$f, v:$v, RC:$?. ; done < <(
  for i in {1..1000};do sleep 3;echo;done)
  U:0.167 S:0.068 R:0.236

  $ time for f in {1..1000};do read -t .01 v; [ $? -ne 142 ] &&
  echo f:$f, v:$v, RC:$?. ; done < <(
  for i in {1..1000};do sleep 3;echo;done)
  U:0.207 S:0.049 R:24.152

  $ time for f in {1..1000};do read -t .01 v; [ $? -ne 142 ] &&
  echo f:$f, v:$v, RC:$?. ; done < <(
  for i in {1..1000};do sleep 3;echo;done)
  U:0.177 S:0.067 R:6.233

Where 6 second seem represent two `sleep 3;echo` ...

Same on my raspberry-pi (BCM2835 ARMv6-compatible rev 7 (v6l) - 512M),
  Machine Type: armv6l-unknown-linux-gnueabihf - 5.1.0.rc1

...but with 100 loop and 10 second sleep:


  $ TIMEFORMAT="U:%U S:%S R:%R"
  $ time for f in {1..100};do read -t .01 v; [ $? -ne 142 ] &&
echo f:$f, v:$v, RC:$?. ; done < <(
for i in {1..100};do sleep 10;echo;done ) 
  U:0.447 S:0.021 R:0.995
  
  $ time for f in {1..100};do read -t .01 v; [ $? -ne 142 ] &&
echo f:$f, v:$v, RC:$?. ; done < <(
for i in {1..100};do sleep 10;echo;done ) 
  U:8.589 S:0.108 R:10.918

  $ time for f in {1..100};do read -t .01 v; [ $? -ne 142 ] &&
echo f:$f, v:$v, RC:$?. ; done < <(
for i in {1..100};do sleep 10;echo;done ) 
  U:0.445 S:0.022 R:0.999

  $ time for f in {1..100};do read -t .01 v; [ $? -ne 142 ] &&
echo f:$f, v:$v, RC:$?. ; done < <(
for i in {1..100};do sleep 10;echo;done ) 
  U:0.540 S:0.065 R:10.958

Again, no output! (Result code seem to be alway 142)

But clearly without `sleep 10; echo` loop as input, this test may hang...
until you hit ...  maybe several times...

Nota: None of the host used to make this tests are really ``quiet'':
 My host loadavg: 0.48 0.78 0.91 2/1074 15425
 Raspbery load  : 0.93 1.01 1.12 2/87 28120

This seem harder to reproduce on quiet system
 Other host load: 0.00 0.04 0.00 1/772 468
 U:0.015 S:0.008 R:0.023
 U:0.021 S:0.000 R:0.023
 U:0.021 S:0.000 R:0.022
 U:0.015 S:0.010 R:3.023
 U:0.015 S:0.004 R:0.020
 U:0.018 S:0.004 R:0.023
 U:0.010 S:0.008 R:0.019
 U:0.010 S:0.012 R:0.023
 U:0.023 S:0.000 R:0.024
 U:0.015 S:0.004 R:0.020
 U:0.019 S:0.000 R:0.020
 U:0.015 S:0.008 R:0.023
 U:0.038 S:0.008 R:3.023
 U:0.023 S:0.000 R:0.023
 U:0.030 S:0.000 R:3.025
 U:0.009 S:0.008 R:0.018
 U:0.020 S:0.007 R:3.031


-- 
 Félix Hauri  --  http://www.f-hauri.ch

read with very small timeout sometime hand on stdin

[felix]

Configuration Information:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2 -Wno-parentheses -Wno-format-security
uname output: Linux medium 4.19.0-9-amd64 #1 SMP Debian 4.19.118-2 (2020-04-29) 
x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 5.1
Patch Level: 0
Release Status: rc1

Description:
Trying to see limits of timeout with `read -t`, I encounter strange
and unconstant result: sometime `read -t .01` don't consider
timeout, when running fastly multiple time.

From help:
  -t timeout time out and return failure... may be a fractional...
 ... exit status is greater than 128 if the timeout is exceeded.

Repeat-By:

$ read -t .001 foo ; echo $?
1

$ read -t .01 foo ;echo $?
142

Ok, then microsecond seem to by smallest value.

$ tot=0; for i in {1..100} ;do
rt=${EPOCHREALTIME//.}
read -t .01 foo
((tot+=${EPOCHREALTIME//.}-rt))
sleep .002
  done; echo ${tot:: -2}.${tot: -2}

If this not end after 1 second, hit , wait 1 second
and repeat (and count); You won't hit 100x !
Maybe this return correctly after ~0.2 seconds. If yes, try
again. Mostly I have to hit 3 - 8x for this 100 loop test.

$ tot=0; for i in {1..100} ;do
rt=${EPOCHREALTIME//.}
read -t .01 foo
((tot+=${EPOCHREALTIME//.}-rt))
done < <(
while :;do sleep 1;echo;done
);echo ${tot:: -2}.${tot: -2}

Correct output must be < 1000


-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Subject: Pressing Ctrl-C during any subshell evaluation terminates the shell

[felix]

Issue 627 in direnv's github project show:

To reproduce this simply:

PROMPT_COMMAND="trap -- '' SIGINT; sleep 0.1; trap SIGINT;"
read foo
^C   -- will exit current shell --



On Mon, Oct 12, 2020 at 09:52:21PM -0700, Daniel Farina wrote:
> On Sun, Oct 11, 2020 at 12:59 PM Chet Ramey  wrote:
> > What commands does this `direnv' command output?
> 
> _direnv_hook() {
>   local previous_exit_status=$?;
>   trap -- '' SIGINT;
>   eval "$("{{.SelfPath}}" export bash)";
>   trap - SIGINT;
>   return $previous_exit_status;
> };

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Bash-5.1-beta available

[felix]

Hi,

On Mon, Sep 21, 2020 at 09:13:13PM -0400, Dale R. Worley wrote:
> Andreas Schwab  writes:
> I assume that if you really want the old effect, you can still do
> 
> exec {dup}<&1
You mean:
exec {dup}<&0

> 
> ... <( ... <$dup ) ...
and:

... <( ... <&$dup ) ...

> 
> exec {dup}<&-
> 
> Dale
Sample:

This worked under previous bash versions (I use 5.0.3(1)-release):

testForkInput () {
local line 
while read line ;do
echo "$line" 
done < <(
sed 's/^/> /'
)
}

But with 5.1.0(1)-beta, I have to replace this with:

testForkInput () {
local dup line
exec {dup}<&0
while read line ;do
echo "$line"
done < <(
sed 's/^/> /' <&$dup
)
exec {dup}>&-
}

testForkInput < <(seq 1 5)
> 1
> 2
> 3
> 4
> 5

On Tue, Sep 22, 2020 at 08:57:57AM +0200, Andreas Schwab wrote:
> That point is that it silently breaks existing scripts.

I agree.

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Undocumented feature: Unnamed fifo '<(:)'

[felix]

Thanks Pierre,

On Sun, Jun 28, 2020 at 10:48:42PM +0200, Pierre Gaston wrote:
> Maybe "coproc" is already the feature you need (limited to only 1 though)?

Correct: I missed this!

This work fine:

  coproc stdbuf -o0 date -f -  +%s 2>&1
  DATEIN=${COPROC[1]} DATEOUT=$COPROC
  echo >&$DATEIN 2009-02-13 23:31:30 UTC;read -u $DATEOUT out;declare -p out
  declare -- out="1234567890"
  echo foo bar >&$DATEIN ;read -u $DATEOUT out;declare -p out
  declare -- out="date: invalid date 'foo bar'"

Unfortunely, coproc don't have option for separated error output...
If I want to handle errors, I could do this by using ``<(:)'' again:

  exec 8<> <(:)
  coproc stdbuf -o0 date -f -  +%s 2>&8
  DATEIN=${COPROC[1]} DATEOUT=$COPROC DATEERR=8
  bound=wrong_date_$(uuidgen)
  date_to_epoch() { 
  local _out
  echo ${@:2}$'\n'$bound 1>&$DATEIN
  read -u $DATEERR _out
  if [ -z "${_out//*$bound*}" ]; then
  read -u $DATEOUT $1
  else
  printf -v $1 %s "$_out"
  read -u $DATEERR _out
  return -1
  fi
  }

  Usage: date_to_epoch  

Then

  if date_to_epoch result 2009-02-13 23:31:30 UTC ;then
echo $result ; else echo ERROR: $result ;fi
  1234567890

  if date_to_epoch result bad entry  ;then
echo $result ; else echo ERROR: $result ;fi
  ERROR: date: invalid date 'bad entry'

Again, I use this for not only with `date` and `bc`, but with `mysql`, `ftp`
 or even `ssh` too.

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Undocumented feature: Unnamed fifo '<(:)'

[felix]

I totally agree, but this work from at least bash-3.2 upto last version!

But if a bug, a feature request:

  exec 8< 9> <>(date -f - +%s)

( ..or, but I don't imagine how this could be implemanted:
  exec 8<> <>(date -f - +%s)
:-p )

...

AND even:

Same `-u` option for `read -u $FD` available for `printf -u $FD`.

On Sun, Jun 28, 2020 at 07:31:53PM +0300, Oğuz wrote:
> 
> Seems like a bug to me. The FIFO created for <(:) shouldn't be writable.

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Undocumented feature: Unnamed fifo '<(:)'

[felix]

In order to prevent ressource eating, once `date -f - +%s` runned as 
background, date_to_epoch will become:

$ exec 8<> <(:)
$ exec 9> >(exec stdbuf -o0 date -f - +%s >&8 2>&8)
$ date_to_epoch() {
echo >&9 ${@:2}
read -t 1 -u 8 ${1:-answeredEpoch}
}

$ date_to_epoch _out 2009-02-13 23:31:30 UTC
$ echo $_out
1234567890

U could test my demo script at 
https://f-hauri.ch/vrac/date1fork-demo.sh.txt

or check for my `shell_connector' at 
https://f-hauri.ch/vrac/shell_connector.sh.txt

This is not only interesting for `date`! I use them with `bc`, `tput`,
and lot of other applications, upto `sqlite`, `mysql` and `postgresql`
with ability of playing with temporary tables and other session related
stuff...

On Sun, Jun 28, 2020 at 10:55:59AM -0500, Dennis Williamson wrote:
> date_to_epoch () {
> date -d "$1" +%s
> }
> 
> _out=$(date_to_epoch "$_string")


> On Sun, Jun 28, 2020, 8:50 AM felix  wrote:
> > ...
> > _fifo=$(mktemp -u /tmp/fifo-)
> > mkfifo $_fifo
> > exec 9> >(exec stdbuf -o0 date -f - +%s >$_fifo 2>&1)
> > exec 8<$_fifo
> > rm $_fifo
> >
> > Then to convert human datetime to UNIX SECONDS:
> > echo >&9 $_string
> > read -t 1 -u 8 _out

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Undocumented feature: Unnamed fifo '<(:)'

[felix]

Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2 -fdebug-prefix-map=/build/bash-2bxm7h/bash-5.0=. 
-fstack-protector-strong -Wformat -Werror=format-security -Wall 
-Wno-parentheses -Wno-format-security
uname output: Linux medium 4.19.0-9-amd64 #1 SMP Debian 4.19.118-2 (2020-04-29) 
x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Versions: 3.2.57(1)-release, 5.0.3(1)-release, 5.1.0(1)-alpha

In order to reduce forks and make some tasks a lot quicker, I run
forked filters as background tasks, with dedicated I/O fd.

For sample, to convert human datetime to UNIX SECONDS, instead of running
   _out=$(date -d "$_string" +%s)
many time in same script, I run something like:

_fifo=$(mktemp -u /tmp/fifo-)
mkfifo $_fifo
exec 9> >(exec stdbuf -o0 date -f - +%s >$_fifo 2>&1)
exec 8<$_fifo
rm $_fifo

Then to convert human datetime to UNIX SECONDS:

echo >&9 $_string
read -t 1 -u 8 _out

become a lot quicker!

But I recently discovered another way for this:

exec 8<> <(:)
exec 9> >(exec stdbuf -o0 date -f - +%s >&8 2>&8)

usable in same way:

echo >&9 $_string
read -t 1 -u 8 _out

maybe a little more quicker...

This was tested using https://f-hauri.ch/vrac/date1fork-demo.sh under
latest version of bash: 5.1.0(1), debian stable version: 5.0.3(1) and
old 3.2: 3.2.57(1).

There is maybe something to document or even create a new feature
about open2 and open3...

(I will have to rewrite https://f-hauri.ch/vrac/shell_connector.sh.txt ;)

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Re: Command substitution

[felix]

Quoting is useless when assigning variable from ouptut of command or
another variable:

$ foo=1 2 3 4 
bash: 2: command not found

Ok. But

$ foo=$(seq 1 3)
$ declare -p foo
declare -- foo="1
2
3"

$ foo="$(seq 1 3)"
$ declare -p foo
declare -- foo="1
2
3"

No difference with or without double-quote.

$ bar=$foo
$ declare -p bar
declare -- bar="1
2
3"

$ bar="$foo"
$ declare -p bar
declare -- bar="1
2
3"

Again, double-quoting is useless.

But for commands:

$ printf "<%s>\n" $foo
<1>
<2>
<3>
$ printf "<%s>\n" "$foo"
<1
2
3>

Same than

$ printf "<%s>\n" $(seq 1 3) 
<1>
<2>
<3>
$ printf "<%s>\n" "$(seq 1 3)"
<1
2
3>

Unfortunely, I don't retrieve this behaviour in man page.

On Tue, Jun 02, 2020 at 09:44:45PM -0400, Dale R. Worley wrote:
> Naively, I expect that
> 
> FOO="$( command2 )"
> command1 $FOO
> 
> has the same effect as
> 
> command1 $( command2 )
> 
> and
> 
> FOO="$( command2 )"
> command1 "$FOO"
> 
> has the same effect as
> 
> command1 "$( command2 )"
> 
> Has anyone pushed the boundaries of this and can tell me whether there
> are gotchas?
> 
> Dale
> 

-- 
 Félix Hauri  --  http://www.f-hauri.ch

Difference between EPOCHREALTIME and EPOCHSECONDS

[felix]

Configuration Information:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2 -Wno-parentheses -Wno-format-security
uname output: Linux medium 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) 
x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 5.0
Patch Level: 16
Release Status: release

Description:
Integer part of $EPOCHREALTIME could increase more than 8000 
microseconds
before $EPOCHSECONDS

Repeat-By:
epochVariableDiff () {
local errcnt=0 lasterrcnt v1 v2 v3 us vals line
while ((errcnt==0)) || ((errcnt>lasterrcnt)); do
lasterrcnt=$errcnt
printf -v vals '%(%s)T %s %s' -1 $EPOCHSECONDS $EPOCHREALTIME
IFS=$' .' read v1 v2 v3 us <<<"$vals"
[ "$v1" = "$v2" ] && [ "$v2" = "$v3" ] || ((errcnt++))
[ $errcnt -eq 1 ] && echo "$line"
printf -v line '%3d %s - %s - %s . %s' $errcnt $v1 $v2 $v3 $us
printf "%s\r" "$line"
((errcnt)) && echo "$line"
read -t ${1:-.0002}
done

epochVariableDiff
  0 1586853481 - 1586853481 - 1586853481 . 40
  1 1586853481 - 1586853481 - 1586853482 . 000320
  2 1586853481 - 1586853481 - 1586853482 . 000691
  3 1586853481 - 1586853481 - 1586853482 . 001059
  4 1586853481 - 1586853481 - 1586853482 . 001429
  5 1586853481 - 1586853481 - 1586853482 . 001854
  6 1586853481 - 1586853481 - 1586853482 . 002220
  7 1586853481 - 1586853481 - 1586853482 . 002672
  8 1586853481 - 1586853481 - 1586853482 . 003113
  9 1586853481 - 1586853481 - 1586853482 . 003530
  9 1586853482 - 1586853482 - 1586853482 . 003889

(My raspberry-pi seem not to be affected)

Was discovered and published at 
https://stackoverflow.com/a/58557346/1765658

Typo in german translation

[felix . ostmann]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' 
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I../. -I.././include -I.././lib  -Wdate-time 
-D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/bash-7fckc0/bash-4.4=. 
-fstack-protector-strong -Wformat -Werror=format-security -Wall -no-pie 
-Wno-parentheses -Wno-format-security
uname output: Linux www44 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u5 
(2017-09-19) x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 4.4
Patch Level: 12
Release Status: release

Description:
There is a typo in the german .po-file: as should be als
./po/de.po-msgid "filename argument required"
./po/de.po:msgstr "Ein Dateiname wird as Argument benötigt."

Repeat-By:
call source without a filename

$ LANG=de_DE.utf8 source
-bash: source: Ein Dateiname wird as Argument benötigt.
source: Aufruf: source Dateiname [Argumente]

Fix:
./po/de.po-msgid "filename argument required"
./po/de.po:msgstr "Ein Dateiname wird als Argument benötigt."

Typo in german translation

[Felix Ostmann]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu'
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL
-DHAVE_CONFIG_H   -I.  -I../. -I.././include -I.././lib  -Wdate-time
-D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/bash-7fckc0/bash-4.4=.
-fstack-protector-strong -Wformat -Werror=format-security -Wall -no-pie
-Wno-parentheses -Wno-format-security
uname output: Linux www44 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u5
(2017-09-19) x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 4.4
Patch Level: 12
Release Status: release

Description:
There is a typo in the german .po-file: as should be als
./po/de.po-msgid "filename argument required"
./po/de.po:msgstr "Ein Dateiname wird as Argument benötigt."

Repeat-By:
call source without a filename

$ LANG=de_DE.utf8 source
-bash: source: Ein Dateiname wird as Argument benötigt.
source: Aufruf: source Dateiname [Argumente]

Fix:
./po/de.po-msgid "filename argument required"
./po/de.po:msgstr "Ein Dateiname wird als Argument benötigt."

Re: Broken PIPESTATUS with --disable-job-control

[Felix Janda]

Chet Ramey wrote:
> On 9/18/16 11:20 PM, Felix Janda wrote:
> 
> >>> Notice that the configure script disables job-control when a run-time
> >>> test (which could easily be a built-time test) fails. So by default,
> >>> a cross-compiled bash will have this bug.
> >>
> >> Which test?
> > 
> > I am referring to BASH_SYS_JOB_CONTROL_MISSING.
> 
> Sure.  I'm asking which part of that run-time test can easily be converted
> to a build-time test that handles conditional compilation and definitions.

below a patch to make things more concrete:

--- a/aclocal.m4
+++ b/aclocal.m4
@@ -1357,7 +1357,7 @@
 [AC_REQUIRE([BASH_SYS_SIGNAL_VINTAGE])
 AC_MSG_CHECKING(for presence of necessary job control definitions)
 AC_CACHE_VAL(bash_cv_job_control_missing,
-[AC_TRY_RUN([
+[AC_TRY_COMPILE([
 #include 
 #ifdef HAVE_SYS_WAIT_H
 #include 
@@ -1367,42 +1367,35 @@
 #endif
 #include 
 
-/* Add more tests in here as appropriate. */
-main()
-{
 /* signal type */
 #if !defined (HAVE_POSIX_SIGNALS) && !defined (HAVE_BSD_SIGNALS)
-exit(1);
+#error
 #endif
 
 /* signals and tty control. */
 #if !defined (SIGTSTP) || !defined (SIGSTOP) || !defined (SIGCONT)
-exit (1);
+#error
 #endif
 
 /* process control */
 #if !defined (WNOHANG) || !defined (WUNTRACED) 
-exit(1);
+#error
 #endif
 
 /* Posix systems have tcgetpgrp and waitpid. */
 #if defined (_POSIX_VERSION) && !defined (HAVE_TCGETPGRP)
-exit(1);
+#error
 #endif
 
 #if defined (_POSIX_VERSION) && !defined (HAVE_WAITPID)
-exit(1);
+#error
 #endif
 
 /* Other systems have TIOCSPGRP/TIOCGPRGP and wait3. */
 #if !defined (_POSIX_VERSION) && !defined (HAVE_WAIT3)
-exit(1);
+#error
 #endif
-
-exit(0);
-}], bash_cv_job_control_missing=present, bash_cv_job_control_missing=missing,
-[AC_MSG_WARN(cannot check job control if cross-compiling -- defaulting to 
missing)
- bash_cv_job_control_missing=missing]
+], bash_cv_job_control_missing=present, bash_cv_job_control_missing=missing
 )])
 AC_MSG_RESULT($bash_cv_job_control_missing)
 if test $bash_cv_job_control_missing = missing; then

Re: Broken PIPESTATUS with --disable-job-control

[Felix Janda]

Chet Ramey wrote:
> On 9/18/16 11:20 PM, Felix Janda wrote:
> 
> >>> Notice that the configure script disables job-control when a run-time
> >>> test (which could easily be a built-time test) fails. So by default,
> >>> a cross-compiled bash will have this bug.
> >>
> >> Which test?
> > 
> > I am referring to BASH_SYS_JOB_CONTROL_MISSING.
> 
> Sure.  I'm asking which part of that run-time test can easily be converted
> to a build-time test that handles conditional compilation and definitions.

Everything could easily be a preprocessor test:


#include 
#ifdef HAVE_SYS_WAIT_H
#include 
#endif
#ifdef HAVE_UNISTD_H
#include 
#endif
#include 

/* signal type */
#if !defined (HAVE_POSIX_SIGNALS) && !defined (HAVE_BSD_SIGNALS)
#error
#endif

/* signals and tty control. */
#if !defined (SIGTSTP) || !defined (SIGSTOP) || !defined (SIGCONT)
#error
#endif

/* process control */
#if !defined (WNOHANG) || !defined (WUNTRACED) 
#error
#endif

/* Posix systems have tcgetpgrp and waitpid. */
#if defined (_POSIX_VERSION) && !defined (HAVE_TCGETPGRP)
#error
#endif

#if defined (_POSIX_VERSION) && !defined (HAVE_WAITPID)
#error
#endif

/* Other systems have TIOCSPGRP/TIOCGPRGP and wait3. */
#if !defined (_POSIX_VERSION) && !defined (HAVE_WAIT3)
#error
#endif


I think that the only reason that it is currently a run-time test is
related to the comment:

/* Add more tests in here as appropriate. */

So it was conceived that in future run-time only tests might be
necessary.


Felix

Re: Broken PIPESTATUS with --disable-job-control

[Felix Janda]

Chet Ramey wrote:
> On 9/17/16 1:27 PM, Felix Janda wrote:
> > Hello,
> > 
> > below this mail you can find a minimal script misbehaving when
> > job-control is configured out (tested on linx with different archs,
> > libc's, and versions (including current git)).
> 
> Yes.  PIPESTATUS doesn't really have any valid values when the shell
> is compiled without job control.  The internal machinery to set it
> doesn't exist.

Ok.

> > 
> > Notice that the configure script disables job-control when a run-time
> > test (which could easily be a built-time test) fails. So by default,
> > a cross-compiled bash will have this bug.
> 
> Which test?

I am referring to BASH_SYS_JOB_CONTROL_MISSING.

Thanks,
Felix

Broken PIPESTATUS with --disable-job-control

[Felix Janda]

Hello,

below this mail you can find a minimal script misbehaving when
job-control is configured out (tested on linx with different archs,
libc's, and versions (including current git)).

Notice that the configure script disables job-control when a run-time
test (which could easily be a built-time test) fails. So by default,
a cross-compiled bash will have this bug.

-- Felix

#!/bin/bash

g() {
p=$PIPESTATUS
if [[ $p -ne 0 ]]
then
echo FAIL!!
fi
}

f() {
false && false
true | true
g
}

f

Re: completion of '../' with symlink component in CWD path

[Felix]

On Fri, 29 Mar 2013 15:30:08 -0600
Bob Proulx  wrote:

> Marcel Giannelia wrote:
> > The problem is that some commands are "smart" and "know" how you
> > got to your current working dir -- bash knows that you're in a
> > symlink and that the parent dir of the *symlink* (not the actual
> > directory you're in) is 'basedir'.
> > 
> > However, this is not the literal meaning of the '..' directory entry
> > according to the filesystem. Some parts of bash use the "smart"
> > behaviour (cd and completion), and some parts use what the
> > filesystem actually says, i.e. that '..' inside 'dir2' is 'dir1',
> > not 'basedir' (output redirection).
> > 
> > ...after thinking about this, I've managed to confuse myself and am
> > not sure which behaviour should be considered correct.
> 
> I consider the 'set -o physical' behavior the canonical and correct
> form.  Try it.  You might like it.  Because it isn't possible to have
> the shell create a 100% correct facade over the top of everything.

Yeah, discovered set -o physical just after posting, and am considering
adopting it... but on the other hand, I'm not so sure the facade
behaviour has to be all that elaborate. For instance, if I give you the
paths '/path/to/directory' and '..' and tell you to append them, you
don't need to know anything special to come up with '/path/to'.

I think I'd be OK with a "always do path arithmetic" mode which behaves
consistently like that, or a "physical" mode.

Things being a mish-mash of aspects of both, though, does seem like a
bug -- I can't think of a conceptual reason why ">" would, by design,
behave differently from tab completion, especially since you can
tab-complete its destination. i.e. if you start typing something like

cat > ../fo

and it helpfully completes to

cat > ../foo

...then that really should work when you press the return key.

~Felix.

completion of '../' with symlink component in CWD path

[Felix]

I just encountered the following behaviour. Set up a directory
structure like:

mkdir basedir
cd basedir
mkdir -p dir1/dir2
ln -s dir1/dir2 dir2link
cd dir2link

now, you're sitting in dir2link, and you type:

cd ../

and hit the Tab key. The possible completions are shown as 'dir1' and
'dir2link', i.e. the contents of basedir. And if you finish the command
as:

cd ../dir1

It will change to dir1. However, if you instead do this while your
current working directory is 'dir2link':

echo foo > ../dir1/bar

It fails with "No such file or directory".

The problem is that some commands are "smart" and "know" how you
got to your current working dir -- bash knows that you're in a symlink
and that the parent dir of the *symlink* (not the actual directory
you're in) is 'basedir'.

However, this is not the literal meaning of the '..' directory entry
according to the filesystem. Some parts of bash use the "smart"
behaviour (cd and completion), and some parts use what the filesystem
actually says, i.e. that '..' inside 'dir2' is 'dir1', not
'basedir' (output redirection).

...after thinking about this, I've managed to confuse myself and am not
sure which behaviour should be considered correct.

~Felix.

show-all-if-ambiguous inserts spurious backslashes

[Felix]

If you have "set show-all-if-ambiguous on" in your .inputrc, typing
this sequence in bash (in a directory containing some non-hidden files):

echo *

Results in the list of possible matches being displayed (as expected),
but the command line is then edited to:

echo \*

Which is probably not what the user wanted.

~Felix.

square bracket vs. curly brace character ranges

[Felix]

I believe I've found an inconsistency in bash or its documentation.

I know the fact that things like [a-c] are highly locale-dependent in
bash (doesn't mean I have to like it, but there it is). Fine. I've
learned to live with it.

But the other day I was on a fresh install (hadn't set
LC_COLLATE=C yet, so I was in en_US.UTF-8), and this happened:

$ touch {a..c}
$ ls
a  b  c
$ touch {A..C}
$ ls
a  A  b  B  c  C
$ ls {a..c}
a  b  c
$ ls [a-c]
a  A  b  B  c

Curly brace range expressions behave differently from square-bracket
ranges. Is this intentional? This is under Arch Linux, bash version
"4.2.37(2)-release (i686-pc-linux-gnu)".

The man page seems to imply that the curly brace behaviour above is a
bug:

"When characters are supplied, the expression expands to each character
lexicographically between x and y, inclusive."

...although this documentation suffers from the same problem as the
passage about character class ranges, namely that it confuses
lexicographic sort order (character collation *weights*) with
character collation *sequence values* (they are not quite the same thing
-- if they were, 'c' and 'C' would *always always always* appear
together in a range expansion, because:
$ touch aa B cd C
$ ls -1
aa
B
C
cd
). The phrases "sorts between" and "lexicographically between" refer to
collation *weights*, but bash clearly uses sequence values.

It's a subtle distinction; I beat it to death in a thread
from 2011, subject "documentation bug re character range expressions",
but I don't think the documentation actually got changed.

It seems the thinking goes something like, "since no one is supposed to
use expressions like [a-c], we don't have precisely
document, care, or even *know* what it means" -- a shame, because with
LC_COLLATE=C set, [a-c] is actually quite useful, and in all other
locales it isn't useful at all (it would be slightly useful if it used
weights like the documentation says because then it would be like a
case-insensitive range, but with it using sequence values instead, it's
useless).

The sheer number of threads we've got complaining about
locale-dependent [a-c] suggests to me that the software should be
changed to just do what people expect, especially since nothing is
really lost by doing so.

Oh well. Dead horses and all that -- but can we at least make the dead
horses consistent? :)

~Felix.

Re: incremental history i/o? (was Re: A Feature Request for History)

[Marcel (Felix) Giannelia]

I wonder how much de-duping the really old history would help. It seems 
that HISTCONTROL='erasedups' only affects the history of the current 
bash process (i.e. commands that were typed since you started that 
shell), and it leaves all the stuff it loaded from .bash_history alone.


As a quick test, removing duplicates from a 4 MB history file reduced 
the number of commands in it from 125236 to 36937, so that file was 
about 70% duplicated data (not quite, 'cause the longer and more 
interesting commands mostly stayed...). Doing that to your 11 MB file 
might get rid of that loading delay.


Of course, de-duplicating the history destroys its role of "accurately 
record everything I've done", so if you also use your history for that 
it's not a good idea. For that latter use though, I can't think of a 
good reason for loading it on shell start, so maybe those roles should 
be split -- .bash_log and .bash_commands? The log is write-only, never 
clobbered, and has the equivalent of a HISTTIMEFORMAT set; the commands 
file is an efficiently stored hash table of unique commands, maybe with 
tweakable parameters for how "interesting" a command has to be to go in 
it (store "mount -o loop,ro,uid=1000 -t vfat /some/file /mnt/temp" but 
ignore "cd ~" 'cause you really don't need Ctrl+R to remember the latter).


~Felix.


On 16/06/11 12:55, Bradley M. Kuhn wrote:

I agree with Marcel's points about keeping a big bash history, although
I wasn't sure if discussing "why" users keep a big bash history was on
topic or not.

Marcel (Felix) Giannelia wrote at 13:16 (EDT) on Tuesday:

A .bash_history file going back years and years is still only a few
megs,

Actually, this relates to a thing I'd been looking into recently.  My
bash history is 11MB now, and on some machines I have a noticeable load
time as it reads the history.  I'd thought about adding support for
incremental read to bash history/readline code.  Basically, it would
load only the parts of the history it needed based on the history
requested.  Obviously running "history" would read it all, but if
reverse-search was requested, it could perhaps be read incrementally
somehow.

Given that this would be a big change (esp. to make it seamless to
existing readline API users), and would provide a feature clearly that
isn't universally desired (ability to have really big history files),
I'm asking, albeit with some trepidation, if such a rewrite of the
history reading/writing code would likely be accepted, and if so what it
would need to look like to be an acceptable patch.

I noticed someone previously attempted to implement mmap() in the
history code, but it's #ifdef'd out (IIRC from my investigations a few
weeks ago).  I theorized that it was #ifdef'd out because implementing
mmap() didn't help anything, since the history reading code immediately
goes through the whole array of history anyway, so the file will be
immediately read in to RAM anyway the way the code currently operates,
even if you mmap() it.  In other words, just slapping mmap() in place
wouldn't work (in fact, it's seem to have been tried and abandoned);
more in-depth changes would be made.

Thoughts on this idea?

Re: A Feature Request for History

[Marcel (Felix) Giannelia]

I do this too -- because we can :)

A .bash_history file going back years and years is still only a few 
megs, and it's faster than Google for those moments like "right... I 
know I've mounted a crypto Luks partition with this particular set of 
nonstandard flags and parameters before, like 6 years ago, what was 
it... Ctrl+r (some short string that you know was in that command 
somewhere) Ah, right, that's it."


~Felix.

On 2011-06-13 21:48, Michael Witten wrote:

On Mon, Jun 13, 2011 at 17:10, Bradley M. Kuhn  wrote:

I have all my bash history going back to
2003-10-15 accessible to me.


Why?

Re: documentation bug re character range expressions

[Marcel (Felix) Giannelia]

On 09/06/11 11:31, Chet Ramey wrote:

[...]
No, it doesn't.  It's not part of any standard, and it's not part of
pattern matching, so I implemented it with the traditional C semantics
because that seemed the most straightforward.


Pity the implementor of character range expressions didn't have the same 
thought ;)



I realize it's pedantic, but documentation should be pedantically accurate
:)  I would be OK with changing the man page so it says, "sorts between
those two characters in a list of single-character strings", as that would
also describe the current behaviour.

But it only matches a single character, by definition.  It should not be
necessary to specify the list of single-character strings part.



I guess that's true. It's still confusing, though -- it would make more 
sense to me if it simply went by sort weights and thus considered 'c' 
and 'C' equivalent... but I guess then you'd get people asking "how come 
[a] is case-sensitive and [a-z] isn't?". Then again, we've already got 
people asking that -- I guess range expressions are just tricky no 
matter how you look at them.



[...]
The bash texinfo documentation says just about the same thing.


There's an 'info bash'?! Oh, so there is. OK, then I revise my suggested 
revision to the man page -- it's missing a mention of the info manual 
under the SEE ALSO section. (Most other commands that have an info 
manual have something like "The full documentation for ls is maintained 
as a  Texinfo  manual. If the info and ls programs are properly 
installed [...]")


Guess it's time I really learned how to navigate texinfo... (Do I come 
across as old-fashioned nowadays for still using man?)


~Felix.

Re: documentation bug re character range expressions

[Marcel (Felix) Giannelia]

On 07/06/11 13:45, Chet Ramey wrote:

[...]
I'm not going to add much to this discussion except to note that I believe
`sorts' is correct.  Consider the following script:

unset LANG LC_ALL LC_COLLATE

export LC_COLLATE=de_DE.UTF-8
printf "%s\n" {A..Z} {a..z} | sort | tr $'\n' ' '
echo
That's really interesting -- and not just your intended point, but what 
happens with those ranges if you take 'sort' out of the pipe. The curly 
brace {A..Z} syntax doesn't obey the locale! Observe:


$ printf "%s\n" {A..Z} {a..z} | sort | tr $'\n' ' '
a A b B c C d D e E f F g G h H i I j J k K l L m M n N o O p P q Q r R 
s S t T u U v V w W x X y Y z Z


(as you expect, but...)

$ printf "%s\n" {A..Z} {a..z} | tr $'\n' ' '
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j 
k l m n o p q r s t u v w x y z


So, if I want C-like behaviour out of "[a-z]*", I can write it as 
"{a..z}*"? Is that a bug or a feature?


It's not quite the same, of course, because the {} syntax expands to 
copies of the entire expression for everything in the range, e.g.:


$ ls {a..f}*
ls: cannot access a*: No such file or directory
ls: cannot access b*: No such file or directory
ls: cannot access c*: No such file or directory
ls: cannot access d*: No such file or directory
ls: cannot access f*: No such file or directory
example.txt

But interesting nonetheless.



[...]

That sure looks like `C' doesn't sort between `a' and `c' in de_DE.UTF-8
and en_GB.UTF-8.
Not in a case like that, with single-character strings. But my point was 
that it's possible for 'C' to sort between 'a' and 'c' in longer 
strings. Try sorting this:


aa
cc
Ca

Because 'c' and 'C' have equal sort weights but 'a' comes before 'c', 
this list will sort as:


aa
Ca
cc

...which has 'C' between 'a' and 'c'.

I realize it's pedantic, but documentation should be pedantically 
accurate :)  I would be OK with changing the man page so it says, "sorts 
between those two characters in a list of single-character strings", as 
that would also describe the current behaviour.

I believe it would also be helpful for the documentation to then go on to
say something like this:

[...]

You might like the text in item 13 of the COMPAT file included in the bash
distribution.  It doesn't take quite so cautionary a tone, but the basic
information is there.
Actually, I discovered that the grep man page says something quite 
similar too, even giving almost the same example I did:


"Within a bracket expression, a range expression consists of two 
characters separated by a hyphen. It matches any single character that 
sorts between the two characters, inclusive, using the locale's 
collating sequence and character set. For example, in the default C 
locale, [a-d] is equivalent to [abcd]. Many locales sort characters in 
dictionary order, and in these locales [a-d] is typically not equivalent 
to [abcd]; it might be equivalent to [aBbCcDd], for example. To obtain 
the traditional interpretation of bracket expressions, you can use the C 
locale by setting the LC_ALL environment variable to the value C."


I would be very happy if the bash man page said that. (It also says 
"sorts between", but because it also says "using the locale's collating 
sequence and character set" it squeaks by as being technically correct 
-- in the sorting example I gave above, where 'C' was between 'a' and 
'c', only collation weights were in play, not sequence numbers, whereas 
if one is forced to sort by comparing 'c' to 'C', collation sequence 
numbers come into play as the tie-breaker. So saying "using the 
collating sequence" implies a similar situation, and would force 'C' to 
come after 'c'.)


~Felix.

Re: documentation bug re character range expressions

[Marcel (Felix) Giannelia]

On Fri, June 3, 2011 10:03, Greg Wooledge wrote:
> On Fri, Jun 03, 2011 at 09:12:07AM -0700, Marcel (Felix) Giannelia wrote:
>
> [...]
>
> In HP-UX's en_US.iso88591 locale, the characters are in a COMPLETELY
> different order.  You can't easily figure out what that order is, because
> it's not documented anywhere, but by using tricks you can beat it into
> submission.  Instead of having two separate ranges from a to z, and from A
> to Z, there's just one big range from A to z (actually þ) which looks
> something like:
>
> A a Á á À à Â â Ä ä Å å Ã ã Æ æ B b C c Ç ç D d Ð ð E e É  Z z Þ þ
>
>
> So when you write A-Z you mean A a Á ... Z.  And a-z means a Á ... Z z.
> In other words, when you tell tr to map from A-Z to a-z all you're
> actually doing is shifting the map one position to the right.  So H becomes
> h, e becomes É, l becomes M, and so on.  Whereas in ASCII, mapping from
> A-Z to a-z shifts everything 32 to the right (the
> difference between 'A' and 'a'), so H becomes h and so on.
>
> The GNU people apparently didn't like this,

See, this is where people like me get confused -- it sounds from your
description above ("...can't easily figure out what this order is...",
"...beat it into submission", and so forth) as if you don't much care for
it either.

It sounds to me like what you're saying is, the *only* uses of bracket
range expressions guaranteed to be "portable" are things like [[:upper:]]
and [[:lower:]]. But I put "portable" in quotation marks just then,
because to my mind the word "portable" implies "has the same behaviour on
all systems", whereas things like [[:upper:]] are locale-dependent; they
change their behaviour depending on system settings.

It would be a bit like "echo $((1 + 1))" printing "2", "4", or "5",
depending on the locale setting (dear lordy I hope it doesn't).

[0-9] presumably still works consistently across all platforms -- I hope?
Or does it map to something like:

0   1
   2...

etc.? I'm just having trouble imagining in what situations this kind of
behaviour is useful. But, from your description, my understanding now is
that it's no longer meant to be useful, i.e. no longer meant to be used
that way.

I think a good solution to this, then, is to just deprecate the use of "-"
in bracket expressions entirely. As you say, it's non-portable and
unpredictable and highly locale- and even OS-dependent (and whether GNU is
in the wrong or HP is the wrong is neither here nor there).

Use the magic of locales to mark "-" as "just another character", and then
[A-Z] means only "A", "-", or "Z" -- at least that would be easier for the
common woman and man to understand than "[A-Z] means 'that long string of
things from the HP ISO 8859-1 code page if you're on an HP-UX system,
uppercase and lowercase letters except lowercase 'a' if  you're on Linux,
uppercase A-Z if you're in the C locale, and so on'."

This way, people would be forced to either use the C locale, or use
[[:upper:]] if they want to match uppercase letters, which is what you've
been saying should happen.

> [...]

~Felix.

Re: documentation bug re character range expressions

[Marcel (Felix) Giannelia]

On 2011-06-03 05:00, Greg Wooledge wrote:

On Fri, Jun 03, 2011 at 12:06:32AM -0700, Marcel (Felix) Giannelia wrote:

Is it really a programmer mistake, though, to assume that [A-Z] is only
capital letters?


Yes, it is.  You should be using [[:upper:]], or you should be setting
LC_COLLATE=C if you insist on using [A-Z].



Alright -- assuming that for the moment, how does one specify 
[ABCDEFGHIJKL] using [[:upper:]]? This is something that I haven't seen 
documented, and I'm genuinely curious.


~Felix.

Re: documentation bug re character range expressions

[Marcel (Felix) Giannelia]

On 2011-06-03 05:09, Greg Wooledge wrote:

Oh, look, there's more!

[...]

See?  Both tr(1) and ls(1) do it too!


Right; forgot about ls (because "alias ls='LC_COLLATE=C ls'" has been in 
my .bashrc for so long that I completely forgot it was there :) ), and 
didn't think to try tr -- but tr appears to be case-sensitive under all 
locales for me, though.


And yours looks broken -- how does
echo Hello World | tr A-Z a-z
result in a bunch of non-ASCII characters? (That's how it looked when it 
got here, at any rate -- maybe one of our mail servers did something.)





Even grep, whose man page says it obeys LC_COLLATE and the locale,
actually has [a-c] equivalent to [abc] on all locales. Someone must have
snuck in and fixed it.


You must live in a strange and peculiar world.

imadev:~/qwerty$ type grep
grep is hashed (/usr/bin/grep)
imadev:~/qwerty$ echo 'brown cow' | grep '[A-C]'
brown cow
imadev:~/qwerty$ echo 'BROWN COW' | grep '[a-c]'
BROWN COW

Is every single bit of your knowledge born out of familiarity with just
ONE operating system with weird extensions?


I tried this on a few systems with different distros on them. The only 
non-Linux systems I have access to are running things so old that they 
only have the C locale, so I can't be sure about them. (For instance, 
the only Solaris system I can get at doesn't support unicode at all.)


Here are a couple:

[c69:~]$ uname -sr
Linux 2.6.32-31-generic
[c69:~]$ type grep
grep is /bin/grep
[c69:~]$ locale | grep LC_COLLATE
LC_COLLATE="en_US.UTF-8"
[c69:~]$ echo 'brown cow' | grep '[A-C]'
[c69:~]$ echo 'BROWN COW' | grep '[a-c]'
[c69:~]$


u-elive ~(0)$ uname -sr
Linux 2.6.32-26-generic
u-elive ~(0)$ locale | grep LC_COL
LC_COLLATE="en_US.UTF-8"
u-elive ~(0)$ type grep
grep is /bin/grep
u-elive ~(0)$ echo 'brown cow' | grep '[A-C]'
u-elive ~(0)$ echo 'BROWN COW' | grep '[a-c]'
u-elive ~(0)$


The university used to have a big AIX system, but again, I think it was 
a version from before the days of unicode locales. Maybe grep working as 
expected is a Linux thing?


And if so, then you're right about me living in a "strange and peculiar 
world" -- it's the world of people and organizations too poor to afford 
proprietary Unices; one would never see HP-UX in a world like that when 
there are cheaper alternatives. I'll spin up some BSD's in a virtual 
machine and check those later -- anything else I should try?




You ought to report the bug in your vendor's grep(1) implementation, if
it is actually broken as you describe.



And no, I'm going to keep very quiet about this "bug" in grep -- because 
it's working the way I want/expect it to, and it'll doubtless break 
many, many shell scripts and cause data loss for a fair number of people 
if it were fixed :)


~Felix.

Re: documentation bug re character range expressions

[Marcel (Felix) Giannelia]

Is it really a programmer mistake, though, to assume that [A-Z] is only 
capital letters? A through Z are a contiguous range in every 
representation system except EBCDIC, and it is even contiguous the 
modern unicode.


In the world of programming characters are numbers, and programmers know 
this (especially if they've ever learned any C). For the example of 
[a-c], programmers are treating letters the way the machine treats them, 
as numbers.


How is the person typing [a-c] the one making the mistake when it 
results in matching against values outside of that range? To make it 
plainer, type it as [\0x61-\0x63] -- if you saw that in a program, you 
would expect that to cover 0x61, 0x62, 0x63, wouldn't you? If you were 
designing a programming language, wouldn't you make it do that?


If person A types [\0x61-\0x63] on software written by person B and it 
comes out matching 0x61, 0x41, 0x62, 0x42, 0x63, and perhaps something 
completely different when the same code is run on a computer in Russia, 
who would you say made the programming mistake? Surely not person A.


This is something that wasn't a "bad programming habit" until somewhere, 
someone made a decision that removed meaning from a sensible, 
logical-looking syntax.


Let's compare the syntaxes:

Under the old notation, there was:

- a succinct way to specify lowercase letters: [a-z]

- likewise for uppercase: [A-Z]

- likewise for case-insensitive: [A-Za-z]

- an easy way to specify ranges of letters of a particular case: [a-m], 
[A-M]


- case-insensitive ranges: [A-Ma-m]

Under the new notation, those things are written as:

- lowercase letters: [[:lower:]] (over twice as long to type)

- uppercase letters: [[:upper:]] (likewise)

- case-insensitive: [[:alpha:]] (not as bad, but still longer)

- how *are* you supposed to specify case-sensitive ranges? 
[abcdefghijklm] looks ridiculous.


- case-insensitive ranges: [a-M] (looks like an error at first glance: 
"why is the M uppercase?" you need to know something about the system 
internals to see why that's not wrong. And that something is a lot more 
complicated to explain than "computers represent letters as numbers")



Bash is a shell. Shells should have a quick, brief, plain language so 
that one can get things done in them. Shells should also be quite 
portable: syntax that works on one system should work on any other as 
much as possible.


[[:alpha:]] is too difficult to type to make it useful for the kind of 
quick pattern-matching that character ranges are used for on the 
interactive shell. Try it. Open-bracket, colon is an awkward sequence 
compared to something like "[a-z]".


But usually one doesn't want all of the alphabet, nor case 
insensitivity. I have actually never had occasion to say [A-Za-z] on the 
command line, or even [A-Ca-c]. I have, however, very often wanted to 
grab everything with a lowercase 'a' through lowercase 'k', for instance.


Previously, that would have been [a-k]. Now I have no way to specify it 
except [abcdefghijk], and I'm not typing that. A useful feature is gone.


You say this is not only a "bash problem" because it's a programmer's 
mistake to assume that [a-c] means the same thing in bash as it does in 
Perl, Python, Java, C/C++ (POSIX regex.h, with system locale set!), 
JavaScript, PHP, sed, grep, and on and on -- you can see why one might 
make this "mistake".


And these aren't historical examples, these are modern implementations 
of these languages that I just tested this on to double-check, on a 
system with its locale set to something that collates 
case-insensitively. Bash is the *only* thing I know of that treats 
character ranges this way, so I would say that does make it "only a bash 
problem".


Even grep, whose man page says it obeys LC_COLLATE and the locale, 
actually has [a-c] equivalent to [abc] on all locales. Someone must have 
snuck in and fixed it. I'm guessing that if grep were to start using 
locale-aware character ranges, a heck of a lot more people would 
complain than do about bash. This is a seldom-used feature in bash but 
many, many people rely on grep being predictable and standard.


~Felix.


On 2011-06-02 22:32, Jan Schampera wrote:

Hi,


just as side note, not meant to touch the maintainer discussion.

This is not only a "Bash problem". The programmer/user mistake to use
[A-Z] for "only capital letters, capital A to capital Z" is a very
common one.

But I'm not sure if every official application-level documentation
should cover those kind of pitfalls. There would be many topics around
"bad programming habbits" that should be documented.

documentation bug re character range expressions

[Marcel (Felix) Giannelia]

Hello,

I realize the issue of character range expressions not working as 
expected (because of locale settings) has been done to death, but I 
thought I should point this out.


The bash man page says:

"A pair of characters separated by a hyphen denotes a range expression; 
any character that ***sorts between those two characters,*** inclusive, 
using the current locale's collating sequence and character set, is 
matched." (emphasis mine)


That is incorrect because, for instance, an uppercase 'C' sorts between 
lowercase 'a' and lowercase 'c' (sometimes), as in this example (locale 
is en_GB.UTF-8):


$ touch aa B cd C
$ ls -1
aa
B
C
cd

However, bash's behaviour does not reflect what the man page says. Observe:

$ touch aa B cd C
$ ls -1 [a-c]*
aa
B
cd

Now, I'm firmly of the opinion that character range expressions paying 
any attention at all to the locale collation settings, in any shape or 
form, is completely broken behaviour. I really wish that [a-c] meant 
[abc] and not [aAbBc].


But, it looks as if that's not going to change, so it is my hope that 
the documentation will at least be updated to reflect what really happens.


Previous posters who've complained about this character range issue have 
been directed to some comments made by Ulrich Drepper (who, I 
understand, is a maintainer of some underlying code that bash uses in 
its evaluation of range expressions?). Those comments include this:


"The strcoll result has nothing whatsoever to do with the range match. 
strcoll uses collation weights, ranges use collation sequence values, 
completely different concept."


I believe that same confusion is behind the problem in that paragraph 
from the man page and has led to the inappropriate use of the phrase 
"sorts between." The bit of man page text I quoted above should read:


"A pair of characters separated by a hyphen denotes a range expression; 
any character that ***occurs between those two characters in collation 
sequence value,*** inclusive, using the current locale's collating 
sequence and character set, is matched."


I believe it would also be helpful for the documentation to then go on 
to say something like this:


"This means that character ranges are neither case-sensitive nor 
case-insensitive in most locales. For instance (in the en_ locales), the 
range [a-c] is equivalent to [aAbBc] (note the absence of uppercase 
'C'!). Thus, sub-ranges of the character class [[:alpha:]] must be used 
with great care, and probably should not be used at all, in locales 
other than C. It is not possible, for example, to specify a range of 
greater than one or fewer than 26 lowercase letters in the en_US.UTF-8 
locale. If you desire to match [abcdefghij] in this locale, you must not 
use a range, but specify all of those characters explicitly, or use 
LC_COLLATE from the C locale."


In closing, it is my fervent hope that the insanity of that last 
paragraph will be recognized (when is [a-c] being equivalent to [aAbBc] 
ever useful?!), and that this will eventually lead to character ranges 
becoming useful again regardless of the current locale.


But in the mean time, I would settle for a documentation change, and 
will continue to "export LC_COLLATE=C"! :)


~Felix.

Re: spaces in the shebang interpreter path

[Felix Schwarz]

Hi Bob, hi Paul,

thanks for your detailed explanations. So I agree that this is not a bash bug 
;-)

Maybe in ten years we will see a Un*x release which will allow the use of 
spaces in the shebang line *dreaming*.


thanks again
fs


smime.p7s
Description: S/MIME Cryptographic Signature

spaces in the shebang interpreter path

[Felix Schwarz]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-redhat-linux-gnu' 
-DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' 
-DSHELL -DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib  -D_GNU_SOURCE  -O2 -g 
-pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector 
--param=ssp-buffer-size=4 -m64 -mtune=generic
uname output: Linux ws2.schwarz.lokal 2.6.21.7-3.fc8xen #1 SMP Fri Mar 28 
20:16:21 CET 2008 x86_64 x86_64 x86_64 GNU/Linux

Machine Type: x86_64-redhat-linux-gnu

Bash Version: 3.2
Patch Level: 33
Release Status: release

Description:
I'm not able to specify an interpreter in a shebang line if the path
to this interpreter contains spaces. I'm not quite sure if this is
considered a bug or if its just there for backwards compatibility but
at least quoting etc. does not work for me.

e.g.
The directory "/home/fs/foo bar" contains the executable bin/python

Now I have this python script
---
#!/home/fs/foo bar/bin/python
print 'foo bar'
---

The shebang above obviously can't work because bash tries to find the
executable named 'foo' and pass 'bar/bin/python' as a parameter.
Therefore I tried to escape the path like in other bash scripts
('#!/home/fs/foo\ bar/bin/python') but this doesn't work either.
Quotes (#!"/home/fs/foo bar/bin/python") don't cut it.

IMHO bash should support every valid unix path for shebang lines.

fs


smime.p7s
Description: S/MIME Cryptographic Signature

Sup

[Felix Mathews]

<>___
Bug-bash mailing list
Bug-bash@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-bash