printf %(fmt)T fails with large times in 64-bit linux.

[Eduardo A . Bustamante López]

printf %(fmt)T fails when the time given is very large in a 64-bit linux
environment. Looping over the powers of two and feeding it to printf shows
the error, which exits the shell with a SIGSEGV

The problem seems to be in builtins/printf.def, as indicated by the backtrace
attached (gdb.txt).


---

I'm running the latest bash from the git repository:

$ echo $BASH_VERSION
4.2.37(3)-release

$ uname -a
Linux claret 3.4.2-2-ARCH #1 SMP PREEMPT Mon Jun 11 22:27:17 CEST 2012
x86_64 GNU/Linux

$ lsb_release -a
LSB Version:n/a
Distributor ID: archlinux
Description:Arch Linux
Release:rolling
Codename:   n/a


---

The following snippet reproduces the problem:

for i in {56..63}; do
printf '%s ' $i;
bash -c 'printf %\(%s\)T\\n $((2**$1))' _ $i;
done

Sample run:

$ for i in {56..63}; do
> printf '%s ' $i;
> bash -c 'printf %\(%s\)T\\n $((2**$1))' _ $i;
> done
56 Segmentation fault
57 Segmentation fault
58 Segmentation fault
59 Segmentation fault
60 Segmentation fault
61 Segmentation fault
62 Segmentation fault
63 Segmentation fault


---

The simple fix is to check if ``tm == NULL``, and show an error message, or
default to some specific value. A 32-bit machine seems to handle overflow as a
zero. I guess defaulting to zero is one option. I'm not sure which one is the
best approach.

I'm attaching the diff generated by git. It just checks if tm is NULL, and
defaults to zero, but it is only to demonstrate that the bug is there, the code
isn't written correctly.

-- 
Eduardo A. Bustamante López
The program being debugged has been started already.
Start it from the beginning? (y or n) Starting program: 
/home/dualbus/local/bin/bash 

Program received signal SIGSEGV, Segmentation fault.
0x77686450 in __strftime_internal () from /lib/libc.so.6
#0  0x77686450 in __strftime_internal () from /lib/libc.so.6
#1  0x776880d6 in strftime_l () from /lib/libc.so.6
#2  0x0047e1b4 in printf_builtin (list=0x72cc08) at ./printf.def:474
#3  0x0042fdbf in execute_builtin (builtin=builtin@entry=0x47d410 
, flags=flags@entry=0, 
subshell=subshell@entry=0, words=) at execute_cmd.c:4109
#4  0x00431998 in execute_builtin_or_function (flags=0, 
fds_to_close=0x76e208, redirects=, var=0x0, 
builtin=0x47d410 , words=0x72c908) at execute_cmd.c:4534
#5  execute_simple_command (simple_command=, 
pipe_in=pipe_in@entry=-1, pipe_out=pipe_out@entry=-1, 
async=async@entry=0, fds_to_close=fds_to_close@entry=0x76e208) at 
execute_cmd.c:3944
#6  0x004329cf in execute_command_internal (fds_to_close=0x76e208, 
pipe_out=-1, pipe_in=-1, asynchronous=0, 
command=0x76e148) at execute_cmd.c:735
#7  execute_command_internal (command=0x76e148, asynchronous=0, pipe_in=-1, 
pipe_out=-1, fds_to_close=0x76e208)
at execute_cmd.c:522
#8  0x00435a4e in execute_command (command=0x76e148) at 
execute_cmd.c:382
#9  0x0041e01d in reader_loop () at eval.c:152
#10 0x0041c5bd in main (argc=1, argv=0x7fffe258, 
env=0x7fffe268) at shell.c:749
diff --git a/builtins/printf.def b/builtins/printf.def
index 71a7c00..94e5af5 100644
--- a/builtins/printf.def
+++ b/builtins/printf.def
@@ -471,6 +471,11 @@ printf_builtin (list)
sv_tz ("TZ");   /* XXX -- just make sure */
 #endif
tm = localtime (&secs);
+if (tm == NULL)
+  {
+secs = 0;
+tm = localtime (&secs);
+  }
n = strftime (timebuf, sizeof (timebuf), timefmt, tm);
free (timefmt);
if (n == 0)

Segmentation fault in arithmetical expression when mixing array variables.

[Eduardo A . Bustamante López]

Hi!

I found an issue while using array variables in an arithmetical
context. I tried to determine where the problem was, but I didn't
understand expr.c. The backtrace points to expr.c's line 556, in
expassing. I tested both the master and devel branches.


---
Script
---
#!/bin/bash

echo "$BASH_VERSION"
echo $(( a=(y[0] + y[1]) & 0xff, b=(y[2] + y[3]) & 0xff, a << 8 | b))
---


---
Results
---
4.2.39(2)-release
bash: line 1: 30526 Segmentation fault  (core dumped) ~/bb/segfault 2>&1
---
4.3.0(2)-devel
bash+: line 1: 30794 Segmentation fault  (core dumped) bash+ ~/bb/segfault 
2>&1
---


---
Backtrace
---
Starting program: /home/dualbus/local/bin/bash+ ~/bb/segfault
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?

Program received signal SIGSEGV, Segmentation fault.
0x77656251 in __strlen_sse2 () from /usr/lib/libc.so.6
#0  0x77656251 in __strlen_sse2 () from /usr/lib/libc.so.6
#1  0x004409c0 in expr_bind_array_element (tok=tok@entry=0x778338 "b", 
ind=ind@entry=1, rhs=rhs@entry=0x778348 "0")
at expr.c:342
#2  0x004422e4 in expassign () at expr.c:556
#3  0x004414c2 in expcomma () at expr.c:453
#4  0x004416d5 in subexpr (expr=0x7af988 " a=(y[0] + y[1]) & 0xff, 
b=(y[2] + y[3]) & 0xff, a << 8 | b") at expr.c:431
#5  subexpr (expr=0x7af988 " a=(y[0] + y[1]) & 0xff, b=(y[2] + y[3]) & 0xff, a 
<< 8 | b") at expr.c:407
#6  0x0044262a in evalexp (expr=expr@entry=0x7af988 " a=(y[0] + y[1]) & 
0xff, b=(y[2] + y[3]) & 0xff, a << 8 | b", 
validp=validp@entry=0x7fffdb2c) at expr.c:396
#7  0x00453f02 in param_expand (
string=string@entry=0x7af588 "$(( a=(y[0] + y[1]) & 0xff, b=(y[2] + y[3]) & 
0xff, a << 8 | b))", 
sindex=sindex@entry=0x7fffdc0c, quoted=quoted@entry=0, 
expanded_something=expanded_something@entry=0x7fffdcc8, 
contains_dollar_at=contains_dollar_at@entry=0x7fffdc18, 
quoted_dollar_at_p=quoted_dollar_at_p@entry=0x7fffdc10, 
had_quoted_null_p=had_quoted_null_p@entry=0x7fffdc14, pflags=0) at 
subst.c:7830
#8  0x00454faa in expand_word_internal (word=0x7acf88, 
quoted=quoted@entry=0, isexp=isexp@entry=0, 
contains_dollar_at=contains_dollar_at@entry=0x7fffdccc, 
expanded_something=expanded_something@entry=0x7fffdcc8)
at subst.c:8272
#9  0x004572dc in shell_expand_word_list (tlist=0x7af0c8, 
eflags=) at subst.c:9407
#10 expand_word_list_internal (list=, eflags=eflags@entry=31) at 
subst.c:9526
#11 0x00457b9a in expand_words (list=) at subst.c:9126
#12 0x00433bbe in execute_simple_command (simple_command=, pipe_in=pipe_in@entry=-1, 
pipe_out=pipe_out@entry=-1, async=async@entry=0, 
fds_to_close=fds_to_close@entry=0x7782f8) at execute_cmd.c:3960
#13 0x004359f0 in execute_command_internal 
(command=command@entry=0x7acfc8, asynchronous=asynchronous@entry=0, 
pipe_in=pipe_in@entry=-1, pipe_out=pipe_out@entry=-1, 
fds_to_close=fds_to_close@entry=0x7782f8) at execute_cmd.c:780
#14 0x00438e1e in execute_command (command=0x7acfc8) at 
execute_cmd.c:390
#15 0x0042105d in reader_loop () at eval.c:160
#16 0x0041f591 in main (argc=2, argv=0x7fffe138, 
env=0x7fffe150) at shell.c:755
---



In case my O/S and hardware are relevant:

$ uname -a
Linux claret 3.6.9-1-ARCH #1 SMP PREEMPT Tue Dec 4 08:04:10 CET 2012 x86_64 
GNU/Linux

$ lscpu
Architecture:  x86_64
CPU op-mode(s):32-bit, 64-bit
Byte Order:Little Endian
CPU(s):2
On-line CPU(s) list:   0,1
Thread(s) per core:1
Core(s) per socket:2
Socket(s): 1
NUMA node(s):  1
Vendor ID: GenuineIntel
CPU family:6
Model: 23
Stepping:  6
CPU MHz:   1600.000
BogoMIPS:      4801.95
Virtualization:VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache:  3072K
NUMA node0 CPU(s): 0,1

-- 
Eduardo A. Bustamante López

Behaviour of brace expansion is different in devel branch.

[Eduardo A . Bustamante López]

The development version of bash seems to handle brace expansion
differently than the stable version.

The relevant code:

---
Code
---
echo "$BASH_VERSION"
echo $RANDOM{,,,}
---


---
master
---
4.2.39(1)-release
30011 30767 22583 12713
---


---
devel
---
4.3.0(2)-devel
14730
---

I would expect these to be similar.

-- 
Eduardo A. Bustamante López

Unexpected behavior of single quotes when used in the patsub PE.

[Eduardo A . Bustamante López]

   }
- else
-   {
-             nestret = ttrans;
- nestlen = ttranslen;
-   }
+  nestret = sh_single_quote (ttrans);
+  free (ttrans);
+  nestlen = strlen (nestret);
  retind -= 2;  /* back up before the $' */
}
  else if MBTEST((tflags & LEX_WASDOL) && ch == '"' && 
(extended_quote || (rflags & P_DQUOTE) == 0))
@@ -3830,17 +3822,9 @@ eof_error:
  ttrans = ansiexpand (nestret, 0, nestlen - 1, &ttranslen);
  xfree (nestret);
 
- if ((rflags & P_DQUOTE) == 0)
-   {
- nestret = sh_single_quote (ttrans);
- free (ttrans);
- nestlen = strlen (nestret);
-   }
- else
-   {
- nestret = ttrans;
- nestlen = ttranslen;
-   }
+nestret = sh_single_quote (ttrans);
+free (ttrans);
+nestlen = strlen (nestret);
  retind -= 2;  /* back up before the $' */
}
  else if MBTEST((tflags & LEX_WASDOL) && ch == '"' && (extended_quote 
|| (rflags & P_DQUOTE) == 0))

-- 
Eduardo A. Bustamante López


pgpcyGieD6Isz.pgp
Description: PGP signature

Re: Unexpected behavior of single quotes when used in the patsub PE.

[Eduardo A . Bustamante López]

On Wed, Mar 20, 2013 at 09:15:24PM -0400, Chet Ramey wrote:

> http://lists.gnu.org/archive/html/bug-bash/2012-02/msg00106.html
Ah, I rembember that thread, but somehow just read the beginning of
it. Until now I realize it developed a lot further.

> You say that, but you don't provide any actual evidence to back is up.
> The behavior with this expansion was the same from bash-3.1 (as far back
> as I went) through bash-4.2.

I'm sorry, I meant the change from 4.2 -> 4.3 (don't know the exact
patch level). Running the test script I provided gives the following
output for bash2 and bash3:


===
bash2 2.00.0(1)-release
---
bash2: line 20: unexpected EOF while looking for matching `''
bash2: line 21: syntax error: unexpected end of file
===
bash3 3.00.0(3)-rc1
---
''\'''  |  '
'  |  '
x  |  x
x  |  x
'  |  '
'  |  '
'  |  '
'  |  '
\'  |  '
'  |  '

But I didn't include it initially, because I didn't thought it was
useful. But since you say it's like that since bash 3.1, I guess I
overestimated the behavior.

> Thanks for the script.  I will look at the fix for the bad substitution
> error (I suspect your fix is not correct because the code should not treat
> single quotes as special when parsing a double-quoted string, and the
> error message is printed elsewhere).
Yes, I suspect it's not correct either, but I didn't understand the
bit about removing quotes if the expression was double quoted, so I
went for the straight change that fixed it apparently.

Thanks!

-- 
Eduardo A. Bustamante López


pgpcqJXxL_8_H.pgp
Description: PGP signature

Re: feature request: file_not_found_handle()

[Eduardo A . Bustamante López]

On Wed, Aug 21, 2013 at 08:39:53PM +0200, Andreas Gregor Frank wrote:
> Hello Greg,
> 
> this is a feature request for no_such_file_or_directory_
> handle(). I do not want to talk about missing quotes in anyone's code
> example!

You are free to send patches with the proposed feature. That way we
would be able to test it, and see if it doesn't conflict with
standards or existing codebases.

There doesn't seem to be much demand for this feature, aside from you
two. Therefore, I don't think it's worth the time of Chet to add it,
when he has clearly better things to do with his time (fixing real
bugs for example). Sure, a lot of things could be added to bash,
Ksh-like discipline functions and compound variables, hey, even FP
things like closures, anonymous functions, ... but the point is, are
they worth the effort? If I want real OOP and FP, I use Python and
Ruby.

So, I repeat. If you're in need of the feature, implement it.
Otherwise, you're asking too much IMHO.

-- 
Eduardo Bustamante

Re: RFE: readarray "-0" (or reciprocal of printf "%s\x00" "${AR[@]}" )

[Eduardo A . Bustamante López]

> However, has any thought (or is there a way already?)
> to read in a bunch of null terminated names from
> the output of such a construct?

Well, there's not a construct to build an array like you mention, but
you can use read and a while loop like this:

while IFS= read -rd '' elem; do
  echo "<$elem>";
done < <(printf '%s\0' a b '1 2 3' $'x y \n z')

Which outputs:



<1 2 3>


Now, in more detail:

#  .- This is used to disable leading/trailing whitespace
#  |   trimming.
#  v
while IFS= read -rd '' elem; do
#^.. When you pass an empty string as a
#"delimiter" for read, it (read) will read
#until it finds a NUL character.
  echo "<$elem>";
done < <(printf '%s\0' a b '1 2 3' $'x y \n z')
#   ^.. we use process substitution ( <(...) ) instead of a
#   normal anonymous pipe in case we want to create some
#   persistent variables.

Just be careful, as written above, it requires each record to end
with a NUL byte. If there's no guarantee of a trailing NUL for each
record, you will have to test if read managed to read one last
element with no trailing NUL, it looks something like this:

while ... read -r ... something; do
  ...
done

[[ $something ]] && ...



-- 
Eduardo Bustamante

Re: -e and permission denied

[Eduardo A . Bustamante López]

If you are using -e to test if a file is readable, then you're asking
the wrong question. If you want to know if a file is readable, use
the -r test.

if [ -r some/file ]; then
  ... do something with file that involves reading it ..
fi

Regarding the 'permission denied' errors, that's an *operating
system* issue. The operating system is (on purpose) not letting you
know if that file exists or not, because, if you don't have the
permission to check it, then you shouldn't get that information!
That's the whole point of having permissions on directories, to avoid
handling the list of files to users that don't have the permissions.

So, at the end, if the file exists or not, you as a user cannot know,
because it might, but you don't have permission to see it. So, don't
expect bash to circumvent O/S security mechanisms.

Better yet, most sane commands return a proper exit code when there
was an error. So, if you try to read the file, but the command
couldn't read it, then the command should return an error exit code,
and you can test things like this:

if program some/file; then
  everything went fine
else
  something went wrong...
fi

-- 
Eduardo A. Bustamante López

Re: Numbers in the begining of variable names generates problem

[Eduardo A. Bustamante López]

The error is very clear. ``2_ave'' is not a valid variable name. Use
a valid name. This is not a bug, it is documented. Check the
DEFINITIONS section of your bash manual, it says:

  name   A word consisting only of alphanumeric characters  and  underscores,  
and
 beginning  with  an alphabetic character or an underscore.  Also 
referred
 to as an identifier.


-- 
Eduardo Alan Bustamante López

Re: bash edited while running

[Eduardo A . Bustamante López]

In this case, the problem is in how you deploy the scripts, and not
with bash. Don't change a running script, just write the new script
to a temp file and then replace the script.

That looks something like this:

  mv newscript oldscript

This way, bash will keep reading from oldscript, but new instances of
the script will use newscript instead. That's not just a bash issue,
but a general problem with deployments where you want to avoid
serving inconsistent content (some files updated and some not),
that's why you usually write everything to a location on the same
filesystem, and just issue a mv, which uses rename(2) under the hood.

Note that replacing a file is not the same as editing the file in
place.

So, fix your environment, it's not a bash issue.

-- 
Eduardo Alan Bustamante López

Re: RFC: turn off word splitting for vars but keep for read

[Eduardo A . Bustamante López]

> Yes, I know, I'd ever done alias for `read`:
> 
> IFS=""
> shopt -s expand_aliases
> alias read='IFS=" " read'
Why do you obfuscate your code with aliases? These just make
debugging issues with your code harder. Remembering to quote
variables and use a localized IFS for read is easier, than
introducing an incompatible behaviour that will just help to make the
confusion of shell scripting bigger.

When giving support, it's easier to tell people: «Just double quote
every parameter expansion when you're in doubt», than to list all the
possible cases where they should and shouldn't. With your proposal,
you're just adding a new variable to the mess...

> But nevertheless, I still find my proposal usable (since word
> splitting for vars is unlikely to be usable in scripts).
Word splitting *IS* being used actively in lots of shell scripts that
aim for POSIX compatibility (because it's the only way of
pseudo-emulating arrays), and even it's used (wrongly) in bash
scripts that do not aim to be compatible with the POSIX shell.

So, if the user puts your shopt in ~/.bashrc and happens to run a
script that relies on that... breakage.


Also, lazyness is not a justification. Shell scripts are written
once, but read and executed many times, so invest a little time into
quoting properly, instead of relying on a modified IFS and set -f. Or
use a language that doesn't have to be compatible with POSIX sh.

Quoting isn't that hard once you get used to it ;)

-- 
Eduardo Alan Bustamante López

Re: problem with pathname expansion

[Eduardo A . Bustamante López]

On Wed, Dec 25, 2013 at 06:41:17PM +0400, vollitwr . wrote:

(...)

> Description:
> The patterns [A-C] and [ABC] work different!  [A-C] ignores case and
> [ABC] does not.
> It is true for any such kind of patterns, e.g., [w-z] vs [wxyz].
> This bug reveals itself with many systems: OpenSuse 12, Mandriva 2010
> (32 bits), Debian 7.1, ...
This is not a bug. It's well known that character set ranges depend
on your locale configuration. You seem to be expecting [A-C] to match
ASCII, but your (unspecified) locale expands differently. In
particular, it's the collation order that specifies which character
comes after which.

> Repeat-By:
> Try 'echo [ABC]*' and 'echo [A-C]*'.  For example, the first command may show
> 'Axe' but second may also show 'axe'.
Try with different locales, like the POSIX locale. Read more about
it: man 7 locale

-- 
Eduardo Alan Bustamante López

DEL character treated specially when preceded by a backslash when used in the RHS of the regex operator ([[ $'\177' =~ $'\\\177' ]])

[Eduardo A . Bustamante López]

The DEL ($'\177') character does not behave like the other control
characters when used with the regex operator inside the test keyword.

This example shows the difference in operation:

ubuntu@ubuntu:~$  for c in $'\001' $'\a' $'\177' $'\377'; do for r in "$c" 
"\\$c" "\\[$c]"; do [[ $c =~ $r ]]; printf '[[ %q =~ %q ]] -> %d\n' "$c" "$r" 
"$?"; done; printf %s\\n ---; done;
[[ $'\001' =~ $'\001' ]] -> 0
[[ $'\001' =~ $'\\\001' ]] -> 0
[[ $'\001' =~ $'\\[\001]' ]] -> 1
---
[[ $'\a' =~ $'\a' ]] -> 0
[[ $'\a' =~ $'\\\a' ]] -> 0
[[ $'\a' =~ $'\\[\a]' ]] -> 1
---
[[ $'\177' =~ $'\177' ]] -> 0
[[ $'\177' =~ $'\\\177' ]] -> 1
[[ $'\177' =~ $'\\[\177]' ]] -> 1
---
[[ $'\377' =~ $'\377' ]] -> 0
[[ $'\377' =~ $'\\\377' ]] -> 0
[[ $'\377' =~ $'\\[\377]' ]] -> 1
---

Notice that only $'\177' seems to fail in the middle case, while the
others work just fine.

I also noticed the following strange behavior in bash under Cygwin:

$ for c in $'\001' $'\a' $'\177' $'\377'; do for r in "$c" "\\$c" "\\[$c]"; do 
[[ $c =~ $r ]]; printf '[[ %q =~ %q ]] -> %d\n' "$c" "$r" "$?"; done; printf 
%s\\n ---; done;
[[ $'\001' =~ $'\001' ]] -> 0
[[ $'\001' =~ $'\\\001' ]] -> 0
[[ $'\001' =~ $'\\[\001]' ]] -> 1
---
[[ $'\a' =~ $'\a' ]] -> 0
[[ $'\a' =~ $'\\\a' ]] -> 0
[[ $'\a' =~ $'\\[\a]' ]] -> 1
---
[[ $'\177' =~ $'\177' ]] -> 0
[[ $'\177' =~ $'\\\177' ]] -> 1
[[ $'\177' =~ $'\\[\177]' ]] -> 1
---
[[ $'\377' =~ $'\377' ]] -> 2
[[ $'\377' =~ $'\\\377' ]] -> 2
[[ $'\377' =~ $'\\[\377]' ]] -> 2
---

Notice the weird return code for non-ASCII characters.

I wonder if this has to do more with the regex library it was
compiled against.

The bash versions:
- cygwin: $ bash --version
GNU bash, version 4.1.10(4)-release (i686-pc-cygwin)
- ubuntu: $ bash --version
GNU bash, version 4.2.25(1)-release (x86_64-pc-linux-gnu)

(seems to be present in 4.3.x, the DEL character issue)

Why is DEL treated specially when preceded by a backslash in the
regex operator?

-- 
Eduardo Alan Bustamante López

Re: DEL character treated specially when preceded by a backslash when used in the RHS of the regex operator ([[ $'\177' =~ $'\\\177' ]])

[Eduardo A . Bustamante López]

On Fri, Jan 17, 2014 at 08:43:46AM -0500, Chet Ramey wrote:
> On 1/16/14 6:46 PM, Eduardo A. Bustamante López wrote:
> > The DEL ($'\177') character does not behave like the other control
> > characters when used with the regex operator inside the test keyword.
> 
> This has to do with the expansion of $r and that $r includes a backslash.
> When combined with the internal quoting bash does, and the fact that the
> backslash is special to pattern matching, we end up with this problem.
> I've only thought about it a little so far, but I don't know if there's a
> quick or simple fix.  This may have to wait until after bash-4.3 is
> released.
> 
> Chet
I understand that the backslash preceding a character *could* make it
to not match, though $'\177' is the *only* non-graphic character that
has this behavior.

This should make it more clear:

ubuntu@ubuntu:~$ for c in $'\001' $'\r' $'\177' $'\277' $'\377'; do
> r="\\$c"; [[ $c =~ $r ]]; printf 'c=%q r=%q %d\n' "$c" "$r" "$?"
> done
c=$'\001' r=$'\\\001' 0
c=$'\r' r=$'\\\r' 0
c=$'\177' r=$'\\\177' 1
c=$'\277' r=$'\\\277' 0
c=$'\377' r=$'\\\377' 0

My issue is more regarding why $'\177' has a different behavior than the other
characters, than if the preceding backslash should make it match or
not.

That is, I would expect either these two outputs:

O1:
c=$'\001' r=$'\\\001' 1
c=$'\r' r=$'\\\r' 1
c=$'\177' r=$'\\\177' 1
c=$'\277' r=$'\\\277' 1
c=$'\377' r=$'\\\377' 1

O2:
c=$'\001' r=$'\\\001' 0
c=$'\r' r=$'\\\r' 0
c=$'\177' r=$'\\\177' 0
c=$'\277' r=$'\\\277' 0
c=$'\377' r=$'\\\377' 0

But the real output shows that the case for c=$'\177' is treated
special:

c=$'\001' r=$'\\\001' 0
c=$'\r' r=$'\\\r' 0
c=$'\177' r=$'\\\177' 1 <-- this one behaves differently.
c=$'\277' r=$'\\\277' 0
c=$'\377' r=$'\\\377' 0


---
Now, regarding the issue of whether the backslash should be treated
in a special way, or treated literally, the only thing I can
contribute is the behavior of GNU sed, which handles non-graphic
characters preceded by a backslash the same as the individual
character:

ubuntu@ubuntu:~$ cat sed
mapfile -t chars < <(
printf '\\x%x\n' {1..255} | while read -r c; do printf "$c"'\n'; done
);

for sed in sed 'sed -r'; do
printf -- '--- sed: %s ---\n' "$sed"
for c in "${chars[@]}"; do
printf '%q > %q\n' "$c" "$(printf %s\\n "$c" | $sed "s/\\$c//" 2>&1)"
done | grep -v "''\$"
done
ubuntu@ubuntu:~$ bash sed
--- sed: sed ---
'' > sed:\ -e\ expression\ #1\,\ char\ 5:\ unterminated\ \`s\'\ command
'' > sed:\ -e\ expression\ #1\,\ char\ 5:\ unterminated\ \`s\'\ command
\' > \'
\( > sed:\ -e\ expression\ #1\,\ char\ 6:\ Unmatched\ \(\ or\ \\\(
\) > sed:\ -e\ expression\ #1\,\ char\ 6:\ Unmatched\ \)\ or\ \\\)
1 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
2 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
3 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
4 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
5 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
6 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
7 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
8 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
9 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
\< > \<
\> > \>
B > B
W > W
\` > \`
a > a
b > b
c > sed:\ -e\ expression\ #1\,\ char\ 6:\ Trailing\ backslash
f > f
n > n
r > r
s > s
t > t
v > v
\{ > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ preceding\ regular\ 
expression
\| > \|
--- sed: sed -r ---
'' > sed:\ -e\ expression\ #1\,\ char\ 5:\ unterminated\ \`s\'\ command
'' > sed:\ -e\ expression\ #1\,\ char\ 5:\ unterminated\ \`s\'\ command
\' > \'
1 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
2 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
3 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
4 > sed:\ -e\ expression\ #1\,\ char\ 6:\ Invalid\ back\ reference
5 > sed:\ -e\ expression\ #1\,\ 

Re: DEL character treated specially when preceded by a backslash when used in the RHS of the regex operator ([[ $'\177' =~ $'\\\177' ]])

[Eduardo A . Bustamante López]

> It's not understanding the problem, or the combination of things that
> causes it, but figuring out the right solution.
> 
> Chet
I see, my mistake. I thought I explained it wrong so I wanted to make
it clear. 

Thanks!

-- 
Eduardo Alan Bustamante López

Re: Login to remote server without coding passwords

[Eduardo A . Bustamante López]

On Thu, Jan 23, 2014 at 12:22:24PM -0800, janeit...@rockingham.k12.va.us wrote:
> I tried setting up ssh-keygen, but if I'm correct, I would need every client 
> machine to generate a key, send the public key to the server, and then add it 
> to the authorized_keys file. It doesn't make sense to do this that way.
This has absolutely nothing to do with bash. You're on the wrong
mailing list. This is about errors in the bash shell (bug-bash), and
there's also a list dedicated to help with programming in the bash
shell, which is help-bash.

-- 
Eduardo Alan Bustamante López

Re: let's establish BASH_MINIMUM_TIME_BETWEEN_INTERACTIVE_COMMAND

[Eduardo A . Bustamante López]

On Thu, Jan 30, 2014 at 10:44:30AM +0800, Chris Down wrote:
> On 2014-01-30 09:18:19 +0800, jida...@jidanni.org wrote:
> > So let's establish BASH_MINIMUM_TIME_BETWEEN_INTERACTIVE_COMMAND=1.0
> 
> Well, you can do this with PROMPT_COMMAND='sleep 1'. I don't see the
> need for another internal variable to do this.
How would that work to avoid filling the prompt with the paste buffer
when I fat finger and paste a whole email to the prompt? It will just
slow down the process, but not cancel it.

What I understand that's being proposed is a feature available in
some IRC clients, where the client prompts you if it detects that
you're inputting more than 2 lines in a short period.

This variable with a really long name would invalidate a sequence of
commands if the commands are input in rapid succesion (as when input
through pasting).


-- 
Eduardo Alan Bustamante López

Re: let's establish BASH_MINIMUM_TIME_BETWEEN_INTERACTIVE_COMMAND

[Eduardo A . Bustamante López]

On Wed, Jan 29, 2014 at 09:03:12PM -0700, Bob Proulx wrote:
> Eduardo A. Bustamante López wrote:
> > Chris Down wrote:
> > > Well, you can do this with PROMPT_COMMAND='sleep 1'. I don't see the
> > > need for another internal variable to do this.
> >
> > How would that work to avoid filling the prompt with the paste buffer
> > when I fat finger and paste a whole email to the prompt? It will just
> > slow down the process, but not cancel it.
> 
> It would give you plenty of time to hit Control-C to interrupt it.
> Try it and you will see.
> 
> Bob
> 
I understand. Indeed it seems that it's a good solution and requires
no additional features. I did test it.

-- 
Eduardo Alan Bustamante López

Invalid byte sequence under UTF-8 locale generates a segmentation fault when using printf %q (ansic_quote)

[Eduardo A . Bustamante López]

Using an invalid byte sequence with printf %q segfaults bash, for a
UTF-8 locale.

Here are the steps to reproduce the fault:

gdb local/bin/bash
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /home/dualbus/local/bin/bash...done.
(gdb) r ./invalid-utf8
Starting program: /home/dualbus/local/bin/bash ./invalid-utf8

Program received signal SIGSEGV, Segmentation fault.
0x004b4bc0 in ansic_quote (str=0x7b0d68 "\031ަ", flags=0, rlen=0x0) at 
strtrans.c:282
282   *r++ = c;
(gdb) bt
#0  0x004b4bc0 in ansic_quote (str=0x7b0d68 "\031ަ", flags=0, rlen=0x0) 
at strtrans.c:282
#1  0x004a4121 in printf_builtin (list=0x7b0da8) at ./printf.def:567
#2  0x00440e37 in execute_builtin (builtin=0x4a2e64 , 
words=0x7b0d48, flags=0, subshell=0)
at execute_cmd.c:4337
#3  0x00441a4a in execute_builtin_or_function (words=0x7b0d48, 
builtin=0x4a2e64 , var=0x0, redirects=0x0, 
fds_to_close=0x7b08a8, flags=0) at execute_cmd.c:4758
#4  0x004408e8 in execute_simple_command (simple_command=0x7b0648, 
pipe_in=-1, pipe_out=-1, async=0, fds_to_close=0x7b08a8)
at execute_cmd.c:4161
#5  0x0043a796 in execute_command_internal (command=0x7b06c8, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x7b08a8)
at execute_cmd.c:787
#6  0x00439d44 in execute_command (command=0x7b06c8) at 
execute_cmd.c:390
#7  0x004255e1 in reader_loop () at eval.c:160
#8  0x00423431 in main (argc=2, argv=0x7fffeab8, 
env=0x7fffead0) at shell.c:755
(gdb) info locals
r = 0x7b2000 
ret = 0x7b0de8 
"$'\\031\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336\336",
 ...
s = 0x7b0d69 "ަ"
l = 0
rsize = 16
c = 222 '\336'
clen = 2
b = 0
wc = 1958 L'ަ'
(gdb) quit
A debugging session is active.

Inferior 1 [process 28162] will be killed.

Quit anyway? (y or n) y
dualbus@debian:~$ cat invalid-utf8
LC_CTYPE=en_US.UTF-8
printf '%q\n' $'\031\336\246'
dualbus@debian:~$ bash invalid-utf8 
Segmentation fault
dualbus@debian:~$ bash --version
GNU bash, version 4.3.0(1)-rc2 (x86_64-unknown-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
dualbus@debian:~$ cat invalid-utf8-c-locale 
LC_CTYPE=C
printf '%q\n' $'\031\336\246'
dualbus@debian:~$ bash invalid-utf8-c-locale 
$'\031\336\246'
dualbus@debian:~$ logout



The commit that introduced the bug is the following:

$ git log -n1 --pretty=medium c920c360
commit c920c360da817d2ee755e8ed94ae7d5b9ce313db
Author: Chet Ramey 
Date:   Mon Jan 9 08:27:00 2012 -0500

commit bash-20110902 snapshot

-- 
Eduardo Alan Bustamante López

Re: Invalid byte sequence under UTF-8 locale generates a segmentation fault when using printf %q (ansic_quote)

[Eduardo A . Bustamante López]

On Thu, Feb 13, 2014 at 11:37:27AM -0500, Chet Ramey wrote:
> On 2/13/14 11:33 AM, Eduardo A. Bustamante López wrote:
> > Using an invalid byte sequence with printf %q segfaults bash, for a
> > UTF-8 locale.
> 
> http://lists.gnu.org/archive/html/bug-bash/2014-02/msg00033.html
Thanks! I remember that issue, but didn't cross my mind that they
might be related. That patch fixes this issue.

-- 
Eduardo Alan Bustamante López

Re: Invalid byte sequence under UTF-8 locale generates a segmentation fault when using printf %q (ansic_quote)

[Eduardo A . Bustamante López]

On Thu, Feb 13, 2014 at 11:37:27AM -0500, Chet Ramey wrote:
> On 2/13/14 11:33 AM, Eduardo A. Bustamante López wrote:
> > Using an invalid byte sequence with printf %q segfaults bash, for a
> > UTF-8 locale.
> 
> http://lists.gnu.org/archive/html/bug-bash/2014-02/msg00033.html
> 
Uhm, apparently the patch doesn't fix the issue entirely. It did fix
the issue for the original payload, but I tested with new payloads,
and it still fails. Found three ways to trigger it:


dualbus@debian:~/nbug$ ls
command-name  invalid-bytes  payloads  printf-q  quote.patch  set-x
dualbus@debian:~/nbug$ cat command-name 
payload=$'\065\247\100\063\231\053\306\123\070\237\242\352\263'
"$payload"
dualbus@debian:~/nbug$ cat printf-q 
payload=$'\065\247\100\063\231\053\306\123\070\237\242\352\263'
printf %q "$payload"
dualbus@debian:~/nbug$ cat set-x 
payload=$'\065\247\100\063\231\053\306\123\070\237\242\352\263'
(set -x; : "$payload")
dualbus@debian:~/nbug$ gdb ~/local/bin/bash
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/dualbus/local/bin/bash...done.
(gdb) r ./printf-q
Starting program: /home/dualbus/local/bin/bash ./printf-q

Program received signal SIGSEGV, Segmentation fault.
0x004b4ba6 in ansic_quote (str=0x7b0ec8 '3' ..., 
flags=0, rlen=0x0) at strtrans.c:279
279 *r++ = c;
(gdb) bt
#0  0x004b4ba6 in ansic_quote (str=0x7b0ec8 '3' ..., 
flags=0, rlen=0x0) at strtrans.c:279
#1  0x004a4121 in printf_builtin (list=0x7b0dc8) at ./printf.def:567
#2  0x00440e37 in execute_builtin (builtin=0x4a2e64 , 
words=0x7b0d88, flags=0, subshell=0)
at execute_cmd.c:4337
#3  0x00441a4a in execute_builtin_or_function (words=0x7b0d88, 
builtin=0x4a2e64 , var=0x0, redirects=0x0, 
fds_to_close=0x7b0ba8, flags=0) at execute_cmd.c:4758
#4  0x004408e8 in execute_simple_command (simple_command=0x7b0708, 
pipe_in=-1, pipe_out=-1, async=0, fds_to_close=0x7b0ba8)
at execute_cmd.c:4161
#5  0x0043a796 in execute_command_internal (command=0x7b0788, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x7b0ba8)
at execute_cmd.c:787
#6  0x00439d44 in execute_command (command=0x7b0788) at 
execute_cmd.c:390
#7  0x004255e1 in reader_loop () at eval.c:160
#8  0x00423431 in main (argc=2, argv=0x7fffeab8, 
env=0x7fffead0) at shell.c:755
(gdb) list
274 }
275   if (l)
276 *r++ = '\\';
277 
278   if (clen == 1)
279 *r++ = c;
280   else
281 {
282   for (b = 0; b < (int)clen; b++)
283 *r++ = (unsigned char)s[b];
(gdb) 
284   s += clen - 1;/* -1 because of the increment above */
285 }
286 }
287 
288   *r++ = '\'';
289   *r = '\0';
290   if (rlen)
291 *rlen = r - ret;
292   return ret;
293 }
(gdb) info locals
r = 0x7b2000 
ret = 0x7b0e48 "$'5\\247@", '3' ...
s = 0x7b1fff "3" 
l = 0
rsize = 56
c = 51 '3'
clen = 1
b = 1
wc = 64 L'@'
(gdb) quit
A debugging session is active.

Inferior 1 [process 2017] will be killed.

Quit anyway? (y or n) y
dualbus@debian:~/nbug$ 


As you can see from the gdb list command, the patch has been applied,
and it still shows the issue. If you are interested, I have a list of
payloads that trigger the bug differently for each of the three
tests (some segfault, some not).

You have to ''set follow-fork-mode child'' for the command-name
example to trace it in gdb.

-- 
Eduardo Alan Bustamante López

Re: Invalid byte sequence under UTF-8 locale generates a segmentation fault when using printf %q (ansic_quote)

[Eduardo A . Bustamante López]

On Thu, Feb 13, 2014 at 04:01:21PM -0500, Chet Ramey wrote:
> On 2/13/14 2:32 PM, Eduardo A. Bustamante López wrote:
> > On Thu, Feb 13, 2014 at 11:37:27AM -0500, Chet Ramey wrote:
> >> On 2/13/14 11:33 AM, Eduardo A. Bustamante López wrote:
> >>> Using an invalid byte sequence with printf %q segfaults bash, for a
> >>> UTF-8 locale.
> 
> I think it depends on your system and locale.  I only had the third
> example produce a seg fault, but it was enough.  Try the attached
> updated patch.
Perfect. This new patch passes all the tests I setup. I confirm that
it works.

Thanks!

-- 
Eduardo Alan Bustamante López

Parent shell gets stopped when a child shell is created with job control but not interactive (bash -mc)

[Eduardo A . Bustamante López]

This works fine:

dualbus@debian:~$ ~/local/bin/bash -s <<< 'for i in . .; do (~/local/bin/bash 
-mic ": & wait") ; done'
[1] 1629
[1]+  Done:
[1] 1631
[1]+  Done:

-
This does not:

dualbus@debian:~$ ~/local/bin/bash -s <<< 'for i in . .; do (~/local/bin/bash 
-mc ": & wait") ; done'
[1]+  Done:

[1]+  Stopped ~/local/bin/bash -s <<< 'for i in . .; do 
(~/local/bin/bash -mc ": & wait") ; done'


-
Here's my OS and bash information (bash is
2e4d3851f76a94dfc21af046e0dfd84b954b0f50, from devel)

dualbus@debian:~$ cat /etc/debian_version 
7.3
dualbus@debian:~$ uname -a
Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64 GNU/Linux
dualbus@debian:~$ ~/local/bin/bash --version
GNU bash, version 4.3.0(1)-release (x86_64-unknown-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.


-
Here's a sample output from strace when running this (I removed lots
of lines, didn't want to flood the list):


strace -fo bash-mc.log ~/local/bin/bash -s <<< 'for i in . .; do 
(~/local/bin/bash -mc ": & wait") ; done'

29747 execve("/home/dualbus/local/bin/bash", ["/home/dualbus/local/bin/bash", 
"-s"], [/* 24 vars */]) = 0
29747 brk(0)= 0x149c000
29747 rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
[...] 1st child
29747 clone(child_stack=0, 
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, 
child_tidptr=0x7f58bff209d0) = 29748
29747 rt_sigprocmask(SIG_SETMASK, [],  
29748 rt_sigprocmask(SIG_SETMASK, [],  
29747 <... rt_sigprocmask resumed> NULL, 8) = 0
29747 rt_sigprocmask(SIG_BLOCK, [CHLD],  
29748 <... rt_sigprocmask resumed> NULL, 8) = 0
29748 rt_sigaction(SIGTSTP, {SIG_DFL, [], SA_RESTORER, 0x7f58bf5914f0},  

29747 <... rt_sigprocmask resumed> [], 8) = 0
29748 <... rt_sigaction resumed> {SIG_DFL, [], 0}, 8) = 0
29747 rt_sigprocmask(SIG_SETMASK, [],  
29748 rt_sigaction(SIGTTIN, {SIG_DFL, [], SA_RESTORER, 0x7f58bf5914f0},  

29747 <... rt_sigprocmask resumed> NULL, 8) = 0
29747 rt_sigprocmask(SIG_BLOCK, [CHLD],  
[...] lots of stuff
29748 open("/home/dualbus/local/share/locale/en_US/LC_MESSAGES/bash.mo", 
O_RDONLY) = -1 ENOENT (No such file or directory)
29748 open("/home/dualbus/local/share/locale/en/LC_MESSAGES/bash.mo", O_RDONLY) 
= -1 ENOENT (No such file or directory)
29748 write(2, "[1]+  Done:\n", 32) = 32
29748 rt_sigprocmask(SIG_BLOCK, [CHLD], [CHLD], 8) = 0
29748 rt_sigprocmask(SIG_SETMASK, [CHLD], NULL, 8) = 0
29748 rt_sigprocmask(SIG_BLOCK, [CHLD], [CHLD], 8) = 0
29748 rt_sigprocmask(SIG_SETMASK, [CHLD], NULL, 8) = 0
29748 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
29748 --- SIGCHLD (Child exited) @ 0 (0) ---
29748 wait4(-1, 0x7fff8fdcad80, WNOHANG|WSTOPPED|WCONTINUED, NULL) = -1 ECHILD 
(No child processes)
29748 rt_sigreturn(0x)  = 0
29748 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
29748 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
29748 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
29748 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
29748 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
29748 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
29748 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
29748 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
29748 setpgid(0, 29746) = 0
29748 exit_group(0) = ?
29747 <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 
29748
29747 rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f58bf5914f0}, 
{0x460380, [], SA_RESTORER, 0x7f58bf5914f0}, 8) = 0
29747 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
29747 --- SIGCHLD (Child exited) @ 0 (0) ---
29747 wait4(-1, 0x7870cd80, WNOHANG, NULL) = -1 ECHILD (No child processes)
29747 rt_sigreturn(0x)  = 0
29747 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
29747 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
29747 rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
29747 clone(child_stack=0, 
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, 
child_tidptr=0x7f58bff209d0) = 29750
29747 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
29747 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
29747 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
29747 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
29747 rt_sigaction(SIGINT, {0x460380, [], SA_RESTORER, 0x7f58bf5914f0}, 
{SIG_DFL, [], SA_RESTORER, 0x7f58bf5914f0}, 8) = 0
29747 wait4(-1,  
29750 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
29750 rt_sigaction(SIGTSTP, {SIG_DFL, [], SA_RESTORER, 0x7f58bf5914f0}, 
{SIG_DFL, [], 0}, 8) = 0
29750 rt_sigaction(SIGTTIN, {SIG_DFL, [], SA_RESTORER, 0x7f58bf5914f0}, 
{SIG_DFL, [], 0}, 8) = 0
29750 rt_sigaction(SIGTTOU, {SIG_DFL, [], SA_RESTORER, 0x7f58bf5914f0}, 
{SIG_DFL, [], 0}, 8) = 0
29750 rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER,

Re: RFE: a way to echo the arguments with quoting

[Eduardo A . Bustamante López]

On Sat, Mar 01, 2014 at 02:34:56PM -0800, Dave Yost wrote:
> I have an ugly function I wrote for zsh that does this:
> 
> Sat 14:17:25 ip2 yost /Users/yost
> 1 634 Z% echo-quoted xyz \$foo  'a b c ' '\n'
> xyz '$foo' 'a b c ' '\n'
> Sat 14:17:53 ip2 yost /Users/yost
> 0 635 Z% 
> 
> It would be nice if there were an easy way to do this in bash.
> 
> Here is my use case:
> 
> echo-command() {
>   echo-n 1>&2 "[ "
>   echo-quoted -n 1>&2$@
>   echo   1>&2 " ]"
> }
> 
> echodo() {
>   echo-command $@
>   eval "$@"
> }
> 
> 1 652 Z% echodo sleep 1 
> [ sleep 1 ]
> 0 653 Z% 
> 
> This is a bit of a hack because when I need to use a pipe character, for 
> example, I have to quote it, and that gets echoed in a way that’s wrong for 
> this purpose.
> 
> 0 656 Z% echodo echo abc \| sed 's,b,_,'
> [ echo abc '|' sed s,b,_, ]
> a_c
> 0 657 Z% 
> 
> A builtin that does my ‘echodo’ without having to escape command-line 
> metacharacters is what I really want.
> 
> Is there such a thing?

(1) This list is for bug reports. If you need help with bash
scripting, consider using the help-bash mailing list.

(2) It's easier to achieve this using the DEBUG trap:

| dualbus@debian:~$ trap 'echo "$BASH_COMMAND"' DEBUG
| dualbus@debian:~$ echo foo >/dev/null
| echo foo > /dev/null
| dualbus@debian:~$ echo hello
| echo hello
| hello

(3) If you need 'eval' to achieve something in the shell, you're most
likely doing it wrong: http://mywiki.wooledge.org/BashFAQ/048

(4) For more advanced discussion regarding how to log commands to be
executed:
http://mywiki.wooledge.org/BashFAQ/050#I_want_a_log_of_my_script.27s_actions

But using the DEBUG trap should suffice, it's really advanced. If you
need to propagate that trap to children, use set -T and read about
shopt -s extdebug


Also, again, use help-bash for scripting questions.

-- 
Eduardo Alan Bustamante López

Re: Parent shell gets stopped when a child shell is created with job control but not interactive (bash -mc)

[Eduardo A . Bustamante López]

On Sun, Mar 02, 2014 at 06:09:40PM -0500, Chet Ramey wrote:
> Thanks for the report.  Pointing out the behavior difference between the
> first case (using -i) and the second (without) was the key.  Here's a
> patch.
> 
> Chet
Thanks! The patch fixes the issue:

dualbus@debian:~$ ~/local/bash-jobs-fix/bin/bash -s <<< 'for i in . .; do 
(~/local/bash-jobs-fix/bin/bash -mc ": & wait") ; done'
[1]+  Done:
[1]+  Done:
dualbus@debian:~$ ~/local/bash-jobs-fix/bin/bash -s <<< 'for i in . .; do 
(~/local/bash-jobs-fix/bin/bash -mic ": & wait") ; done'
[1] 26773
[1]+  Done:
[1] 26776
[1]+  Done:

-- 
Eduardo Alan Bustamante López

Executing 'return' inside RETURN trap causes function to recurse infinitely

[Eduardo A . Bustamante López]

WARNING: the codes given below cause the shell to enter an infinite
loop.


Both:
dualbus@debian:~$ bash -Tc 'f(){ :; }; trap return RETURN; f'
^C

and:
dualbus@debian:~$ bash -c 'f(){ trap return RETURN; }; f'
^C

Cause the function call to recurse infinitely. I would understand if
this is labeled as a feature, rather than a bug (in the sense that
the user shouldn't be putting return inside a return trap :), so one
can safely assume that if they did it, it's because they wanted the
infinite recursion).

Though, I think the following 2 approaches would be better/more
consistent.

(1) treat the execution of return inside a RETURN trap as an error,
like when 'return' is called outside a function/sourced script.

(2) treat return specially when it's executed inside a return trap,
making it return immediately from the trap.



dualbus@debian:~$ bash --version
GNU bash, version 4.3.0(1)-rc2 (x86_64-unknown-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later


This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

-- 
Eduardo Alan Bustamante López

Re: Executing 'return' inside RETURN trap causes function to recurse infinitely

[Eduardo A . Bustamante López]

On Fri, Mar 07, 2014 at 11:38:21AM -0800, Eduardo A. Bustamante López wrote:
> WARNING: the codes given below cause the shell to enter an infinite
> loop.
> 
> 
> Both:
> dualbus@debian:~$ bash -Tc 'f(){ :; }; trap return RETURN; f'
> ^C
> 
> and:
> dualbus@debian:~$ bash -c 'f(){ trap return RETURN; }; f'
> ^C
> 
> Cause the function call to recurse infinitely. I would understand if

By the way, I forgot to add the reason why I found this. POSIX states
the following:

| EXIT STATUS
| 
| The value of the special parameter '?' shall be set to n, an unsigned
| decimal integer, or to the exit status of the last command executed
| if n is not specified. If the value of n is greater than 255, the
| results are undefined. When return is executed in a trap action, the
| last command is considered to be the command that executed
| immediately preceding the trap action.

Reference: 
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_24

The interesting bit is 'when return is executed in a trap action,
...'. So I wanted to test that, and the easiest way I thought I could
verify if that matched, was to execute return in a RETURN trap. It's
worth noting that using --posix does not affect the outcome.

-- 
Eduardo Alan Bustamante López

Re: pattern substitution expands "~" even in quoted variables

[Eduardo A . Bustamante López]

On Fri, Mar 07, 2014 at 05:21:53PM +0100, Lars Wendler wrote:
> Configuration Information 
> [Automatically generated, do not change]:
> Machine: x86_64
> OS: linux-gnu
> Compiler: x86_64-pc-linux-gnu-gcc
> Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
> -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' 
> -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
> -DHAVE_CONFIG_H   -I. -I./include -I. -I./include -I./lib  
> -DDEFAULT_PATH_VALUE='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
>  -DSTANDARD_UTILS_PATH='/bin:/usr/bin:/sbin:/usr/sbin' 
> -DSYS_BASHRC='/etc/bash/bashrc' -DSYS_BASH_LOGOUT='/etc/bash/bash_logout' 
> -DNON_INTERACTIVE_LOGIN_SHELLS -DSSH_SOURCE_BASHRC -march=barcelona 
> -mtune=barcelona -O2 -pipe
> uname output: Linux shanghai 3.12.13 #2 SMP PREEMPT Wed Mar 5 17:26:09 CET 
> 2014 x86_64 Quad-Core AMD Opteron(tm) Processor 2389 AuthenticAMD GNU/Linux
> Machine Type: x86_64-pc-linux-gnu
> 
> Bash Version: 4.3
> Patch Level: 0
> Release Status: release
> 
> Description:
> 
> bash-4.3 seems to expand a "~" (tilde character) with full homepath in a 
> pattern substitution even when the variable is embraced by double quotes.
> 
> Repeat-By:
> 
> replace_with_tilde() { P="foo_bar" ; echo ${P/_/~} ;  echo "${P/_/~}" ; }
> 
> 
> bash-4.2.45:
> 
> ~% replace_with_tilde
> foo/home/polynomial-cbar
> foo~bar
> 
> 
> bash-4.3.0:
> 
> ~% replace_with_tilde
> foo/home/polynomial-cbar
> foo/home/polynomial-cbar

(1) You're not quoting it right:

dualbus@debian:~$ for shell in /bin/bash ~/local/bin/bash; do "$shell" -c 
'p=foo_bar; echo "${p/_/\~} $BASH_VERSION"'; done
foo\~bar 4.2.37(1)-release
foo~bar 4.3.0(2)-release

(2) In fact, this change in 4.3 is a fix to an old (and a bit weird)
behavior, see:

- http://lists.gnu.org/archive/html/bug-bash/2012-02/msg00106.html
- http://lists.gnu.org/archive/html/bug-bash/2013-03/msg00099.html

If you want the old behavior:

dualbus@debian:~$ ~/local/bin/bash -c 'shopt -s compat42; p=foo_bar; echo 
"${p/_/\~} $BASH_VERSION"'
foo\~bar 4.3.0(2)-release

-- 
Eduardo Alan Bustamante López

Re: Executing 'return' inside RETURN trap causes function to recurse infinitely

[Eduardo A . Bustamante López]

> While it was clearly not the original intention of the code, the current
> behavior has been in place for many years.  I know how to change it if
> and when I want, but for now I will leave it alone.
> 
> Chet
I don't see any problem with keeping the current behavior (since it's
been like that for so long). Though it would nice if it were
documented behavior.

-- 
Eduardo Alan Bustamante López

Bash does not follow POSIX when return is called during the action of a trap

[Eduardo A . Bustamante López]

According to POSIX:

| The value of the special parameter '?' shall be set to n, an
| unsigned decimal integer, or to the exit status of the last command
| executed if n is not specified. If the value of n is greater than
| 255, the results are undefined. When return is executed in a trap
| action, the last command is considered to be the command that
| executed immediately preceding the trap action.

Source (EXIT STATUS section):
  
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#return


So, what I understand from this:

(1) When return is called without a numeric argument, the code
returned is that of the `last command'.

(2) The `last command' is defined as: ``[...] the command that
executed immediately preceding the trap action''.


Taking the SYNOPSIS for the trap builtin:

| trap n [condition...] 
| trap [action condition...]

and from DESCRIPTION:

| Each time trap is invoked, the action argument shall be processed in
| a manner equivalent to:
| 
| eval action

Source:
  http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#trap


So as I read it, `action' refers to the whole string.



Now, this means, taking the following pseudo-code:

|  trap '(exit BEFORE-RETURN); return' SIGNAL
|
|  fn() {
|(exit BEFORE-ACTION); -block here waiting for signal-
|  }

If that script receives SIGNAL, it should return the BEFORE-ACTION
exit code, and not the BEFORE-RETURN exit code.



Testing this is a bit tricky, because there's no simple way of
blocking to wait for a signal in a way that it doesn't affect our
testing, so the bes I could come up with is this:


### begin test script
code='trap "(exit 2); return" USR1
f() {
{ echo; kill -USR1 $$; } | exit 3
return 5
}

(exit 7); f
'

shells=(
bash
'bash --posix'
ksh
mksh
dash
   'busybox sh'
zsh
jsh
)

for attempt in {1..1000}; do
for shell in "${shells[@]}"; do
printf '%s: %s\n' "$shell" "$($shell -c "$code"; echo $?)"
done
done | sort | uniq -c
### end test script



And sample output from this script:

969 bash: 2
 31 bash: 5
979 bash --posix: 2
 21 bash --posix: 5
   1000 busybox sh: 5
971 dash: 3
 29 dash: 5
118 jsh: 3
882 jsh: 5
  1 ksh: 0
999 ksh: 3
970 mksh: 3
 30 mksh: 5
  6 zsh: 2
994 zsh: 3


As you can see from the results, both bash and zsh, when signaled at
the right time, return `BEFORE-RETURN', which in the script was set
as 2. zsh sometimes returns `BEFORE-ACTION' though. dash, jsh, ksh,
and mksh follow POSIX, in the sense that they usually return 3,
(BEFORE-ACTION). And busybox is just playing alone in there, it
appears as it cannot handle that trap correctly.


The versions tested are:


bash --version|head -n1: 
  GNU bash, version 4.3.0(2)-release (x86_64-unknown-linux-gnu)
zsh --version|head -n1: 
  zsh 4.3.17 (x86_64-unknown-linux-gnu)
ksh --version|head -n1: 
version sh (AT&T Research) 93u+ 2012-02-29
mksh -c 'echo "$KSH_VERSION"': 
  @(#)MIRBSD KSH R40 2012/07/20 Debian-7
apt-cache policy dash|grep Installed: 
Installed: 0.5.7-3
apt-cache policy busybox|grep Installed: 
Installed: 1:1.20.0-7
head -n3 ~/local/src/heirloom-sh/CHANGES: 
  Release ...
* A bug in the supplied realloc() replacement could result in heap
  corruption. (No resulting failures have been observed with sh so far.)


-- 
Eduardo Alan Bustamante López

Re: printf cannot write more than one line to /proc/pid/uid_map

[Eduardo A . Bustamante López]

On Tue, Mar 25, 2014 at 08:24:13PM +0900, Kusanagi Kouichi wrote:
> Configuration Information [Automatically generated, do not change]:
> Machine: x86_64
> OS: linux-gnu
> Compiler: gcc
> Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
> -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' 
> -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
> -DHAVE_CONFIG_H   -I.  -I../. -I.././include -I.././lib  -D_FORTIFY_SOURCE=2 
> -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat 
> -Werror=format-security -Wall
> uname output: Linux crescent 3.14.0-rc7 #1 SMP Mon Mar 17 07:17:51 UTC 2014 
> x86_64 GNU/Linux
> Machine Type: x86_64-pc-linux-gnu
> 
> Bash Version: 4.3
> Patch Level: 0
> Release Status: release
> 
> Description:
>   Bash's builtin printf cannot write more than one line to
>   /proc/pid/uid_map because printf writes one line at a time
>   and uid_map can be written only once.
I don't see why this is a bash bug though. printf works as
documented. The fact that the way it works is incompatible with this
/proc thing is *not* bash's fault. It just means that it's not
flexible enough to tell it to write all at once, but that's not a
bug.

> Repeat-By:
>   shell 1
>   $ unshare -U
>   $ echo $$
>   31861
> 
>   shell 2
>   # printf '0 0 1\n1 1 1' > /proc/31861/uid_map
>   printf: write error: Operation not permitted
> 

-- 
Eduardo Alan Bustamante López

Re: improve performance of a script

[Eduardo A . Bustamante López]

(I forgot to CC the list in my first reply)

On Tue, Mar 25, 2014 at 07:12:16AM -0700, xeon Mailinglist wrote:
> For each file inside the directory $output, I do a cat to the file and 
> generate a sha256 hash. This script takes 9 minutes to read 105 files, with 
> the total data of 556MB and generate the digests. Is there a way to make this 
> script faster? Maybe generate digests in parallel?
> 
> for path in $output
> do
> # sha256sum
> digests[$count]=$( $HADOOP_HOME/bin/hdfs dfs -cat "$path" | sha256sum | 
> awk '{ print $1 }')
> (( count ++ ))
> done
> 
> 
> Thanks,
You were already told in #bash at Freenode that this is not a bash
issue, and yet, you report it as a bug.

Once bash runs the commands, it has no relation at all with their
performance.

Rather, ask the Hadoop people and also maybe the support for your
operating system to see what you can do to optimize that. Maybe it
cannot be optimized... it depends on what the bottleneck is (disk,
network, etc.)

-- 
Eduardo Alan Bustamante López

Re: ls doesn't work in if statements in bash 4.3

[Eduardo A . Bustamante López]

This is a "user" problem. You are using the wrong features for the
task, your code should read:

| for f in *; do ./test "$f"; done

and quote all other variable expansions.

NEVER do: for foo in `...`ORfor foo in $(...(

This is wrong, because you're relying on word splitting and glob
expansion, which is wrong 99% of the cases.

Read:

- http://mywiki.wooledge.org/WordSplitting
- http://mywiki.wooledge.org/Quotes


It is totally not a bash bug (read the manual, it's documented).

On Wed, Mar 26, 2014 at 05:30:12PM -0700, billyco...@gmail.com wrote:
> I tested on bash 4.3 and 3.0
> 
> testing]$ bash --version
> bash --version
> GNU bash, version 4.3.0(1)-release (x86_64-unknown-linux-gnu)
> 
> In a directory I have:
> 
> testing]$ ls -l
> total 16
> -rw-r--r-- 1 hpierce hpierce 77 Mar 26 20:09 dog1
> -rw-r--r-- 1 hpierce hpierce 77 Mar 26 20:09 dog2
> -rw-r--r-- 1 hpierce hpierce 77 Mar 26 20:09 dog3
> -rw-r--r-- 1 hpierce hpierce  0 Mar 26 20:07 dog4
> -rwxr-xr-x 1 hpierce hpierce 80 Mar 26 20:02 test
> 
> dog1, dog2, and dog3 have content.  dog4 is empty.
> 
> test is a simple script:
> 
> testing]$ cat test
> #!/bin/bash
> FILE=$1
> echo $FILE
> if [ ! -e $FILE ]
> then
>   echo "Usage: $0 "
>   exit 1
> else
> echo $FILE exists
> fi
> 
> Here's a regular run:
> 
> testing]$ for f in *; do ./test $f; done
> dog1
> dog1 exists
> dog2
> dog2 exists
> dog3
> dog3 exists
> dog4
> dog4 exists
> test
> test exists
> 
> Now I add a ls:
> 
> testing]$ for f in `ls dog*`; do ./test $f; done
> dog1
> Usage: ./test 
> dog2
> Usage: ./test 
> dog3
> Usage: ./test 
> dog4
> Usage: ./test 
> 
> So I moved it to an earlier version of bash
> 
> testing]$ bash --version
> bash --version
> GNU bash, version 3.00.15(1)-release (x86_64-redhat-linux-gnu)
> 
> testing]$ for f in `ls dog*`; do ./test $f; done
> dog1
> dog1 exists
> dog2
> dog2 exists
> dog3
> dog3 exists
> dog4
> dog4 exists
> 
> 
> 

-- 
Eduardo Alan Bustamante López

Re: ls doesn't work in if statements in bash 4.3

[Eduardo A . Bustamante López]

> These restrictions that you talk about are important, but you don't seem to 
> hear me when I say "My filenames never have spaces".  My directories never 
> have spaces.  I am meticulous about my filesystems.  On one hand I realize 
> you think you're being helpful to someone who has just started bash 
> programming, but since I've been at this for > 12 years it has gotten a 
> little condescending.  I solved the issue myself about 10 minutes later.  I 
> thought it was interesting that with the problematic dir_colors file I was 
> unable to use 'ls' but 'find' worked.  I work as a system administrator so I 
> understand that people are at different levels in terms of technical 
> knowledge.  It's all about reading comprehension.  If you had read "My 
> filenames never have spaces" part and actually paid attention you would see 
> that the problem had nothing to do with spaces.  But no.  Everyone wants to 
> show how smart they are that spaces can mess things up.  Wow!  Thanks!  My 
> guess is that you'll breeze right past this post and continue to give me 
> helpful hints.  Thanks guys!
> 
> --Snarky System Administrator (Is there any other kind?)
Who are you replying to? Since you do not address a specific person,
or at least keep part of the email you're replying to.

And I read the whole thread, and the only thing you mention is:

> I have a script which goes out and converts all my filenames (/home/user/) 
> with 
> spaces into filenames with underscores.  I also convert them to lower case.  

Well, that's cool. But spaces are not the only characters that break
that for loop. But, OK, lets pretend your environment is perfect,
that your filenames don't have spaces, tabs, newlines, asterisks,
question marks, and so on.

Even with all this in mind, don't rely on word splitting. There's a
simpler, easier to read way of doing it, which is: let the shell
expand the globs!


And this whole thread is just ironic. You're defending that your
approach is right, but using that same flawed approach caused you
this issue in the first place! (the dir_colors issue most likely
caused terminal sequences to be captured by `...`, messing with your
filenames).



Anyways. Documented behavior, so, no bash bug. Try to use help-bash
next time though, for scripting issues (that aren't bash bugs). Also,
I recommend you to use 'set -x' more often, it's awesome to catch
these types of bugs caused by non-graphic stuff.

-- 
Eduardo Alan Bustamante López

Re: ls doesn't work in if statements in bash 4.3

[Eduardo A . Bustamante López]

> Your response is "I recommend you to use 'set -x'"?  Nice.  Unfortunately it 
> also means you're a troll, so further discussion with you is pointless.
>   

Well, care to explain why I'm labeled as a troll?


I ran the following:

  | bash-4.3$ mkdir a b c
  | bash-4.3$ touch 1 2 3
  | bash-4.3$ (set -x; echo '===1'; for file in `ls --color=never`; do [ -e 
"$file" ] && echo "$file"; done; echo '===2'; for file in `ls --color=always`; 
do [ -e "$file" ] && echo "$file"; done;)
  | + echo ===1
  | ===1
  | ++ ls --color=never
  | + for file in '`ls --color=never`'
  | + '[' -e 1 ']'
  | + echo 1
  | 1
  | + for file in '`ls --color=never`'
  | + '[' -e 2 ']'
  | + echo 2
  | 2
  | + for file in '`ls --color=never`'
  | + '[' -e 3 ']'
  | + echo 3
  | 3
  | + for file in '`ls --color=never`'
  | + '[' -e a ']'
  | + echo a
  | a
  | + for file in '`ls --color=never`'
  | + '[' -e b ']'
  | + echo b
  | b
  | + for file in '`ls --color=never`'
  | + '[' -e c ']'
  | + echo c
  | c
  | + echo ===2
  | ===2
  | ++ ls --color=always
  | + for file in '`ls --color=always`'
  | + '[' -e 1 ']'
  | + echo 1
  | 1
  | + for file in '`ls --color=always`'
  | + '[' -e 2 ']'
  | + echo 2
  | 2
  | + for file in '`ls --color=always`'
  | + '[' -e 3 ']'
  | + echo 3
  | 3
  | + for file in '`ls --color=always`'
  | + '[' -e 'a' ']'
  | + for file in '`ls --color=always`'
  | + '[' -e 'b' ']'
  | + for file in '`ls --color=always`'
  | + '[' -e 'c' ']'


Due to limitations on what can be sent by email, you won't be able to
notice that the last three entries are shown in color. But running
this on a terminal will make that evident quickly. And this would've
hinted you immediately what the issue was.

(but hey, I'm a troll ;) )




---
Also, on a slightly different issue, please... really, please, use
Google Groups properly. It's VERY annoying to have to skip through
pages of text just to see your 1 line reply. Also, it appears as if
you're replying to yourself. This makes it hard to know who you're
talking to.

Follow the suggestions here to make the experience less painful to us
who have to read you: https://wiki.python.org/moin/GoogleGroupsPython

-- 
Eduardo Alan Bustamante López

Re: 4.3 crash entering q! and exiting the shell

[Eduardo A . Bustamante López]

On Thu, Mar 27, 2014 at 07:36:56PM +0100, Matthias Klose wrote:
> seen when entering q!, then leaving the shell with ^D
> 
> $ gdb bash
> (gdb) run
> Starting program: /bin/bash
> doko@gb:~$ q!
> q!: Befehl nicht gefunden.
> doko@gb:~$
> malloc: .././parse.y:2314: assertion botched
> realloc: start and end chunk sizes differ
> last command: q!
> Aborting...
> Program received signal SIGABRT, Aborted.
> 0x7761df79 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> (gdb) bt
> #0  0x7761df79 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1  0x77621388 in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2  0x0044054f in programming_error ()
> #3  0x004b4bc0 in ?? ()
> #4  0x00472bb6 in sh_xrealloc ()
> #5  0x00423879 in ?? ()
> #6  0x00426612 in ?? ()
> #7  0x00429c39 in yyparse ()
> #8  0x00420ebb in parse_command ()
> #9  0x00420f8c in read_command ()
> #10 0x00421189 in reader_loop ()
> #11 0x0041f729 in main ()
> (gdb) quit
> 

I noticed that this just happens for a single character, followed by
the '!', when history expansion is enabled:

| dualbus@debian ~ % bash
| bash-4.3$ a!
| bash: a!: command not found
| bash-4.3$ 
| malloc: ./parse.y:2314: assertion botched
| realloc: start and end chunk sizes differ
| last command: a!
| Aborting...zsh: abort  bash
| dualbus@debian ~ % bash
| bash-4.3$ aa!
| bash: aa!: command not found
| bash-4.3$ exit
| dualbus@debian ~ % bash
| bash-4.3$ set +H
| bash-4.3$ a!
| bash: a!: command not found
| bash-4.3$ exit


This is the change that introduced the error:

| % git show 36eb585cfa52fbb6cd0c324c628593ea856a50a5 -- parse.y


And I've attached a patch (well, it's basically just a revert of part of that
change), that "fixes" the issue. I blindly reverted, so I don't know
what error was that specific patch supposed to fix in the first
place. Take my patch just as a pointer of where the issue is.

-- 
Eduardo Alan Bustamante López
diff --git a/parse.y b/parse.y
index 83d5b9f..5e3ccde 100644
--- a/parse.y
+++ b/parse.y
@@ -2288,30 +2288,7 @@ shell_getc (remove_quoted_newline)
 	  continue;
 	}
 
-	  /* Theoretical overflow */
-	  /* If we can't put 256 bytes more into the buffer, allocate
-	 everything we can and fill it as full as we can. */
-	  /* XXX - we ignore rest of line using `truncating' flag */
-	  if (shell_input_line_size > (SIZE_MAX - 256))
-	{
-	  size_t n;
-
-	  n = SIZE_MAX - i;	/* how much more can we put into the buffer? */
-	  if (n <= 2)	/* we have to save 1 for the newline added below */
-		{
-		  if (truncating == 0)
-		internal_warning("shell_getc: shell_input_line_size (%zu) exceeds SIZE_MAX (%llu): line truncated", shell_input_line_size, SIZE_MAX);
-		  shell_input_line[i] = '\0';
-		  truncating = 1;
-		}
-	  if (shell_input_line_size < SIZE_MAX)
-		{
-		  shell_input_line_size = SIZE_MAX;
-		  shell_input_line = xrealloc (shell_input_line, shell_input_line_size);
-		}
-	}
-	  else
-	RESIZE_MALLOCED_BUFFER (shell_input_line, i, 2, shell_input_line_size, 256);
+	  RESIZE_MALLOCED_BUFFER (shell_input_line, i, 2, shell_input_line_size, 256);
 
 	  if (c == EOF)
 	{
@@ -2424,7 +2401,7 @@ shell_getc (remove_quoted_newline)
 	 not already end in an EOF character.  */
   if (shell_input_line_terminator != EOF)
 	{
-	  if (shell_input_line_size < SIZE_MAX && shell_input_line_len > shell_input_line_size - 3)
+	  if (shell_input_line_len + 3 > shell_input_line_size)
 	shell_input_line = (char *)xrealloc (shell_input_line,
 	1 + (shell_input_line_size += 2));
 

Re: /dev/fd/62: No such file or directory

[Eduardo A . Bustamante López]

On Fri, Mar 28, 2014 at 08:21:40AM -0700, Linda Walsh wrote:
>   I just checked... it is partly my fault in that I
> upgraded my bash from 4.2 -> 4.3 to get around a limitation in 4.2.
> Am not looking forward to upgrading to 4.4, since both times I've tried,
> autocompletion completely broke and performance was seriously dog'ed even
> in non-debug versions.
There's no bash 4.4.

> Isn't it only things that are like "read xxx < <(cmd)" ? or is there
> something else that uses  process substitution??
<(cmd) and >(cmd) are process substitutions. If you want to work
around, use named pipes. You won't have issues with these.

-- 
Eduardo Alan Bustamante López

Issue when using job control and SIGCHLD with bash 4.3 (trap is not being run)

[Eduardo A . Bustamante López]

Hi,

I was playing with set -m and trapping CHLD in a script, and found
the following error:

$ cat setm-bad.bash
set -m
i=0
trap ':; ((++i<5)) && { f & }' CHLD
f(){ echo x; }
f &
wait

$ cat setm.bash
set -m
i=0
trap '((++i<5)) && { f & }' CHLD
f(){ echo x; }
f &
wait

$ cat reproduce.bash  
for bash in /bin/bash ~/local/bin/bash ~/bb/prefix/bin/bash; do
echo "$bash"
"$bash" --version |& head -n1
echo setm.bash
"$bash" setm.bash
echo setm-bad.bash
"$bash" setm-bad.bash
echo =
done
$ bash reproduce.bash
/bin/bash
GNU bash, version 4.2.37(1)-release (x86_64-pc-linux-gnu)
setm.bash
x
x
x
x
x
setm-bad.bash
x
x
x
x
x
=
/home/dualbus/local/bin/bash
GNU bash, version 4.3.0(1)-rc2 (x86_64-unknown-linux-gnu)
setm.bash
x
x
x
x
x
setm-bad.bash
x
x
x
=
/home/dualbus/bb/prefix/bin/bash
GNU bash, version 4.3.0(1)-maint (x86_64-unknown-linux-gnu)
setm.bash
x
x
x
x
x
setm-bad.bash
x
x
x
x
x
=


The bb/prefix version is the devel branch after applying the attached
patch.


As you can notice, bash 4.3 exits earlier. It should do 5 iterations,
but instead, it only completes 3. I found that is has something to do
with using queue_sigchld_trap instead of run_sigchld_trap, which I
assume fixes another issue, but the fix introduced this error.

-- 
Eduardo A. Bustamante López
diff --git a/jobs.c b/jobs.c
index 4982019..35f3f9d 100644
--- a/jobs.c
+++ b/jobs.c
@@ -3346,6 +3346,9 @@ itrace("waitchld: waitpid returns %d block = %d", pid, block);
 	  if (sigchld == 0)
 	longjmp (wait_intr_buf, 1);
 	}
+  else
+	run_sigchld_trap (children_exited);	/* XXX */
+#if 0
   /* If not in posix mode and not executing the wait builtin, queue the
 	 signal for later handling.  Run the trap immediately if we are
 	 executing the wait builtin, but don't break out of `wait'. */
@@ -3357,6 +3360,7 @@ itrace("waitchld: waitpid returns %d block = %d", pid, block);
 	run_sigchld_trap (children_exited);	/* XXX */
   else
 	queue_sigchld_trap (children_exited);
+#endif
 }
 
   /* We have successfully recorded the useful information about this process

Re: Command name dequote does not work

[Eduardo A . Bustamante López]

On Tue, Apr 15, 2014 at 03:53:42PM +0200, ingo.kra...@eoa.de wrote:
> I finally found it. I forgot to check the function names. It's a function of 
> bash-completion, though I don't think that such a function name should be 
> exported.
bash-completion is a package apart from bash. So, you should report
that to them instead.

Re: Issue when using job control and SIGCHLD with bash 4.3 (trap is not being run)

[Eduardo A . Bustamante López]

On Tue, Apr 15, 2014 at 04:26:50PM -0400, Chet Ramey wrote:
> Here's a patch.
> 
> Chet
Hi Chet, I compiled bash from the latest devel branch and I confirm
that the issue is gone:

dualbus@debian ~/local/src/bash % ~/bash/bin/bash bad
x
x
x
x
x


Thanks!

Re: test for "command not found" before expanding shell parameters

[Eduardo A . Bustamante López]

On Sun, Apr 20, 2014 at 08:05:08PM +0200, Toralf Förster wrote:
> If "greo" does not exist - will bash in that case expand /var/db/pkg/*/*/USE 
> neverttheless ?
I understand that you want this for *interactive* stuff. Storing the
command in a variable and then test with 'command -v' surely is too
much hassle for this to be practical. If you're going to put the name
in a variable, then test if the command exists, and then run the
command, I find it easier to not make typos on the first place ;)

But well, regarding your original question, I wrote this:

tl;dr: Your request would break the standard, but you can use the
DEBUG trap to inspect $BASH_COMMAND and see if there's a valid
command name there.

It is mandated by POSIX that Pathname expansion occurs before the
shell determines if a word corresponds to a command name.

Read here:


It says:

| 2.9.1 Simple Commands
| 
| A "simple command" is a sequence of optional variable assignments and
| redirections, in any sequence, optionally followed by words and redirections,
| terminated by a control operator.
| 
| When a given simple command is required to be executed (that is, when any
| conditional construct such as an AND-OR list or a case statement has not
| bypassed the simple command), the following expansions, assignments, and
| redirections shall all be performed from the beginning of the command text to
| the end:
| 
| 1. The words that are recognized as variable assignments or redirections
| according to Shell Grammar Rules are saved for processing in steps 3 and 4.
| 
| 2. The words that are not variable assignments or redirections shall be
| expanded. If any fields remain following their expansion, the first field
| shall be considered the command name and remaining fields are the arguments
| for the command.
| 
| 3. Redirections shall be performed as described in Redirection.
| 
| 4. Each variable assignment shall be expanded for tilde expansion, parameter
| expansion, command substitution, arithmetic expansion, and quote removal
| prior to assigning the value.
| 
| In the preceding list, the order of steps 3 and 4 may be reversed if no
| command name results from step 2 or if the command name matches the name of a
| special built-in utility; see Special Built-In Utilities.

If you modify 2 to fit your request:

- Break POSIX compatibility (which is OK if the option can be set via
  shopt)
- It'll have to take Pathname expansion as a special case, because
  I've seen many scripts depending on other expansions performed in
  step two, for example, scripts that store command lines in
  variables.


Also, you don't have to patch bash to achieve this. You can use the
DEBUG trap. This approach must have some problems though, but it
should be enough for a pratical use: 

| dualbus@debian:~/y$ shopt -s extglob extdebug
| dualbus@debian:~/y$ cat ~/trap 
| trap '(
| get_command() {
|   local p="+([!=])=*([!=])[[:space:]]";
| 
|   if [[ $1 = $p* ]]; then
| get_command "${1#$p}";
|   else
| printf %s "$1";
|   fi;
| };
| command=$(get_command "$BASH_COMMAND");
| command=${command%%[[:space:]]*}
| if type -t "$command"  >/dev/null 2>&1; then
|   exit 0
| else
|   printf \
| "DEBUG: command not found: %s\n" \
| "$command"
|   exit 1
| fi
| )' DEBUG
| dualbus@debian:~/y$ . ~/trap 
| dualbus@debian:~/y$ ls
| dualbus@debian:~/y$ lsa
| DEBUG: command not found: lsa
| dualbus@debian:~/y$ time lsa
| DEBUG: command not found: lsa
| 
| real  0m0.003s
| user  0m0.004s
| sys 0m0.000s
| dualbus@debian:~/y$ time lsa $(sleep 1)
| DEBUG: command not found: lsa
| 
| real  0m0.002s
| user  0m0.000s
| sys 0m0.000s
| dualbus@debian:~/y$ time lsa $(sleep 5)
| DEBUG: command not found: lsa
| 
| real  0m0.002s
| user  0m0.000s
| sys 0m0.000s
| dualbus@debian:~/y$ time ls $(sleep 5)
| 
| real  0m5.005s
| user  0m0.000s
| sys 0m0.000s

Re: Tilde expansion in ${var/x/y}

[Eduardo A . Bustamante López]

On Sun, Apr 20, 2014 at 01:09:35PM -0700, Jo Liss wrote:
> Is this an intentional change, or is it a bug?
> 
> Cheers,
> Jo
Just to expand on Chet's comment, here's the specific thread:



tl;dr: It's not a bug. You can use a compatibility setting to force
bash behave in the old way, or there are other workarounds.

bug-bash@gnu.org

[Eduardo A . Bustamante López]

> However, the last paragraph does not describe the form "n>&-" (which
> does close descriptor n).  Perhaps that is implied by "similarly", but
> it would be better to spell it out:
> 
>is used similarly to duplicate output file descriptors.  If
>word expands to one or more digits, the file descriptor denoted
>by n is made to be a copy of that file descriptor.  If the
>digits in word do not specify a file descriptor open for
>output, a redirection error occurs.  If word expands to -,
>file descriptor n is closed.  If n is not specified, the
>standard output (file descriptor 1) is used.  As a special
>case, if n is omitted, and word does not expand to one or more
>digits, the standard output and standard error are redirected
>as described previously.
Hmm, I'd recommend to always check the most recent bash version. I
don't have the most recent, but I already have this:

|   Duplicating File Descriptors
|   The redirection operator
|
|  [n]<&word
|
|   is  used  to  duplicate input file descriptors.  If word expands to one 
or more digits, the file descriptor denoted by n is
|   made to be a copy of that file descriptor.  If the digits in word do 
not specify a file descriptor open for input, a  redi‐
|   rection  error  occurs.   If  word  evaluates to -, file descriptor n 
is closed.  If n is not specified, the standard input
|   (file descriptor 0) is used.
|
|   The operator
|
|  [n]>&word
|
|   is used similarly to duplicate output file descriptors.  If n is not 
specified, the standard output (file descriptor 1)  is
|   used.   If the digits in word do not specify a file descriptor open for 
output, a redirection error occurs.  If word evalu‐
|   ates to -, file descriptor n is closed.  As a special case, if n is 
omitted, and word does not expand to one or more digits
|   or -, the standard output and standard error are redirected as 
described previously.

Note the "If word evaluates to -," in both cases. Also, I wouldn't be
that picky on "evaluate" vs "expand", because they can be used
interchangeable here.

Re: Some kind of file descriptor overflow

[Eduardo A . Bustamante López]

On Fri, Jun 13, 2014 at 09:52:49AM -0300, Jorge Sivil wrote:
> The script is in the answer:
> 
> http://stackoverflow.com/questions/24192459/bash-running-out-of-file-descriptors
Can't you reduce the script to a minimum reproducible case? To be
honest, it smells like a scripting error and not a bug, but the code
in that answer is too large and with too many dependencies to be even
worth the time to execute.

Re: Base conversion within Base

[Eduardo A . Bustamante López]

> That said, I have no objection to Chet adding such a feature to Bash,
> if he can think of a sane way to do it.  If you have a patch that
> implements it, it's possible he'll be willing to review it.
I don't really see the point, that'd be just bloating the code base. That task
is easy to accomplish using the features that bash already offers:

dualbus@dualbus ~ % cat foo
convert() {
local base=$1 numerals=$2 number=$3
local result

while ((number >= base)); do
 result=${numerals:number % base:1}$result
((number = number / base))
done
result=${numerals:number:1}$result

echo "$result"
}

convert 23 0123456789abcdefghijklm 1040
convert 16 0123456789abcdef 255
convert  6 012345 123
convert  2 .- 10
convert  2 01 5
convert  8 01234567 16
dualbus@dualbus ~ % bash foo
1m5
ff
323
-.-.
101
20

The point of features is to accomplish things that are not already
possible, or that are possible, but with an insane amount of work. An
8 line function does what you want, with the features already
available.

Re: Issues with exported functions

[Eduardo A . Bustamante López]

> Not quite.  While autoloaded functions are lazily evaluated, you have to
> pay the price of searching $FPATH and loading them in every shell, and
> there still has to be a mechanism to indicate which functions should be
> autoloaded in each shell.
How about lazy loading functions using the following mechanism:

# code
foo arg1 arg2 arg3

1) Search for an alias called 'foo'
2) Search for a function named 'foo'
3) Search for a variable named 'foo' with '(){ ...; }' as contents
4) Search for a builtin ...
...

So far, this is compatible with what we already have, right?

Now, this method (of skipping the function definition until we
attempt to use it) could allow for a special shopt, which disables
the step '3'.

Something like: shopt -s nofunexp, which changes step 3 to

  3) Search for a variable named 'foo' with '(){ ...; }' as contents
  IF nofunexp is disabled.

So, I can now put: shopt -s nofunexp in my scripts proactively, to
avoid function exports at all.

Also, a special array variable could be introduced, like:

BASH_ALLOWED_FUNEXP=(foo bar baz), which acts as a whitelist to allow
only a certain set of functions to be taken from the environment.


So, this way, we keep backwards compatibility, but we also have the
opportunity to disallow exported functions.

-- 
Eduardo Bustamante

Re: Bash-4.3 Official Patch 25 Bug 896776 - (CVE-2014-6271)

[Eduardo A . Bustamante López]

On Thu, Sep 25, 2014 at 05:33:38PM +0200, ralf.naeg...@she.net wrote:
> Hello,
> 
> I've downloaded the source for bash 4.3 and all patches, patched the source 
> to Patch 25. 
> But according some description I've found (http://heise.de/-2403305 sorry, 
> only in German
> available), you can test with the command
> 
> env x='() { :;}; echo vulnerable' bash -c "echo this is a test"
Did you install the patched bash properly? If you forgot to install
it, calling 'bash' will launch the still vulnerable version.

Re: REGRESSION: shellshock patch rejects valid function names

[Eduardo A . Bustamante López]

Well, what did you expect? You're relying on undocumented features.

Relevant sections from the manual:

Definition of 'name':
|  name   A word consisting only of alphanumeric characters and 
underscores, and beginning with an alphabetic character or  an
| underscore.  Also referred to as an identifier.

Definition of shell functions:
|   Shell Function Definitions
|   A  shell function is an object that is called like a simple command and 
executes a compound command with a new set of posi‐
|   tional parameters.  Shell functions are declared as follows:
|
|   name () compound-command [redirection]
|   function name [()] compound-command [redirection]
|  This defines a function named name.  The reserved word function 
is optional.  If the function reserved word is  sup‐
|  plied,  the  parentheses  are optional.  The body of the 
function is the compound command compound-command (see Com‐
|  pound Commands above).  That command is usually a list of 
commands between { and }, but may be  any  command  listed
|  under Compound Commands above.  compound-command is executed 
whenever name is specified as the name of a simple com‐
|  mand.  When in posix mode, name may not be the name of one of 
the POSIX special builtins.  Any redirections (see RE‐
|  DIRECTION  below) specified when a function is defined are 
performed when the function is executed.  The exit status
|  of a function definition is zero unless a syntax error occurs or 
a readonly function  with  the  same  name  already
|  exists.   When  executed, the exit status of a function is the 
exit status of the last command executed in the body.
|  (See FUNCTIONS below.)

It's common knowledge that if you rely on undocumented stuff, your
code will eventually break, like it did now. It's not a regression
though, nowhere in the manual you'll find that colons are allowed in
function names.

So, fix your scripts, perhaps?

Re: Testing for Shellshock ... combinatorics and latest(Shellshock) Bash Vulnerability...(attn: Chet Ramey)

[Eduardo A . Bustamante López]

Wow, that's a very long email.

While it's possible to fuzzy test bash, the problem is, first, you
have to find a way to generate strings that maximize the chance of
being a genuine command or a command that triggers a bug. This is
very expensive...

Second, once you generate a command, how will your test program know
if it found a bug? It's easy when bash segfaults, but in the case of
shellshock, it wasn't a crash. It was rather that bash was executing
code where it shouldn't. Not even humans were able to tell that for
more than 20 years. I'm sure Stephane wasn't the first one to notice
that, but he was the first one to realize the huge problem that this
represented.

Also, you have to find a way to isolate your machine/OS from the
testing, because bash has access to the file system, and other
things, that could destroy the operating system in the case of a
command gone wrong. (I suggest looking at 'shbot' for this particular
issue).

Other than that, I guess, good luck. I also played with this idea for
some time in my mind, but, didn't bother implementing it, because of
the second point, mainly.

I guess you could feed your command generator a large corpus of
scripts, and use a Markov model to generate sequences of commands,
that could be better than the combinatorics.

Re: Issue with Bash-4.3 Official Patch 27

[Eduardo A . Bustamante López]

On Wed, Oct 15, 2014 at 03:38:01PM +0200, lorenz.bucher@rohde-schwarz.com 
wrote:
> Hello,
> in refer to 
> http://lists.gnu.org/archive/html/bug-bash/2014-09/msg00278.html variables 
> with suffix "%%" can't be set/exported.
> This makes problems restoring environments which where saved by external 
> programs like printenv (see example below)

I'm not sure if this is a bug, because I doubt that function
exporting was intended to use that way. Just source a file with the
function definitions...

If you want to save the function definitions, use:

  declare -f > func-defs

or:

  declare -f func1 func2 > func-defs

And then source that file. And, bug gone ;)


I'm pretty sure it's not a bug, because you're just supposed to use
(my guess) function exports between live processes, not save and then
restore.

Re: Fwd: Testing for Shellshock...

[Eduardo A . Bustamante López]

> I believe it would be interesting to measure the combinatorial coverage of
> the fuzz tests.
You already asked this... why didn't you follow up in the thread that
you started?

> [...] So we would be able to say what percentage of 2-way,
> 3-way, etc. combinations are covered,  which would be useful in
> understanding the difficulty of finding the bug using tests.  For example,
> it may be that the test script produces a high level of 4-way combination
> coverage, suggesting that the bug is complex and that a high proportion of
> the possible input space needs to be covered by tests to detect the problem
> (that we would need 4-way or 5-way testing).
Uh, what? This seems like a school/research project for a course that
I obviously don't know, but would you mind explainin what n-way
combinations are and why should we care?

> This may depend on how tailored the fuzz tests are for finding this
> particular bug.  Obviously they could be very narrowly tailored and not
> cover much of the input space.  We would have to measure the coverage to
> see.
Uh, but, you see... they already found bugs with that approach.
Either we fiddle with that approach to increase the number of tests
and possibly uncover more bugs, or make the approach more general,
but I'm not sure this is feasable. What I remember from the person that
run a fuzzer on bash, sie specifically reduced the possibilities by
testing just a small part of bash (exported functions).

> I’m asking this group for feedback... do you think the fuzz test scripts
> that have been developed could be analyzed in this way?   To measure the
> combination coverage, we just need test values in a matrix or spreadsheet
> format, where each row is a test and each column represents a parameter.
Uh, again, what? Also, again, follow up in the original thread.

> Could we run the fuzz test scripts and produce such a matrix?
Who is 'we'? If you're willing to invest some CPU cycles on that,
please do it :)

Re: Practical use of the C preprocessor

[Eduardo A . Bustamante López]

On Mon, Oct 20, 2014 at 04:14:04PM -0700, Flávio Medeiros wrote:
> Hi Bash developers,
Please, don't spam this list. The topic is about *bash*.

There are better and less annoying ways of doing these kinds of
surveys.

Re: feature: time builtin and file descriptor

[Eduardo A . Bustamante López]

> Sounds the there is not much enthusiasm about making this sort of
> feature to work. This proposal belongs to archived never implemented
> ideas area. Good that effort wasting was kept minimal.
Remember that working patches are better than just requesting
features. There are lots of pending features that are requested more
frequently, so that's a lot of work for a single person to handle.

Re: feature: time builtin and file descriptor

[Eduardo A . Bustamante López]

> OK.  It doesn't sound like this feature is of general interest.  Since
> you can control when you open and close file descriptors, you might look
> at $SECONDS when the file is opened and when it's closed and using the
> difference to see how long it was open.

If I understand correctly, the original issue wasn't exactly to
measure the time FDs were open, but to use that measure to time the
execute of code blocks. That is, timing open FDs was just a tool, not
the goal itself. Since we can already time blocks, I think this
alternative is not needed. I wrote a detailed explanation here:


dualbus@hp:~$ bash ./script
processing $i=1
item $i=1 took 4.003
processing $i=2
item $i=2 took 6.003
processing $i=3
item $i=3 took 8.003
all together took 18.014

dualbus@hp:~$ cat ./script
#!/bin/bash

# set TIMEFORMAT to show only the 'elapsed time in seconds'. (see man bash)
TIMEFORMAT=%R

# The key points are:
# 1. You can 'time' a block, so, stuff like: time { ...; } and time for ...; do 
...
#are valid. You don't need to trace an open file descriptor to time a block
#of commands.
# 2. Capturing the output of 'time' is tricky, but it can be done, follow the
#indications here: http://mywiki.wooledge.org/BashFAQ/032 (or read my
#attempt at explaining it at the end)

{ g=$(
{ time for i in 1 2 3; do
echo "processing \$i=$i"

{ t=$(
  { time {
  # item commands
  sleep "$i"
  sleep "$((i+2))"
}
  } 2>&1 >&3
);} 3>&1

echo "item \$i=$i took $t"
  done
} 2>&1 >&3
);} 3>&1

echo "all together took $g"


# How to do what I think you want to do:
#
# 1. Change TIMEFORMAT to output just the number you want (easy)
# 2. Now, the tricky part, capture the output of time:
#- You can't just do: var=$(time ...) and hold the output of time in var,
#  because 'time' writes to stderr. So the logical thing follows.
#- We try: var=$(time ... 2>&1 >/dev/null) (we discard stdout for
#  simplicity), but that still doesn't work, because that redirection
#  doesn't seem to apply to 'time', instead, it applies to the command
#  we're timing.
#- So, we try: var=$({ time ...; } 2>&1 >/dev/null), and... that works.
#  By doing the redirection on a group that consists only of 'time', we
#  managed to capture its output.
#
#  You can try this at home:
#
# $ TIMEFORMAT=%R; var=$({ time sleep 3.1416;} 2>&1 >/dev/null); echo "$var"
# 3.143
#
# 3. So, we have the elapsed time... but, what if we also want to keep stdout?
#We have to do some tricky redirections:
#   
#{ var=$({ time ...; } 2>&1 >&3); } 3>&1
#
#Don't panic, it looks horrible, but it's actually easy to understand. The
#first thing we should know is that redirections apply following these two
#rules:
#
#a. If you have nested blocks, it will apply the outer redirections first.
#   So, in { { { foo >a; } >b; } >c; } >d
#   It will first redirect to 'd', then to 'c', then to 'b', ...
#b. Redirections done at the same level are executed from left to right.
#
#With these two details we can proceed to understand:
#
#{ var=$({ time ...; } 2>&1 >&3); } 3>&1
#
#Since we execute the outer redirections first, '3>&1' is the first
#redirection, so now there's a file descriptor 3, open for the whole block
#delimited by { ... }. Ah, and that FD3 is a clone of the script's stdout,
#which could be a file, the terminal, etc. Then, inside var=$({...} redirs)
#we have another set of redirections, '2>&1' and '>&3'. They get executed
#from left to right, so '2>&1' goes first. What this does is to send stderr
#(time's output), to stdout *but* in this case, stdout goes to $(...),
#which is captured in 'var' (it doesn't go to the main script stdout). And
#then we have '>&3', which does: send stdout (which currently was going to
#$(...) ) to the main script's stdout (&3 is a clone of it).
#
#Sorry for the complex explanation.
#
#And, if I read your original email correctly, this is what you wanted to 
do:
#
#  > This idea came to my mind while writing a script that runs multiple
#  > commands, and I simply wanted to know how long they are busy.  I am
#
#So, now you can run 'time' multiple times, even nested, and find out how
#long they are busy, without having to patch bash. (Sure, the code is ugly, 
#but, it's bash, it's expected to be like that ;) ).
#
#  > What do you think, useful feature or unnecessary bloat?
#   My opinion is: unnecessary bloat, since we can work-around that specific
#   case without new features. Sure, the work-around is ugly, but, it's
#   consistent, and, with a bit of effort, it can do almost everything your
#   proposal attempts to achieve.

Re: Odd bash behaviour with time:

[Eduardo A . Bustamante López]

On Fri, Oct 31, 2014 at 11:00:22PM +0100, Piotr Grzybowski wrote:
> Hi,
> 
>  it is actually built-in time that you run:
Hm, read again. It is stated that the external command doesn't have
that issue. This is precisely related to the time builtin and why it
would cause a syntax error in this case.

Re: Odd bash behaviour with time:

[Eduardo A . Bustamante López]

>  well, help time clearly states how it should be used.
You are clearly not understanding the point.

The point is: why does

  time

work, but

  time ; somecommand

doesn't.

It's that simple. It's not a usage question, I'm very aware of how to
use time. Stop being condescending.

Re: Odd bash behaviour with time:

[Eduardo A . Bustamante López]

To Piotr:

On Sat, Nov 01, 2014 at 12:35:47AM +0100, Piotr Grzybowski wrote:
>  well, thats what I have been trying to say, before Eduardo almost killed me 
> :)
>  help time clearly states that is requires a pipeline, NULL pipeline
> is something that does not exist. it is just a bad usage of time
> built-in.
Can you cut the crap? It's you not understanding the question and
pretending that you do. Also, don't send mail to me privately.

I don't take back the condescending part, because that's what you're doing,
pretending that you know more than the OP and I, without even actually
researching what's being discussed. It would be better if you just stepped
back, and waited for Chet's reply, who is the only one who has something
interesting to say about this.

Also, quit the broken Spanish and the references to killing. That's very
distasteful.


Evidently, you haven't read the manual:

| When the shell is in posix mode, time may be followed by a newline.  In this 
case, the shell displays the
| total user and system time consumed by the shell and its children.  The 
TIMEFORMAT variable may  be  used
| to specify the format of the time information.

That is: time\n

Hint: it's not followed by a pipeline.

It doesn't say about bash not running in posix mode, but, I'd expect that to be
the same behavior.



To jon:

> > Interesting interpreter question, but does "time" on its own have any
> > meaning? - if not "time" with no arguments should generate an error
> > rather than an output as it does currently.
I agree, if

  time

doesn't generate an error, why

  time; command

does...


That's a question for Chet to answer, or by someone that understands
what the parser is doing here.

Re: Odd bash behaviour with time:

[Eduardo A . Bustamante López]

> Two comments.
> 
> 1) The shell is not in posix mode.
Yep.

According to my tests, it also fails like OP reported in posix mode:

| dualbus@dualbus ~ % bash -c $'for ((i=0;i<1;i++)); do :; done\ntime'
| 
| real0m0.000s
| user0m0.000s
| sys 0m0.000s
| dualbus@dualbus ~ % bash --posix -c $'for ((i=0;i<1;i++)); do :; 
done\ntime'
| user0m0.06s
| sys 0m0.00s
| dualbus@dualbus ~ % bash --posix -c $'head -c100 /dev/null\ntime'
| user0m0.00s
| sys 0m0.09s
| dualbus@dualbus ~ % bash -c $'head -c100 /dev/null\ntime'
| 
| real0m0.000s
| user0m0.000s
| sys 0m0.000s
| dualbus@dualbus ~ % bash -c $'time;ls'
| bash: -c: line 0: syntax error near unexpected token `ls'
| bash: -c: line 0: `time;ls'
| dualbus@dualbus ~ % bash --posix -c $'time;ls'
| bash: -c: line 0: syntax error near unexpected token `ls'
| bash: -c: line 0: `time;ls'


> 2) In non posix mode time with no arguments is not meaningful. Best I
> can tell it always returns 0, I still maintain it should return an
> error.  
> 
> # while :; do echo Hello; done
> Let it run and run and run
> # ^C
> # time
> 
> real0m0.000s
> user0m0.000s
> sys 0m0.000s
> 
> 
> 
> Maybe it should be more like this:
> 
> # time
> Error, 'time' with no arguments is only meaningful in posix mode
I agree with you on this.

But, the thing is... it shouldn't be a syntax error, right? My
opinion is that

$ time
bash: usage error: blah blah

$ time; ls
bash: usage error: blah blah
foo bar baz

Should be how bash treats this, even if time is a special keyword.

Or... why not have time behave the same in non-posix mode?...
backwards compatibility is not an issue, since it's not useful right
now. And the posix mode behavior is kind of useful.

Segmentation fault when running recursive traps

[Eduardo A . Bustamante López]

I know that a recursive trap in a script is probably a bug, but I think that it
shouldn't make bash segfault:

| dualbus@hp ...local/src/bash % ./bash -c 'trap "kill \$\$" TERM; kill $$'
| [...]
| ./bash: line 1: warning: run_pending_traps: recursive invocation while 
running trap for signal 15
| ./bash: line 1: warning: run_pending_traps: recursive invocation while 
running trap for signal 15
| ./bash: line 1: warning: run_pending_traps: recursive invocation while 
running trap for signal 15
| ./bash: line 1: warning: run_pending_traps: recursive invocation while 
running trap for signal 15
| ./bash: line 1: warning: run_pending_traps: recursive invocation while 
running trap for signal 15
| zsh: segmentation fault  ./bash -c 'trap "kill \$\$" TERM; kill $$'

Other shells that also segfault:
- dash:
| dualbus@hp ~/local/src/dash-0.5.8
|  % ./src/dash -c 'trap "kill \$\$" TERM; kill $$' 
|  zsh: segmentation fault  ./src/dash -c 'trap "kill \$\$" TERM; kill $$'
- mksh:
| dualbus@hp ~ % mksh -c 'echo $KSH_VERSION'
| @(#)MIRBSD KSH R50 2014/10/19
- busybox sh:
|dualbus@hp ~ % busybox sh -c 'trap "kill \$\$" TERM; kill $$' 
|zsh: segmentation fault  busybox sh -c 'trap "kill \$\$" TERM; kill $$'
|dualbus@hp ~ % busybox|head -n1
|BusyBox v1.22.1 (Debian 1:1.22.0-13) multi-call binary.

Shells that do not:
- zsh
- ksh93
- heirloom's

(BTW, I do know that I should be doing: 
| -c 'trap "trap - TERM; kill \$\$"; kill $$',
but, my point is that it should probably have a recursion limit, just like
functions have)

Improper array name validation for the 'mapfile' builtin

[Eduardo A . Bustamante López]

Hi Chet,

It seems like the variable name validation for the mapfile builtin is a bit
wrong. I hope the patch below fixes it.

Demonstration:

| dualbus@hp ...local/src/bash % : before patch
| dualbus@hp ...local/src/bash % /bin/bash -c 'mapfile -t "a[\$(ls)]" <<< ""; 
declare -p'|grep -F '[$('
| declare -a a[$(ls)]='([0]="")'
| 
| dualbus@hp ...local/src/bash % : after patch
| dualbus@hp ...local/src/bash % ./bash -c 'mapfile -t "a[\$(ls)]" <<< ""; 
declare -p'|grep -F '[$('
| ./bash: line 0: mapfile: `a[$(ls)]': not a valid identifier


Patch:

diff --git a/builtins/mapfile.def b/builtins/mapfile.def
index 5e258bb..51e37c6 100644
--- a/builtins/mapfile.def
+++ b/builtins/mapfile.def
@@ -349,7 +349,7 @@ mapfile_builtin (list)
   else
 array_name = list->word->word;
   
-  if (legal_identifier (array_name) == 0 && valid_array_reference (array_name) 
== 0)
+  if (legal_identifier (array_name) == 0)
 {
   sh_invalidid (array_name);
   return (EXECUTION_FAILURE);

Re: test '-v' - associative vs. normal array discrepancy - a bug ?

[Eduardo A . Bustamante López]

People, are we forgetting that this is supposed to work in a function, by
passing the name of a variable?

i.e. it has to look like this:

a=
b=x

is_defined a -> yes
is_defined b -> yes
is_defined c -> no

The 'length of array' expansion doesn't work when you're given a name.


So far, the best I could think of is 'is_defined3':

dualbus@hp ~ % ./test2
functiona   b   c   A   B   C   D   AA  BB  CC  DD
is_defined1 1   0   1   1   0   1   0   1   0   1   1
is_defined2 1   0   1   1   0   1   0   1   0   1   1
is_defined3 0   0   1   0   0   1   0   0   0   1   0


note: 0 : yes, 1 : no



Greg, close your eyes, you will *not* like this :)



dualbus@hp ~ % cat ./test2
#!/bin/bash

# note for is_defined[12]: if you expand an associative array with "${aa[@]}";
# it will expand to its values.

is_defined1() {
local an="$1[@]"
local -a a=("${!an}")

[[ ${!1+x} = x ]] || [[ "${#a}" -gt 0 ]]
}

is_defined2() {
local an="$1[@]"
local -a a=("${!an}")

[[ -v "$1" ]] || [[ "${#a}" -gt 0 ]]
}

is_defined3() {
{ declare -p -- "$1" && ! declare -fp -- "$1"; } 2>/dev/null >&2
}

a=
b=x

A=()
B=(X)
D=('')

declare -A AA=()
declare -A BB=(['x']=x)
declare -A DD=(['x']=)

declare -A is_defined{1..3}
vars=(a b c A B C D AA BB CC DD)

for var in "${vars[@]}"; do 
is_defined1 "$var"; is_defined1["$var"]=$?
is_defined2 "$var"; is_defined2["$var"]=$?
is_defined3 "$var"; is_defined3["$var"]=$?
done

printf '%s' function
for var in "${vars[@]}"; do
printf '\t%s' "$var"
done; printf \\n

for result in is_defined{1..3}; do
printf '%s' "$result"
for var in "${vars[@]}"; do
element="$result[\"$var\"]"
printf '\t%s' "${!element}"
done; printf \\n
done

Re: test '-v' - associative vs. normal array discrepancy - a bug ?

[Eduardo A . Bustamante López]

On Wed, Nov 19, 2014 at 10:32:52PM +0100, Piotr Grzybowski wrote:
>  I do understand your point. Dont you understand the need to check in
> a simple (I know, nothing works, nothing is simple ;-)) way that given
> variable has value? no matter what it is?

there:

is_defined3() {
{ declare -p -- "$1" && ! declare -fp -- "$1"; } 2>/dev/null >&2
}

works for strings, arrays and associative arrays. No need for hacks like eval,
${!var}, or a need to add a new operator to bash's test.

Re: test '-v' - associative vs. normal array discrepancy - a bug ?

[Eduardo A . Bustamante López]

> So far, so good.  But
> 
> imadev:~$ foo() { echo foo; }
> imadev:~$ foo=bar
> imadev:~$ is_defined3 foo ; echo $?
> 1

Ouch! Last try:

is_defined4() {
declare -p -- "$1" 2>/dev/null >&2
}

But I agree with you, the shell is tricky.

Performance regression for hash_insert when using bash's internal malloc

[Eduardo A . Bustamante López]

I noticed that bash from the devel branch was taking too much time to start
when invoked as an interactive shell. I happen to have bash-completion
installed.

See this:

| dualbus@hp:~$ for sh in /bin/bash /tmp/bash/devel-O2/bin/bash; do time "$sh" 
-i <<< ''; done
| dualbus@hp:~$ 
| dualbus@hp:~$ exit
| 
| real0m0.071s
| user0m0.060s
| sys 0m0.004s
| dualbus@hp:~$ 
| dualbus@hp:~$ exit
| 
| real0m2.497s
| user0m2.484s
| sys 0m0.012s


>From 0.071s to 2.497s, that's a lot of time...

I managed to track the issue to the calls to 'complete ... NAME', which do a
hash_insert for each name (bash-completion does a lot of 'complete' calls).

So, in order to try and isolate the issue, I wrote the following test script
(the relevant part is highlighted):



dualbus@hp ~ % cat test
#!/bin/bash

: <<'REQUIRES'
* moreutils (for 'ts')
* gnuplot
REQUIRES

unset tmpdir tmpimg
trap 'rm -rf "$tmpdir"' EXIT

tmpdir=$(mktemp -d)
tmpimg=$(mktemp)

shells=(
/tmp/bash/master-O2/bin/bash
/tmp/bash/4.3-rc2-O2/bin/bash
/tmp/bash/devel-O2/bin/bash
/tmp/bash/devel-nobashmalloc-O2/bin/bash
)

cd "$tmpdir" || exit 1

for i in "${!shells[@]}"; do
sh=${shells[i]}
for _ in {1..20}; do
"$sh" -c '

# sh -c code
# THIS IS THE INTERESTING PART
declare -A a;

for ((i = 0; i < 3000; i++)); do
a["$i"]=.; echo "$i";
done
# END INTERESTING PART

' | ts -s '%.s'
done | awk '

# we have: $timestamp $i
# we want: $delta $i

BEGIN { p_ts = 0 }

{ 
if(p_ts > $1) {
p_ts = 0
}

if(! N[$2]) {
S[l++] = $2
}

X[$2] += ($1 - p_ts)
N[$2]++

p_ts = $1
}

END {
for(i = 0; i < l; i++) {
k = S[i]
printf "%f\t%s\n", X[k]/N[k], k
}
}
' > "sh-$i.dat"
done 

gnuplot /dev/stdin <<'PLOT' > "$tmpimg"
reset
set terminal png notransparent size 1200,960 

set xtics 512
set xlabel "item"

set ylabel "time (ms)"

set title "hash_insert performance"

set style data linespoints

plot "sh-0.dat" using 2:($1*1000) title "master", \
 "sh-1.dat" using 2:($1*1000) title "4.3", \
 "sh-2.dat" using 2:($1*1000) title "devel", \
 "sh-3.dat" using 2:($1*1000) title "devel no-bash-malloc"
PLOT

echo "image: $tmpimg"




I compiled the shells like this:

/tmp/bash/master-O2/bin/bash:
CFLAGS='-O2' ./configure --prefix=/tmp/bash/master-O2 && make install
/tmp/bash/4.3-rc2-O2/bin/bash:
CFLAGS='-O2' ./configure --prefix=/tmp/bash/4.3-rc2-O2 && make install
/tmp/bash/devel-O2/bin/bash:
CFLAGS='-O2' ./configure --prefix=/tmp/bash/devel-O2 && make install
/tmp/bash/devel-nobashmalloc-O2/bin/bash:
CFLAGS='-O2' ./configure --with-bash-malloc=no 
--prefix=/tmp/bash/devel-nobashmalloc-O2 && make install


I attach the graph that results from running the script. And for those who
check this through the archive, I leave a link to it here:

http://i.imgur.com/ir8zJ6s.png

As you can notice, there's a weird spike for both the 4.3 and devel versions
which use the internal bash malloc around 2560.

Re: Performance regression for hash_insert when using bash's internal malloc

[Eduardo A . Bustamante López]

Nevermind, I was told by someone from #bash in IRC that this issue has been
raised multiple times:

* http://lists.gnu.org/archive/html/bug-bash/2014-01/msg00126.html
* http://lists.gnu.org/archive/html/bug-bash/2011-05/msg00020.html
* http://lists.gnu.org/archive/html/bug-bash/2013-05/msg3.html



Compiling with:
CFLAGS='-O2' ./configure --with-bash-malloc=yes 
--prefix=/tmp/bash/devel-nodebug-O2 && make DEBUG= MALLOC_DEBUG= install

Changing the test:

dualbus@hp ~ % diff -u test.old test 
--- test.old2014-11-22 17:09:35.413970597 -0600
+++ test2014-11-22 17:10:50.730915097 -0600
@@ -16,6 +16,7 @@
 /tmp/bash/4.3-rc2-O2/bin/bash
 /tmp/bash/devel-O2/bin/bash
 /tmp/bash/devel-nobashmalloc-O2/bin/bash
+/tmp/bash/devel-nodebug-O2/bin/bash
 )
 
 cd "$tmpdir" || exit 1
@@ -80,7 +81,8 @@
 plot "sh-0.dat" using 2:($1*1000) title "master", \
  "sh-1.dat" using 2:($1*1000) title "4.3", \
  "sh-2.dat" using 2:($1*1000) title "devel", \
- "sh-3.dat" using 2:($1*1000) title "devel no-bash-malloc"
+ "sh-3.dat" using 2:($1*1000) title "devel no-bash-malloc", \
+ "sh-4.dat" using 2:($1*1000) title "devel no-debug"
 PLOT
 
 echo "image: $tmpimg"

Gives: http://imgur.com/LDBWeSr

(also attached)


So, in conclusion: It's not a regression, you can easily turn off the
instrumentation to get the normal performance.


Sorry for the noise :)

Re: BASH_FUNC__ appearing in environment

[Eduardo A . Bustamante López]

On Thu, Nov 27, 2014 at 03:43:05AM -0800, steveT wrote:
> I am not sure if this is the correct place to raise this - I have tried 
> specific Fedora and bash forums, but with no joy so far.
Bash forums :-)? I'm interested on knowing which ones!

> This may be expected behaviour, but it seems so random. The above seems to be 
> related to rcs, but I also intermittently get similar entries for 
> BASH_FUNC__sudo. I am not sure if these functions appearing in my environment 
> poses any sort of issue - but I have never seen such entries appear before.
> 
> Any ideas as to what they are and why/when/how they are appearing in my 
> environment?
This is expected. Where were you when the shellshock crisis happened? Before
shellshock, bash would export functions by using something like this:

$ name='(){ echo foo; }' bash -c name
foo

But, this turned out to be a big issue, because of a bug in how bash parsed
that function definition, which allowed bad people to do nasty stuff, like:

USER_AGENT='(){ echo foo; }; cat /etc/passwd' ...

(bash was being used for CGI scripts, which pass around some HTTP headers
as environment variables)

This became a mess very quickly, with more and more parser bugs coming out
that would make that bug even more dangerous, so a solution was introduced
by RedHat variants (including Fedora, I guess), of prefixing these
function definitions with the special 'BASH_FUNC_'. This helped reduce the
attack surface. In the end, the official bash patch took a similar path, by
using that prefix, but also adding a %% suffix:

dualbus@hp:~$ f(){ echo foo; }; export -f f; env|grep %% -A1
BASH_FUNC_f%%=() {  echo foo
}


Wikipedia has an entry, if you want to read more:
http://en.wikipedia.org/wiki/Shellshock_%28software_bug%29

Re: Inconsistent behaviour of +=() and existing array keys

[Eduardo A . Bustamante López]

On Fri, Nov 28, 2014 at 03:47:49PM -0500, Maarten Billemont wrote:
> What I would expect is for += inside +=() to behave as = does now, and = to
> behave as it does outside of +=(), which is to "set" the value, not
> append.  Ergo:
> 
> declare -a indexed_array=( [0]=a )
> indexed_array=( [0]=b ) #=> [0]=b  -- because we unset the array and set
> the element so those given.
> indexed_array+=( [0]=b ) #=> [0]=b -- because we mutate the array by
> setting the element to that given.
> indexed_array=( [0]+=b ) #=> [0]=b -- because we unset the array and set
> the element by appending the given to nothing.
> indexed_array+=( [0]+=b ) #=> [0]=bb -- because we mutate the array by
> appending the given to what is already there.

Further expanding on this:

| dualbus@hp ~ % cat array 
| #!/bin/bash
| 
| for shell in \
| ksh93 \
| ~/local/bin/bash \
| /tmp/bash/bin/bash; do
| echo "$shell"
| "$shell" -c '
| typeset -A A
| A=( ["k"]=a )
|  echo "A=([\"k\"]=b)"
|  A=(["k"]=b);  typeset -p A
| A=( ["k"]=a )
| echo "A+=([\"k\"]=b)"
| A+=(["k"]=b);  typeset -p A
| A=( ["k"]=a )
|  echo "A=([\"k\"]+=b)"
|  A=(["k"]+=b); typeset -p A
| A=( ["k"]=a )
| echo "A+=([\"k\"]+=b)"
| A+=(["k"]+=b); typeset -p A
| '
| done


| dualbus@hp ~ % ./array   
| ksh93
| A=(["k"]=b)
| typeset -A A=([k]=b)
| A+=(["k"]=b)
| typeset -A A=([k]=b)
| A=(["k"]+=b)
| typeset -A A=([k]=b)
| A+=(["k"]+=b)
| typeset -A A=([k]=ab)
| /home/dualbus/local/bin/bash
| A=(["k"]=b)
| declare -A A='([k]="b" )'
| A+=(["k"]=b)
| declare -A A='([k]="ab" )'
| A=(["k"]+=b)
| declare -A A='([k]="b" )'
| A+=(["k"]+=b)
| declare -A A='([k]="ab" )'
| /tmp/bash/bin/bash
| A=(["k"]=b)
| declare -A A='([k]="b" )'
| A+=(["k"]=b)
| declare -A A='([k]="b" )'
| A=(["k"]+=b)
| declare -A A='([k]="b" )'
| A+=(["k"]+=b)
| declare -A A='([k]="ab" )'


~/local/bin/bash is bash from the devel branch. /tmp/bash/bin/bash is bash with
the attached patch applied. The attached patch makes bash behave like ksh93, at
least for the associative arrays case, because in the case of normal arrays,
ksh93 does some pretty funny things:

| dualbus@hp ~ % ksh93 -c 'a=(); a+=([0]=b); typeset -p a'
| typeset -A a=([0]=b) # Ha! Now 'a' is an associative array...
diff --git a/arrayfunc.c b/arrayfunc.c
index 804e6da..0f900aa 100644
--- a/arrayfunc.c
+++ b/arrayfunc.c
@@ -498,7 +498,7 @@ assign_compound_array_list (var, nlist, flags)
 
   for (list = nlist; list; list = list->next)
 {
-  iflags = flags;
+  iflags = (flags & ~ASS_APPEND);
   w = list->word->word;
 
   /* We have a word of the form [ind]=value */

Re: bash 4.3 compilation

[Eduardo A . Bustamante López]

On Mon, Dec 01, 2014 at 04:55:15PM +0530, Dhiraj Bhor wrote:
[...]
> *./builtins/libbuiltins.a(shopt.o): In function
> `shopt_enable_hostname_completion':shopt.c:(.text+0x945): undefined
> reference to `enable_hostname_completion'./lib/glob/libglob.a(gmisc.o): In
> function `glob_dirscan':gmisc.c:(.text+0x7df): undefined reference to
> `glob_patscan'collect2: ld returned 1 exit statusmake: *** [bash] Error
> 1make: Leaving directory `/root/bash-4.3.30'*
> 
> 
> Can someone point out what happened wrong here?
> When I compiled my previous bash-3.0, it compiled successfully.
Please see this:
https://lists.gnu.org/archive/html/bug-bash/2014-09/msg00267.html

Re: Bash crashes on detaching Screen

[Eduardo A . Bustamante López]

On Tue, Dec 02, 2014 at 11:29:59PM +0530, Darshit Shah wrote:
> On 12/02, Darshit Shah wrote:
[...]
> >When I attempt to detach a running Screen session using `screen -Drr
> >,session name` command, the original shell which contained the screen
> >session does not return to a prompt. It instead remains hung and a while
> >later I get the message:
> >
> >Warning: Program '/bin/bash' crashed

You're using screen's power detach (-D -r), from 'man screen':
|   -D -r   Reattach a session. If necessary detach and logout remotely 
first.

Notice the *logout* part.

If you check screen.c (from GNU screen's source code), you will find:
| 1896  *D_POWERSIG_POWER_BYE  power detach -- attacher kills his parent
| 1897  *D_REMOTE_POWER SIG_POWER_BYE  remote power detach -- both

Then, in attacher.c, function Attach(how):
 185   if (ret == SIG_POWER_BYE)
 186 {
 187   int ppid;
 188   if (setgid(real_gid) || setuid(real_uid))
 189 Panic(errno, "setuid/gid");
 190   if ((ppid = getppid()) > 1)
 191 Kill(ppid, SIGHUP); /* notice this part */
 192   exit(0);
 193 }

It sends a SIGHUP to the parent process, i.e. bash.


So, now that we know that, let's test it:

Shell 1:
dualbus@hp:~$ PS1='remote> '
remote> echo "$BASH_VERSION"
4.3.30(1)-maint
remote> screen -S bug -s ~/local/bin/bash
[... screen issues a clear screen... ]
dualbus@hp:~$

Shell 2:
dualbus@hp:~$ PS1='attacher> '
attacher> screen -Drr bug
[... gets the cleared screen from before ...]
dualbus@hp:~$

Shell 1:
[remote power detached from 3739.bug]
zsh: hangup ~/local/bin/bash
 ^ this tells us that bash was killed by a SIGHUP


Now, let's try again, ignoring SIGHUP:

Shell 1:
dualbus@hp:~$ PS1='$$ remote> '
3994 remote> trap '' HUP # ignore SIGHUP
3994 remote> screen -S bug -s ~/local/bin/bash
[.. again screen clears the terminal ..]
dualbus@hp:~$

Shell 2:
dualbus@hp:~$ PS1='$$ attacher> '
3954 attacher> screen -Drr bug
[.. we get the cleared screen ..]
dualbus@hp:~$

Shell 1:
[remote power detached from 4043.bug]
3994 remote> echo I am alive
I am alive


#1 shell survived the power detach, because we ignored the SIGHUP sent by
screen to its parent. Now... what does this mean?


I means that:
* It's not a bash bug. Bash is supposed to die from SIGHUP if you're not doing
  anything to handle it. In fact, if you're using that screen's feature, it's
  *supposed* to log out/kill its parent shell.
* It happens to other shells.


> I just recompiled my Bash with debugging symbols and attempted to run Bash
> under gdb. Here is the trace I received:
> 
> 
> [remote power detached from 19442.pts-1.mordor]
> 
> Program received signal SIGHUP, Hangup.
Yep, dies from SIGHUP. Don't use screen's power detach (-D -r) if you don't
want this to happen. I normally use (-d -r), which doesn't have this effect.

Re: bug with multiline strings parsing and single-quoted !

[Eduardo A . Bustamante López]

On Wed, Dec 03, 2014 at 08:57:26AM +0100, Piotr Grzybowski wrote:
> Hey,
> 
>  I think in this case, history_expand lib/readline/histexpand.c:905 is
> invoked with hstring="b\" | echo '!'", and goes wrongly into history
> expansion.
>  Not that I know how to fix it ;-)
Yes, the issue is tricky, because you don't have access to the whole buffer
there (shell_getc), just to the current line (shell_input_line), but the
current line is a fragment of the whole command.

What I don't understand is why bash does history expansion inside shell_getc,
shouldn't it be easier to do it once you have the whole thing?

zsh does that:

| dualbus@hp ~ % echo x
| x
| dualbus@hp ~ % echo "
| dualbus@hp ~ dquote> !!
| dualbus@hp ~ dquote> "
| echo "
| echo x
| "
| 
| echo x


Also tcsh:

| dualbus@hp ~ % tcsh
| hp:~> echo x
| x
| hp:~> echo "\
| ? !!\
| ? "
| echo "\
| echo x\
| "
| 
| echo x
| 
| hp:~> 

vs bash:

| dualbus@hp:~$ echo x
| x
| dualbus@hp:~$ echo "
| > !!
| echo "
| 
| echo 
| dualbus@hp:~$ 

It seems hard to change that, though.

Re: while loop read file problem

[Eduardo A . Bustamante López]

This is well know, and not a bug. Please read:
http://mywiki.wooledge.org/BashFAQ/024

when you run:

command | while-loop

the 'while-loop' part is executed in a subshell (actually, both commands are).
This means that they're no longer the same process as the main shell, and the
consequence is that they do not share variables.

There are several workarounds, all explained in the Bash FAQ from Wooledge.

Re: Memory continusely increase

[Eduardo A . Bustamante López]

On Mon, Dec 24, 2018 at 10:51:00AM +0800, chen liu wrote:
> Chet Ramey  于2018年12月22日周六 上午12:51写道：
(...)
> > What is the hard limit on the number of processes for a process started in
> > this environment? (The value of `ulimit -n'.)

Ah, interesting. Thanks for the pointer Chet.

(...)
> Below is all of the values of the bash ulimit optionals, this issue
> present in this environment
> --
> root@localhost:/root> cat ulimit.txt
(...)
> max user processes (-u) 516046
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
> ---
> But found change the -u optionals value through the ( ulimit -u 1024)
> will let this issue disapeared.

As you probably found out already, the changes between 4.3 and 4.4 that make
this issue "disappear" are related to how bash stores the exit status of
child processes.

The threads that prompted this change provide some insight:

- https://lists.gnu.org/archive/html/bug-bash/2015-04/msg00069.html
- http://lists.gnu.org/archive/html/bug-bash/2015-04/msg00075.html

And the note in the changelog:

http://git.savannah.gnu.org/cgit/bash.git/tree/CWRU/old-changelogs/CWRU.chlog.v15?h=ab8ded9c30b51a6bc0c8145553263da044b565ea#n9811

> jobs.[ch]
>   - bgpids: new implementation from a patch from John Fremlin
> , uses an array for the list of the last CHILD_MAX
> terminated background pids, and a separate hash table to search it.
> The storage can be freed as a unit, and the size of the hash table
> (currently 4096) is independent of the size of the bgpids table

http://git.savannah.gnu.org/cgit/bash.git/tree/CHANGES-4.4?h=ab8ded9c30b51a6bc0c8145553263da044b565ea#n220

> e.  There is a new implementation of the code that saves the last CHILD_MAX
> exited background pids so their status can be queried later.

Re: difference between /tmp and other directory for loadable mkdir?

[Eduardo A . Bustamante López]

On Wed, Dec 26, 2018 at 12:40:09PM -0600, Peng Yu wrote:
> Hi,
> 
> I can not mkdir -p . in /tmp/ via the loadable mkdir. What is the
> difference between /tmp/ and other directories? I am on Mac OS X. Is
> this a bug in mkdir?
> 
> $ cd /tmp
> $ mkdir -p -- .
> -bash: mkdir: .: Operation not permitted

I can reproduce this in Linux
[Linux debian 4.18.0-3-amd64 #1 SMP Debian 4.18.20-2 (2018-11-23) x86_64 
GNU/Linux]

dualbus@debian:~/src/gnu/bash$ strace -e trace=%file -f ./bash -c 'enable -f 
examples/loadables/mkdir mkdir && mkdir -p -- /tmp'
execve("./bash", ["./bash", "-c", "enable -f examples/loadables/mkd"...], 
0x7fff7c9b1c18 /* 33 vars */) = 0
(...)
stat("/home/dualbus/src/gnu/bash", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/home", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/home/dualbus", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/home/dualbus/src", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/home/dualbus/src/gnu", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/home/dualbus/src/gnu/bash", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/home/dualbus/src/iovisor/bcc", {st_mode=S_IFDIR|0755, st_size=4096, 
...}) = 0
openat(AT_FDCWD, "examples/loadables/mkdir", O_RDONLY|O_CLOEXEC) = 3
getcwd("/home/dualbus/src/gnu/bash", 128) = 27
stat("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=4096, ...}) = 0
chmod("/tmp", 0755) = -1 EPERM (Operation not permitted)
openat(AT_FDCWD, "/usr/share/locale/en_CA/LC_MESSAGES/libc.mo", O_RDONLY) = -1 
ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 
ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/local/share/locale/en_CA/LC_MESSAGES/bash.mo", O_RDONLY) 
= -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/local/share/locale/en/LC_MESSAGES/bash.mo", O_RDONLY) = 
-1 ENOENT (No such file or directory)
./bash: line 0: mkdir: /tmp: Operation not permitted
+++ exited with 1 +++

dualbus@debian:~/src/gnu/bash$ stat /tmp
  File: /tmp
  Size: 4096Blocks: 8  IO Block: 4096   directory
Device: fe01h/65025dInode: 25427969Links: 18
Access: (1777/drwxrwxrwt)  Uid: (0/root)   Gid: (0/root)
Access: 2018-12-26 11:08:30.062681076 -0800
Modify: 2018-12-26 11:37:56.826984422 -0800
Change: 2018-12-26 11:37:56.826984422 -0800
 Birth: -


If you look at the source, it calls the mkdir() function to create the 
directory.

dualbus@debian:~/src/gnu/bash$ cat -n examples/loadables/mkdir.c | sed -n 
'174,180p'
   174if (stat (npath, &sb) != 0)
   175  {
   176if (mkdir (npath, parent_mode))  <- this
   177  {
   178builtin_error ("cannot create directory `%s': %s", npath, 
strerror (errno));
   179umask (original_umask);
   180free (npath);

Ref: 
http://git.savannah.gnu.org/cgit/bash.git/tree/examples/loadables/mkdir.c?h=6870125961b85baa9628ce4c5fbbca94e8046656#n174


Comparing to GNU coreutils' mkdir (which uses gnulib):

dualbus@debian:~$ strace -e trace=%file -f mkdir -p /tmp/
execve("/bin/mkdir", ["mkdir", "-p", "/tmp/"], 0x7fff4f7a11b8 /* 33 vars */) = 0
(...)
mkdir("/tmp/", 0777)= -1 EEXIST (File exists)
stat("/tmp/", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=4096, ...}) = 0
+++ exited with 0 +++


My guess is that the "problem" has to do with how bash computes the 
`parent_mode':

dualbus@debian:~/src/gnu/bash$ cat -n examples/loadables/mkdir.c | sed -n 
'106,114p'
   106/* Make the new mode */
   107original_umask = umask (0);
   108umask (original_umask);
   109  
   110nmode = (S_IRWXU | S_IRWXG | S_IRWXO) & ~original_umask;
   111parent_mode = nmode | (S_IWUSR|S_IXUSR);  /* u+wx */
   112  
   113/* Adjust new mode based on mode argument */
   114nmode &= omode;

Ref: 
http://git.savannah.gnu.org/cgit/bash.git/tree/examples/loadables/mkdir.c?h=6870125961b85baa9628ce4c5fbbca94e8046656#n106


In gnulib, this is what is used to create the parent directories:

Ref: 
http://git.savannah.gnu.org/cgit/gnulib.git/tree/lib/mkdir-p.c?h=95c96b631f3676f72ed044d0c493ab5642d8#n116

Re: "COMMAND 2>&1 > PATH" doesn't work

[Eduardo A . Bustamante López]

On Sun, Dec 30, 2018 at 10:10:42PM +0100, Dušan Kreheľ wrote:
> Hello.
> 
> If I try in bash this command ex. "rmdir somethingA > somethingA.out
> 2>&1" work right. But, if I try "rmdir somethingB 2>&1 >
> somethingB.out" that work wrong.
> 
> That work's wrong in Bash version 4.4.23(1) and too in 5.0.0(1)-rc1:

Why do you expect these two to behave in the same way? They are clearly
distinct operations.


 COMMAND >FILE 2>&1

is not the same as:

 COMMAND 2>&1 >FILE

The order of redirections matter, they are processed left-to-right. Let's look
at these examples in more detail.


Example 1.

 COMMAND >FILE 2>&1

Let's break it down into tokens and see what happens at each step.

a) COMMAND

file descriptor 0 -> terminal
file descriptor 1 -> terminal
file descriptor 2 -> terminal

This is the "normal" state in an interactive shell, the three standard file
descriptors (input: 0, output: 1, and error: 2) are pointing to the terminal
device.

b) >FILE

file descriptor 0 -> terminal
file descriptor 1 -> FILE
file descriptor 2 -> terminal

The `>FILE' redirection is processed, causing file descriptor 1 (standard
output) to be directed to the file named `FILE'.

c) 2>&1

file descriptor 0 -> terminal
file descriptor 1 -> FILE
file descriptor 2 -> FILE

The `2>&1' redirection is processed, causing file descriptor 2 (the `2>' bit) to
be directed to "wherever-fd1-points-to" (i.e. the `&1' bit). FD1 currently
points to `FILE', so FD2 is updated to point to `FILE' too.


VS

Example 2.

 COMMAND 2>&1 FILE 

a) COMMAND

file descriptor 0 -> terminal
file descriptor 1 -> terminal
file descriptor 2 -> terminal

same as example 1.

b) 2>&1

file descriptor 0 -> terminal
file descriptor 1 -> terminal
file descriptor 2 -> terminal * no change

The `2>&1' redirection is processed, causing file descriptor 2 to be directed to
"wherever-fd1-points-to". FD1 currently points to the terminal. FD2 points to
the terminal too, so it stays the same.

c) >FILE

file descriptor 0 -> terminal
file descriptor 1 -> FILE
file descriptor 2 -> terminal

The `>FILE' redirection is processed, causing file descriptor 1 (standard
output) to be directed to the file named `FILE'.



So, this is not a bug. It's the expected behavior of redirections. If you want
to read more about it, I recommend:

- https://mywiki.wooledge.org/BashFAQ/055
- 
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07

[PATCH 2/3] Fix `hashtest' target in Makefile

[Eduardo A . Bustamante López]

- Adds the dependency on the Bash's libmalloc
- Removes the libintl dependency, since it's unused
- Adds the `running_trap' global variable, since libmalloc depends on
  it when the `SHELL' macro is defined (see [1])
- Removes compiler warning due to missing return type in main function

After these changes, the `hashtest' target can be built with:

| dualbus@system76-pc:~/src/gnu/bash$ CC=gcc ./configure --silent 
--with-bash-malloc && make -j$(nproc) -s hashtest
|
| Beginning configuration for bash-5.0-rc1 for x86_64-pc-linux-gnu
|
| config.status: creating po/POTFILES
| config.status: creating po/Makefile
| ./parse.y: warning: 1 shift/reduce conflict [-Wconflicts-sr]
|
|   ***
|   * *
|   * GNU bash, version 5.0.0(1)-rc1 (x86_64-pc-linux-gnu)
|   * *
|   ***
|
| make[1]: warning: -j16 forced in submake: resetting jobserver mode.
|
| dualbus@system76-pc:~/src/gnu/bash$ echo hi | ./hashtest >/dev/null 2>&1; 
echo $?
| 0

It doesn't work if `--without-bash-malloc' is specified.

[1]

/usr/bin/ld: ./lib/malloc/libmalloc.a(malloc.o): in function `morecore':
/home/dualbus/src/gnu/bash/lib/malloc/malloc.c:602: undefined reference to 
`running_trap'
---
 Makefile.in | 4 ++--
 hashlib.c   | 2 ++
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/Makefile.in b/Makefile.in
index 5fcb44b0..76a51b19 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -967,8 +967,8 @@ depends: force
$(Program) $(SUPPORT_SRC)mkdep -c ${CC} -- ${CCFLAGS} ${CSOURCES}
 
  PRIVATE TARGETS 
-hashtest:  hashlib.c
-   $(CC) -DTEST_HASHING $(CCFLAGS) $(TEST_NBUCKETS) -o $@ 
$(srcdir)/hashlib.c xmalloc.o $(INTL_LIB)
+hashtest:  hashlib.c xmalloc.o $(MALLOC_LIBRARY)
+   $(CC) -DTEST_HASHING $(CCFLAGS) $(TEST_NBUCKETS) -o $@ 
$(srcdir)/hashlib.c xmalloc.o $(MALLOC_LIBRARY)
 
  DEPENDENCIES ###
 
diff --git a/hashlib.c b/hashlib.c
index 8adbe221..f8e3b09a 100644
--- a/hashlib.c
+++ b/hashlib.c
@@ -392,6 +392,7 @@ hash_pstats (table, name)
 HASH_TABLE *table, *ntable;
 
 int interrupt_immediately = 0;
+int running_trap = 0;
 
 int
 signal_is_trapped (s)
@@ -417,6 +418,7 @@ internal_warning (const char *format, ...)
 {
 }
 
+int
 main ()
 {
   char string[256];
-- 
2.20.1

[PATCH 1/3] Fix implicit declaration of abort()

[Eduardo A . Bustamante López]

gcc version 8.2.0 (Debian 8.2.0-13) x86_64-linux-gnu

```
malloc.c:333:3: warning: incompatible implicit declaration of built-in function 
‘abort’
malloc.c:333:3: note: include ‘’ or provide a declaration of ‘abort’
```
---
 lib/malloc/malloc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/malloc/malloc.c b/lib/malloc/malloc.c
index 5621adf4..8f641036 100644
--- a/lib/malloc/malloc.c
+++ b/lib/malloc/malloc.c
@@ -83,6 +83,7 @@
 #endif
 #include 
 #include 
+#include 
 
 #if defined (HAVE_MMAP)
 #include 
-- 
2.20.1

[PATCH 3/3] Fix incompatible pointer type warning in unicode.c

[Eduardo A . Bustamante López]

The warning is raised by Clang (7.0.1-4) when sizeof(wchar_t) is 4

| dualbus@system76-pc:~/src/gnu/bash/lib/sh$ make unicode.o
| clang -c   -I. -I../.. -I../.. -I../../lib -I../../include -I.  
-DHAVE_CONFIG_H -DSHELL  -ggdb -O0 -Wno-parentheses -Wno-format-security   
unicode.c
| unicode.c:262:69: warning: incompatible pointer types passing 'wchar_t [3]' 
to parameter of type 'unsigned short *' [-Wincompatible-pointer-types]
|   else if (sizeof (wchar_t) == 2 && c <= 0x10 && u32toutf16 (c, ws))
| ^~
| 1 warning generated.

dualbus@system76-pc:~/src/gnu/bash$ clang -v 2>&1 | head -n2
clang version 7.0.1-4 (tags/RELEASE_701/final)
Target: x86_64-pc-linux-gnu

In practice, this isn't really a problem because the compiler should optimize
away the `sizeof(wchar_t) == 2' branch. Still, it's easy to fix.

I think that maybe there should be some sort of compile-time assertion inside
`u32toutf16`, to ensure it's only used when `sizeof(wchar_t) == 2', but I don't
know how to do that.
---
 lib/sh/unicode.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/sh/unicode.c b/lib/sh/unicode.c
index fe13c4a0..99c422ab 100644
--- a/lib/sh/unicode.c
+++ b/lib/sh/unicode.c
@@ -216,21 +216,21 @@ u32toutf8 (wc, s)
 int
 u32toutf16 (c, s)
  u_bits32_t c;
- unsigned short *s;
+ wchar_t *s;
 {
   int l;
 
   l = 0;
   if (c < 0x0d800 || (c >= 0x0e000 && c <= 0x0))
 {
-  s[0] = (unsigned short) (c & 0x);
+  s[0] = (wchar_t) (c & 0x);
   l = 1;
 }
   else if (c >= 0x1 && c <= 0x010)
 {
   c -= 0x01;
-  s[0] = (unsigned short)((c >> 10) + 0xd800);
-  s[1] = (unsigned short)((c & 0x3ff) + 0xdc00);
+  s[0] = (wchar_t)((c >> 10) + 0xd800);
+  s[1] = (wchar_t)((c & 0x3ff) + 0xdc00);
   l = 2;
 }
   s[l] = 0;
-- 
2.20.1

Re: Identical function names in bash source code

[Eduardo A . Bustamante López]

On Sat, Jan 05, 2019 at 08:19:38AM -0600, Peng Yu wrote:
> Hi,
> 
> It is not uncommon to see the same name is used to defined functions
> in different .c files in bash source code.
> 
> For example, sh_single_quote is defined in both lib/readline/shell.c
> and lib/sh/shquote.c with the exact same signature. The two pieces of
> code are slightly different. Do they do the exact same things or do
> something different?
> 
> In either case, is having the same name for different functions a good
> practice? This will make the linked binary dependent on the order of
> the corresponding .a files specified. Or if linked via .o files, then
> one function will shadow the others. See 1) and 2) below for minimal
> working examples. Neither cases seem to be good and could be avoided
> easily by giving the functions unique names.
> 
> So, should such functions with the same name be named differently? Thanks.

This mailing list (bug-bash) is for reporting BUGS. If you must, at least please
use help-bash for generic questions?

Although you should probably be asking this to a C programming forum...

Segmentation fault in lib/readline/text.c rl_change_case

[Eduardo A . Bustamante López]

I found the issue with AFL (http://lcamtuf.coredump.cx/afl/).

The crash itself happens due to the following:

dualbus@system76-pc:~/src/gnu/bash$ cat -n lib/readline/text.c | sed -n 
'1455,1460p'
  1455mlen = wcrtomb (mb, nwc, &mps);   // <- mlen is -1, due 
to `nwc' being an invalid character.
  1456if (mlen > 0)
  1457  mb[mlen] = '\0';
  1458/* what to do if m != mlen? adjust below */
  1459/* m == length of old char, mlen == length of new char */
  1460s = rl_line_buffer + start;

dualbus@system76-pc:~/src/gnu/bash$ cat -n lib/readline/text.c | sed -n 
'1475,1480p'
  1475rl_extend_line_buffer (mlen - m + 1);
  1476memmove (s + mlen, s + m, (e - s) - m);
  1477memcpy (s, mb, mlen); // <- mlen is negative here, 
thus it's an invalid `memcpy'
  1478next += mlen - m; /* next char changes */
  1479end += mlen - m;  /* end of word changes */
  1480rl_end += mlen - m;   /* end of line changes */

I'm not really sure how to fix this problem. I worked around it by changing the
condition in L1456 to just return early from the function.

The file I used as input:

debian@debian-fuzz:/mnt$ xxd rl_change_case
: f2b1 a8b1 011b 55..U

debian@debian-fuzz:/mnt$ base64 < rl_change_case
8rGosQEbVQ==

debian@debian-fuzz:/mnt$ cat -A rl_change_case
M-rM-1M-(M-1^A^[U

I patched bash to remove the isatty() check for `read -e', and ran (with: 
LC_ALL=zh_CN.gbk):

(...)
(gdb) r
Starting program: /home/debian/build-gdb/bash --noprofile --norc -c PATH=\ 
read\ -e\ \<\ rl_change_case
hi
򱨱
Program received signal SIGSEGV, Segmentation fault.
__memmove_avx_unaligned_erms () at 
../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:490
490 ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file 
or directory.

(gdb) bt
#0  __memmove_avx_unaligned_erms () at 
../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:490
#1  0x5565b668 in rl_change_case (count=1, op=1) at 
../../../bash-5.0-rc1/lib/readline/text.c:1477
#2  0x5565b201 in rl_upcase_word (count=1, key=117) at 
../../../bash-5.0-rc1/lib/readline/text.c:1358
#3  0x55639e9a in _rl_dispatch_subseq (key=117, map=0x556ac220 
, got_subseq=0) at 
../../../bash-5.0-rc1/lib/readline/readline.c:852
#4  0x55639c11 in _rl_dispatch (key=1433811945, map=0x556ac220 
) at ../../../bash-5.0-rc1/lib/readline/readline.c:798
#5  0x55639dd7 in _rl_dispatch_subseq (key=85, map=0x556ac220 
, got_subseq=0) at 
../../../bash-5.0-rc1/lib/readline/readline.c:841
#6  0x5563a35f in _rl_dispatch_subseq (key=27, map=0x556ab200 
, got_subseq=0) at 
../../../bash-5.0-rc1/lib/readline/readline.c:986
#7  0x55639c11 in _rl_dispatch (key=1433811945, map=0x556ab200 
) at ../../../bash-5.0-rc1/lib/readline/readline.c:798
#8  0x55639894 in readline_internal_char () at 
../../../bash-5.0-rc1/lib/readline/readline.c:632
#9  0x556398ef in readline_internal_charloop () at 
../../../bash-5.0-rc1/lib/readline/readline.c:659
#10 0x5563990f in readline_internal () at 
../../../bash-5.0-rc1/lib/readline/readline.c:671
#11 0x5563932d in readline (prompt=0x55680f84 "") at 
../../../bash-5.0-rc1/lib/readline/readline.c:377
#12 0x55611b95 in edit_line (p=0x55680f84 "", itext=0x0) at 
../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:1107
#13 0x556108be in read_builtin (list=0x0) at 
../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:566
#14 0x555a5afa in execute_builtin (builtin=0x5560fa39 
, words=0x55761ea8, flags=0, subshell=0) at 
../bash-5.0-rc1/execute_cmd.c:4706
#15 0x555a6aa2 in execute_builtin_or_function (words=0x55761ea8, 
builtin=0x5560fa39 , var=0x0, redirects=0x55761c08, 
fds_to_close=0x55761be8, flags=0)
at ../bash-5.0-rc1/execute_cmd.c:5214
#16 0x555a5365 in execute_simple_command 
(simple_command=0x55761ac8, pipe_in=-1, pipe_out=-1, async=0, 
fds_to_close=0x55761be8) at ../bash-5.0-rc1/execute_cmd.c:4476
#17 0x5559e9f4 in execute_command_internal (command=0x55761a88, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x55761be8) at 
../bash-5.0-rc1/execute_cmd.c:842
#18 0x55608550 in parse_and_execute (string=0x557616c8 "PATH= read 
-e < rl_change_case", from_file=0x556690f0 "-c", flags=4) at 
../../bash-5.0-rc1/builtins/evalstring.c:436
#19 0x5558564a in run_one_command (command=0x7fffe280 "PATH= read 
-e < rl_change_case") at ../bash-5.0-rc1/shell.c:1426
#20 0x55584789 in main (argc=5, argv=0x7fffdfd8, 
env=0x7fffe008) at ../bash-5.0-rc1/shell.c:741

(gdb) frame 1
#1  0x5565b668 in rl_change_case (count=1, op=1) at 
../../../bash-5.0-rc1/lib

Recursive execution of expand_prompt_string

[Eduardo A . Bustamante López]

The expand_prompt_string function will call itself recursively in some
situations:

dualbus@system76-pc:~$ bash -c 'q="\${q@P}"; echo "${q@P}"'
Segmentation fault

My guess is that this is expected behavior. I'm reporting this because it's
affecting my fuzzing effort.

realloc: start and end chunk sizes differ - rl_extend_line_buffer in lib/readline/util.c

[Eduardo A . Bustamante López]

Found by fuzzing with AFL

debian@debian-fuzz:/mnt$ cat -A rl_extend_line_buffer 
00^[^?000^?^X^E^_^Y^Y^Y^Y^Y^Y^Y^Y

debian@debian-fuzz:/mnt$ base64 < rl_extend_line_buffer 
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwG38wMDAwMDAwMDAwMDAwMDB/GAUfGRkZMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAZGRkZGQ==

(gdb) r
Starting program: /home/debian/build-gdb/bash --noprofile --norc -c PATH=\ 
read\ -e\ \<\ rl_extend_line_buffer
hi
00
[Detaching after fork from child process 21638]
/home/debian/build-gdb/bash: emacs: No such file or directory
^@^@0^@0^@0^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@0^@0^@0^@0
malloc: unknown:0: assertion botched
malloc: 0x55769408: allocated: last allocated from unknown:0
realloc: start and end chunk sizes differ
Aborting...
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.

(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x77df4535 in __GI_abort () at abort.c:79
#2  0x555b39b5 in programming_error (format=0x55686c98 "realloc: 
start and end chunk sizes differ") at ../bash-5.0-rc1/error.c:175
#3  0x556651dc in xbotch (mem=0x55769408, e=8, s=0x55686c98 
"realloc: start and end chunk sizes differ", file=0x0, line=0) at 
../../../bash-5.0-rc1/lib/malloc/malloc.c:354
#4  0x55666993 in internal_realloc (mem=0x55769408, n=512, 
file=0x0, line=0, flags=0) at ../../../bash-5.0-rc1/lib/malloc/malloc.c:1091
#5  0x556670c8 in realloc (mem=0x55769408, nbytes=512) at 
../../../bash-5.0-rc1/lib/malloc/malloc.c:1381
#6  0x555fff94 in xrealloc (pointer=0x55769408, bytes=512) at 
../bash-5.0-rc1/xmalloc.c:135
#7  0x556547d8 in rl_extend_line_buffer (len=273) at 
../../../bash-5.0-rc1/lib/readline/util.c:169
#8  0x556594a5 in rl_insert_text (string=0x5576d888 '0' ) at ../../../bash-5.0-rc1/lib/readline/text.c:95
#9  0x55655a81 in rl_yank (count=1, key=25) at 
../../../bash-5.0-rc1/lib/readline/kill.c:494
#10 0x55639ed4 in _rl_dispatch_subseq (key=25, map=0x556ab200 
, got_subseq=0) at 
../../../bash-5.0-rc1/lib/readline/readline.c:852
#11 0x55639c4b in _rl_dispatch (key=-136275877, map=0x556ab200 
) at ../../../bash-5.0-rc1/lib/readline/readline.c:798
#12 0x556398ce in readline_internal_char () at 
../../../bash-5.0-rc1/lib/readline/readline.c:632
#13 0x55639929 in readline_internal_charloop () at 
../../../bash-5.0-rc1/lib/readline/readline.c:659
#14 0x55639949 in readline_internal () at 
../../../bash-5.0-rc1/lib/readline/readline.c:671
#15 0x55639367 in readline (prompt=0x55680f84 "") at 
../../../bash-5.0-rc1/lib/readline/readline.c:377
#16 0x55611bcf in edit_line (p=0x55680f84 "", itext=0x0) at 
../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:1107
#17 0x556108f8 in read_builtin (list=0x0) at 
../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:566
#18 0x555a5afa in execute_builtin (builtin=0x5560fa73 
, words=0x55764dc8, flags=0, subshell=0) at 
../bash-5.0-rc1/execute_cmd.c:4706
#19 0x555a6aa2 in execute_builtin_or_function (words=0x55764dc8, 
builtin=0x5560fa73 , var=0x0, redirects=0x55764b48, 
fds_to_close=0x55764a48, flags=0)
at ../bash-5.0-rc1/execute_cmd.c:5214
#20 0x555a5365 in execute_simple_command 
(simple_command=0x557648c8, pipe_in=-1, pipe_out=-1, async=0, 
fds_to_close=0x55764a48) at ../bash-5.0-rc1/execute_cmd.c:4476
#21 0x5559e9f4 in execute_command_internal (command=0x55764888, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x55764a48) at 
../bash-5.0-rc1/execute_cmd.c:842
#22 0x5560858a in parse_and_execute (string=0x55764008 "PATH= read 
-e < rl_extend_line_buffer", from_file=0x556690f0 "-c", flags=4)
at ../../bash-5.0-rc1/builtins/evalstring.c:436
#23 0x5558564a in run_one_command (command=0x7fffe294 "PATH= read 
-e < rl_extend_line_buffer") at ../bash-5.0-rc1/shell.c:1426
#24 0x55584789 in main (argc=5, argv=0x7fffdff8, 
env=0x7fffe028) at ../bash-5.0-rc1/shell.c:741

Re: realloc: start and end chunk sizes differ - rl_extend_line_buffer in lib/readline/util.c

[Eduardo A . Bustamante López]

On Sun, Jan 06, 2019 at 07:18:27PM -0800, Eduardo A. Bustamante López wrote:
(...)
> malloc: unknown:0: assertion botched
> malloc: 0x55769408: allocated: last allocated from unknown:0
> realloc: start and end chunk sizes differ

OK, I think I know what the problem is.

I noticed that `rl_insert_text' writes past the end of the rl_line_buffer 
buffer:

| Hardware watchpoint 1: rl_line_buffer[255]
| 
| Old value = -33 '\337'
| New value = 48 '0'
| __strncpy_ssse3 () at ../sysdeps/x86_64/multiarch/strcpy-ssse3.S:2399
| 2399../sysdeps/x86_64/multiarch/strcpy-ssse3.S: No such file or directory.
| 
| (gdb) bt
| #0  __strncpy_ssse3 () at ../sysdeps/x86_64/multiarch/strcpy-ssse3.S:2399
| #1  0x55659522 in rl_insert_text (string=0x5576bc88 '0' ) at text.c:99
| #2  0x55655a9f in rl_yank (count=1, key=25) at kill.c:494
| #3  0x55639ef2 in _rl_dispatch_subseq (key=25, map=0x556aa200 
, got_subseq=0) at readline.c:852
| #4  0x55639c69 in _rl_dispatch (key=1433844896, map=0x556aa200 
) at readline.c:798
| #5  0x556398ec in readline_internal_char () at readline.c:632
| #6  0x55639947 in readline_internal_charloop () at readline.c:659
| #7  0x55639967 in readline_internal () at readline.c:671
| #8  0x55639385 in readline (prompt=0x556809dc "") at 
readline.c:377
| #9  0x55611bdd in edit_line (p=0x556809dc "", itext=0x0) at 
./read.def:1107
| #10 0x55610906 in read_builtin (list=0x0) at ./read.def:566
| #11 0x555a5b3f in execute_builtin (builtin=0x5560fa81 
, words=0x55762d08, flags=0, subshell=0) at execute_cmd.c:4706
| #12 0x555a6ae7 in execute_builtin_or_function (words=0x55762d08, 
builtin=0x5560fa81 , var=0x0, redirects=0x557629c8, 
fds_to_close=0x55762a08, flags=0)
| at execute_cmd.c:5214
| #13 0x555a53aa in execute_simple_command 
(simple_command=0x55762948, pipe_in=-1, pipe_out=-1, async=0, 
fds_to_close=0x55762a08) at execute_cmd.c:4476
| #14 0x5559ea39 in execute_command_internal (command=0x55762908, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x55762a08) at 
execute_cmd.c:842
| #15 0x55608586 in parse_and_execute (string=0x5574b268 "read -e < 
~/xxx", from_file=0x556690f0 "-c", flags=4) at evalstring.c:436
| #16 0x55585632 in run_one_command (command=0x7fffec14 "read -e < 
~/xxx") at shell.c:1416
| #17 0x5558477d in main (argc=3, argv=0x7fffe948, 
env=0x7fffe968) at shell.c:735
| 
| (gdb) frame 1
| #1  0x55659522 in rl_insert_text (string=0x5576bc88 '0' ) at text.c:99
| 99strncpy (rl_line_buffer + rl_point, string, l);
| 
| (gdb) l
| 94if (rl_end + l >= rl_line_buffer_len)
| 95  rl_extend_line_buffer (rl_end + l);
| 96
| 97for (i = rl_end; i >= rl_point; i--)
| 98  rl_line_buffer[i + l] = rl_line_buffer[i];
| 99strncpy (rl_line_buffer + rl_point, string, l);
| 100
| 101   /* Remember how to undo this if we aren't undoing something. */
| 102   if (_rl_doing_an_undo == 0)
| 103 {
| 
| (gdb) p rl_end
| $1 = 213
| (gdb) p l
| $2 = 30
| (gdb) p rl_end + l
| $3 = 243
| (gdb) p string
| $4 = 0x5576bc88 '0' 
| (gdb) p strlen(string)
| $5 = 30
| (gdb) p rl_point
| $6 = 227

So somehow rl_point ended up with an invalid value. Tracing that problem:

| Hardware watchpoint 5: (rl_point > 0 && rl_point > rl_end + 1)
| 
| Old value = 0
| New value = 1
| rl_do_undo () at undo.c:199
| 199   rl_insert_text (rl_undo_list->text);
| (gdb) p rl_end
| $12 = 0
| (gdb) p rl_point
| $13 = 14
| (gdb) bt
| #0  rl_do_undo () at undo.c:199
| #1  0x5565685c in rl_undo_command (count=1, key=31) at undo.c:356
| #2  0x55639ef2 in _rl_dispatch_subseq (key=31, map=0x556aa200 
, got_subseq=0) at readline.c:852
| #3  0x55639c69 in _rl_dispatch (key=1433812680, map=0x556aa200 
) at readline.c:798
| #4  0x556398ec in readline_internal_char () at readline.c:632
| #5  0x55639947 in readline_internal_charloop () at readline.c:659
| #6  0x55639967 in readline_internal () at readline.c:671
| #7  0x55639385 in readline (prompt=0x556809dc "") at 
readline.c:377
| #8  0x55611bdd in edit_line (p=0x556809dc "", itext=0x0) at 
./read.def:1107
| #9  0x55610906 in read_builtin (list=0x0) at ./read.def:566
| #10 0x555a5b3f in execute_builtin (builtin=0x5560fa81 
, words=0x55762d08, flags=0, subshell=0) at execute_cmd.c:4706
| #11 0x555a6ae7 in execute_builtin_or_function (words=0x55762d08, 
builtin=0x5560fa81 , var=0x0, redirects=0x557629c8, 
fds_to_close=0x55762a08, flags=0)
| at execute_cmd.c:5214
| #12 0x555a53aa in execute_simple_command 
(simple_command=0x5576

Re: $RANDOM not Cryptographically secure pseudorandom number generator

[Eduardo A . Bustamante López]

On Mon, Jan 07, 2019 at 08:15:12AM +0100, Ole Tange wrote:
> On Mon, Jan 7, 2019 at 12:08 AM Chet Ramey  wrote:
> >
> > On 1/5/19 3:12 PM, Eduardo A. Bustamante López wrote:
> > > On Fri, Dec 28, 2018 at 10:24:50AM +0100, Ole Tange wrote:
> > > (...)
> > >> Patch attached.
> :
> > > - Does the new RNG generate uniformly distributed numbers? (Yes)
> > > - What is the performance impact (roughly 2X slower)
> 
> The impact is not from Salsa20/ChaCha20, though, if you compare
> patched vs. unpatched code.
> 
> On my system BASH_RANDOM_16 is a tiny bit faster than the original,
> while BASH_RANDOM_32 is a tiny bit slower.

I find that really hard to believe. The new RNG is clearly doing more work, so
it should be slower. And that's OK. I just wanted to know by how much.

What numbers are you seeing and how did you measure them?

Segmentation fault in lib/readline/undo.c - rl_do_undo

[Eduardo A . Bustamante López]

I found this with AFL. I think it's related to the problem reported here:
http://lists.nongnu.org/archive/html/bug-bash/2018-09/msg00045.html

debian@debian-fuzz:/mnt$ cat -A rl_do_undo
^RM-CM-!M-CM-CM-!M-C^[.^[^[0^P^@^P^Q0^[-^P^Q0^[^W0^@0&/^[^[^[--^W^_~0^@0^@-^L^D^@^@'/^[B^@0^B^@M-
 
^[^[M-^T^[M-mM-^?^[F-^W^_0^[M-^@0^P^@^@^@^@^D^I^@^[M-UM-UM-UM-NM-U^@M-^@^@M-=$^@01^@01^["0^?M-^?M-^?M-^?0M-r0^@'0M-^?^@^@^@M-CM-CM-!M-C^[.^[^[--^W00^P^@00(-^P^Q;^[-^P^Q0^[^W0^@n&/^[^[^[--^W^_~0^@0^@-^L^D^@^@'/^[B^@M-^T^B^@M-
 
^[^[M-^T^[M-mM-^?^[F-^W^_0^[M-^@0^P^@^@^@^@^D^I^@^[M-UM-UM-UM-NM-U^@M-^@^@M-=$^@01^@01^["0^?M-^?M-^?M-^?0M-r0^@'0M-^?^@^@^@@^N\0^[11#0-^P^@^@^@^@^D^I^@^[M-UM-=M-UM-NM-U^@M-^@^@M-=$^@J^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^PM-eM-KM-YM-@M-nM-n^ZM-{0M-xM-|^@}}^L0#^A^Cd^\#^@^X^E^X^E^@M-^?M-^?^[^I^I^@^@^@M-^?\^O^@000M-^R00M-,0^@^@M-^?\^O^@qq0M-^Dq^@0^P^I^[^I^I^E^@M-^?M-^?0M-v^P^P^P^P^P^P^PM-eM-KM-YM-@M-nM-n^ZM-{0M-xM-|^@}}^L0#^A^Cd^\#^@^X^E^X^E^@M-^?M-^?^[^I^@^@M-^@^@0^@^@^@M-^?\^O^@0^?0M-^R00M-,0^@^@M-^?\^O^@0^@0^P^I^[^I^Iu000^E^@M-^?M-^?0M-vM-Q^A^@0^P^I^]0^I00^@^@^@M-^?\^O^GM-^?\^Oq0q^[^I^I^I^@^@M-h^C@^N\0^[11#0-^P^@^@^@^@^D^I^@^[M-UM-UM-UM-NM-U^@M-^@^@M-=$^@0^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^PM-eM-KM-YM-@M-nM-n^ZM-{0M-xM-|^@00^L0#^A^Cd^\#^@^X^E^X^E^@M-^?M-^?^[^I^I^@^@^@M-^?\^O^@000M-^R00M-,0^@^@M-^?\^O^@qq0M-^Dq^@0^P^I^[^I^Iu000^E^@M-^?M-^?0M-v^P^P^P^P^P^P^PM-eM-KM-YM-@M-nM-n^ZM-{0M-xM-|^@}}^L0#^A^Cd^\#^@^X^E^X^E^@M-^?M-^?^[^I^I^@^@^@M-^?\^O^@00M-^?M-^R00M-,0^@^@M-^?\^O^@0^@0^P^I^[^I^I^E^@M-^?M-^?0M-vM-Q^A^@0^P^I^]0^I00^@^@^@M-^?\^O^GM-^?\^Oq0q^[^I^I^I^@^@M-h^C^E^@000M-v

debian@debian-fuzz:/mnt$ base64 < rl_do_undo
EsOhw8OhwxsuGxswMDAwMBAAMDAwMBARMBstEBEwGxcwADAmLxsbGy0tFx9+MAAwAC0MBAAAJy8b
QgAwAgCgGxuUG+3/G0YtFx8wG4AwEAAECQAb1dXVztUAgAC9JAAwMQAwMRsiMH8w8jAA
JzD/w8OhwxsuGxstLRcwMBAAMDAoLRAROxstEBEwGxcwAG4mLxsbGy0tFx9+MAAwAC0MBAAA
Jy8bQgCUAgCgGxuUG+3/G0YtFx8wG4AwEAAECQAb1dXVztUAgAC9JAAwMQAwMRsiMH8w
8jAAJzD/QA5cMBsxMSMwLRAABAkAG9W91c7VAIAAvSQAShAQEBAQEBAQEBAQEBAQEBAQ
EBAQ5cvZwO7uGvsw+PwAfX0MMCMBA2QcIwAYBRgFAP//GwkJMDAwMP9cDwAwMDCSMDCsMAAA
/1wPAHFxMIRxADAQCRsJCTAwMDAFAP//MPYQEBAQEBAQ5cvZwO7uGvsw+PwAfX0MMCMBA2QcIwAY
BRgFAP//GwkAAIAAMP9cDwAwfzCSMDCsMAAA/1wPADAwMDAwADAQCRsJCXUwMDAFAP//MPbR
AQAwEAkdMAkwMDAwMDD/XA8H/1wPcTBxGwkJCQAA6ANADlwwGzExIzAtEAAECQAb1dXV
ztUAgAC9JAAwEBAQEBAQEBAQEBAQEBAQEBAQEBDly9nA7u4a+zD4/AAwMAwwIwEDZBwjABgFGAUA
//8bCQkwMDAw/1wPADAwMJIwMKwwAAD/XA8AcXEwhHEAMBAJGwkJdTAwMAUA//8w9hAQEBAQ
EBDly9nA7u4a+zD4/AB9fQwwIwEDZBwjABgFGAUA//8bCQkwMDAw/1wPADAw/5IwMKwwAAD/
XA8AMDAwMDAAMBAJGwkJMDAwMAUA//8w9tEBADAQCR0wCTAwMDAwMP9cDwf/XA9xMHEbCQkJ
AADoAwUAMDAw9g==

debian@debian-fuzz:/mnt$ LC_ALL=zh_CN.gbk  ~/build/bash --noprofile --norc -c 
'PATH= read -e < rl_do_undo' >/dev/null 2>&1; echo $?
Segmentation fault
139


And the backtrace:

(gdb) bt
#0  0x55656672 in rl_do_undo () at 
../../../bash-5.0-rc1/lib/readline/undo.c:255
#1  0x55656807 in rl_revert_line (count=1, key=0) at 
../../../bash-5.0-rc1/lib/readline/undo.c:339
#2  0x5563956b in readline_internal_teardown (eof=0) at 
../../../bash-5.0-rc1/lib/readline/readline.c:471
#3  0x5563995c in readline_internal () at 
../../../bash-5.0-rc1/lib/readline/readline.c:672
#4  0x55639367 in readline (prompt=0x55680f84 "") at 
../../../bash-5.0-rc1/lib/readline/readline.c:377
#5  0x55611bcf in edit_line (p=0x55680f84 "", itext=0x0) at 
../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:1107
#6  0x556108f8 in read_builtin (list=0x0) at 
../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:566
#7  0x555a5afa in execute_builtin (builtin=0x5560fa73 
, words=0x55761ea8, flags=0, subshell=0) at 
../bash-5.0-rc1/execute_cmd.c:4706
#8  0x555a6aa2 in execute_builtin_or_function (words=0x55761ea8, 
builtin=0x5560fa73 , var=0x0, redirects=0x55761c08, 
fds_to_close=0x55761be8, flags=0)
at ../bash-5.0-rc1/execute_cmd.c:5214
#9  0x555a5365 in execute_simple_command 
(simple_command=0x55761ac8, pipe_in=-1, pipe_out=-1, async=0, 
fds_to_close=0x55761be8) at ../bash-5.0-rc1/execute_cmd.c:4476
#10 0x5559e9f4 in execute_command_internal (command=0x55761a88, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x55761be8) at 
../bash-5.0-rc1/execute_cmd.c:842
#11 0x5560858a in parse_and_execute (string=0x557616c8 "PATH= read 
-e < rl_do_undo", from_file=0x556690f0 "-c", flags=4) at 
../../bash-5.0-rc1/builtins/evalstring.c:436
#12 0x5558564a in run_one_command (command=0x7fffe284 "PATH= read 
-e < rl_do_undo") at ../bash-5.0-rc1/shell.c:1426
#13 0x55584789 in main (argc=5, argv=0x7fffdfe8, 
env=0x7fffe018) at ../bash-5.0-rc1/shell.c:741
(gdb) x/12xb &search
0x7fffd7f8: 0x300x300x300x300x300x300x000x07
0x7fffd800: 0x080x3c0x76 

Re: realloc: start and end chunk sizes differ - rl_extend_line_buffer in lib/readline/util.c

[Eduardo A . Bustamante López]

On Sun, Jan 06, 2019 at 08:46:33PM -0800, Eduardo A. Bustamante López wrote:
> On Sun, Jan 06, 2019 at 07:18:27PM -0800, Eduardo A. Bustamante López wrote:
> (...)
> > malloc: unknown:0: assertion botched
> > malloc: 0x55769408: allocated: last allocated from unknown:0
> > realloc: start and end chunk sizes differ
> 
> OK, I think I know what the problem is.
(...)
> I still don't know how to trigger this with "human" input, but I think the
> problem is that rl_point should be bounded by the value of rl_end, thus the
> following patch makes the problem go away:
> 
> dualbus@system76-pc:~/src/gnu/bash$ git diff -- lib/readline/undo.c
> diff --git a/lib/readline/undo.c b/lib/readline/undo.c
> index ae65d380..12952555 100644
> --- a/lib/readline/undo.c
> +++ b/lib/readline/undo.c
> @@ -196,6 +196,8 @@ rl_do_undo (void)
> /* Undoing deletes means inserting some text. */
> case UNDO_DELETE:
>   rl_point = start;
> + if (rl_point > rl_end)
> +   rl_point = rl_end;
>   rl_insert_text (rl_undo_list->text);
>   xfree (rl_undo_list->text);
>   break;

I missed a spot, updated patch:

diff -ruN bash-5.0-rc1.orig/lib/readline/undo.c bash-5.0-rc1/lib/readline/undo.c
--- bash-5.0-rc1.orig/lib/readline/undo.c   2019-01-03 13:14:43.428392927 
-0800
+++ bash-5.0-rc1/lib/readline/undo.c2019-01-07 01:28:08.288255650 -0800
@@ -196,6 +196,8 @@
/* Undoing deletes means inserting some text. */
case UNDO_DELETE:
  rl_point = start;
+ if (rl_point > rl_end)
+   rl_point = rl_end;
  rl_insert_text (rl_undo_list->text);
  xfree (rl_undo_list->text);
  break;
@@ -204,6 +206,8 @@
case UNDO_INSERT:
  rl_delete_text (start, end);
  rl_point = start;
+ if (rl_point > rl_end)
+   rl_point = rl_end;
  break;
 
/* Undoing an END means undoing everything 'til we get to a BEGIN. */

Buffer overflow in string_extract_double_quoted - subst.c

[Eduardo A . Bustamante López]

Found by fuzzing `read -e' with AFL:

debian@debian-fuzz:/mnt$ cat -A dispose_word 
"^[^EM-b^_M-u$$(M-J^^_^Q$
^[^E

debian@debian-fuzz:/mnt$ base64 < dispose_word 
IhsF4h/1JCQoyl4fEQobBQ==

debian@debian-fuzz:/mnt$ LC_ALL=zh_CN.gbk ~/build-gdb/bash --noprofile --norc 
-c 'PATH= read -e < dispose_word' 
hi
"��$$(
TRACE: pid 15530: xparse_dolparen:0: base[5] != RPAREN (40), base = `"��$$(
'
TRACE: pid 15530: xparse_dolparen:0: *indp (5) < orig_ind (6), orig_string = `
'

malloc: ../bash-5.0-rc1/dispose_cmd.c:249: assertion botched
malloc: 0x55d956dc4de8: allocated: last allocated from 
../bash-5.0-rc1/subst.c:866
free: start and end chunk sizes differ
Aborting...Aborted

(...)

Aborting...
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x77df4535 in __GI_abort () at abort.c:79
#2  0x555b39b5 in programming_error (format=0x55686bd8 "free: start 
and end chunk sizes differ") at ../bash-5.0-rc1/error.c:175
#3  0x5566523d in xbotch (mem=0x55765da8, e=8, s=0x55686bd8 
"free: start and end chunk sizes differ", file=0x5566c268 
"../bash-5.0-rc1/dispose_cmd.c", line=249)
at ../../../bash-5.0-rc1/lib/malloc/malloc.c:354
#4  0x5566648e in internal_free (mem=0x55765da8, 
file=0x5566c268 "../bash-5.0-rc1/dispose_cmd.c", line=249, flags=1) at 
../../../bash-5.0-rc1/lib/malloc/malloc.c:960
#5  0x55667006 in sh_free (mem=0x55765da8, file=0x5566c268 
"../bash-5.0-rc1/dispose_cmd.c", line=249) at 
../../../bash-5.0-rc1/lib/malloc/malloc.c:1321
#6  0x556001f4 in sh_xfree (string=0x55765da8, file=0x5566c268 
"../bash-5.0-rc1/dispose_cmd.c", line=249) at ../bash-5.0-rc1/xmalloc.c:223
#7  0x5559d860 in dispose_word (w=0x55761da8) at 
../bash-5.0-rc1/dispose_cmd.c:249
#8  0x555d6ef5 in expand_word_internal (word=0x55761e08, quoted=0, 
isexp=0, contains_dollar_at=0x0, expanded_something=0x0) at 
../bash-5.0-rc1/subst.c:10189
#9  0x555c84dd in call_expand_word_internal (w=0x55761e08, q=0, 
i=0, c=0x0, e=0x0) at ../bash-5.0-rc1/subst.c:3684
#10 0x555c8b94 in expand_word (word=0x55761e08, quoted=0) at 
../bash-5.0-rc1/subst.c:3978
#11 0x555f35b6 in shell_expand_line (count=1, ignore=5) at 
../bash-5.0-rc1/bashline.c:2755
#12 0x55639ed4 in _rl_dispatch_subseq (key=5, map=0x556ac220 
, got_subseq=0) at 
../../../bash-5.0-rc1/lib/readline/readline.c:852
#13 0x5563a399 in _rl_dispatch_subseq (key=27, map=0x556ab200 
, got_subseq=0) at 
../../../bash-5.0-rc1/lib/readline/readline.c:986
#14 0x55639c4b in _rl_dispatch (key=-136275877, map=0x556ab200 
) at ../../../bash-5.0-rc1/lib/readline/readline.c:798
#15 0x556398ce in readline_internal_char () at 
../../../bash-5.0-rc1/lib/readline/readline.c:632
#16 0x55639929 in readline_internal_charloop () at 
../../../bash-5.0-rc1/lib/readline/readline.c:659
#17 0x55639949 in readline_internal () at 
../../../bash-5.0-rc1/lib/readline/readline.c:671
#18 0x55639367 in readline (prompt=0x55680f84 "") at 
../../../bash-5.0-rc1/lib/readline/readline.c:377
#19 0x55611bcf in edit_line (p=0x55680f84 "", itext=0x0) at 
../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:1107
#20 0x556108f8 in read_builtin (list=0x0) at 
../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:566
#21 0x555a5afa in execute_builtin (builtin=0x5560fa73 
, words=0x55761e68, flags=0, subshell=0) at 
../bash-5.0-rc1/execute_cmd.c:4706
#22 0x555a6aa2 in execute_builtin_or_function (words=0x55761e68, 
builtin=0x5560fa73 , var=0x0, redirects=0x55761bc8, 
fds_to_close=0x55761ba8, flags=0)
at ../bash-5.0-rc1/execute_cmd.c:5214
#23 0x555a5365 in execute_simple_command 
(simple_command=0x55761a88, pipe_in=-1, pipe_out=-1, async=0, 
fds_to_close=0x55761ba8) at ../bash-5.0-rc1/execute_cmd.c:4476
#24 0x5559e9f4 in execute_command_internal (command=0x55761a48, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x55761ba8) at 
../bash-5.0-rc1/execute_cmd.c:842
#25 0x5560858a in parse_and_execute (string=0x55761688 "PATH= read 
-e < dispose_word", from_file=0x556690f0 "-c", flags=4) at 
../../bash-5.0-rc1/builtins/evalstring.c:436
#26 0x5558564a in run_one_command (command=0x7fffe28c "PATH= read 
-e < dispose_word") at ../bash-5.0-rc1/shell.c:1426
#27 0x55584789 in main (argc=5, argv=0x7fffdfe8, 
env=0x7fffe018) at ../bash-5.0-rc1/shell.c:741

(gdb) frame 7
#7  0x5559d860 in dispose_word (w=0x55761da8) at 
../bash-5.0-rc1/dispose_cmd.c:249
249   FREE (w->word);
(gdb) l
244 /* How to free a WORD_DESC. */
245 void
246 d

Re: Segmentation fault in lib/readline/undo.c - rl_do_undo

[Eduardo A . Bustamante López]

On Mon, Jan 07, 2019 at 01:16:05AM -0800, Eduardo A. Bustamante López wrote:
> I found this with AFL. I think it's related to the problem reported here:
> http://lists.nongnu.org/archive/html/bug-bash/2018-09/msg00045.html
> 
> debian@debian-fuzz:/mnt$ cat -A rl_do_undo
> ^RM-CM-!M-CM-CM-!M-C^[.^[^[0^P^@^P^Q0^[-^P^Q0^[^W0^@0&/^[^[^[--^W^_~0^@0^@-^L^D^@^@'/^[B^@0^B^@M-
>  
> ^[^[M-^T^[M-mM-^?^[F-^W^_0^[M-^@0^P^@^@^@^@^D^I^@^[M-UM-UM-UM-NM-U^@M-^@^@M-=$^@01^@01^["0^?M-^?M-^?M-^?0M-r0^@'0M-^?^@^@^@M-CM-CM-!M-C^[.^[^[--^W00^P^@00(-^P^Q;^[-^P^Q0^[^W0^@n&/^[^[^[--^W^_~0^@0^@-^L^D^@^@'/^[B^@M-^T^B^@M-
>  
> ^[^[M-^T^[M-mM-^?^[F-^W^_0^[M-^@0^P^@^@^@^@^D^I^@^[M-UM-UM-UM-NM-U^@M-^@^@M-=$^@01^@01^["0^?M-^?M-^?M-^?0M-r0^@'0M-^?^@^@^@@^N\0^[11#0-^P^@^@^@^@^D^I^@^[M-UM-=M-UM-NM-U^@M-^@^@M-=$^@J^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^PM-eM-KM-YM-@M-nM-n^ZM-{0M-xM-|^@}}^L0#^A^Cd^\#^@^X^E^X^E^@M-^?M-^?^[^I^I^@^@^@M-^?\^O^@000M-^R00M-,0^@^@M-^?\^O^@qq0M-^Dq^@0^P^I^[^I^I^E^@M-^?M-^?0M-v^P^P^P^P^P^P^PM-eM-KM-YM-@M-nM-n^ZM-{0M-xM-|^@}}^L0#^A^Cd^\#^@^X^E^X^E^@M-^?M-^?^[^I^@^@M-^@^@0^@^@^@M-^?\^O^@0^?0M-^R00M-,0^@^@M-^?\^O^@0^@0^P^I^[^I^Iu000^E^@M-^?M-^?0M-vM-Q^A^@0^P^I^]0^I00^@^@^@M-^?\^O^GM-^?\^Oq0q^[^I^I^I^@^@M-h^C@^N\0^[11#0-^P^@^@^@^@^D^I^@^[M-UM-UM-UM-NM-U^@M-^@^@M-=$^@0^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^P^PM-eM-KM-YM-@M-nM-n^ZM-{0M-xM-|^@00^L0#^A^Cd^\#^@^X^E^X^E^@M-^?M-^?^[^I^I^@^@^@M-^?\^O^@000M-^R00M-,0^@^@M-^?\^O^@qq0M-^Dq^@0^P^I^[^I^Iu000^E^@M-^?M-^?0M-v^P^P^P^P^P^P^PM-eM-KM-YM-@M-nM-n^ZM-{0M-xM-|^@}}^L0#^A^Cd^\#^@^X^E^X^E^@M-^?M-^?^[^I^I^@^@^@M-^?\^O^@00M-^?M-^R00M-,0^@^@M-^?\^O^@0^@0^P^I^[^I^I^E^@M-^?M-^?0M-vM-Q^A^@0^P^I^]0^I00^@^@^@M-^?\^O^GM-^?\^Oq0q^[^I^I^I^@^@M-h^C^E^@000M-v
> 

Heh, I forgot to minimize the test case:

debian@debian-fuzz:/mnt$ cat -A rl_do_undo 
0^X^E0^P^P^P^X^E\^O^P^P

debian@debian-fuzz:/mnt$ base64 < rl_do_undo 
MBgFMBAQEBgFXA8QEA==

Also, running it with ASAN provides more information:

debian@debian-fuzz:/mnt$ ~/build-asan/bash --noprofile --norc -c 'PATH= read -e 
< rl_do_undo'
hi
0
/home/debian/build-asan/bash: emacs: No such file or directory
0
/home/debian/build-asan/bash: emacs: No such file or directory
\
0
=
==29290==ERROR: AddressSanitizer: heap-use-after-free on address 0x60304018 
at pc 0x007d7508 bp 0x7ffe9530f5c0 sp 0x7ffe9530f5b8
READ of size 4 at 0x60304018 thread T0
#0 0x7d7507 in rl_do_undo 
/home/debian/build-asan/lib/readline/../../../bash-5.0-rc1/lib/readline/undo.c:188:25
#1 0x7d8682 in rl_revert_line 
/home/debian/build-asan/lib/readline/../../../bash-5.0-rc1/lib/readline/undo.c:341:2
#2 0x767dd3 in readline_internal_teardown 
/home/debian/build-asan/lib/readline/../../../bash-5.0-rc1/lib/readline/readline.c:471:7
#3 0x7678b0 in readline_internal 
/home/debian/build-asan/lib/readline/../../../bash-5.0-rc1/lib/readline/readline.c:672:11
#4 0x7676ba in readline 
/home/debian/build-asan/lib/readline/../../../bash-5.0-rc1/lib/readline/readline.c:377:11
#5 0x6fe637 in edit_line 
/home/debian/build-asan/builtins/../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:1107:9
#6 0x6fa7d5 in read_builtin 
/home/debian/build-asan/builtins/../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:566:16
#7 0x592620 in execute_builtin 
/home/debian/build-asan/../bash-5.0-rc1/execute_cmd.c:4706:13
#8 0x5910a7 in execute_builtin_or_function 
/home/debian/build-asan/../bash-5.0-rc1/execute_cmd.c:5214:14
#9 0x579877 in execute_simple_command 
/home/debian/build-asan/../bash-5.0-rc1/execute_cmd.c:4476:13
#10 0x5701d2 in execute_command_internal 
/home/debian/build-asan/../bash-5.0-rc1/execute_cmd.c:842:4
#11 0x6dd393 in parse_and_execute 
/home/debian/build-asan/builtins/../../bash-5.0-rc1/builtins/evalstring.c:436:17
#12 0x51d4f4 in run_one_command 
/home/debian/build-asan/../bash-5.0-rc1/shell.c:1426:12
#13 0x518ec9 in main /home/debian/build-asan/../bash-5.0-rc1/shell.c:741:7
#14 0x7f194ff8c09a in __libc_start_main 
(/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
#15 0x43fa39 in _start (/home/debian/build-asan/bash+0x43fa39)

0x60304018 is located 24 bytes inside of 32-byte region 
[0x60304000,0x60304020)
freed by thread T0 here:
#0 0x4e7502 in __interceptor_free (/home/debian/build-asan/bash+0x4e7502)
#1 0x6c2bbf in xfree /home/debian/build-asan/../bash-5.0-rc1/xmalloc.c:150:5
#2 0x7d7036 in _rl_free_undo_list 
/home/debian/build-asan/lib/readline/../../../bash-5.0-rc1/lib/readline/undo.c:113:7
#3 0x7d7070 in rl_free_undo_list 
/home/debian/build-asan/lib/readline/../../../bash-5.0-rc1/lib/readline/undo.c:124:3
#4 0x767e6a in readline_internal_teardown 
/home/debian/build-asan/lib/readline/../../../bash-5.0-rc1/lib/readline/readline.c:485:5
#5 0x7678b0 in r

Re: realloc: start and end chunk sizes differ - rl_extend_line_buffer in lib/readline/util.c

[Eduardo A . Bustamante López]

On Mon, Jan 07, 2019 at 01:30:46AM -0800, Eduardo A. Bustamante López wrote:
(...)
> I missed a spot, updated patch:
> 
> diff -ruN bash-5.0-rc1.orig/lib/readline/undo.c 
> bash-5.0-rc1/lib/readline/undo.c
> --- bash-5.0-rc1.orig/lib/readline/undo.c   2019-01-03 13:14:43.428392927 
> -0800
> +++ bash-5.0-rc1/lib/readline/undo.c2019-01-07 01:28:08.288255650 -0800
> @@ -196,6 +196,8 @@
> /* Undoing deletes means inserting some text. */
> case UNDO_DELETE:
>   rl_point = start;
> + if (rl_point > rl_end)
> +   rl_point = rl_end;
>   rl_insert_text (rl_undo_list->text);
>   xfree (rl_undo_list->text);
>   break;
> @@ -204,6 +206,8 @@
> case UNDO_INSERT:
>   rl_delete_text (start, end);
>   rl_point = start;
> + if (rl_point > rl_end)
> +   rl_point = rl_end;
>   break;
>  
> /* Undoing an END means undoing everything 'til we get to a BEGIN. */

Missed 2 actually, this is another case, in lib/readline/search.c now:

Hardware watchpoint 1: (rl_point > 0 && rl_point > rl_end + 1)

Old value = 0
New value = 1
_rl_nsearch_abort (cxt=0x557caf08) at 
../../../bash-5.0-rc1/lib/readline/search.c:258
258   rl_mark = cxt->save_mark;
(gdb) bt
#0  _rl_nsearch_abort (cxt=0x557caf08) at 
../../../bash-5.0-rc1/lib/readline/search.c:258
#1  0x5563fc73 in _rl_nsearch_dispatch (cxt=0x557caf08, c=3) at 
../../../bash-5.0-rc1/lib/readline/search.c:300
#2  0x5563fe2d in noninc_search (dir=1, pchar=0) at 
../../../bash-5.0-rc1/lib/readline/search.c:389
#3  0x5563fea9 in rl_noninc_forward_search (count=1, key=110) at 
../../../bash-5.0-rc1/lib/readline/search.c:405
#4  0x55639ed4 in _rl_dispatch_subseq (key=110, map=0x556ac220 
, got_subseq=0) at 
../../../bash-5.0-rc1/lib/readline/readline.c:852
#5  0x5563a399 in _rl_dispatch_subseq (key=27, map=0x556ab200 
, got_subseq=0) at 
../../../bash-5.0-rc1/lib/readline/readline.c:986
#6  0x55639c4b in _rl_dispatch (key=-135542668, map=0x556ab200 
) at ../../../bash-5.0-rc1/lib/readline/readline.c:798
#7  0x556398ce in readline_internal_char () at 
../../../bash-5.0-rc1/lib/readline/readline.c:632
#8  0x55639929 in readline_internal_charloop () at 
../../../bash-5.0-rc1/lib/readline/readline.c:659
#9  0x55639949 in readline_internal () at 
../../../bash-5.0-rc1/lib/readline/readline.c:671
#10 0x55639367 in readline (prompt=0x55680f84 "") at 
../../../bash-5.0-rc1/lib/readline/readline.c:377
#11 0x55611bcf in edit_line (p=0x55680f84 "", itext=0x0) at 
../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:1107
#12 0x556108f8 in read_builtin (list=0x0) at 
../../bash-5.0-rc1/builtins/../../bash-5.0-rc1/builtins/read.def:566
#13 0x555a5afa in execute_builtin (builtin=0x5560fa73 
, words=0x55764cc8, flags=0, subshell=0) at 
../bash-5.0-rc1/execute_cmd.c:4706
#14 0x555a6aa2 in execute_builtin_or_function (words=0x55764cc8, 
builtin=0x5560fa73 , var=0x0, redirects=0x55764b08, 
fds_to_close=0x55764a28, flags=0)
at ../bash-5.0-rc1/execute_cmd.c:5214
#15 0x555a5365 in execute_simple_command 
(simple_command=0x55764908, pipe_in=-1, pipe_out=-1, async=0, 
fds_to_close=0x55764a28) at ../bash-5.0-rc1/execute_cmd.c:4476
#16 0x5559e9f4 in execute_command_internal (command=0x557648c8, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x55764a28) at 
../bash-5.0-rc1/execute_cmd.c:842
#17 0x5560858a in parse_and_execute (string=0x5574c2a8 "PATH= read 
-e < bar", from_file=0x556690f0 "-c", flags=4) at 
../../bash-5.0-rc1/builtins/evalstring.c:436
#18 0x5558564a in run_one_command (command=0x7fffe2a6 "PATH= read 
-e < bar") at ../bash-5.0-rc1/shell.c:1426
#19 0x55584789 in main (argc=5, argv=0x7fffe008, 
env=0x7fffe038) at ../bash-5.0-rc1/shell.c:741
(gdb) l
253 _rl_nsearch_abort (_rl_search_cxt *cxt)
254 {
255   rl_maybe_unsave_line ();
256   rl_clear_message ();
257   rl_point = cxt->save_point;
258   rl_mark = cxt->save_mark;
259   rl_restore_prompt ();
260
261   RL_UNSETSTATE (RL_STATE_NSEARCH);
262 }
(gdb) c
Continuing.

malloc: unknown:0: assertion botched
malloc: 0x55769408: allocated: last allocated from unknown:0
realloc: start and end chunk sizes differ
Aborting...
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.


diff -ruN bash-5.0-rc1.orig/lib/readline/search.c 
bash-5.0-rc1/lib/readline/search.c
--- bash-5.0-rc1.orig/lib/readline/search.c 2019-01-03 13:14:43.42839

Re: bug: illegal function name?

[Eduardo A . Bustamante López]

On Sun, Jan 20, 2019 at 06:26:43PM +0300, Andrey Butirsky wrote:
> Andreas, I know it will work with the '-f' flag.
> But for others function names, the '-f' unset flag is not required.
> Moreover, it seem confronts with Open Group Base Specification.
> So I consider it as a bug still.
> 

It is intentional. If you want bash to run in POSIX mode, you should use the 
`POSIXLY_CORRECT' variable or the `--posix'
flag.

| dualbus@system76-pc:~$ POSIXLY_CORRECT=y bash -c 'function 1a() { :; }'
| bash: `1a': not a valid identifier

And as Andreas mentioned, in general you should use `unset -f' if you want to 
operate on functions. Otherwise you risk
conflict with a variable name.

Re: [bug-bash] $RANDOM not Cryptographically secure pseudorandom number generator

[Eduardo A . Bustamante López]

On Sun, Jan 20, 2019 at 05:22:12PM -0500, Chet Ramey wrote:
> On 1/20/19 4:54 PM, Chet Ramey wrote:
> 
> >> As an aside, I can confirm the findings of a performance difference
> >> between 4.4 and 5.0 when running the script provided earlier in the
> >> discussion. At first glance it seems to be due to the switch from the
> >> old LCG to the current MINSTD RNG, 
(...)
> So I ran a quick test.
> 
> $ ./bash ./x3
> iterations: 100
> BASH_VERSION: 5.0.2(4)-maint
> time: 9.684
> $ ../bash-5.0/bash ./x3
> iterations: 100
> BASH_VERSION: 5.0.0(1)-release
> time: 9.749
> $ ../bash-5.0-patched/bash ./x3
> iterations: 100
> BASH_VERSION: 5.0.2(3)-release
> time: 9.840
> $ ../bash-4.4-patched/bash ./x3
> iterations: 100
> BASH_VERSION: 4.4.23(7)-release
> time: 11.365
> $ ../bash-4.4-patched/bash ./x3
> iterations: 100
> BASH_VERSION: 4.4.23(7)-release
> time: 11.235
> jenna.local(1)
> 
> Where the script is Eduardo's iterator that just expands $RANDOM
> N times.
> 
> The random number generator has been the same since bash-4.0.

I'm sorry, my tests were wrong. I built bash using the default `./configure'
behavior for the `devel' branch, which I always forget, uses the internal
allocator with debugging enabled, and thus, all of my times were off due to the
additional malloc overhead.

I rebuilt it with `../bash/configure --without-bash-malloc', which causes it to
use the system's allocator, and surely enough, the timings make more sense now:

(`build-bash-devel-malloc' is `configure --with-bash-malloc',
 `build-bash-devel' is `configure --without-bash-malloc')

| dualbus@system76-pc:~/src/gnu/build-bash-devel-malloc$ ./bash ~/test-speed.sh 
| iterations: 100
| BASH_VERSION: 5.0.0(1)-maint
| time: 8.765
| 
| dualbus@system76-pc:~/src/gnu/build-bash-devel-malloc$ 
../build-bash-devel/bash ~/test-speed.sh 
| iterations: 100
| BASH_VERSION: 5.0.0(1)-maint
| time: 3.431
| 
| dualbus@system76-pc:~/src/gnu/build-bash-devel-malloc$ bash ~/test-speed.sh 
| iterations: 100
| BASH_VERSION: 5.0.0(1)-release
| time: 3.435
| 
| dualbus@system76-pc:~/src/gnu/build-bash-4.4$ ./bash ~/test-speed.sh 
| iterations: 100
| BASH_VERSION: 4.4.0(1)-release
| time: 3.443

Re: bug: illegal function name?

[Eduardo A . Bustamante López]

On Mon, Jan 21, 2019 at 09:26:59AM +0700, Robert Elz wrote:
(...)
> I think his point is that if unset "unset f" (no flags) works to unset 
> function f, if f is not a (set) variable, then it should work every time
> "f" is not a set variable, not only the times when the word "f" happens
> to be of the correct syntax to be a variable name, but happens not
> to be.
> 
> I think that is a good point.
> 
> In the NetBSD sh we avoid that issue completely, the only way to
> unset a function is with "unset -f", a simple "unset f" only ever
> unsets variables.

I think that's a good point too. But how would you fix this particular issue?

Changing the behavior of `unset f' to only ever unset variables means
potentially breaking existing scripts. Is the inconsistency reported severe
enough to make this change?

Re: RETURN trap will inherit last RETURN trap cmd though set +T

[Eduardo A . Bustamante López]

On Wed, Jan 23, 2019 at 03:59:12PM +, Shen Herbert wrote:
(...)
> Repeat-By:
> Run this script:
> ```
> set +T
> one { trap 'echo in one' RETURN; }
> all {
> trap 'echo in all' RETURN
> one
> }
> all
> ```

Hm, the script above is a bit odd, these aren't syntactically valid function 
definitions:

  dualbus@system76-pc:~$ cat > file
  set +T
  one { trap 'echo in one' RETURN; }
  all {
  trap 'echo in all' RETURN
  one
  }
  all
  
  dualbus@system76-pc:~$ bash file 
  file: line 2: syntax error near unexpected token `}'
  file: line 2: `one { trap 'echo in one' RETURN; }'


Do you have another example we can use?

Re: Simple exit trap doesn't run in newer versions of bash

[Eduardo A . Bustamante López]

On Fri, Jan 25, 2019 at 11:22:07AM -0400, Brad Spencer wrote:
(...)
> Now consider this command:
> 
> bash -c 'trap "echo WORKS" EXIT && touch x'
> 
> On newer versions of bash, it produces no output.  Substituting different
> commands in the trap or tracing it seems to indicate that the trap never
> runs.

I can reproduce the problem, my bash is: GNU bash, version 5.0.2(1)-release 
(x86_64-pc-linux-gnu)

| dualbus@system76-pc:~$ rm x
| dualbus@system76-pc:~$ bash -c 'trap "echo WORKS" EXIT && touch x'
| dualbus@system76-pc:~$ ls -l x
| -rw-r--r-- 1 dualbus dualbus 0 Jan 25 08:26 x

> Is this a bug?  If not, can someone explain why the last command behaves
> differently?

I think it is a bug. My guess is that bash is optimizing the fork, and it's
forgetting to run the `EXIT' trap in that case, or at least that seems to be the
problem when I run this through a debugger.

| dualbus@system76-pc:~/src/gnu/build-bash-devel$ gdb --args ./bash -c 'trap 
"echo WORKS" EXIT && touch x'
| GNU gdb (Debian 8.2-1) 8.2
| (...)
| (gdb) b execute_cmd.c:4537
| Breakpoint 1 at 0x515d2: file ../bash/execute_cmd.c, line 4537.
| (gdb) r
| Starting program: /home/dualbus/src/gnu/build-bash-devel/bash -c trap\ 
\"echo\ WORKS\"\ EXIT\ \&\&\ touch\ x
| 
| Breakpoint 1, execute_simple_command (simple_command=0x55761a48, 
pipe_in=-1, pipe_out=-1, async=0, fds_to_close=0x557619a8) at 
../bash/execute_cmd.c:4537
| 4537  execute_from_filesystem:
| (gdb) p *simple_command->words->word
| $1 = {word = 0x55761908 "touch", flags = 0}
| (gdb) n
| 4538if (command_line == 0)
| (gdb)
| 4539  command_line = savestring (the_printed_command_except_trap ? 
the_printed_command_except_trap : "");
| (gdb)
| 4545if (already_forked == 0 && (simple_command->flags & CMD_NO_FORK) && 
fifos_pending() > 0)
| (gdb)
| 4548result = execute_disk_command (words, simple_command->redirects, 
command_line,
| (gdb)
| process 26087 is executing new program: /bin/touch
| 
| [1]+  Stopped gdb --args ./bash -c 'trap "echo WORKS" EXIT && 
touch x'

AddressSanitizer: heap-use-after-free on (...) in rl_do_undo ../../../bash-devel/lib/readline/undo.c:188

[Eduardo A . Bustamante López]

I found another issue in rl_do_undo, but I haven't been successful in figuring 
out how it happens.

I've been working with the `devel' branch, commit 
`8a9718cfc93958b34e205d0507c3bbf64cba6db5'

Here's how I built the binaries I use below:

debian@debian-fuzz:~/tmp$ cat ~/build.sh 
#!/bin/bash
mkdir build-devel{,-asan,-gdb}
(cd build-devel  && CFLAGS='' CC='afl-clang-fast'   
   ../bash-devel/configure --silent --without-bash-malloc && make 
-sj$(nproc))
(cd build-devel-asan && CFLAGS='-O0 -ggdb -fno-omit-frame-pointer 
-fsanitize=address ' ../bash-devel/configure --silent --without-bash-malloc && 
make -sj$(nproc))
(cd build-devel-gdb  && CFLAGS='-O0 -ggdb -fno-omit-frame-pointer ' 
   ../bash-devel/configure --silent --without-bash-malloc && make 
-sj$(nproc))

This is the input to `read -e':

debian@debian-fuzz:~/tmp$ base64 < o
MBgFEBAQDhUwEBgoHx8wEDAYRRQbEDAYBTAYRQ4=

debian@debian-fuzz:~/tmp$ cat -A o
0^X^E^P^P^P^N^U0^P^X(^_^_0^P0^XE^T^[^P0^X^E0^XE^N


My poor attempt of a trace under GDB:

debian@debian-fuzz:~/tmp$ gdb --batch --command=trace_rl_undo.gdb --args 
~/build-devel-gdb/bash --noprofile --norc -c 'EDITOR=: PATH= read -e < o' > 
out.txt 2>&1
(no output)
---

dualbus@system76-pc:~/src/dualbus/bash-fuzzing/tmp/tmp$ cat out.txt 
0
0
/tmp/bash-fc.9lfkNE: line 1: 0: No such file or directory
000
000
/tmp/bash-fc.HkVuNJ: line 1: 000: No such file or directory
readline: maximum macro execution nesting level exceeded

munmap_chunk(): invalid pointer
---


dualbus@system76-pc:~/src/dualbus/bash-fuzzing/tmp/tmp$ cat gdb.txt 
Temporary breakpoint 1 at 0x2eea9: file ../bash-devel/shell.c, line 392.

Temporary breakpoint 1, main (argc=5, argv=0x7fffdfd8, env=0x7fffe008) 
at ../bash-devel/shell.c:392
392   code = setjmp_nosigs (top_level);
Breakpoint 2 at 0x5565004c: file ../../../bash-devel/lib/readline/undo.c, 
line 177.
[Detaching after fork from child process 588]

Breakpoint 2, rl_do_undo () at ../../../bash-devel/lib/readline/undo.c:177
(... snip ...)

Breakpoint 2, rl_do_undo () at ../../../bash-devel/lib/readline/undo.c:177
177   start = end = waiting_for_begin = 0;
$625 = "~~~ print_rl_undo_list>"
$626 = 0
$627 = (UNDO_LIST *) 0x556f9e50
$628 = (struct undo_list *) 0x556fa6f0
$629 = 0x0
$630 = 1
$631 = (struct undo_list *) 0x556fa6f0
$632 = (struct undo_list *) 0x0
$633 = 0x0
$634 = "~~~ print_the_history>"
$635 = 0
$636 = (HIST_ENTRY *) 0x556fa720
$637 = 0x556debd0 "000"
$638 = 0
$639 = (UNDO_LIST *) 0x556fa6f0
$640 = (struct undo_list *) 0x0
$641 = 0x0
$642 = 1
$643 = (HIST_ENTRY *) 0x556fa7b0
$644 = 0x556f89a0 "0"
$645 = 0
$646 = (UNDO_LIST *) 0x556fa330
$647 = (struct undo_list *) 0x556fa3d0
$648 = 0x0
$649 = 1
$650 = (struct undo_list *) 0x556fa3d0
$651 = (struct undo_list *) 0x556f9f00
$652 = 0x556de940 "\360\237oUUU"
$653 = 2
$654 = (struct undo_list *) 0x556f9f00
$655 = (struct undo_list *) 0x0
$656 = 0x556fa750 " "
$657 = 2
$658 = (HIST_ENTRY *) 0x556fa2c0
$659 = 0x556f85f0 ""
$660 = 0
$661 = (UNDO_LIST *) 0x556f90e0
$662 = (struct undo_list *) 0x0
$663 = 0x0
$664 = 3
$665 = (HIST_ENTRY *) 0x556f9130
$666 = 0x556fa6d0 ""
$667 = 0
$668 = (UNDO_LIST *) 0x556f90b0
$669 = (struct undo_list *) 0x0
$670 = 0x0
#0  rl_do_undo () at ../../../bash-devel/lib/readline/undo.c:177
#1  0x556504fd in rl_undo_command (count=1, key=31) at 
../../../bash-devel/lib/readline/undo.c:358
#2  0x55633b75 in _rl_dispatch_subseq (key=31, map=0x5569f200 
, got_subseq=0) at 
../../../bash-devel/lib/readline/readline.c:852
#3  0x556338ec in _rl_dispatch (key=1433265360, map=0x5569f200 
) at ../../../bash-devel/lib/readline/readline.c:798
#4  0x5563356f in readline_internal_char () at 
../../../bash-devel/lib/readline/readline.c:632
#5  0x556335ca in readline_internal_charloop () at 
../../../bash-devel/lib/readline/readline.c:659
#6  0x556335ea in readline_internal () at 
../../../bash-devel/lib/readline/readline.c:671
#7  0x55633008 in readline (prompt=0x556767bc "") at 
../../../bash-devel/lib/readline/readline.c:377
#8  0x5560c4cc in edit_line (p=0x556767bc "", itext=0x0) at 
../../bash-devel/builtins/../../bash-devel/builtins/read.def:1107
#9  0x5560b2b2 in read_builtin (list=0x0) at 
../../bash-devel/builtins/../../bash-devel/builtins/read.def:566
#10 0x555a3bc5 in execute_builtin (builtin=0x5560a45d 
, words=0x556c1030, flags=0, subshell=0) at 
../bash-devel/execute_cmd.c:4709
#11 0x555a4ae9 in execute_builtin_or_function (words=0x556c1030, 
builtin=0x5560a45d , var=0x0, redirects=0x556bf720, 
fds_to_close=0x556c0cb0, flags=0) at ../bash-devel/execute_cmd.c:5217
#12 0x5

configure: gl_PTHREAD_RWLOCK_RDLOCK_PREFER_WRITER: command not found

[Eduardo A . Bustamante López]

Commit `be4078d25ff3af268a6ef7ab56c8121f9a8dfb36' introduced the following 
issue:

| dualbus@system76-pc:~/src/gnu/bash$ ./configure --silent
| 
| Beginning configuration for bash-5.0-maint for x86_64-pc-linux-gnu
| 
| ./configure: line 9614: gl_PTHREAD_RWLOCK_RDLOCK_PREFER_WRITER: command not 
found
| config.status: creating po/POTFILES
| config.status: creating po/Makefile

There's no mention of these changes in the changelog, so I don't really 
understand what's broken or how to fix it.


I patched bash and ran `autoconf' to regenerate the `configure' file, and it 
/seems/ to work, but I don't really know
what I'm doing:


diff --git a/MANIFEST b/MANIFEST
index cf957d3e..e30536f6 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -502,6 +502,7 @@ m4/nls.m4   f
 m4/po.m4   f
 m4/printf-posix.m4 f
 m4/progtest.m4 f
+m4/pthread_rwlock_rdlock.m4f
 m4/size_max.m4 f
 m4/stdint_h.m4 f
 m4/threadlib.m4f
diff --git a/configure.ac b/configure.ac
index ab8bc23f..81816ce8 100644
--- a/configure.ac
+++ b/configure.ac
@@ -712,6 +712,7 @@ m4_include([m4/nls.m4])
 m4_include([m4/po.m4])
 m4_include([m4/printf-posix.m4])
 m4_include([m4/progtest.m4])
+m4_include([m4/pthread_rwlock_rdlock.m4])
 m4_include([m4/size_max.m4])
 m4_include([m4/stdint_h.m4])
 m4_include([m4/threadlib.m4])
diff --git a/m4/pthread_rwlock_rdlock.m4 b/m4/pthread_rwlock_rdlock.m4
new file mode 100644
index ..3c1d6456
--- /dev/null
+++ b/m4/pthread_rwlock_rdlock.m4
@@ -0,0 +1,165 @@
+# pthread_rwlock_rdlock.m4 serial 2
+dnl Copyright (C) 2017-2019 Free Software Foundation, Inc.
+dnl This file is free software; the Free Software Foundation
+dnl gives unlimited permission to copy and/or distribute it,
+dnl with or without modifications, as long as this notice is preserved.
+
+dnl From Bruno Haible.
+dnl Inspired by
+dnl 
https://github.com/linux-test-project/ltp/blob/master/testcases/open_posix_testsuite/conformance/interfaces/pthread_rwlock_rdlock/2-2.c
+dnl by Intel Corporation.
+
+dnl Test whether in a situation where
+dnl   - an rwlock is taken by a reader and has a writer waiting,
+dnl   - an additional reader requests the lock,
+dnl   - the waiting writer and the requesting reader threads have the same
+dnl priority,
+dnl the requesting reader thread gets blocked, so that at some point the
+dnl waiting writer can acquire the lock.
+dnl Without such a guarantee, when there a N readers and each of the readers
+dnl spends more than 1/Nth of the time with the lock held, there is a high
+dnl probability that the waiting writer will not get the lock in a given finite
+dnl time, a phenomenon called "writer starvation".
+dnl Without such a guarantee, applications have a hard time avoiding writer
+dnl starvation.
+dnl
+dnl POSIX:2017 makes this requirement only for implementations that support TPS
+dnl (Thread Priority Scheduling) and only for the scheduling policies 
SCHED_FIFO
+dnl and SCHED_RR, see
+dnl 
http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_rwlock_rdlock.html
+dnl but this test verifies the guarantee regardless of TPS and regardless of
+dnl scheduling policy.
+dnl Glibc currently does not provide this guarantee, see
+dnl https://sourceware.org/bugzilla/show_bug.cgi?id=13701
+AC_DEFUN([gl_PTHREAD_RWLOCK_RDLOCK_PREFER_WRITER],
+[
+  AC_REQUIRE([gl_THREADLIB_EARLY])
+  AC_CACHE_CHECK([whether pthread_rwlock_rdlock prefers a writer to a reader],
+[gl_cv_pthread_rwlock_rdlock_prefer_writer],
+[save_LIBS="$LIBS"
+ LIBS="$LIBS $LIBMULTITHREAD"
+ AC_RUN_IFELSE(
+   [AC_LANG_SOURCE([[
+#include 
+#include 
+#include 
+#include 
+
+#define SUCCEED() exit (0)
+#define FAILURE() exit (1)
+#define UNEXPECTED(n) (exit (10 + (n)))
+
+/* The main thread creates the waiting writer and the requesting reader threads
+   in the default way; this guarantees that they have the same priority.
+   We can reuse the main thread as first reader thread.  */
+
+static pthread_rwlock_t lock;
+static pthread_t reader1;
+static pthread_t writer;
+static pthread_t reader2;
+static pthread_t timer;
+/* Used to pass control from writer to reader2 and from reader2 to timer,
+   as in a relay race.
+   Passing control from one running thread to another running thread
+   is most likely faster than to create the second thread.  */
+static pthread_mutex_t baton;
+
+static void *
+timer_func (void *ignored)
+{
+  /* Step 13 (can be before or after step 12):
+ The timer thread takes the baton, then waits a moment to make sure
+ it can tell whether the second reader thread is blocked at step 12.  */
+  if (pthread_mutex_lock (&baton))
+UNEXPECTED (13);
+  usleep (10);
+  /* By the time we get here, it's clear that the second reader thread is
+ blocked at step 12.  This is the desired behaviour.  */
+  SUCCEED ();
+}
+
+static void *
+reader2_func (void *ignored)
+{
+  int err;
+
+  /* Step 8 (can be before or after step 7):
+ The second reader 

Re: General Associative Array problem -- Was: UUID as Array Keys strangely not possible

[Eduardo A . Bustamante López]

On Sat, Feb 23, 2019 at 01:48:08AM +, Robert White wrote:
(...)
> Your syntax checker is straight tripping on that SC1036 error dude. Array
> assignment using ARRAYNAME=( expression ) is completely legal and correct
> with or without the eval. The structure even allows for line continuation
> just like pipelines

To be fair, I didn't say the code was wrong. I just wanted to illustrate that 
the code you sent wasn't particularly easy
to work with (from my perspective). Which is fine, we found the problem 
anyways. But in the future it might help you
(and us), to reduce the test case to the most minimal script that still shows 
the behavior. That way we can easily run
it in our machines and see exactly what you see, and we can also focus on the 
stuff that matters (i.e. the `blkid',
`sed', `eval', etc. are all distractions from the actual problem).

I usually practice the code reduction approach myself and I think it's quite 
effective at helping me understand my own
mistakes :-).

Re: "here strings" and tmpfiles

[Eduardo A . Bustamante López]

On Mon, Mar 18, 2019 at 05:18:10PM -0400, Daniel Kahn Gillmor wrote:
> hi bash developers--
(...)
>  a) use socketpair(2) or pipe(2) instead of making a tmpfile.  this has
> the potential downside that the semantics of access to the remaining
> file descriptor would be subtly different from "regular file"
> semantics.

Disclaimer: not a bash developer, just a user.

I don't think the implementation details of herestrings are documented anywhere,
and I'm not too sure if they should (i.e. IMO if you need that degree of control
over the implementation details, then you should use something other than
shell).


Having said that, have you tried process substitution as an option?

You should be able to do something like:


  mycommand < <(printf %s 'super secret')


That will:

- not write the 'super secret' string to the file-system, nor
- show the mentioned string in the process tree (because printf is a bash
  built-in command, and thus, does not require a fork).

Re: Bug? Bash manual not indexable by search engines

[Eduardo A . Bustamante López]

On Sat, May 25, 2019 at 02:56:43PM -0400, Richard Marmorstein wrote:
> There was discussion on Twitter today
> (https://twitter.com/PttPrgrmmr/status/1132351142938185728) about how the
> Bash manual appears to not be indexable by search engines.
> 
> https://www.gnu.org/software/bash/manual/bashref.html
> redirects to
> https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html, and
> www.gnu.org/robots.txt
> has
> "Disallow: /savannah-checkouts/"
> 
> We reasoned that this probably wasn't deliberate and wanted to report it.

Hmmm, interesting. How did you get to 
?

I went to: , which is the "landing page" (?)
for Bash. That has:

> Documentation
> 
> *Documentation for Bash* is available online, as is documentation for most GNU
> software. You may also find more information about Bash by running info bash 
> or
> man bash, or by looking at /usr/doc/bash/, /usr/local/doc/bash/, or similar
> directories on your system. A brief summary is available by running bash 
> --help.

The "Documentation for Bash" text includes a link to:
, which then links to:
 (i.e. it's "bash.html", 
not "bashref.html").

Furthermore, if I search for "bash manual" in Google (i.e.
), the top three results (for me)
are:

1. 
2. 
3. 

So, it looks like the manual IS indexable?


I searched for "https://www.gnu.org/software/bash/manual/bashref.html"; in Google
too, and I can see it's referenced from a couple of 

user submitted posts, but that's it.

Segmentation fault in pat_subst

[Eduardo A . Bustamante López]

Bash `devel' crashes under the following circumstances:

| dualbus@system76-pc:/tmp/build-bash-devel$ CFLAGS='-O0 -ggdb' 
~/src/gnu/bash/configure --with-bash-malloc
| (...)
| dualbus@system76-pc:/tmp/build-bash-devel$ make -j$(nproc)
| (...)
| dualbus@system76-pc:/tmp/build-bash-devel$ ./bash -c $'x=0; : 
${x/#[0\xef\xbf\xbd\\Z[:]]}'
| Segmentation fault (core dumped)


Here's the stack trace:

| dualbus@system76-pc:/tmp/build-bash-devel$ gdb ./bash --args ./bash -c $'x=0; 
: ${x/#[0\xef\xbf\xbd\\Z[:]]}'
| GNU gdb (Debian 8.2.1-2+b1) 8.2.1
| (...)
| Reading symbols from ./bash...done.
| (gdb) r
| Starting program: /tmp/build-bash-devel/bash -c x=0\;\ :\ 
\$\{x/\#\[0�\\Z\[:\]\]\}
| 
| Program received signal SIGSEGV, Segmentation fault.
| 0x555d1fae in pat_subst (string=0x5575f298 "0", 
pat=0x55764509 "[0�\\Z[:]]", rep=0x0, mflags=1) at 
/home/dualbus/src/gnu/bash/subst.c:8136
| 8136  if (str && *str)
| (gdb) bt
| #0  0x555d1fae in pat_subst (string=0x5575f298 "0", 
pat=0x55764509 "[0�\\Z[:]]", rep=0x0, mflags=1) at 
/home/dualbus/src/gnu/bash/subst.c:8136
| #1  0x555d250b in parameter_brace_patsub (varname=0x5575f248 "x", 
value=0x5575f288 "0", ind=0, patsub=0x55763f48 "#[0�\\Z[:]]", quoted=0, 
pflags=0, flags=0)
| at /home/dualbus/src/gnu/bash/subst.c:8306
| #2  0x555d47e2 in parameter_brace_expand (string=0x55763f28 
"${x/#[0�\\Z[:]]}", indexp=0x7fffe0d8, quoted=0, pflags=0, 
quoted_dollar_atp=0x7fffe1d4,
| contains_dollar_at=0x7fffe1cc) at 
/home/dualbus/src/gnu/bash/subst.c:9028
| #3  0x555d5ae8 in param_expand (string=0x55763f28 
"${x/#[0�\\Z[:]]}", sindex=0x7fffe1d8, quoted=0, 
expanded_something=0x7fffe374, contains_dollar_at=0x7fffe1cc,
| quoted_dollar_at_p=0x7fffe1d4, had_quoted_null_p=0x7fffe1d0, 
pflags=0) at /home/dualbus/src/gnu/bash/subst.c:9557
| #4  0x555d6ed8 in expand_word_internal (word=0x55763f68, 
quoted=0, isexp=0, contains_dollar_at=0x7fffe370, 
expanded_something=0x7fffe374)
| at /home/dualbus/src/gnu/bash/subst.c:10125
| #5  0x555da0b6 in shell_expand_word_list (tlist=0x55763f88, 
eflags=31) at /home/dualbus/src/gnu/bash/subst.c:11504
| #6  0x555da3bb in expand_word_list_internal (list=0x55763948, 
eflags=31) at /home/dualbus/src/gnu/bash/subst.c:11628
| #7  0x555d95b9 in expand_words (list=0x55763948) at 
/home/dualbus/src/gnu/bash/subst.c:11148
| #8  0x555a51d9 in execute_simple_command 
(simple_command=0x557639c8, pipe_in=-1, pipe_out=-1, async=0, 
fds_to_close=0x55763a88)
| at /home/dualbus/src/gnu/bash/execute_cmd.c:4334
| #9  0x5559ed6b in execute_command_internal (command=0x55763988, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x55763a88)
| at /home/dualbus/src/gnu/bash/execute_cmd.c:823
| #10 0x555a2116 in execute_connection (command=0x55763a48, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x55763a88) at 
/home/dualbus/src/gnu/bash/execute_cmd.c:2707
| #11 0x5559f134 in execute_command_internal (command=0x55763a48, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x55763a88)
| at /home/dualbus/src/gnu/bash/execute_cmd.c:996
| #12 0x55609e4d in parse_and_execute (string=0x557632c8 "x=0; : 
${x/#[0�\\Z[:]]}", from_file=0x5566b0f0 "-c", flags=4)
| at /home/dualbus/src/gnu/bash/builtins/evalstring.c:458
| #13 0x55585632 in run_one_command (command=0x7fffebdc "x=0; : 
${x/#[0�\\Z[:]]}") at /home/dualbus/src/gnu/bash/shell.c:1424
| #14 0x5558477d in main (argc=3, argv=0x7fffe8f8, 
env=0x7fffe918) at /home/dualbus/src/gnu/bash/shell.c:735
| 
| (gdb) p str
| $1 = 0xdfdfdfdfdfdfdfdf 

I have been looking around but I don't understand what's going on. I can see
that the value of `str' comes from `e', which in turn comes from
`match_pattern', but it's not clear to me why this is happening.

Re: Segmentation fault in pat_subst

[Eduardo A . Bustamante López]

On Sun, Jul 21, 2019 at 06:56:09PM -0400, Chet Ramey wrote:
(...)
> Thanks for the report. Look at match_wpattern and consider what happens if
> wmatchlen returns something longer than the string length. It will be fixed
> for the next devel branch push.

Aha! I see.

| Starting program: /tmp/bash/bash -c x=0\;\ :\ \$\{x/\#\[0�\\Z\[:\]\]\}
| 
| Breakpoint 1, match_wpattern (wstring=0x556c1d90 L"0", 
indices=0x556c1e20, wstrlen=1, wpat=0x556c1d00 L"[0�\\Z[:]]", mtype=1, 
sp=0x7fffd918, ep=0x7fffd910)
| at /home/dualbus/src/gnu/bash/subst.c:4963
| 4963  if (mlen > wstrlen)
| (gdb) p mlen
| $1 = 2
| (gdb) p wstrlen
| $2 = 1

Interesting, I can see how that `mlen = 2' can cause trouble. Thank you for the 
pointer and the fix!