RE: Bash crash
Hi Piotr, Actually when process com is not needed, "com.sh" will not be launched instead of commenting one line. I have never tried that. From the coredump log of 4.3, seems it crashed before line 187. (crashed at 114). Thank you. -Original Message- From: Piotr Grzybowski [mailto:narsil...@gmail.com] Sent: 2015年10月23日 1:08 To: Kai Wang X Cc: chet.ra...@case.edu; Aharon Robbins; bug-bash@gnu.org Subject: Re: Bash crash do I understand correctly, that when you comment the line 187 the issue is not existent? cheers, pg On Thu, Oct 22, 2015 at 4:39 AM, Kai Wang X wrote: > Hi all, > > Thank you all! > > The issue happens since we added a new process launched by a bash script. > Before that, no "sbrk issues" were found and hundreds of process including > scripts were running in my equipment. So it is hard for me to believe that > there are RAM or sbrk issues already exist on my system. Also bash is used by > more users world-wide, so ... I am being confused... > > The process named "com" launched by script command "com.sh start". Pls refer > to the attached files. It looks easy, doesn't it? > > @Piotr, > > Sure. Every equipment has the possibility of the issue. > > > -Original Message- > From: Piotr Grzybowski [mailto:narsil...@gmail.com] > Sent: 2015年10月21日 19:27 > To: Kai Wang X > Subject: Re: Bash crash > > hello, > > does the problem also appear on other pieces of equipment? I mean, different > phisical machine, of the same sort? > > pg > > > > > > On Wed, Oct 21, 2015 at 4:29 AM, Kai Wang X wrote: >> Hi Chet, >> >> Thank you for your response. >> >> But it does not make sense since sbrk failure will be checked: >> >> mp = (union mhead *) sbrk (sbrk_amt); >> >> /* Totally out of memory. */ >> if ((long)mp == -1) >> goto morecore_done; >> >> The script just runs when my equipment boots up. Also it is hard to >> reproduce in my environment. Only every few times of my equipment booting >> up, it generates a coredump file. >> >> -Original Message- >> From: Chet Ramey [mailto:chet.ra...@case.edu] >> Sent: 2015年10月20日 21:31 >> To: Kai Wang X; bug-bash@gnu.org >> Cc: chet.ra...@case.edu >> Subject: Re: Bash crash >> >> On 10/19/15 10:47 PM, Kai Wang X wrote: >>> Dear, >>> >>> >>> >>> We have two products which are using bash 4.2 and 4.3 separately. >>> They all meet bash crash issue. Please refer to the attached files. >>> >>> It is hard for me to understand the bash source code to find the >>> root cause out. >> >> It really looks like sbrk(2) is failing here, but since I don't have any way >> to reproduce it, that may not be it. This could be caused by your process >> exceeding its memory resource limit or your system's swap space being >> exhausted. >> >> Chet >> >> -- >> ``The lyf so short, the craft so long to lerne.'' - Chaucer >> ``Ars longa, vita brevis'' - Hippocrates >> Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Inconsistency in the handling of variables inside a function when using a tempenv variable
I found this some time ago, but didn't report it because I couldn't come up
with a patch:
dualbus@hp ~ % for sh in bash mksh zsh ksh93 dash; do $sh -c
't=${KSH_VERSION+typeset}; f() { x=3; ${t:-local} x; echo $x; }; [ "$(f)" =
"$(x=4 f)" ]'; echo $sh $?; done
bash 1
mksh 0
zsh 0
ksh93 0
dash 0
dualbus@hp ~ % bash -xc 'f() { x=3; local x; echo $x; }; f; x=4 f'
+ f
+ x=3
+ local x
+ echo
+ x=4
+ f
+ x=3
+ local x
+ echo 3
3
I'm putting it here so I don't forget about it.
Bash is the only one that handles this: tmpenv - global - local variable chain
in that way.
--
Eduardo Bustamante
https://dualbus.me/
Re: Bash-4.4-beta available for FTP
On 2015-10-15 16:23, Chet Ramey wrote: The first beta release of bash-4.4 is now available with the URL ftp://ftp.cwru.edu/pub/bash/bash-4.4-beta.tar.gz First attempt at build on AIX configure warns that bison is not available - good. configure finishes - good make fails immediately because yacc is not installed - not good make distclean; ./configure; ./make fails - need to reinstall tarball, so make distclean cleans up something ./configure does not prepare. Will continue to research and post more asap.
Re: Bash crash
do I understand correctly, that when you comment the line 187 the issue is not existent? cheers, pg On Thu, Oct 22, 2015 at 4:39 AM, Kai Wang X wrote: > Hi all, > > Thank you all! > > The issue happens since we added a new process launched by a bash script. > Before that, no "sbrk issues" were found and hundreds of process including > scripts were running in my equipment. So it is hard for me to believe that > there are RAM or sbrk issues already exist on my system. Also bash is used by > more users world-wide, so ... I am being confused... > > The process named "com" launched by script command "com.sh start". Pls refer > to the attached files. It looks easy, doesn't it? > > @Piotr, > > Sure. Every equipment has the possibility of the issue. > > > -Original Message- > From: Piotr Grzybowski [mailto:narsil...@gmail.com] > Sent: 2015年10月21日 19:27 > To: Kai Wang X > Subject: Re: Bash crash > > hello, > > does the problem also appear on other pieces of equipment? I mean, different > phisical machine, of the same sort? > > pg > > > > > > On Wed, Oct 21, 2015 at 4:29 AM, Kai Wang X wrote: >> Hi Chet, >> >> Thank you for your response. >> >> But it does not make sense since sbrk failure will be checked: >> >> mp = (union mhead *) sbrk (sbrk_amt); >> >> /* Totally out of memory. */ >> if ((long)mp == -1) >> goto morecore_done; >> >> The script just runs when my equipment boots up. Also it is hard to >> reproduce in my environment. Only every few times of my equipment booting >> up, it generates a coredump file. >> >> -Original Message- >> From: Chet Ramey [mailto:chet.ra...@case.edu] >> Sent: 2015年10月20日 21:31 >> To: Kai Wang X; bug-bash@gnu.org >> Cc: chet.ra...@case.edu >> Subject: Re: Bash crash >> >> On 10/19/15 10:47 PM, Kai Wang X wrote: >>> Dear, >>> >>> >>> >>> We have two products which are using bash 4.2 and 4.3 separately. >>> They all meet bash crash issue. Please refer to the attached files. >>> >>> It is hard for me to understand the bash source code to find the root >>> cause out. >> >> It really looks like sbrk(2) is failing here, but since I don't have any way >> to reproduce it, that may not be it. This could be caused by your process >> exceeding its memory resource limit or your system's swap space being >> exhausted. >> >> Chet >> >> -- >> ``The lyf so short, the craft so long to lerne.'' - Chaucer >> ``Ars longa, vita brevis'' - Hippocrates >> Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Dynamic variable failure & equiv-const strings compare unequal
On 10/22/15 8:13 AM, Linda Walsh wrote: > > > Oleg Popov wrote: >> On Thu, Oct 22, 2015 at 03:01:06AM -0700, Linda Walsh wrote: >>> [cut] >>> I.e. test output was: >>> Case 2 got/Expected: >>> "222" >>> "1\ 222\ .3\ .4" >>> [cut] >> >> You didn't initialize the array. By the time you do "parts[1]=222" it's >> still empty. And in your previous message you tried to initialize it in a >> subshell. Variables don't retain their values after returning from >> subshells. > > I was testing if dynamic scoping included subshells, I didn't think so, > but that doesn't mean I don't test it. I removed > it though, as it confused the example. You don't show what you `removed', so I am looking at the original script you posted. > ip and 'parts' are both initialized in global. Yes. parts is assigned the empty array at the global scope. > testor calls (tst0, tst1, tst2 & tst3). In subshells started to run command substitution. Those subshells modify `parts', but the changed value is not reflected in the parent shell. > > tst0 & tst1 both call "assignparts" which uses the global > value of $ip to set the global value of parts. I.e. since > neither "ip" nor 'parts' are declared inside of any of the functions, they > should use the top-level global values, no? You. Run. Your. Tests. In. Subshells. > tst2, using the last global value set in tst1, only tries to > change 1 value in 'parts'... i.e. why would 'ip' reference the > global value of 'ip', but not parts? You modify parts in a subshell. The global value of parts is initialized to and remains an empty array. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Dynamic variable failure & equiv-const strings compare unequal
Oleg Popov wrote: $(...) is a subshell. Variables cannot be passed back from a subshell, no matter how and where they are declared. --- Um... oh.. in testor, still calling that way. I missed that. This is even more annoying for passing back results than I thought. Grrr.
Re: Dynamic variable failure & equiv-const strings compare unequal
On Thu, Oct 22, 2015 at 03:01:06AM -0700, Linda Walsh wrote:
> [cut]
> I.e. test output was:
> Case 2 got/Expected:
> "222"
> "1\ 222\ .3\ .4"
> [cut]
You didn't initialize the array. By the time you do "parts[1]=222" it's
still empty. And in your previous message you tried to initialize it in
a subshell. Variables don't retain their values after returning from
subshells.
> [cut]
> test 3 --- was supposed to check the values
> of the hash field-by-field, but first I thought to check
> them when printed out in a specific order as 2 identical
> strings. The strings look identical, yet don't compare equal.
> so it seemed pointless to try to break down test 3's output
> into a field-by-field comparison when I couldn't even get
> the identical strings that contained the fields to compare.
> [cut]
> [cut]
>if [[ $out != $exp ]]; then
> [cut]
Bash uses unquoted characters on the right side of == and != as wildcard
patterns. For example, [ipaddr] in $exp means "any of i, p, a, d, r".
You should quote the right operand.
> #!/bin/bash
>
> shopt -s expand_aliases ; alias my=declare
> alias array='my -a' int='my -i'
>
> ip=10.20.30.40
> array parts=()
> array answers=( '10 20 30 40' '1 .2 .3 .4' '1 222 .3 .4'
> '192.168.0.1/24::{ [ipaddr]="192.168.0.1/24" [address]="192.168.0.1"
> [prefixlen]="24" ; }'
> )
>
>
> array addr_fields=(ipaddr address prefixlen)
>
> assignparts() { parts=( $ip ) ; }
>
> tst0 () { my IFS=. ; assignparts ; echo -E "${parts[@]}"; }
> tst1 () { my IFS=0 ; assignparts ; echo -E "${parts[@]}"; }
> tst2 () { parts[1]=222; echo -E "${parts[@]}" ; }
>
> tst3 () {
> my ipaddr="$1"; shift;
> int status=0
> my pat='^([^/]+)/([0-9]+)\s*$'
> my address prefixlen
> if [[ ! $ipaddr =~ $pat ]]; then
>echo >&2 "Error in ip/netsize format: \"$ipaddr\""
>status=1
> else
>address=${BASH_REMATCH[1]}
>prefixlen=${BASH_REMATCH[2]}
>my out=""
>for flds in "${addr_fields[@]}"; do
> out+="[$flds]=\"${!flds}\" "
>done
>printf "{ %s; }" "$out"
> fi
> return $status
> }
>
>
>
> int passno=0 tests=0
> my fmt="Case %d got/Expected:\n \"%q\"\n \"%q\"\n"
>
>
> testor () {
> int tstno
> my out=""
> for tstno in {0..3}; do
>tests+=1
>exp="${answers[tstno]}"
>if [[ $exp =~ :: ]]; then
> my args="${exp%::*}"
> exp="${exp#*::}"
> out="$(tst$tstno $args)"
>else
> out="$(tst$tstno)"
>fi
>if [[ $out != $exp ]]; then
> printf >&2 "$fmt" "$tstno" "$out" "$exp"
> continue
>fi
>passno+=1
> done
> }
>
> testor
> echo "Passed $passno/$tests tests."
>
>
> output:
> Case 2 got/Expected:
> "222"
> "1\ 222\ .3\ .4"
> Case 3 got/Expected:
> "\{\ \[ipaddr\]=\"192.168.0.1/24\"\ \[address\]=\"192.168.0.1\"\
> \[prefixlen\]=\"24\"\ \;\ \}"
> "\{\ \[ipaddr\]=\"192.168.0.1/24\"\ \[address\]=\"192.168.0.1\"\
> \[prefixlen\]=\"24\"\ \;\ \}"
> Passed 2/4 tests.
>
> The outputs for case 3 look identical -- I was using %s to print
> them out, but switched to "%q", to ensure no hidden chars...
>
> case 2 --
> ??? Why didn't it only change the 1 member?
>
>
>
>
>
>
>
Re: Dynamic variable failure & equiv-const strings compare unequal
On Thu, Oct 22, 2015 at 05:13:45AM -0700, Linda Walsh wrote: > Oleg Popov wrote: > > On Thu, Oct 22, 2015 at 03:01:06AM -0700, Linda Walsh wrote: > >> [cut] > >> I.e. test output was: > >> Case 2 got/Expected: > >> "222" > >> "1\ 222\ .3\ .4" > >> [cut] > > > > You didn't initialize the array. By the time you do "parts[1]=222" it's > > still empty. And in your previous message you tried to initialize it in > > a subshell. Variables don't retain their values after returning from > > subshells. > > I was testing if dynamic scoping included subshells, I > didn't think so, but that doesn't mean I don't test it. I removed > it though, as it confused the example. > > ip and 'parts' are both initialized in global. > > testor calls (tst0, tst1, tst2 & tst3). > > tst0 & tst1 both call "assignparts" which uses the global > value of $ip to set the global value of parts. I.e. since > neither "ip" nor 'parts' are declared inside of any of the > functions, they should use the top-level global values, no? > > tst2, using the last global value set in tst1, only tries to > change 1 value in 'parts'... i.e. why would 'ip' reference the > global value of 'ip', but not parts? > > ip and parts are declared at the same scope (global), so why > wouldn't the global 'parts' be initialized as well? $(...) is a subshell. Variables cannot be passed back from a subshell, no matter how and where they are declared. > > Bash uses unquoted characters on the right side of == and != as > > wildcard patterns. For example, [ipaddr] in $exp means "any of i, p, > > a, d, r". You should quote the right operand. > --- > Ahh... or if I quote both sides > using 'printf "%q"' first, that should do the same, no? No. Just use double quotes: [[ $var1 == "$var2" ]] > So, ok, I get that one -- but the first looks sketchy, > as ip and parts are both, only defined at the global level, > thus my expectation that just like it used the global value of > 'ip' for tst0 & tsts1 -- it should have stored the split version > of it in the global value of parts... I really don't get why > it would use the global value as a dynamic in 1 case but not > the other...?
Re: Bash crash
On Thu, Oct 22, 2015 at 02:39:43AM +, Kai Wang X wrote:
> The process named "com" launched by script command "com.sh start". Pls refer
> to the attached files. It looks easy, doesn't it?
It looks like a horrible mish-mash of legacy Bourne shell syntax,
edited later by another person using certain post-Bourne shell constructs
(note the inconsistent use of test and [ commands), with quoting failures
thrown in for good measure (lots of [ -e ${foo} ] commands).
I cringed at the `expr` all over the place, and stopped reading halfway
through.
The shebang is #!/bin/sh and it's using half-Bourne half-POSIX code, so
if you suspect a bug in bash, why not just let it run under some other
shell? /bin/sh does not link to bash on most operating systems. Not
even on all of the Linux-based operating systems.
Re: language inconsistency(wart) & RFE
On Wed, Oct 21, 2015 at 07:19:30PM -0700, Linda Walsh wrote:
> Not to mention your statement doesn't work:
My code worked. I tested it. If you take working code and mangle it
according to your incredibly bizarre notions of how bash code should
look, and it doesn't work, then the problem is on your end.
> function nint {
> local i j k
> for ((i=0; i<10; ++i)) ; do j+=i; k+=j ;done
THAT. IS. NOT. HOW. YOU. DO. ARITHMETIC.
You want to perform integer addition? Then you use the syntax that
specifies integer addition.
j+=i is the syntax that specifies string concatenation.
To perform arithmetic, you use $((..)) or ((..)) or let.
nint() {
local i j k
for ((i=0; i<10; i++)); do ((j+=i, k+=j)); done
declare -p i j k
}
imadev:~$ nint
declare -- i="10"
declare -- j="45"
declare -- k="165"
I have stated this before, but apparently it didn't sink it, so I'll
try again in all caps: DO NOT USE declare -i EVER.
If you use declare -i then you BREAK the ability to read your script and
see what a command is doing. You pervert the string concatention syntax
into performing integer addition, but only some of the time, and you can't
tell which times that is. (OO people call this "overloading" and think
that it's a feature rather than an abomination.) (I do not and never
will understand OO people. They must have been brainwashed.)
So now that you know not to use declare -i, all of your other problems
just go away!
You use local when you want local variables, and you don't use any
declaration at all when you want global (or dynamically scoped)
variables.
That provides the ability you have been asking for: to decide on a
case-by-case basis which variables will be local and which will not.
The end.
Re: Dynamic variable failure & equiv-const strings compare unequal
Oleg Popov wrote: On Thu, Oct 22, 2015 at 03:01:06AM -0700, Linda Walsh wrote: [cut] I.e. test output was: Case 2 got/Expected: "222" "1\ 222\ .3\ .4" [cut] You didn't initialize the array. By the time you do "parts[1]=222" it's still empty. And in your previous message you tried to initialize it in a subshell. Variables don't retain their values after returning from subshells. I was testing if dynamic scoping included subshells, I didn't think so, but that doesn't mean I don't test it. I removed it though, as it confused the example. ip and 'parts' are both initialized in global. testor calls (tst0, tst1, tst2 & tst3). tst0 & tst1 both call "assignparts" which uses the global value of $ip to set the global value of parts. I.e. since neither "ip" nor 'parts' are declared inside of any of the functions, they should use the top-level global values, no? tst2, using the last global value set in tst1, only tries to change 1 value in 'parts'... i.e. why would 'ip' reference the global value of 'ip', but not parts? ip and parts are declared at the same scope (global), so why wouldn't the global 'parts' be initialized as well? Bash uses unquoted characters on the right side of == and != as wildcard patterns. For example, [ipaddr] in $exp means "any of i, p, a, d, r". You should quote the right operand. --- Ahh... or if I quote both sides using 'printf "%q"' first, that should do the same, no? So, ok, I get that one -- but the first looks sketchy, as ip and parts are both, only defined at the global level, thus my expectation that just like it used the global value of 'ip' for tst0 & tsts1 -- it should have stored the split version of it in the global value of parts... I really don't get why it would use the global value as a dynamic in 1 case but not the other...?
Dynamic variable failure & equiv-const strings compare unequal
Greg Wooledge wrote:
Also I think you are completely misrepresenting the dynamic variable
scope system that bash uses. Variables are not just global or local.
There's an entire stack of them. When you reference a
variable (let's say i) inside a function, bash searches up
through the call stack looking for a variable named
i until it finds one.
So where in the manpage is the behavior documented -- I'd like
to reread it to ensure I don't have any misconceptions.
The reason being is that out of 4 tests below,
2 fail. It appears test 2 failing is a failure of the
dynamic scoping system.
Test 3 was meant to be an extension, as I have some
code where I change a member in a hash, and try to compare
the final result with a known correct value, but as in test-2
all the other members of the hash disappear
Notice that 'parts' is defined at the global level as an array.
in tst0 & tst1, I locally change the value of IFS, and
call "assignparts" that splits the value "$ip" based on
the "localized" values of IFS -- tst0 + tst1 work as I'd
expect with dynamic scoping. Since parts was declared 'global',
the different splitting caused by differing 'IFS' values is
reflected in the global.
However, in tst3, I only change 1 member in the parts
array, via "parts[1]=222".
before the assignment, parts=(1 .2 .3 .4)
but after the assignment "parts[1]=222",
the entire array was wiped.
I.e. test output was:
Case 2 got/Expected:
"222"
"1\ 222\ .3\ .4"
instead of it only replacing the 2nd member of parts,
the entire array was wiped.
The same happens if parts is a 'hash' -- trying
to only replace 1 member of the hash results in the
entire hash being wiped.
I don't see how this is correct function for
"dynamically scoped" variables -- which is why
I need to reread (or read) the manpage descriptions of
how dynamic vars are *supposed* to work, as they don't
seem to be working by "unwinding" the call stack and using
the last definition of 'parts' (which is at the global level).
test 3 --- was supposed to check the values
of the hash field-by-field, but first I thought to check
them when printed out in a specific order as 2 identical
strings. The strings look identical, yet don't compare equal.
so it seemed pointless to try to break down test 3's output
into a field-by-field comparison when I couldn't even get
the identical strings that contained the fields to compare.
tests 2 & 3 work backward from a program where I try
to change 1 value in a hash (vs. the array in test2), but
end up with the local assignments of values->[keys] not
being propagated, at all, to the global frame where the has
is declared.
I.e. these examples of dynamic variable declarations and usage
don't seem to work as I understand the concept, but there is
nothing about bash's 'dynamic variables' in the manpage.
-l
-
#!/bin/bash
shopt -s expand_aliases ; alias my=declare
alias array='my -a' int='my -i'
ip=10.20.30.40
array parts=()
array answers=( '10 20 30 40' '1 .2 .3 .4' '1 222 .3 .4'
'192.168.0.1/24::{ [ipaddr]="192.168.0.1/24" [address]="192.168.0.1"
[prefixlen]="24" ; }'
)
array addr_fields=(ipaddr address prefixlen)
assignparts() { parts=( $ip ) ; }
tst0 () { my IFS=. ; assignparts ; echo -E "${parts[@]}"; }
tst1 () { my IFS=0 ; assignparts ; echo -E "${parts[@]}"; }
tst2 () { parts[1]=222; echo -E "${parts[@]}" ; }
tst3 () {
my ipaddr="$1"; shift;
int status=0
my pat='^([^/]+)/([0-9]+)\s*$'
my address prefixlen
if [[ ! $ipaddr =~ $pat ]]; then
echo >&2 "Error in ip/netsize format: \"$ipaddr\""
status=1
else
address=${BASH_REMATCH[1]}
prefixlen=${BASH_REMATCH[2]}
my out=""
for flds in "${addr_fields[@]}"; do
out+="[$flds]=\"${!flds}\" "
done
printf "{ %s; }" "$out"
fi
return $status
}
int passno=0 tests=0
my fmt="Case %d got/Expected:\n \"%q\"\n \"%q\"\n"
testor () {
int tstno
my out=""
for tstno in {0..3}; do
tests+=1
exp="${answers[tstno]}"
if [[ $exp =~ :: ]]; then
my args="${exp%::*}"
exp="${exp#*::}"
out="$(tst$tstno $args)"
else
out="$(tst$tstno)"
fi
if [[ $out != $exp ]]; then
printf >&2 "$fmt" "$tstno" "$out" "$exp"
continue
fi
passno+=1
done
}
testor
echo "Passed $passno/$tests tests."
output:
Case 2 got/Expected:
"222"
"1\ 222\ .3\ .4"
Case 3 got/Expected:
"\{\ \[ipaddr\]=\"192.168.0.1/24\"\ \[address\]=\"192.168.0.1\"\
\[prefixlen\]=\"24\"\ \;\ \}"
"\{\ \[ipaddr\]=\"192.168.0.1/24\"\ \[address\]=\"192.168.0.1\"\
\[prefixlen\]=\"24\"\ \;\ \}"
Passed 2/4 tests.
The outputs for case 3 look identical -- I was using %s to print
them out, but switched to "%q", to ensure no hidden chars...
case 2 --
??? Why didn't it only change the 1 member?
Re: Design question(s), re: why use of tmp-files or named-pipes(/dev/fd/N) instead of plain pipes?
On Thu, Oct 22, 2015 at 5:57 AM, Linda Walsh wrote: > > But only as a pointer to something one can do I/O on. > You can't set any file attributes or metadata on "pipe:[]" It's not a > real file somewhere. > Yes, it's not a regular file, but it not the less true that <( ) gives you a string that can be used by applications that were written to deal with regular files whose names are passed as an argument., or in cases where you need more than one stream. eg you can compare the output of two commands like: cmp <(sort fileA) <(sort fileB) cmp will receive 2 strings as arguments, use these strings as filenames and will open the special files just like if they were regular files. Yes they are not regular, yes they can not bee seeked and the trick will not work in all cases, but for cmp it will work just as well as with regular files.
