Doc of "set -e" should mention non-locality
Configuration Information [Automatically generated, do not change]:
Machine: i586
OS: linux-gnu
Compiler: gcc -I/usr/src/packages/BUILD/bash-3.2
-L/usr/src/packages/BUILD/bash-3.2/../readline-5.2
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i586'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i586-suse-linux-gnu'
-DCONF_VENDOR='suse' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL
-DHAVE_CONFIG_H -I. -I. -I./include -I./lib -march=i586 -mtune=i686
-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector
-funwind-tables -fasynchronous-unwind-tables -g -D_GNU_SOURCE -DRECYCLES_PIDS
-Wall -g -std=gnu89 -Wextra -Wno-unprototyped-calls -Wno-switch-enum -pipe
-fPIE -fprofile-use
uname output: Linux spatzenhorst 2.6.27.23-0.1-pae #1 SMP 2009-05-26 17:02:05
-0400 i686 i686 i386 GNU/Linux
Machine Type: i586-suse-linux-gnu
Bash Version: 3.2
Patch Level: 39
Release Status: release
Description:
The implementation of "set -e" does not respect "lexical nesting".
This can be very surprising. The documentation should explicitly
and clearly explain the behavior.
In a context of "set -e", with function "f() { x; false; y; }",
"f" will run only "x", while "f || true" will also run "y".
This is highly surprising.
The documentation states that "shell does not exit if the command that
fails is part of [a boolean expression]" (paraphrased). The problem
here is the phrase "is part of". At first glance, this appears to mean
"is lexically part of", because the wording refers to a number of
syntactical constructs. This is the relevant sentence in full:
The shell does not exit if the command that fails is part of the
command list immediately following a while or until keyword, part of
the test in an if statement, part of a && or || list, or if the
command's return value is being inverted via !.
However, "is part of" apparently does not mean "is lexically part of".
Instead, it means "is executed in the execution context of", as in "is
executed when a preceding stack frame represents".
Repeat-By:
set -e; f() { echo before; false; echo after; }; f || true; echo rc=$?;
prints:
before
after
rc=0
while many if not most programmers would expect it to print
before
rc=0
Fix:
If the effect is intended, clearly document it.
If not, change the implementation.
Note that some other shells, in particular ksh, implement the same
behavior.
--
NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone!
Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a
Re: Doc of "set -e" should mention non-locality
On Thursday, June 28, 2012 02:37:17 PM Rainer Blome wrote:
> The implementation of "set -e" does not respect "lexical nesting".
> This can be very surprising.
None of the "set" options do, nor does the ERR trap. That would make this the
exception. Here's a workaround (untested).
sete() {
[[ $- == *e* ]] && return 1
trap "$(
signature.asc
Description: This is a digitally signed message part.
Re: Doc of "set -e" should mention non-locality
Original-Nachricht
> Datum: Fri, 29 Jun 2012 18:03:13 -0500
> Von: Dan Douglas
> An: bug-bash@gnu.org
> CC: Rainer Blome
Remember that my main suggestion is to clearly document the intended
behavior (see the subject). This could mean to add a generic
paragraph to the documentation of "set" that describes the scope
and extent for all options. Something like this:
In Bash, "set" options act on a global scope per shell process,
they do not respect shell function scope.
The effect of turning on a "set" option starts after the "set -" command
completes.
The effect of turning off a "set" option starts after the "set +" command
completes.
For "set -e", I would add this:
Conditional statements such as while and if have the effect of temporarily
suppressing the effect of any "set -e"
in effect while the condition is evaluated.
Thus, when a shell function is run as part of the evaluation of a condition,
"set -e" will not be in effect while in the function.
Any use of "set -e" within such a shell function immediately changes
the global flags ($-), but will otherwise
take effect only after the condition is evaluated.
> On Thursday, June 28, 2012 02:37:17 PM Rainer Blome wrote:
> > The implementation of "set -e" does not respect "lexical nesting".
> > This can be very surprising.
>
> None of the "set" options do, nor does the ERR trap.
That may very well be. Is this documented anywhere?
It can still be surprising, for several reasons:
* The wording in the current documentation is easy to misunderstand.
It is not clear what "part" means.
In other programming languages, "part" often respects lexical nesting.
This makes it easy to make a false assumption.
See my first mail.
* Different POSIX shell implementations are free to behave
differently, because the behavior is not specified exactly.
http://pubs.opengroup.org/onlinepubs/009695399/utilities/set.html
does not mention scoping or extent, except in the section about "-e".
The section does not specify whether the commands that make up
a shell function are considered part of the calling command or not.
At least I did not see anything like that.
In my view, this point is under-specified,
or at least specified not clearly enough.
* Some shell implementations *do* behave differently from Bash
in this regard. At least on AIX, ksh and sh do so.
* The average shell programmer probably seldomly encounters
the difference, because it rarely matters.
Because of this, the issue is not well-known.
> That would make this the exception.
As I said, I suggest to clarify the documentation.
Regards, Rainer
PS: Your suggested "workaround" is, well, hard to understand.
Reminds me of the way people extend FORTH by massaging the stacks.
You have to know exactly what is parsed, substituted and evaluated
when in order to understand this (if it even works, did not try it).
I would not dare use this in production for fear of
receiving a beating from colleagues over hard to maintain code. ;-)
> Here's a workaround (untested).
>
> sete() {
> [[ $- == *e* ]] && return 1
> trap "$( } < if [[ $FUNCNAME != \$FUNCNAME ]]; then
> set +e
> trap - RETURN
> else
> set -e
> fi
> EOF
>
> This will "set -e" in only the scope of the caller. It gets a bit more
> complicated if you want to use this RETURN feature more extensively and
> preserve other possible RETURN traps, but most people won't have to worry
> about that.
Re: Doc of "set -e" should mention non-locality
On 07/04/2012 09:37 AM, Rainer Blome wrote:
>
> * Different POSIX shell implementations are free to behave
> differently, because the behavior is not specified exactly.
Actually, that's where you are wrong. POSIX _does_ specify bash's
current behavior (in fact, bash changed behavior to match POSIX)
> http://pubs.opengroup.org/onlinepubs/009695399/utilities/set.html
http://austingroupbugs.net/view.php?id=52
which even includes this example:
Application writers should avoid relying on set -e within
functions. For example, in the following script:
set -e
start() {
some_server
echo some_server started successfully
}
start || echo >&2 some_server failed
the -e setting is ignored within the function body (because the
function is a command in an AND-OR list other than the last).
Therefore if some_server fails, the function carries on to
echo "server started successfully", and the exit status of the
function is zero (which means "server failed" is not output).
> * Some shell implementations *do* behave differently from Bash
> in this regard. At least on AIX, ksh and sh do so.
Only because they haven't been patched to obey POSIX yet.
>
> * The average shell programmer probably seldomly encounters
> the difference, because it rarely matters.
Actually, the average shell programmer should avoid 'set -e', because of
its historical warts and non-intuitive behavior.
--
Eric Blake ebl...@redhat.com+1-919-301-3266
Libvirt virtualization library http://libvirt.org
signature.asc
Description: OpenPGP digital signature
Re: Doc of "set -e" should mention non-locality
Thanks for pointing me to that issue report. Original-Nachricht > Datum: Wed, 04 Jul 2012 10:47:11 -0600 > Von: Eric Blake > An: Rainer Blome > CC: Dan Douglas , bug-bash@gnu.org > On 07/04/2012 09:37 AM, Rainer Blome wrote: > > * Different POSIX shell implementations are free to behave > > differently, because the behavior is not specified exactly. > > http://pubs.opengroup.org/onlinepubs/009695399/utilities/set.html > Actually, that's where you are wrong. POSIX _does_ specify bash's > current behavior (in fact, bash changed behavior to match POSIX) Is POSIX an open and public standard? Can you please provide a link to where this is documented? The link I gave is apparently not the right one. > http://austingroupbugs.net/view.php?id=52 Thanks for pointing to that issue record. Is this part of the standard yet? > which even includes this example: > [clear doc omitted...] That is what I was suggesting in the first place, clear documentation. Hope it makes it into the standard docs, if it hasn't already. > > * Some shell implementations *do* behave differently from Bash > > in this regard. At least on AIX, ksh and sh do so. > > Only because they haven't been patched to obey POSIX yet. OK. Rainer
Re: Doc of "set -e" should mention non-locality
On 07/04/2012 12:03 PM, Rainer Blome wrote: > Thanks for pointing me to that issue report. > >>> * Different POSIX shell implementations are free to behave >>> differently, because the behavior is not specified exactly. >>> http://pubs.opengroup.org/onlinepubs/009695399/utilities/set.html >> Actually, that's where you are wrong. POSIX _does_ specify bash's >> current behavior (in fact, bash changed behavior to match POSIX) > > Is POSIX an open and public standard? Yes, anyone can contribute. From the POSIX document itself: http://pubs.opengroup.org/onlinepubs/9699919799/frontmatter/preface.html "Anyone wishing to participate in the Austin Group should contact the chair with their request. There are no fees for participation or membership. You may participate as an observer or as a contributor. You do not have to attend face-to-face meetings to participate; electronic participation is most welcome. For more information on the Austin Group and how to participate, see www.opengroup.org/austin." I personally call in to the majority of the meetings (generally a 90 minute teleconference about 2 or 3 times a month), while others on this list (such as Chet) take a more passive role and only subscribe to the mailing list to read the meeting minutes and provide feedback when directly asked how a potential change to the standard would impact existing implementations. > Can you please provide a link to where this is documented? > The link I gave is apparently not the right one. > The link you gave is the online version of POSIX 2008 (the .pdf is the canonical version; the online rendition is for convenience); but you should be aware that the Technical Corrigendum 1 for POSIX 2008 is due out later this year. >> http://austingroupbugs.net/view.php?id=52 This is one of the bugs that was approved for inclusion in the Technical Corrigendum, currently under balloting (you can see the full list of bugs that will be folded into TC1 by filtering the list to those bugs with the 'tc1-2008' tag). > > Thanks for pointing to that issue record. > Is this part of the standard yet? Assuming balloting goes well, it will be later this year, but bash proactively already implemented it. -- Eric Blake ebl...@redhat.com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: Doc of "set -e" should mention non-locality
On Wednesday, July 04, 2012 05:37:25 PM Rainer Blome wrote:
> Original-Nachricht
> > Datum: Fri, 29 Jun 2012 18:03:13 -0500
> > Von: Dan Douglas
> > An: bug-bash@gnu.org
> > CC: Rainer Blome
>
> Remember that my main suggestion is to clearly document the intended
> behavior (see the subject). This could mean to add a generic
> paragraph to the documentation of "set" that describes the scope
> and extent for all options.
I'm all for better documentation. Scope in Bash is a complex subject. Almost
none of it is documented, and there is little standardization around how
function scope is supposed to work anyway. I'd call set -e a low priority
relative to documenting what scope behaviors are actually in place.
> > On Thursday, June 28, 2012 02:37:17 PM Rainer Blome wrote:
> > > The implementation of "set -e" does not respect "lexical nesting".
> > > This can be very surprising.
> >
> > None of the "set" options do, nor does the ERR trap.
>
> That may very well be. Is this documented anywhere?
About the one thing you can count on with regards to scope in Bash, is that it
won't be lexical. But that's true of the majority of shells that have any kind
of scoping features at all beyond positional parameters and environment
variables.
> PS: Your suggested "workaround" is, well, hard to understand.
> Reminds me of the way people extend FORTH by massaging the stacks.
> You have to know exactly what is parsed, substituted and evaluated
> when in order to understand this (if it even works, did not try it).
> I would not dare use this in production for fear of
> receiving a beating from colleagues over hard to maintain code. ;-)
That's basically what it is (and this is yet another an undocumented scope
thing). Setting this trap within a function sets a hook on the function and
all of its callers which essentially runs eval on the given string upon
returning. To make matters more confusing, also potentially on callees if they
have the trace attribute set.
Other than that there's not much to understand or maintain. It'll set -e when
returning from the first function and set +e when returning from the second,
then unset itself on any further callers.
> > Here's a workaround (untested).
> >
> > sete() {
> > [[ $- == *e* ]] && return 1
> > trap "$( > } < > if [[ $FUNCNAME != \$FUNCNAME ]]; then
> > set +e
> > trap - RETURN
> > else
> > set -e
> > fi
> > EOF
> >
--
Dan Douglas
signature.asc
Description: This is a digitally signed message part.
Re: Doc of "set -e" should mention non-locality
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 7/5/12 11:55 AM, Dan Douglas wrote: >> Remember that my main suggestion is to clearly document the intended >> behavior (see the subject). This could mean to add a generic >> paragraph to the documentation of "set" that describes the scope >> and extent for all options. > > I'm all for better documentation. Scope in Bash is a complex subject. Almost > none of it is documented, and there is little standardization around how > function scope is supposed to work anyway. I'd call set -e a low priority > relative to documenting what scope behaviors are actually in place. There really isn't that much scope. Posix barely acknowledges the existence of scopes, period. It doesn't have local variables, options, or traps. The paragraph in the bash man page describing the behavior of the ERR, DEBUG, and RETURN traps, and the various descriptions of local variables, are about it. Bash's use of dynamic scoping is where the complexity comes in, I think, but the implementation is, I hope, consistent. Chet - -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/1u9AACgkQu1hp8GTqdKs19QCfRvkBQ0rUxgNklTCZYhliQv4j d8MAoIfmHmDq9y3EP3y2wxz0fCshO+bo =EmUI -END PGP SIGNATURE-
