Re: -e does not take effects in subshell
On jue, 20-08-2015 a las 17:38 -0700, Linda Walsh wrote: > Functions are a collection of many commands. They are not a single, > simple statement. I remember using their return value in some cases. > > With the change, I couldn't run a func and have it return the > value in $? (1 byte, I know) -- but about 128x as flexible > as "ok"/"die" (i.e. any non-zero value triggers the behavior > now, but before, it didn't). > > My simplistic view was that -e was there to auto-exit if an external > command failed because they are "out of your control". But if > I write a function -- I design the behavior. Even if I designed in > a bug -- it's still "my code" that has the problem not some > -_e_xternal command If you design the function, you can put an exit 0 on them, so they never return a non-zero status. It is completely sensible that if "echo foo > /dev/full" fails, it behaves the same way no matter if it's /bin/echo or a builtin what is run by "echo". Note that Windows has a similar deviation between binaries and batch scripts. A .bat containing: foo bar will run foo.exe then bar.exe if there's an executable named foo.exe, but only foo.bat will be run if it happens to be a .bat script (ie. acts as if prefixed by exec(1) when it's a .bat). And you can't know beforehand (even worse, someone may have installed a .bat with the same name as a .exe in order to slightly augment it). Treating all of them consistently is the way to go. > cf. perl's option "autodie" -- you can say you want the "open > builtin" to just die on errors, then for simple scripts you don't > have > to put extra code to handle each problem. I.e. -- it's not really > something you want in production code, but I write far more throw > away quick and dirty scripts than production ones. I would to like to have a way to automatically set -e all functions defined after that, but it's orthogonal to treating them as commands. >
Re: -e does not take effects in subshell
On 8/20/15 8:38 PM, Linda Walsh wrote: > > > Chet Ramey wrote: >>> The earlier spec had -e only exit a script if a *simple* (external) >>> command failed. It didn't include builtins nor functions. >> >> This is not; builtins and functions are simple commands. > --- > The builtins are _complex_ binary blobs that replace external commands. > > Functions are a collection of many commands. They are not a single, > simple statement. I remember using their return value in some cases. `Simple command' is a specific term with a specific meaning. It doesn't have anything to do with perceived implementation complexity. > > With the change, I couldn't run a func and have it return the > value in $? (1 byte, I know) -- but about 128x as flexible as "ok"/"die" > (i.e. any non-zero value triggers the behavior > now, but before, it didn't). That's fine, even clever, but fundamentally incompatible with the idea that 0 means success and everything else means failure. > My simplistic view was that -e was there to auto-exit if an external > command failed because they are "out of your control". That's an incomplete view. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: -e does not take effects in subshell
Chet Ramey wrote: The earlier spec had -e only exit a script if a *simple* (external) command failed. It didn't include builtins nor functions. This is not; builtins and functions are simple commands. --- The builtins are _complex_ binary blobs that replace external commands. Functions are a collection of many commands. They are not a single, simple statement. I remember using their return value in some cases. With the change, I couldn't run a func and have it return the value in $? (1 byte, I know) -- but about 128x as flexible as "ok"/"die" (i.e. any non-zero value triggers the behavior now, but before, it didn't). My simplistic view was that -e was there to auto-exit if an external command failed because they are "out of your control". But if I write a function -- I design the behavior. Even if I designed in a bug -- it's still "my code" that has the problem not some -_e_xternal command cf. perl's option "autodie" -- you can say you want the "open builtin" to just die on errors, then for simple scripts you don't have to put extra code to handle each problem. I.e. -- it's not really something you want in production code, but I write far more throw away quick and dirty scripts than production ones. I tried to write a more complex bash script one time -- that met my expectations. W/o error testing the code was about 150 lines. With error testing and _helpful_ error messages it was over 1000. But that's the difference between quick&dirty vs. production. -e was useful for Q&D, IMO
Re: -e does not take effects in subshell
On 8/19/15 5:58 PM, Linda Walsh wrote: > > > Greg Wooledge wrote: >> >> (Wow, how did we get here from "-e does not take effects in subshell"?) >> > --- > because the POSIX spec changed and bash's handling of "-e" > changed to follow the new spec. This is true, though I would have used `revised' instead of `new'. > The earlier spec had -e only exit a script if a *simple* (external) > command failed. It didn't include builtins nor functions. This is not; builtins and functions are simple commands. The Posix spec changed because it didn't accurately reflect historical behavior. There was a *lot* of discussion about how to accurately describe the desired behavior, but everyone agreed that restricting it to simple commands was not how the `base implementations' behaved. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: -e does not take effects in subshell
On Tue, Aug 18, 2015 at 03:31:10PM -0700, Linda Walsh wrote: > with 'rm' functionality to remove '/' '.' and '..' was prohibited > by POSIX, though the coreutils version still allows the choice > of the more dangerous removal of '/' with with the --[no-]preserve-root. > > But the more useful "rm -fr ." [...] OK. I think you are saying that the POSIX specification sentence If either of the files dot or dot-dot are specified as the basename portion of an operand (that is, the final pathname component) or if an operand resolves to the root directory, rm shall write a diagnostic message to standard error and do nothing more with such operands. conflicts with your prior use of the GNU rm --one-file-system extension as a shorthand for "find . -xdev -delete". Since GNU already has a --no-preserve-root extension, as you pointed out, I don't see what would stop them from adding another extension to permit GNUrm -rfx . to work. Perhaps you should come up with an appropriate syntax for it and submit a patch to the GNU coreutils maintainers. This is not related to bash. (Wow, how did we get here from "-e does not take effects in subshell"?)
Re: -e does not take effects in subshell
Greg Wooledge wrote: (Wow, how did we get here from "-e does not take effects in subshell"?) --- because the POSIX spec changed and bash's handling of "-e" changed to follow the new spec. The earlier spec had -e only exit a script if a *simple* (external) command failed. It didn't include builtins nor functions.
Re: -e does not take effects in subshell
Greg Wooledge wrote:
On Tue, Aug 18, 2015 at 01:49:53PM -0700, Linda Walsh wrote:
Ex: rmx -fr (alias to rm --one-file-system -fr, since rm lacks the
-x switch like 'find, cp, mv, et al.) no longer works to clean
out a directory && stay on *one* file system.
When did POSIX or any historical Unix rm have a --one-file-system option?
You say "no longer works" as if it had EVER worked in the past.
---
Historically, linux had it going back to early 2000's
(linux being a *nix platform) -- but historically, it wasn't
so easy to have features like 'bind/rbind', snapshots, multiple
virtual machines that need their own root (or chroot), etc. If
you go back far enough symlinks weren't even around.
I'm only talking about POSIX ~2001 or before. After that
it started changing. So it depends on how historical you are talking.
POSIX cmd language started with POSIX.2 in 1992, before that it
was purely a programming API. It started including the cmd's as
a way of providing portable shell scripts. Not as a way of
restricting users.
While POSIX changed the 'rm' algorithm to no longer do
depth-first removal (now it's 2-pass, depth-first permissions check,
then depth-first removal). But that's not the behavior of the historical
'rm'.
Various "one-file-system" cp -x, find -x, du -x were added after
it became common to allow more complicated mount structures.
I remember an early version of cygwin-coreutils-rm on Win7 that
didn't recognize symlinks or mountpoints (linkd/junctions) wandering
up out of the "C:\recycle bin" over to a documents folder on another
computer...
Daily-backups do come in handy.
And yes, the standard way to do this (the only way with traditional
tools) would use find ... -xdev ... -exec rm {} +
---
Which won't reliably work if your starting path is "pathname/."
but would with an rm -frx (or rmx -fr path/.").
Re: -e does not take effects in subshell
Andreas Schwab wrote:
Linda Walsh writes:
Ex: rmx -fr (alias to rm --one-file-system -fr, since rm lacks the
-x switch like 'find, cp, mv, et al.) no longer works to clean
out a directory && stay on *one* file system.
Now rm will delete things on any number of file systems, as long
as they correspond to a cmdline argument.
That's the only sensible way to implement it. Which, incidentally,
works exactly like find -xdev.
---
with 'find', you can specify -xdev with a starting path of "."
with 'rm' functionality to remove '/' '.' and '..' was prohibited
by POSIX, though the coreutils version still allows the choice
of the more dangerous removal of '/' with with the --[no-]preserve-root.
But the more useful "rm -fr ." or the variant "rm -fr dir/." so you
know you are removing the contents of "dir", no matter where or what
"dir" is... with find, that doesn't work -- if 'dir' is a symlink
to /tmp/dir/.., find won't remove anything. Any other solution
from POSIX added complication. I was told by a BSD fanatic that 'rm'
was changed because after the SysV companies left POSIX (as most of
them had disappeared), BSD'ers gained a majority and could redirect
the standard as they pleased. Disallowing students playing around with
"rm -fr {/,dir/,}{.,..}" apparently was a big thing @Berkeley. Being
able to force the removal of such options from everyone's "rm" was
an huge win, they considered (this is from a discussion w/one fanatic,
but boy, was it memorable). Disallowing any option or ENV(POSIX_CORRECTLY)
option to "re-allow" the feature has been continuously shot down by
'rm' maintainers (even though they keep in their own alias-able switches
to all removal of '/').
Now please explain what this has anything to do with POSIX.
It apparently was the POSIX 2008 standard that prohibited the historical
behavior (on linux -- removed dir contents, and failed on current-dir
because it made no sense -- but did so *quietly* and after following
the depth first design.
Re: -e does not take effects in subshell
Linda Walsh writes: > Ex: rmx -fr (alias to rm --one-file-system -fr, since rm lacks the > -x switch like 'find, cp, mv, et al.) no longer works to clean > out a directory && stay on *one* file system. > > Now rm will delete things on any number of file systems, as long > as they correspond to a cmdline argument. That's the only sensible way to implement it. Which, incidentally, works exactly like find -xdev. Now please explain what this has anything to do with POSIX. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Re: -e does not take effects in subshell
On Tue, Aug 18, 2015 at 01:49:53PM -0700, Linda Walsh wrote:
> Ex: rmx -fr (alias to rm --one-file-system -fr, since rm lacks the
> -x switch like 'find, cp, mv, et al.) no longer works to clean
> out a directory && stay on *one* file system.
When did POSIX or any historical Unix rm have a --one-file-system option?
You say "no longer works" as if it had EVER worked in the past.
And yes, the standard way to do this (the only way with traditional
tools) would use find ... -xdev ... -exec rm {} +
Re: -e does not take effects in subshell
Eric Blake wrote: Like it or not, it is the historical behavior standardized by POSIX. This is not true. POSIX no longer documents historical behavior, but now dictates new, historically-incompatible behaviors for a variety of features in a variety of products (not just BASH). As such, since the original mission statement of POSIX was to be *descriptive* of what was (so a compatible standard could be provided), and that is NOT what the new POSIX (post 2001-2003) has as a mission statement, I assert the new "POSIX" is simply a new organization that got the rights to use the name but use it as a "club" to force products to their new, dumbed-down and maladaptive behaviors. Ex: rmx -fr (alias to rm --one-file-system -fr, since rm lacks the -x switch like 'find, cp, mv, et al.) no longer works to clean out a directory && stay on *one* file system. Now rm will delete things on any number of file systems, as long as they correspond to a cmdline argument. Many people said to use rm -xfr * to delete contents... but each object in 'rm' can be on a different file system. Worse "rm -xfr **". The workaround -- to use non-posix options of 'find' (or have find call 'rm' for each qualified object. Please don't spread the lies that the *current* POSIX specs only reflect historical behavior because it is not true. It is NOT intuitive, and our advice is "DON'T USE set -e - IT WON'T DO WHAT YOU WANT". We can't change the behavior, because it would break scripts that rely on the POSIX-specified behavior. === I used the old behavior for over 10 years in various SH-compat shells, and it *WAS* useful. POSIX changed it to be unuseful.
Re: -e does not take effects in subshell
On 8/13/15 1:35 AM, isabella parakiss wrote: > On 8/13/15, Eric Blake wrote: >> Like it or not, it is the historical behavior standardized by POSIX. It >> is NOT intuitive, and our advice is "DON'T USE set -e - IT WON'T DO WHAT >> YOU WANT". We can't change the behavior, because it would break scripts >> that rely on the POSIX-specified behavior. >> > POSIX didn't say anything about shopt set-e-that-actually-makes-sense. Sure, all it needs is a specification with the same rigor as Posix's. There have been a couple of attempts to do that so far, but they haven't been articulated well enough to differentiate from set -e or to implement. I encourage folks to take another shot at specifying their requirements for something `that actually makes sense', and `not sucking' isn't sufficient. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: -e does not take effects in subshell
On 8/13/15, Eric Blake wrote: > Like it or not, it is the historical behavior standardized by POSIX. It > is NOT intuitive, and our advice is "DON'T USE set -e - IT WON'T DO WHAT > YOU WANT". We can't change the behavior, because it would break scripts > that rely on the POSIX-specified behavior. > POSIX didn't say anything about shopt set-e-that-actually-makes-sense. --- xoxo iza
Re: -e does not take effects in subshell
On 08/12/2015 09:54 PM, PRC wrote: > 1. If the function is complicated and contains lots of commands, it > would be more troublesome appending && to every command rather > than simply using one single "set -e" at the front. > 2. It is not > reasonable that suppression of '-e' is applied as well inside the > function. My intention is if my_build succeeds continue to do other > stuff. Does anyone like the rule that a function never fails only if it is > part of a > compound command? Like it or not, it is the historical behavior standardized by POSIX. It is NOT intuitive, and our advice is "DON'T USE set -e - IT WON'T DO WHAT YOU WANT". We can't change the behavior, because it would break scripts that rely on the POSIX-specified behavior. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
RE: -e does not take effects in subshell
1. If the function is complicated and contains lots of commands, it
would be more troublesome appending && to every command rather
than simply using one single "set -e" at the front.
2. It is not
reasonable that suppression of '-e' is applied as well inside the
function. My intention is if my_build succeeds continue to do other
stuff. Does anyone like the rule that a function never fails only if it is part
of a
compound command?
> Date: Tue, 11 Aug 2015 09:50:56 -0400
> From: wool...@eeg.ccf.org
> To: ijk...@msn.com
> CC: bug-bash@gnu.org
> Subject: Re: -e does not take effects in subshell
>
> On Tue, Aug 11, 2015 at 11:42:29AM +, PRC wrote:
> > mybuild()
> > {
> > (
> > set -e
> > make
> > echo "build okay"
> > )
> > }
> >
> > mybuild && do_other_stuff
>
> http://mywiki.wooledge.org/BashFAQ/105
>
> Since mybuild is invoked as part of a compound command, set -e is
> suppressed. I guess this applies not only to set -e that's invoked
> beforehand, but even to set -e that's set within the compound command.
>
> Stop using set -e and all of these problems simply go away.
>
> mybuild() {
> make && echo "build okay"
> }
>
> mybuild && do_other_stuff
Re: -e does not take effects in subshell
On Tue, Aug 11, 2015 at 11:42:29AM +, PRC wrote:
> mybuild()
> {
> (
> set -e
> make
> echo "build okay"
> )
> }
>
> mybuild && do_other_stuff
http://mywiki.wooledge.org/BashFAQ/105
Since mybuild is invoked as part of a compound command, set -e is
suppressed. I guess this applies not only to set -e that's invoked
beforehand, but even to set -e that's set within the compound command.
Stop using set -e and all of these problems simply go away.
mybuild() {
make && echo "build okay"
}
mybuild && do_other_stuff
-e does not take effects in subshell
I have a export function looking like:
mybuild()
{
(
set -e
make
echo "build okay"
)
}
I wish to use this function this way:
mybuild && do_other_stuff
But whatever (success or failure) make returns "build okay" is always printed
and do_other_stuff always gets executed, which is my expectation.
I have found below descriptions on bash manual.
-e Exit immediately if a pipeline (which may consist of a
single simple command), a subshell command enclosed in parentheses, or one of
the
commands executed as part of a command list enclosed by
braces (see SHELL GRAMMAR above) exits with a non-zero status. The shell does
not
exit if the command that fails is part of the command
list immediately following a while or until keyword, part of the test following
the if
or elif reserved words, part of any command executed in a
&& or || list except the command following the final && or ||, any command in
a
pipeline but the last, or if the command's return
value is being inverted with !. A trap on ERR, if set, is executed before the
shell
exits. This option applies to the shell environment and
each subshell environment separately (see COMMAND EXECUTION ENVIRONMENT above),
and
may cause subshells to exit before executing all the
commands in the subshell.
To my understanding, the purpose of these limitations is to prevent the shell
exiting if a command fails in conditional testing context such as if, elif,
while, isn't it?
My confusion is why the && operator disables `-e' effects in the subshell. It
does not make sense to continue executing remaining commands since I wish the
subshell to stop on error.
Or I am wrong and the results match the original designs?
BR
