Re: /bin/sh => STDIN & functions, var scope messing
On 31 May 2013 20:50, "Dan Nelson" wrote: > > In the last episode (May 31), Reid Linnemann said: > > On Fri, May 31, 2013 at 1:12 PM, Teske, Devin wrote: > > > If you're arguing we have to change sh's behavior to be more compliant, > > > jilles already quoted XCU 2.12 (our shell is well within its right to > > > run any/all lvalue/rvalue operands of a pipe in a sub-shell without > > > contradicting the guidelines). > > > > > > But if you're arguing that it has to change to make things better or > > > easier... I don't know about that. Might just make people lulled into > > > using a style that's non-portable. I'd like to keep things the way they > > > are so that if you program for FreeBSD, you're inherently going to > > > program in a fashion that is more portable. > > > > FWIW bash (invoked as sh) on RHEL-based linux systems exhibits the same > > behavior- > > > > sh-3.2$ var=1 > > sh-3.2$ yes|var=2 > > sh-3.2$ echo $var > > 1 > > sh-3.2$ > > > > If my opinion matters at all, I would agree that for the sake of > > portability that behavior be consistent with the majority of sh > > implementations rather than "right" according to arbitrary ruling. > > On the other hand, zsh runs the last component of a pipeline in the parent > shell. The usual model is "do work in pipeline, process results in final > component", and being able to simply set variables there that can be used in > the rest of the script is very elegant. Right... But it's not portable. Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: /bin/sh => STDIN & functions, var scope messing
In the last episode (May 31), Reid Linnemann said: > On Fri, May 31, 2013 at 1:12 PM, Teske, Devin > wrote: > > If you're arguing we have to change sh's behavior to be more compliant, > > jilles already quoted XCU 2.12 (our shell is well within its right to > > run any/all lvalue/rvalue operands of a pipe in a sub-shell without > > contradicting the guidelines). > > > > But if you're arguing that it has to change to make things better or > > easier... I don't know about that. Might just make people lulled into > > using a style that's non-portable. I'd like to keep things the way they > > are so that if you program for FreeBSD, you're inherently going to > > program in a fashion that is more portable. > > FWIW bash (invoked as sh) on RHEL-based linux systems exhibits the same > behavior- > > sh-3.2$ var=1 > sh-3.2$ yes|var=2 > sh-3.2$ echo $var > 1 > sh-3.2$ > > If my opinion matters at all, I would agree that for the sake of > portability that behavior be consistent with the majority of sh > implementations rather than "right" according to arbitrary ruling. On the other hand, zsh runs the last component of a pipeline in the parent shell. The usual model is "do work in pipeline, process results in final component", and being able to simply set variables there that can be used in the rest of the script is very elegant. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: /bin/sh => STDIN & functions, var scope messing
On Fri, May 31, 2013 at 1:12 PM, Teske, Devin wrote: > > If you're arguing we have to change sh's behavior to be more compliant, > jilles already quoted XCU 2.12 (our shell is well within its right to run > any/all lvalue/rvalue operands of a pipe in a sub-shell without > contradicting the guidelines). > > But if you're arguing that it has to change to make things better or > easier… I don't know about that. Might just make people lulled into using a > style that's non-portable. I'd like to keep things the way they are so that > if you program for FreeBSD, you're inherently going to program in a fashion > that is more portable. > -- > Devin > FWIW bash (invoked as sh) on RHEL-based linux systems exhibits the same behavior- sh-3.2$ var=1 sh-3.2$ yes|var=2 sh-3.2$ echo $var 1 sh-3.2$ If my opinion matters at all, I would agree that for the sake of portability that behavior be consistent with the majority of sh implementations rather than "right" according to arbitrary ruling. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: /bin/sh => STDIN & functions, var scope messing
On May 31, 2013, at 10:59 AM,
wrote:
> Redirections > and >> don't put it in a subshell.
Correct. (note: I made no such insinuation; But thanks for clarifying for
others that perhaps were not aware).
> Only pipe |, which means only STDIN affects/triggers this behaviour.
> Does < operator also does it, as it is also STDIN?
>
> Anyway, I don't care for executing binaries, but I do care if that is part of
> sh's code, as function is.
> It messes var scopes.
>
> And finally if we take into account this:
>
>> For example… in pc-sysinstall…
>>
>> ./backend/functions-extractimage.sh- tar cvf - . 2>/dev/null | tar -xpv
>> -C ${FSMNT} ${TAROPTS} -f - 2>&1 | tee -a ${FSMNT}/.tar-extract.log
>> ./backend/functions-extractimage.sh: if [ $? -ne 0 ]
>>
>> That's a big no-no.
>>
>> If your first tar (the initial lvalue to the first pipe) fails… but your
>> second tar succeeds (rvalue to the first pipe)… you won't catch that failure
>> in your checking of $?
>>
>> Also, if the first tar succeeds, but the second tar fails, AND the final
>> rvalue (the tee) succeeds… you also miss the error code.
>>
>> I call this the "piped return-status conflation issue". Basically… anytime
>> you want to check the return-status in shell… and you care about
>> lvalue-failures in a pipe-chain… you must rewrite it to either:
>>
>> (a) be aware of the problem (I've in the past written wrappers that will
>> test the previous return status and abort the chain in such an event)
>>
>> (b) undo the pipe-chain and store your results for incremental processing…
>> checking the return status after each filter.
>>
>> --
>> Devin
>
>
> sh's behaviour must be changed regarding piping
>
If you're arguing we have to change sh's behavior to be more compliant, jilles
already quoted XCU 2.12 (our shell is well within its right to run any/all
lvalue/rvalue operands of a pipe in a sub-shell without contradicting the
guidelines).
But if you're arguing that it has to change to make things better or easier… I
don't know about that. Might just make people lulled into using a style that's
non-portable. I'd like to keep things the way they are so that if you program
for FreeBSD, you're inherently going to program in a fashion that is more
portable.
--
Devin
>>> Curious. Which of the two behaviours is POSIXly correct?
>>
>> Both. As per XCU 2.12 Shell Execution Environment, each command in a
>> multi-command pipeline may or may not be executed in a subshell
>> environment.
>>
>> Behaviour different from our sh is most often encountered in the various
>> versions of the real Korn shell (ksh88 and ksh93), which execute the
>> last command in a pipeline in the current shell environment.
>>
>> If things like jobs | cat work, that can also be explained using this
>> rule.
>>
>> --
>> Jilles Tjoelker
>>
>
>
> Domagoj Smolčić
>
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
_
The information contained in this message is proprietary and/or confidential.
If you are not the intended recipient, please: (i) delete the message and all
copies; (ii) do not disclose, distribute or use the message in any manner; and
(iii) notify the sender immediately. In addition, please be aware that any
message addressed to our domain is subject to archiving and review by persons
other than the intended recipient. Thank you.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: /bin/sh => STDIN & functions, var scope messing
Redirections > and >> don't put it in a subshell.
Only pipe |, which means only STDIN affects/triggers this behaviour.
Does < operator also does it, as it is also STDIN?
Anyway, I don't care for executing binaries, but I do care if that is part of
sh's code, as function is.
It messes var scopes.
And finally if we take into account this:
> For example… in pc-sysinstall…
>
> ./backend/functions-extractimage.sh- tar cvf - . 2>/dev/null | tar -xpv
> -C ${FSMNT} ${TAROPTS} -f - 2>&1 | tee -a ${FSMNT}/.tar-extract.log
> ./backend/functions-extractimage.sh: if [ $? -ne 0 ]
>
> That's a big no-no.
>
> If your first tar (the initial lvalue to the first pipe) fails… but your
> second tar succeeds (rvalue to the first pipe)… you won't catch that failure
> in your checking of $?
>
> Also, if the first tar succeeds, but the second tar fails, AND the final
> rvalue (the tee) succeeds… you also miss the error code.
>
> I call this the "piped return-status conflation issue". Basically… anytime
> you want to check the return-status in shell… and you care about
> lvalue-failures in a pipe-chain… you must rewrite it to either:
>
> (a) be aware of the problem (I've in the past written wrappers that will test
> the previous return status and abort the chain in such an event)
>
> (b) undo the pipe-chain and store your results for incremental processing…
> checking the return status after each filter.
>
> --
> Devin
sh's behaviour must be changed regarding pipeing
> > Curious. Which of the two behaviours is POSIXly correct?
>
> Both. As per XCU 2.12 Shell Execution Environment, each command in a
> multi-command pipeline may or may not be executed in a subshell
> environment.
>
> Behaviour different from our sh is most often encountered in the various
> versions of the real Korn shell (ksh88 and ksh93), which execute the
> last command in a pipeline in the current shell environment.
>
> If things like jobs | cat work, that can also be explained using this
> rule.
>
> --
> Jilles Tjoelker
>
Domagoj Smolčić
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: /bin/sh => STDIN & functions, var scope messing
On Tue, May 28, 2013 at 11:48:47AM +0200, Václav Zeman wrote: > On 27 May 2013 21:58, Reid Linnemann wrote: > > from SH(1) > > "Note that unlike some other shells, sh executes each process in a pipe- > > line with more than one command in a subshell environment and as a > > child > > of the sh process." > > I'm taking this to mean that redirecting to sh_f has sh_f execute in > > a subshell in which global_scope_var changes, but the original > > shell's copy is uncahnged. > Curious. Which of the two behaviours is POSIXly correct? Both. As per XCU 2.12 Shell Execution Environment, each command in a multi-command pipeline may or may not be executed in a subshell environment. Behaviour different from our sh is most often encountered in the various versions of the real Korn shell (ksh88 and ksh93), which execute the last command in a pipeline in the current shell environment. If things like jobs | cat work, that can also be explained using this rule. -- Jilles Tjoelker ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: /bin/sh => STDIN & functions, var scope messing
On May 28, 2013, at 8:07 AM, Reid Linnemann wrote:
>
> On May 28, 2013, at 7:00 AM, Ryan Stone wrote:
>
>> On Tue, May 28, 2013 at 5:48 AM, Václav Zeman wrote:
>> Curious. Which of the two behaviours is POSIXly correct?
>>
>> I believe that /bin/sh's behaviour is correct. I don't know what shell the
>> manpage is referring to, but it's not bash (bash does the same thing in a
>> pipeline). Perhaps it's referring to csh? If that is the case that line is
>> probably causing more confusion rather than alleviating it.
>
> I believe it's referring to csh, possible ksh as well. I tried a similar
> experiment with tcsh:
>
> #!/bin/tcsh
> alias fn set var=12
> set var=
> echo $var
> yes | fn
> echo $var
I wonder why the sh(1) manual would be referring to csh(1).
CSH does more than this, so you know…
#!/bin/csh
true | true | false | true
# returns false
#!/bin/sh
true | true | false | true
# returns true
So not only must you be aware that sh(1) throws away variables assigned within
a shell running as an rvalue to any pipe (because said statements are run in a
sub-shell with a transient namespace initialized from the parent)…
You must also be aware that return status of an lvalue operand of a pipe is not
retained along the chain. The entire chain is traversed (all rvalues of all
pipes are invoked), but in sh(1) only the return status of last rvalue is
returned.
I see this problem running rampant in the pc-sysinstall code.
For example… in pc-sysinstall…
./backend/functions-extractimage.sh- tar cvf - . 2>/dev/null | tar -xpv -C
${FSMNT} ${TAROPTS} -f - 2>&1 | tee -a ${FSMNT}/.tar-extract.log
./backend/functions-extractimage.sh: if [ $? -ne 0 ]
That's a big no-no.
If your first tar (the initial lvalue to the first pipe) fails… but your second
tar succeeds (rvalue to the first pipe)… you won't catch that failure in your
checking of $?
Also, if the first tar succeeds, but the second tar fails, AND the final rvalue
(the tee) succeeds… you also miss the error code.
I call this the "piped return-status conflation issue". Basically… anytime you
want to check the return-status in shell… and you care about lvalue-failures in
a pipe-chain… you must rewrite it to either:
(a) be aware of the problem (I've in the past written wrappers that will test
the previous return status and abort the chain in such an event)
(b) undo the pipe-chain and store your results for incremental processing…
checking the return status after each filter.
--
Devin
NOTE: I'm responding on another thread that is related to pc-sysinstall
changes. In that thread, I didn't mention the above faux-pas of pc-sysinstall
because I really do consider things like this to be secondary to the
over-arching namespace and debugging issues.
_
The information contained in this message is proprietary and/or confidential.
If you are not the intended recipient, please: (i) delete the message and all
copies; (ii) do not disclose, distribute or use the message in any manner; and
(iii) notify the sender immediately. In addition, please be aware that any
message addressed to our domain is subject to archiving and review by persons
other than the intended recipient. Thank you.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: /bin/sh => STDIN & functions, var scope messing
On May 28, 2013, at 7:00 AM, Ryan Stone wrote: > On Tue, May 28, 2013 at 5:48 AM, Václav Zeman wrote: > Curious. Which of the two behaviours is POSIXly correct? > > I believe that /bin/sh's behaviour is correct. I don't know what shell the > manpage is referring to, but it's not bash (bash does the same thing in a > pipeline). Perhaps it's referring to csh? If that is the case that line is > probably causing more confusion rather than alleviating it. I believe it's referring to csh, possible ksh as well. I tried a similar experiment with tcsh: #!/bin/tcsh alias fn set var=12 set var= echo $var yes | fn echo $var ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: /bin/sh => STDIN & functions, var scope messing
On Tue, May 28, 2013 at 5:48 AM, Václav Zeman wrote: > Curious. Which of the two behaviours is POSIXly correct? > I believe that /bin/sh's behaviour is correct. I don't know what shell the manpage is referring to, but it's not bash (bash does the same thing in a pipeline). Perhaps it's referring to csh? If that is the case that line is probably causing more confusion rather than alleviating it. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: /bin/sh => STDIN & functions, var scope messing
On 27 May 2013 21:58, Reid Linnemann wrote: > from SH(1) > > "Note that unlike some other shells, sh executes each process in a pipe- > line with more than one command in a subshell environment and as a > child > of the sh process." > > I'm taking this to mean that redirecting to sh_f has sh_f execute in a > subshell in which global_scope_var changes, but the original shell's copy > is uncahnged. Curious. Which of the two behaviours is POSIXly correct? -- VZ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: /bin/sh => STDIN & functions, var scope messing
from SH(1)
"Note that unlike some other shells, sh executes each process in a pipe-
line with more than one command in a subshell environment and as a
child
of the sh process."
I'm taking this to mean that redirecting to sh_f has sh_f execute in a
subshell in which global_scope_var changes, but the original shell's copy
is uncahnged.
On Mon, May 27, 2013 at 2:42 PM, wrote:
> 9.1-RELEASE-p3
> ---
> #!/bin/sh
>
> sh_f ()
> {
> global_scope_var=7463457
> }
>
> yes | sh_f
> echo "$global_scope_var"
>
> echo '
> Now without /usr/bin/yes (maybe it is STDIN issue, instead) ?!?
> '
>
> sh_f
> echo "$global_scope_var"
> ---
>
>
> Domagoj Smolčić
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
