Re: bug-report: debug trap messes up pipestatus if containing command substitution
Thanks for the fix! +Shaoyi Peng +端端 On Tue, May 28, 2019, 6:07 PM Chet Ramey wrote: > On 5/23/19 5:11 PM, Hengyang Zhao wrote: > > Awesome! Could you please cc the fix to this thread once it's done? > Thank you! > > It's in the devel branch. I've attached the relevant piece. > > This is more confirmation that it was a mistake to apply the DEBUG trap to > simple commands in pipelines. > > > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ >
Re: bug-report: debug trap messes up pipestatus if containing command substitution
Awesome! Could you please cc the fix to this thread once it's done? Thank
you!
Hengyang
On Wed, May 22, 2019, 7:27 AM Chet Ramey wrote:
> On 5/21/19 12:51 AM, Hengyang Zhao wrote:
> > Dear bash developers,
> >
> > I found an unexpected behavior when engineering some code in a callback
> of
> > DEBUG trap. The code has been distilled to expose this behavior:
> >
> > ### BEGIN
> > my_func() {
> > local x=$(:)
> > }
> > trap my_func DEBUG
> > false | true | false | true | false
> > echo ${PIPESTATUS[@]}
> > # expects 1 0 1 0 1
> > # gets 0 0 0 0 1
> > ### END
>
> Thanks for the report. This appears to be a Linux-only problem (at least I
> can't reproduce it on Mac OS X). I'll fix it for the next devel push.
>
> Chet
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
> ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
>
Re: bug-report: debug trap messes up pipestatus if containing command substitution
I missed a segment of my config when copy-pasting:
Bash Version: 5.0
Patch Level: 7
Release Status: release
On Mon, May 20, 2019 at 9:57 PM Hengyang Zhao wrote:
> +speng...@ucr.edu +yunduan@gmail.com
>
> On Mon, May 20, 2019 at 9:51 PM Hengyang Zhao
> wrote:
>
>> Dear bash developers,
>>
>> I found an unexpected behavior when engineering some code in a callback
>> of DEBUG trap. The code has been distilled to expose this behavior:
>>
>> ### BEGIN
>> my_func() {
>> local x=$(:)
>> }
>> trap my_func DEBUG
>> false | true | false | true | false
>> echo ${PIPESTATUS[@]}
>> # expects 1 0 1 0 1
>> # gets 0 0 0 0 1
>> ### END
>>
>> Configuration Information:
>> Machine: x86_64
>> OS: linux-gnu
>> Compiler: gcc
>> Compilation CFLAGS: -O2 -g -pipe -Wall -Werror=format-security
>> -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions
>> -fstack-protector-strong -grecord-gcc-switches
>> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
>> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
>> -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
>> -Wno-parentheses -Wno-format-security
>> uname output: Linux fedora 5.0.9-301.fc30.x86_64 #1 SMP Tue Apr 23
>> 23:57:35 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
>> Machine Type: x86_64-redhat-linux-gnu
>>
>> This also happens on many of my OSes running bash 4.x.
>>
>> I'm happy to provide further information if needed.
>>
>> Thank you,
>> Hengyang
>>
>
Re: bug-report: debug trap messes up pipestatus if containing command substitution
+speng...@ucr.edu +yunduan@gmail.com
On Mon, May 20, 2019 at 9:51 PM Hengyang Zhao wrote:
> Dear bash developers,
>
> I found an unexpected behavior when engineering some code in a callback of
> DEBUG trap. The code has been distilled to expose this behavior:
>
> ### BEGIN
> my_func() {
> local x=$(:)
> }
> trap my_func DEBUG
> false | true | false | true | false
> echo ${PIPESTATUS[@]}
> # expects 1 0 1 0 1
> # gets 0 0 0 0 1
> ### END
>
> Configuration Information:
> Machine: x86_64
> OS: linux-gnu
> Compiler: gcc
> Compilation CFLAGS: -O2 -g -pipe -Wall -Werror=format-security
> -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions
> -fstack-protector-strong -grecord-gcc-switches
> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
> -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
> -Wno-parentheses -Wno-format-security
> uname output: Linux fedora 5.0.9-301.fc30.x86_64 #1 SMP Tue Apr 23
> 23:57:35 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> Machine Type: x86_64-redhat-linux-gnu
>
> This also happens on many of my OSes running bash 4.x.
>
> I'm happy to provide further information if needed.
>
> Thank you,
> Hengyang
>
bug-report: debug trap messes up pipestatus if containing command substitution
Dear bash developers,
I found an unexpected behavior when engineering some code in a callback of
DEBUG trap. The code has been distilled to expose this behavior:
### BEGIN
my_func() {
local x=$(:)
}
trap my_func DEBUG
false | true | false | true | false
echo ${PIPESTATUS[@]}
# expects 1 0 1 0 1
# gets 0 0 0 0 1
### END
Configuration Information:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -O2 -g -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions
-fstack-protector-strong -grecord-gcc-switches
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
-Wno-parentheses -Wno-format-security
uname output: Linux fedora 5.0.9-301.fc30.x86_64 #1 SMP Tue Apr 23 23:57:35
UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-redhat-linux-gnu
This also happens on many of my OSes running bash 4.x.
I'm happy to provide further information if needed.
Thank you,
Hengyang
Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
Hi Greg,
Thanks for your explanation! Now it solved my problem. I did misuse the
"builtin" builtin when using "jobs". Now I only need to enforce that "jobs"
is "enabled", which is not a walkaround :-)
Except one thing that bothers me a tiny bit is that, how about I wrote a
"jobs" function and still want to pipe "builtin jobs" somewhere? But I
believe I won't make such a use in the near future. Anyways, let me read
the code first :-)
Thanks for your time!
Sincerely,
Hengyang
On Tue, Mar 21, 2017 at 10:08 AM Greg Wooledge wrote:
> On Tue, Mar 21, 2017 at 04:53:59PM +, Hengyang Zhao wrote:
> > But back to the user's perspective, as I looked up "help jobs" or "help
> > builtin", the sematics of "builtin" is only for forcing the shell to use
> > the builtin version, right? Actually, I was writing a script that needs
> to
> > secure the use of the builtin jobs, but now I need to seek for a reliable
> > walkaround instead of using "builtin".
>
> A builtin is always used by preference over an external command of the
> same name. You don't need to specify "builtin jobs" to be sure you're
> using the builtin. Just use "jobs".
>
> The only time you need to use the "builtin" command is when you're
> defining a function by the same name, and you want bash to use its
> builtin instead of your function. In practice, this should be quite rare.
> You'd really only use it if you are creating a wrapper function. For
> example:
>
> cd() {
> local where=${1-$HOME}
> # set xterm title bar
> printf '\e]2;%s\a' "$where"
> builtin cd "$where"
> }
>
> The "builtin cd" at the end prevents bash from recursively calling the
> function.
>
> > So if we don't treat it as a bug, is
> > it still a good suggestion that we write a caveat info the "builtin" help
> > info?
>
> The code that makes bash behave differently when "jobs" is one of the
> commands in a pipeline/subshell is kind of a hack. It's probably not
> extremely well known outside of this mailing list, but I suspect many
> people have used it without realizing what it is. It's a fairly intuitive
> hack.
>
> I've got no strong opinions about whether the "jobs hack" should be
> documented. I don't think the "builtin" command needs any further
> explanation, though. The "help builtin" text already contains a terse
> version of the explanation I gave up above.
>
--
Hengyang Zhao
Ph.D. Candidate, Electrical Engineering
University of California, Riverside
Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
Dear Chet, Thanks for your reply and I apologyze for the mistake I made in the "buggy behavior" section. I didn't realize that a builtin is forced to run in a subshell until you told me. Yes, as I tried, "jobs" consumes no PID, but even "jobs | true" consumes two. I'll look into the code following the clue you gave me. But back to the user's perspective, as I looked up "help jobs" or "help builtin", the sematics of "builtin" is only for forcing the shell to use the builtin version, right? Actually, I was writing a script that needs to secure the use of the builtin jobs, but now I need to seek for a reliable walkaround instead of using "builtin". So if we don't treat it as a bug, is it still a good suggestion that we write a caveat info the "builtin" help info? Thanks again for your detailed reply! It helps me a lot :-) Sincerely, Hengyang On Tue, Mar 21, 2017 at 8:13 AM Chet Ramey wrote: > On 3/20/17 7:47 PM, Hengyang Zhao wrote: > > Hello, > > > > I am experiencing an unexpected behavior when using "builtin jobs -l". It > > seems that it's output fd is not as same as "jobs -l" without the builtin > > prefix. So when I piped it into wc, I got different results (as seen in > the > > code I pasted below). > > > > Bash version: GNU bash, version 4.3.43(1)-release > (x86_64-redhat-linux-gnu) > > OS: Fedora 25, kernel 4.9.13-200.fc25.x86_64 > > > > Buggy behavior: > > when we have one or more background processes: > > jobs -l | wc# produces 0 0 0 > > builtin -l | wc # produces non-zero non-zero non-zero > > (This is actually the opposite of what happens, as shown by your examples > below.) > > > I tried to fix the bug, but after I looked at the code, the jobs printing > > procedure clearly prints to stdout. So I think it's not a easy work to > > track down the bug. > > It's not a bug. The first part of a pipeline is always run in a subshell. > That subshell doesn't really have any jobs, since none of the parent's > jobs are children of that shell -- you can't wait for them, for instance. > > There is special-case code that attempts to detect when the `jobs' builtin > is running (execute_cmd.c:execute_subshell_builtin_or_function) and doesn't > remove the jobs from the jobs table in that one case, specifically to allow > the output of `jobs' to be piped. Running the `builtin' builtin defeats > that. > > Chet > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRUc...@case.edu > http://cnswww.cns.cwru.edu/~chet/ > -- Hengyang Zhao Ph.D. Candidate, Electrical Engineering University of California, Riverside
Bug: different behavior between "jobs -l" and "builtin jobs -l"
Hello, I am experiencing an unexpected behavior when using "builtin jobs -l". It seems that it's output fd is not as same as "jobs -l" without the builtin prefix. So when I piped it into wc, I got different results (as seen in the code I pasted below). Bash version: GNU bash, version 4.3.43(1)-release (x86_64-redhat-linux-gnu) OS: Fedora 25, kernel 4.9.13-200.fc25.x86_64 Buggy behavior: when we have one or more background processes: jobs -l | wc# produces 0 0 0 builtin -l | wc # produces non-zero non-zero non-zero I tried to fix the bug, but after I looked at the code, the jobs printing procedure clearly prints to stdout. So I think it's not a easy work to track down the bug. Way to reproduce: bash-4.3$ bash-4.3$ enable -a | grep jobs enable jobs bash-4.3$ cat ^Z [1]+ Stopped cat bash-4.3$ jobs -l [1]+ 14030 Stopped cat bash-4.3$ jobs -l | wc 1 4 39 bash-4.3$ builtin jobs -l | wc 0 0 0 bash-4.3$ type jobs jobs is a shell builtin bash-4.3$ type -a jobs jobs is a shell builtin jobs is /usr/bin/jobs jobs is /bin/jobs bash-4.3$ Thank you! Sincerely, Hengyang -- Hengyang Zhao Ph.D. Candidate, Electrical Engineering University of California, Riverside
