Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
On 3/21/17 4:27 PM, Peter & Kelly Passchier wrote: > On 22/03/2560 01:32, Martijn Dekker wrote: >> A workaround for the original poster's problem could be: >> >> (unset -f jobs; unalias jobs; eval 'jobs -l') | wc > > 'unset -f jobs' is not guaranteed to unset jobs, it might set jobs to a > function definition in an earlier scope..! There are no local functions. All functions exist at the global scope. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
On 22/03/2560 01:32, Martijn Dekker wrote: > A workaround for the original poster's problem could be: > > (unset -f jobs; unalias jobs; eval 'jobs -l') | wc 'unset -f jobs' is not guaranteed to unset jobs, it might set jobs to a function definition in an earlier scope..!
Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
On 3/21/17 2:38 PM, Greg Wooledge wrote: > On Tue, Mar 21, 2017 at 07:32:27PM +0100, Martijn Dekker wrote: >> A workaround for the original poster's problem could be: >> >> (unset -f jobs; unalias jobs; eval 'jobs -l') | wc >> >> The 'eval' is to stop the alias from being expanded at parse time before >> you have a chance to unalias it. This should be about as robust as >> running 'builtin jobs -l', except it still doesn't check whether the >> builtin might have been disabled. > > Does this still trigger the "jobs hack"? Given that the shell is allowed > to use simple lexical analysis to determine whether we're just running > one jobs (or trap) command in the subshell, my first guess would *not* be > that this would work. (Not least because we are definitely *not* just > running one jobs command; there are at least three commands.) It doesn't, but the text Eric quoted doesn't really apply here anyway, since it only mentions command substitution. Arbitrary subshells' behavior is up to the implementation, and bash does what it does only to allow the output of jobs to be piped (e.g., `(jobs)' doesn't work the same way). -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
On Tue, Mar 21, 2017 at 07:32:27PM +0100, Martijn Dekker wrote: > A workaround for the original poster's problem could be: > > (unset -f jobs; unalias jobs; eval 'jobs -l') | wc > > The 'eval' is to stop the alias from being expanded at parse time before > you have a chance to unalias it. This should be about as robust as > running 'builtin jobs -l', except it still doesn't check whether the > builtin might have been disabled. Does this still trigger the "jobs hack"? Given that the shell is allowed to use simple lexical analysis to determine whether we're just running one jobs (or trap) command in the subshell, my first guess would *not* be that this would work. (Not least because we are definitely *not* just running one jobs command; there are at least three commands.)
Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
Op 21-03-17 om 18:08 schreef Greg Wooledge: > A builtin is always used by preference over an external command of the > same name. You don't need to specify "builtin jobs" to be sure you're > using the builtin. Just use "jobs". > > The only time you need to use the "builtin" command is when you're > defining a function by the same name, and you want bash to use its > builtin instead of your function. You may also need to use it if you're writing a function or script used by another program you don't control, or in another user's interactive shell environment. In that case you have no way to be sure whether 'jobs' might already be used as an alias or function name. A workaround for the original poster's problem could be: (unset -f jobs; unalias jobs; eval 'jobs -l') | wc The 'eval' is to stop the alias from being expanded at parse time before you have a chance to unalias it. This should be about as robust as running 'builtin jobs -l', except it still doesn't check whether the builtin might have been disabled. - M.
Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
On 03/21/2017 12:08 PM, Greg Wooledge wrote: > The code that makes bash behave differently when "jobs" is one of the > commands in a pipeline/subshell is kind of a hack. It's probably not > extremely well known outside of this mailing list, but I suspect many > people have used it without realizing what it is. It's a fairly intuitive > hack. And it's for more than just 'jobs' - at least 'trap' relies on it, too. > > I've got no strong opinions about whether the "jobs hack" should be > documented. POSIX documents the 'jobs'/'trap' hack thus: When a subshell is entered, traps that are not being ignored shall be set to the default actions, except in the case of a command substitution containing only a single trap command, when the traps need not be altered. Implementations may check for this case using only lexical analysis; for example, if `trap` and $( trap -- ) do not alter the traps in the subshell, cases such as assigning var=trap and then using $($var) may still alter them. For more background on this, see: http://austingroupbugs.net/view.php?id=53 although that bug specifically mentions 'trap', and concludes that a separate bug may need to be opened (which I could not find) giving the same treatment to 'jobs'. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
Hi Greg,
Thanks for your explanation! Now it solved my problem. I did misuse the
"builtin" builtin when using "jobs". Now I only need to enforce that "jobs"
is "enabled", which is not a walkaround :-)
Except one thing that bothers me a tiny bit is that, how about I wrote a
"jobs" function and still want to pipe "builtin jobs" somewhere? But I
believe I won't make such a use in the near future. Anyways, let me read
the code first :-)
Thanks for your time!
Sincerely,
Hengyang
On Tue, Mar 21, 2017 at 10:08 AM Greg Wooledge wrote:
> On Tue, Mar 21, 2017 at 04:53:59PM +, Hengyang Zhao wrote:
> > But back to the user's perspective, as I looked up "help jobs" or "help
> > builtin", the sematics of "builtin" is only for forcing the shell to use
> > the builtin version, right? Actually, I was writing a script that needs
> to
> > secure the use of the builtin jobs, but now I need to seek for a reliable
> > walkaround instead of using "builtin".
>
> A builtin is always used by preference over an external command of the
> same name. You don't need to specify "builtin jobs" to be sure you're
> using the builtin. Just use "jobs".
>
> The only time you need to use the "builtin" command is when you're
> defining a function by the same name, and you want bash to use its
> builtin instead of your function. In practice, this should be quite rare.
> You'd really only use it if you are creating a wrapper function. For
> example:
>
> cd() {
> local where=${1-$HOME}
> # set xterm title bar
> printf '\e]2;%s\a' "$where"
> builtin cd "$where"
> }
>
> The "builtin cd" at the end prevents bash from recursively calling the
> function.
>
> > So if we don't treat it as a bug, is
> > it still a good suggestion that we write a caveat info the "builtin" help
> > info?
>
> The code that makes bash behave differently when "jobs" is one of the
> commands in a pipeline/subshell is kind of a hack. It's probably not
> extremely well known outside of this mailing list, but I suspect many
> people have used it without realizing what it is. It's a fairly intuitive
> hack.
>
> I've got no strong opinions about whether the "jobs hack" should be
> documented. I don't think the "builtin" command needs any further
> explanation, though. The "help builtin" text already contains a terse
> version of the explanation I gave up above.
>
--
Hengyang Zhao
Ph.D. Candidate, Electrical Engineering
University of California, Riverside
Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
On Tue, Mar 21, 2017 at 04:53:59PM +, Hengyang Zhao wrote:
> But back to the user's perspective, as I looked up "help jobs" or "help
> builtin", the sematics of "builtin" is only for forcing the shell to use
> the builtin version, right? Actually, I was writing a script that needs to
> secure the use of the builtin jobs, but now I need to seek for a reliable
> walkaround instead of using "builtin".
A builtin is always used by preference over an external command of the
same name. You don't need to specify "builtin jobs" to be sure you're
using the builtin. Just use "jobs".
The only time you need to use the "builtin" command is when you're
defining a function by the same name, and you want bash to use its
builtin instead of your function. In practice, this should be quite rare.
You'd really only use it if you are creating a wrapper function. For
example:
cd() {
local where=${1-$HOME}
# set xterm title bar
printf '\e]2;%s\a' "$where"
builtin cd "$where"
}
The "builtin cd" at the end prevents bash from recursively calling the
function.
> So if we don't treat it as a bug, is
> it still a good suggestion that we write a caveat info the "builtin" help
> info?
The code that makes bash behave differently when "jobs" is one of the
commands in a pipeline/subshell is kind of a hack. It's probably not
extremely well known outside of this mailing list, but I suspect many
people have used it without realizing what it is. It's a fairly intuitive
hack.
I've got no strong opinions about whether the "jobs hack" should be
documented. I don't think the "builtin" command needs any further
explanation, though. The "help builtin" text already contains a terse
version of the explanation I gave up above.
Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
Dear Chet, Thanks for your reply and I apologyze for the mistake I made in the "buggy behavior" section. I didn't realize that a builtin is forced to run in a subshell until you told me. Yes, as I tried, "jobs" consumes no PID, but even "jobs | true" consumes two. I'll look into the code following the clue you gave me. But back to the user's perspective, as I looked up "help jobs" or "help builtin", the sematics of "builtin" is only for forcing the shell to use the builtin version, right? Actually, I was writing a script that needs to secure the use of the builtin jobs, but now I need to seek for a reliable walkaround instead of using "builtin". So if we don't treat it as a bug, is it still a good suggestion that we write a caveat info the "builtin" help info? Thanks again for your detailed reply! It helps me a lot :-) Sincerely, Hengyang On Tue, Mar 21, 2017 at 8:13 AM Chet Ramey wrote: > On 3/20/17 7:47 PM, Hengyang Zhao wrote: > > Hello, > > > > I am experiencing an unexpected behavior when using "builtin jobs -l". It > > seems that it's output fd is not as same as "jobs -l" without the builtin > > prefix. So when I piped it into wc, I got different results (as seen in > the > > code I pasted below). > > > > Bash version: GNU bash, version 4.3.43(1)-release > (x86_64-redhat-linux-gnu) > > OS: Fedora 25, kernel 4.9.13-200.fc25.x86_64 > > > > Buggy behavior: > > when we have one or more background processes: > > jobs -l | wc# produces 0 0 0 > > builtin -l | wc # produces non-zero non-zero non-zero > > (This is actually the opposite of what happens, as shown by your examples > below.) > > > I tried to fix the bug, but after I looked at the code, the jobs printing > > procedure clearly prints to stdout. So I think it's not a easy work to > > track down the bug. > > It's not a bug. The first part of a pipeline is always run in a subshell. > That subshell doesn't really have any jobs, since none of the parent's > jobs are children of that shell -- you can't wait for them, for instance. > > There is special-case code that attempts to detect when the `jobs' builtin > is running (execute_cmd.c:execute_subshell_builtin_or_function) and doesn't > remove the jobs from the jobs table in that one case, specifically to allow > the output of `jobs' to be piped. Running the `builtin' builtin defeats > that. > > Chet > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRUc...@case.edu > http://cnswww.cns.cwru.edu/~chet/ > -- Hengyang Zhao Ph.D. Candidate, Electrical Engineering University of California, Riverside
Re: Bug: different behavior between "jobs -l" and "builtin jobs -l"
On 3/20/17 7:47 PM, Hengyang Zhao wrote: > Hello, > > I am experiencing an unexpected behavior when using "builtin jobs -l". It > seems that it's output fd is not as same as "jobs -l" without the builtin > prefix. So when I piped it into wc, I got different results (as seen in the > code I pasted below). > > Bash version: GNU bash, version 4.3.43(1)-release (x86_64-redhat-linux-gnu) > OS: Fedora 25, kernel 4.9.13-200.fc25.x86_64 > > Buggy behavior: > when we have one or more background processes: > jobs -l | wc# produces 0 0 0 > builtin -l | wc # produces non-zero non-zero non-zero (This is actually the opposite of what happens, as shown by your examples below.) > I tried to fix the bug, but after I looked at the code, the jobs printing > procedure clearly prints to stdout. So I think it's not a easy work to > track down the bug. It's not a bug. The first part of a pipeline is always run in a subshell. That subshell doesn't really have any jobs, since none of the parent's jobs are children of that shell -- you can't wait for them, for instance. There is special-case code that attempts to detect when the `jobs' builtin is running (execute_cmd.c:execute_subshell_builtin_or_function) and doesn't remove the jobs from the jobs table in that one case, specifically to allow the output of `jobs' to be piped. Running the `builtin' builtin defeats that. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Bug: different behavior between "jobs -l" and "builtin jobs -l"
Hello, I am experiencing an unexpected behavior when using "builtin jobs -l". It seems that it's output fd is not as same as "jobs -l" without the builtin prefix. So when I piped it into wc, I got different results (as seen in the code I pasted below). Bash version: GNU bash, version 4.3.43(1)-release (x86_64-redhat-linux-gnu) OS: Fedora 25, kernel 4.9.13-200.fc25.x86_64 Buggy behavior: when we have one or more background processes: jobs -l | wc# produces 0 0 0 builtin -l | wc # produces non-zero non-zero non-zero I tried to fix the bug, but after I looked at the code, the jobs printing procedure clearly prints to stdout. So I think it's not a easy work to track down the bug. Way to reproduce: bash-4.3$ bash-4.3$ enable -a | grep jobs enable jobs bash-4.3$ cat ^Z [1]+ Stopped cat bash-4.3$ jobs -l [1]+ 14030 Stopped cat bash-4.3$ jobs -l | wc 1 4 39 bash-4.3$ builtin jobs -l | wc 0 0 0 bash-4.3$ type jobs jobs is a shell builtin bash-4.3$ type -a jobs jobs is a shell builtin jobs is /usr/bin/jobs jobs is /bin/jobs bash-4.3$ Thank you! Sincerely, Hengyang -- Hengyang Zhao Ph.D. Candidate, Electrical Engineering University of California, Riverside
