Re: [Parameter Expansion] bug in ${variable% *}
On Mon, Feb 11, 2013 at 06:50:49PM +0100, Dashing wrote: $ ./pe 'mplayer foo1\ foo2\ foo3\ 4\ 5\ foo6\ 7' :--Mistake--: :--Mistake--: Whatever you're doing, it's wrong. ./pe mplayer foo1\ foo2\ foo3\ 4\ 5\ foo6\ 7 #!/bin/bash prog=$1 shift exec $prog extra args go here $@ THAT is how a wrapper should be done. Do not combine a program and its arguments all together into a single argument and then try to parse it apart.
Re: [Parameter Expansion] bug in ${variable% *}
On Tue, Feb 12, 2013 at 02:37:03PM +0100, Dashing wrote:
For my purposes this is irrelevant, because the nature of the
script from which my example code derived is tab completion.
READLINE_LINE will contain mplayer foo1\ foo2\ etc.
Then you are still doing something wrong.
imadev:~$ args Tori\ Amos\ -\ Mr.\ Zebra.ogg
1 args: Tori Amos - Mr. Zebra.ogg
There are no literal backslashes in an argument that is produced by
tab completion. The backslashes are a form of quoting, and they are
removed by the calling shell during the quote removal phase.
If you believe you have found a bug in parameter expansion, then
simplify the example so that it directly demonstrates the bug. For
example,
foo='some thing with \ \ \ \ backslashes'
echo ${foo##* }
echo I expected to see 'blah'
If you are playing around with read -e, then your script should CONTAIN
that. And if you are reading shell commands in a shell script using
read -e and then attempting to parse them the way a shell would, then you
should reevaluate your goals, or the methods you are attempting to use to
attain them, because writing a shell in a shell is kinda wonky, isn't it?
Re: [Parameter Expansion] bug in ${variable% *}
On Tue, 12 Feb 2013 14:45:26 +0100 Greg Wooledge wool...@eeg.ccf.org wrote: There are no literal backslashes in an argument that is produced by tab completion. The backslashes are a form of quoting, and they are removed by the calling shell during the quote removal phase. When bind -x sets READLINE_LINE backslashes are kept intact. What I meant was that I wrote my own tab completion. If you believe you have found a bug in parameter expansion, then simplify the example so that it directly demonstrates the bug. As I showed in the output, it does not happen all the time. Even the same value of $rest one time works and another time does not. Therefore, looping over a bunch of items is required.
Re: eval doesn't close file descriptor?
Ok, but I see the same behaviour when eval runs in a subshell:
$ bash -c 'llfd () { echo pid:$BASHPID 2; ls -l /proc/$BASHPID/fd/ 2;
}; x=3; eval exec $x/dev/null; llfd; echo | eval llfd $x-'
[same output, fd 10 open, pointing to /dev/null, even though it's a
subshell]
$ bash -c 'llfd () { echo pid:$BASHPID 2; ls -l /proc/$BASHPID/fd/ 2;
}; x=3; eval exec $x/dev/null; llfd; echo | llfd 3-'
[not the same output; no fds pointing to /dev/null, as expected]
When eval is run by the main shell and I want $x closed, I can just do
'eval exec $x-'. However, I cannot do that with eval runs in a
subshell. In my script I needed $x open for other processes in that
pipeline.
On a different but related note, I hate having to do eval to manipulate an
fd stored in a variable. Why doesn't 'llfd $x-' work, especially since
'llfd $x' works just fine... so by the time is handled, the variable
substitutions seem to be done already.
On Tue, Feb 12, 2013 at 2:13 AM, Pierre Gaston pierre.gas...@gmail.comwrote:
On Tue, Feb 12, 2013 at 1:54 AM, matei.da...@gmail.com wrote:
With the script below, I'd expect any fd pointing to /dev/null to be
closed when the second llfd() is executed. Surprisingly, fd 3 is closed,
but fd 10 is now open, pointing to /dev/null, as if eval copied it instead
of closing it. Is this a bug?
Thanks,
M
$ bash -c 'llfd () { ls -l /proc/$BASHPID/fd/; }; x=3; eval exec
$x/dev/null; llfd; eval llfd $x-'
total 0
lrwx-- 1 matei matei 64 Feb 11 18:36 0 - /dev/pts/2
lrwx-- 1 matei matei 64 Feb 11 18:36 1 - /dev/pts/2
lrwx-- 1 matei matei 64 Feb 11 18:36 2 - /dev/pts/2
l-wx-- 1 matei matei 64 Feb 11 18:36 3 - /dev/null
lr-x-- 1 matei matei 64 Feb 11 18:36 8 - /proc/4520/auxv
total 0
lrwx-- 1 matei matei 64 Feb 11 18:36 0 - /dev/pts/2
lrwx-- 1 matei matei 64 Feb 11 18:36 1 - /dev/pts/2
l-wx-- 1 matei matei 64 Feb 11 18:36 10 - /dev/null
lrwx-- 1 matei matei 64 Feb 11 18:36 2 - /dev/pts/2
lr-x-- 1 matei matei 64 Feb 11 18:36 8 - /proc/4520/auxv
$ bash --version
GNU bash, version 4.2.24(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
http://gnu.org/licenses/gpl.html
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$
Note that the same happens without using eval:
$ llfd 3-
total 0
lrwx-- 1 pgas pgas 64 Feb 12 08:00 0 - /dev/pts/0
lrwx-- 1 pgas pgas 64 Feb 12 08:00 1 - /dev/pts/0
l-wx-- 1 pgas pgas 64 Feb 12 08:00 10 - /dev/null
lrwx-- 1 pgas pgas 64 Feb 12 08:00 2 - /dev/pts/0
lrwx-- 1 pgas pgas 64 Feb 12 08:00 255 - /dev/pts/0
But you need to consider what process you are examining, you use a
function and you examine the file descriptors of the process where this
function runs.
A function runs in the same process as the parent shell, if it simply
closes 3 then there will be no more fd opened on /dev/null in the parent
shell when the function returns
So what bash does is a little juggling with the file descriptors, moving 3
temporarily to be able to restore it.
Re: eval doesn't close file descriptor?
On Tue, Feb 12, 2013 at 11:07:06AM -0500, Matei David wrote:
On a different but related note, I hate having to do eval to manipulate an
fd stored in a variable. Why doesn't 'llfd $x-' work, especially since
'llfd $x' works just fine... so by the time is handled, the variable
substitutions seem to be done already.
Each redirection that may be preceded by a file descriptor number may
instead be preceded by a word of the form {varname}. In this case,
for each redirection operator except - and -, the shell will
allocate a file descriptor greater than 10 and assign it to varname.
If - or - is preceded by {varname}, the value of varname defines
the file descriptor to close.
This was added in Bash 4.1.
Re: [Parameter Expansion] bug in ${variable% *}
Greg Wooledge wool...@eeg.ccf.org writes: Then you are still doing something wrong. No matter how badly the script is written, bash should execute it properly, which it doesn't. Try running it under valgrind in a multibyte locale and you will see the error immediately: ==18677== Conditional jump or move depends on uninitialised value(s) ==18677==at 0x4504B6: xdupmbstowcs (xmbsrtowcs.c:240) ==18677==by 0x42A43F: remove_pattern (subst.c:3964) ==18677==by 0x42F3B7: parameter_brace_expand (subst.c:4543) ==18677==by 0x466E48: param_expand (subst.c:7651) ==18677==by 0x467C91: expand_word_internal (subst.c:8139) ==18677==by 0x467F24: expand_word_internal (subst.c:8285) ==18677==by 0x42F981: call_expand_word_internal.constprop.15 (subst.c:3197) ==18677==by 0x42FD94: expand_string_assignment (subst.c:3283) ==18677==by 0x466BB3: expand_string_if_necessary (subst.c:3046) ==18677==by 0x47632F: do_assignment_internal (subst.c:2772) ==18677==by 0x469726: expand_word_list_internal (subst.c:2866) ==18677==by 0x45CCB3: execute_simple_command (execute_cmd.c:3776) ==18677== Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 And now for something completely different.
Re: eval doesn't close file descriptor?
On Tue, Feb 12, 2013 at 6:07 PM, Matei David matei.da...@gmail.com wrote:
Ok, but I see the same behaviour when eval runs in a subshell:
$ bash -c 'llfd () { echo pid:$BASHPID 2; ls -l /proc/$BASHPID/fd/
2; }; x=3; eval exec $x/dev/null; llfd; echo | eval llfd $x-'
[same output, fd 10 open, pointing to /dev/null, even though it's a
subshell]
eval runs in a subshell, but it's the same thing inside this subshell.
eg you could have: echo | { eval llfd $x-; echo blah 3; }
Bash could optimize this once it realizes there's only one command, but
it's probably not that simple to implement.
Try with a function that spawns a subshell eg:
llfd () ( echo pid:$BASHPID 2; ls -l /proc/$BASHPID/fd/ 2; )
or llfd () { bash -c 'ls -l /proc/$$/fd' ; }
Re: [Parameter Expansion] bug in ${variable% *}
On 2/12/13 11:37 AM, Andreas Schwab wrote: Greg Wooledge wool...@eeg.ccf.org writes: Then you are still doing something wrong. No matter how badly the script is written, bash should execute it properly, which it doesn't. Try running it under valgrind in a multibyte locale and you will see the error immediately: There is an error in xdupmbstowcs, but beyond that the output from valgrind is fairly useless, all the more so because you didn't post the version (most likely the 4.2 patch level) you're using. If you're using patch 42, the line number corresponds to a return of a value that's always initialized. In any event, the problem is in a different place, and I've fixed it for the next version. Chet ==18677== Conditional jump or move depends on uninitialised value(s) ==18677==at 0x4504B6: xdupmbstowcs (xmbsrtowcs.c:240) ==18677==by 0x42A43F: remove_pattern (subst.c:3964) ==18677==by 0x42F3B7: parameter_brace_expand (subst.c:4543) ==18677==by 0x466E48: param_expand (subst.c:7651) ==18677==by 0x467C91: expand_word_internal (subst.c:8139) ==18677==by 0x467F24: expand_word_internal (subst.c:8285) ==18677==by 0x42F981: call_expand_word_internal.constprop.15 (subst.c:3197) ==18677==by 0x42FD94: expand_string_assignment (subst.c:3283) ==18677==by 0x466BB3: expand_string_if_necessary (subst.c:3046) ==18677==by 0x47632F: do_assignment_internal (subst.c:2772) ==18677==by 0x469726: expand_word_list_internal (subst.c:2866) ==18677==by 0x45CCB3: execute_simple_command (execute_cmd.c:3776) ==18677== Andreas. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: [Parameter Expansion] bug in ${variable% *}
Bash version: 4.2.042
I have a script that behaves erratically:
Thanks for the report. The reason it is erratic and not always
reproducible in the same spot is that it is the result of an array
bounds error, and so depends on the contents of memory beyond the
end of a string.
I've fixed the problem, and the fix will be in the next release. If
you're interested in the fix, I've attached a patch.
Chet
*** ../bash-4.2-patched/lib/glob/xmbsrtowcs.c 2012-07-08 21:53:19.0
-0400
--- lib/glob/xmbsrtowcs.c 2013-02-12 12:00:39.0 -0500
***
*** 217,220
--- 217,226
n = mbsnrtowcs(wsbuf+wcnum, p, nms, wsbuf_size-wcnum, state);
+ if (n == 0 p == 0)
+ {
+ wsbuf[wcnum] = L'\0';
+ break;
+ }
+
/* Compensate for taking single byte on wcs conversion failure above. */
if (wcslength == 1 (n == 0 || n == (size_t)-1))
***
*** 222,226
state = tmp_state;
p = tmp_p;
! wsbuf[wcnum++] = *p++;
}
else
--- 228,238
state = tmp_state;
p = tmp_p;
! wsbuf[wcnum] = *p;
! if (*p == 0)
! break;
! else
! {
! wcnum++; p++;
! }
}
else
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: [Parameter Expansion] bug in ${variable% *}
On 11.02.2013 18:50, Dashing wrote:
Bash version: 4.2.042
I have a script that behaves erratically:
=
#! /bin/bash
last=${1##* }
rest=${1% *}
while [[ ${rest: -1} == '\' ]]; do
last=${rest##* } $last
oldrest=$rest
rest=${rest% *}
if [[ $oldrest == $rest ]]; then
echo :--Mistake--:
# sleep 0.01
# rest=${rest% *}
# if [[ $oldrest == $rest ]]; then
# echo 'unable to interpret'
# break
# fi
fi
done
echo REST:$rest:
echo LAST:$last:
=
$ ./pe 'mplayer foo1\ foo2\ foo3\ 4\ 5\ foo6\ 7'
:--Mistake--:
:--Mistake--:
REST:mplayer:
LAST:foo1\ foo1\ foo1\ foo2\ foo3\ 4\ 5\ foo6\ 7:
=
What happens is that rest=${rest% *} doesn't always update $rest,
even when there are spaces left in $rest.
Meanwhile last=${rest##* } $last works every time.
In the above example it got stuck twice when $rest was mplayer
foo1\. It tends to vary wildly.
The script runs fine if you omit the superfluous blank in the ${rest:
-1} expression:
while [[ ${rest:-1} == '\' ]]; do
If the script behaves indeterministic, it is probably a bash bug.
Regards,
Bernd
--
www.sudrala.de
Re: eval doesn't close file descriptor?
Wow, thanks, I didn't know that. So this syntax achieves a bit more than I
asked for- it allows you to get a new unused file descriptor, right? It
seems that the only useful way to use a non-closing form (-, -) is with
exec, as in 'exec {new_fd}2'. (Why would I want the fd in a variable
otherwise.)
Too bad the natural syntax 'llfd $x-' doesn't work, but I guess this
will do.
On Tue, Feb 12, 2013 at 11:12 AM, Greg Wooledge wool...@eeg.ccf.org wrote:
On Tue, Feb 12, 2013 at 11:07:06AM -0500, Matei David wrote:
On a different but related note, I hate having to do eval to manipulate
an
fd stored in a variable. Why doesn't 'llfd $x-' work, especially since
'llfd $x' works just fine... so by the time is handled, the variable
substitutions seem to be done already.
Each redirection that may be preceded by a file descriptor number may
instead be preceded by a word of the form {varname}. In this case,
for each redirection operator except - and -, the shell will
allocate a file descriptor greater than 10 and assign it to varname.
If - or - is preceded by {varname}, the value of varname defines
the file descriptor to close.
This was added in Bash 4.1.
Re: [Parameter Expansion] bug in ${variable% *}
On Tue, 12 Feb 2013 18:02:05 +0100 Chet Ramey
I've fixed the problem, and the fix will be in the next release.
Thank you, Chet!
On Tue, 12 Feb 2013 10:13:46 +0100 Bernd Eggink
superfluous blank in the ${rest: -1} expression
Bernd, this blank makes a big difference:
${rest: -1} last character of $rest
${rest:-1} $rest or 1 if $rest is empty
Re: printf question
On Tue, Feb 12, 2013 at 04:20:53AM -0800, laurent.tes...@gmail.com wrote:
liste=`ls *.T02`
This is broken because filenames may contain spaces, newlines, etc.
Use an array instead:
liste=(*.T02)
See http://mywiki.wooledge.org/BashPitfalls for an explanation of some
of the issues.
This type of question is more appropriate for the help-bash mailing list,
by the way.
1 teqc.exe `ls *.T02` out.T02
2 teqc.exe $liste out.T02
3 teqc.exe $listeout.T02
All of those are wrong, each in its own way.
What is the proper way of doing this ?
Well, the easiest way would be:
teqc.exe *.T02 out.T02
If you insist on storing the filenames in advance for some reason, use
an array:
files=(*.T02)
teqc.exe ${files[@]} out.T02
If you run into command line length issues, then you may have to process
the array in several chunks. See http://mywiki.wooledge.org/BashFAQ/095
for examples of that.
Re: eval doesn't close file descriptor?
So in other words, you're saying I should use '... | eval exec $x-;
llfd' instead of '... | eval llfd $x-'. This way the subshell won't be
assuming I might use $x later. That works, but I still find it
counterintuitive that with the original syntax the subshell doesn't realize
there's nothing left to execute after $x-.
Also, I tried your other suggestions. The second one 'llfd () { bash -c
}' works, but the other 'llfd () ( ... )' doesn't! I tried to
understand why...
Looking at this:
$ bash -c 'llfd () { echo pid:$BASHPID 2; ls -gG /proc/$BASHPID/fd 2;
}; f () { llfd; (llfd); bash -c echo pid:\$\$ 2; ls -gG /proc/\$\$/fd
2; }; x=3; exec 4/tmp/fd_4; coproc cat; eval exec $x/tmp/fd_3; llfd;
echo | eval f $x-'
pid:14920
total 0
lrwx-- 1 64 Feb 12 14:03 0 - /dev/pts/2
lrwx-- 1 64 Feb 12 14:03 1 - /dev/pts/2
lrwx-- 1 64 Feb 12 14:03 2 - /dev/pts/2
l-wx-- 1 64 Feb 12 14:03 3 - /tmp/fd_3
l-wx-- 1 64 Feb 12 14:03 4 - /tmp/fd_4
l-wx-- 1 64 Feb 12 14:03 60 - pipe:[5010928]
lr-x-- 1 64 Feb 12 14:03 63 - pipe:[5010927]
lr-x-- 1 64 Feb 12 14:03 8 - /proc/4520/auxv
pid:14924
total 0
lr-x-- 1 64 Feb 12 14:03 0 - pipe:[5007145]
lrwx-- 1 64 Feb 12 14:03 1 - /dev/pts/2
l-wx-- 1 64 Feb 12 14:03 10 - /tmp/fd_3
lrwx-- 1 64 Feb 12 14:03 2 - /dev/pts/2
l-wx-- 1 64 Feb 12 14:03 4 - /tmp/fd_4
lr-x-- 1 64 Feb 12 14:03 8 - /proc/4520/auxv
pid:14926
total 0
lr-x-- 1 64 Feb 12 14:03 0 - pipe:[5007145]
lrwx-- 1 64 Feb 12 14:03 1 - /dev/pts/2
l-wx-- 1 64 Feb 12 14:03 10 - /tmp/fd_3
lrwx-- 1 64 Feb 12 14:03 2 - /dev/pts/2
l-wx-- 1 64 Feb 12 14:03 4 - /tmp/fd_4
lr-x-- 1 64 Feb 12 14:03 8 - /proc/4520/auxv
pid:14928
total 0
lr-x-- 1 64 Feb 12 14:03 0 - pipe:[5007145]
lrwx-- 1 64 Feb 12 14:03 1 - /dev/pts/2
lrwx-- 1 64 Feb 12 14:03 2 - /dev/pts/2
l-wx-- 1 64 Feb 12 14:03 4 - /tmp/fd_4
lr-x-- 1 64 Feb 12 14:03 8 - /proc/4520/auxv
... there seem to be not 2 but 3(!) types of file descriptors:
1. fds which are copied across both subshells and exec; like 4
2. fds which are not copied across subshells; like 6063
3. fds which are copied across subshells, but not exec; like 10
I knew about types 12, but not about type 3. Apparently with your first
suggestion, fd 10 is created and survives a subshell creation. Is this
correct??
On Tue, Feb 12, 2013 at 11:40 AM, Pierre Gaston pierre.gas...@gmail.comwrote:
On Tue, Feb 12, 2013 at 6:07 PM, Matei David matei.da...@gmail.comwrote:
Ok, but I see the same behaviour when eval runs in a subshell:
$ bash -c 'llfd () { echo pid:$BASHPID 2; ls -l /proc/$BASHPID/fd/
2; }; x=3; eval exec $x/dev/null; llfd; echo | eval llfd $x-'
[same output, fd 10 open, pointing to /dev/null, even though it's a
subshell]
eval runs in a subshell, but it's the same thing inside this subshell.
eg you could have: echo | { eval llfd $x-; echo blah 3; }
Bash could optimize this once it realizes there's only one command, but
it's probably not that simple to implement.
Try with a function that spawns a subshell eg:
llfd () ( echo pid:$BASHPID 2; ls -l /proc/$BASHPID/fd/ 2; )
or llfd () { bash -c 'ls -l /proc/$$/fd' ; }
Re: eval doesn't close file descriptor?
On 2/12/13 2:07 PM, Matei David wrote: ... there seem to be not 2 but 3(!) types of file descriptors: 1. fds which are copied across both subshells and exec; like 4 2. fds which are not copied across subshells; like 6063 3. fds which are copied across subshells, but not exec; like 10 I knew about types 12, but not about type 3. Apparently with your first suggestion, fd 10 is created and survives a subshell creation. Is this correct?? Yes, file descriptors used to save the state of other file descriptors are set close-on-exec. `man fcntl' for a description. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: eval doesn't close file descriptor?
Hi Chet, Conceptually, the availability of a single flag close-on-exec (BTW, is there a way to check those flags in bash or using /proc?) should create only 2, not 3 types of file descriptors- that's what I find confusing. Does subprocess creation as in '(llfd)' entail an execve() call? My guess is that no, because the extended environment like arrays and function definitions would not survive it, so you'd have to copy them by hand. Either way: If subprocess creation DOES NOT perform execve(), then I don't understand fd's of type 2, like 6063: they should continue to exist in a subshell. If subprocess creation DOES perform execve(), then I don't understand fd's of type 3: they should not exist in a subshell. My best guess here is that type 2 is non-standard? Meaning that they are closed by bash during subprocess creation, even though there is no execve()? I only came across those when trying to use coproc... I wonder why this decision was made. Generally speaking, it took me quite some time to figure out how to properly create a double pipe without using any intermediate files or fifos. The concept is very easy: in - tee - two parallel, independent pipes - join - out. A first complication is the limited pipe capacity, and the possibility of one side getting stuck if the other stops pulling in data. I then wrote a cat-like program which doesn't block on stdout, but keeps reading stdin, buffering in as much data as needed. I used it at the very end of either pipe. But then I had to create the actual processes, and then I stumbled upon all these issues with coproc and file descriptors. You leave the wrong one open and the thing gets stuck... I wish there was a howto on this subject. Thanks, M On Tue, Feb 12, 2013 at 2:50 PM, Chet Ramey chet.ra...@case.edu wrote: On 2/12/13 2:07 PM, Matei David wrote: ... there seem to be not 2 but 3(!) types of file descriptors: 1. fds which are copied across both subshells and exec; like 4 2. fds which are not copied across subshells; like 6063 3. fds which are copied across subshells, but not exec; like 10 I knew about types 12, but not about type 3. Apparently with your first suggestion, fd 10 is created and survives a subshell creation. Is this correct?? Yes, file descriptors used to save the state of other file descriptors are set close-on-exec. `man fcntl' for a description. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.edu http://cnswww.cns.cwru.edu/~chet/
Re: eval doesn't close file descriptor?
On 2/12/13 2:07 PM, Matei David wrote: OK. Let me try this again. One key to the explanations is that child processes inherit all of the parent's open file descriptors. ... there seem to be not 2 but 3(!) types of file descriptors: 1. fds which are copied across both subshells and exec; like 4 File descriptors that are explicitly created by user redirections are inherited and not set close-on-exec. 2. fds which are not copied across subshells; like 6063 You created a coproc; bash opened pipes to the coproc and is careful to close those when child processes are created. Having, say, the read end of a pipe open in multiple processes really messes with SIGPIPE generation. Multiple fds open for write means that read will never see EOF, even if the process you intend to write to it exits. This is true for pipes in general, so bash is very careful to keep track of file descriptors open on pipes and close them in child processes. 3. fds which are copied across subshells, but not exec; like 10 File descriptors which are used internally to save state of file descriptors used in redirections are set close-on-exec. I knew about types 12, but not about type 3. Apparently with your first suggestion, fd 10 is created and survives a subshell creation. Is this correct?? Yes. A subshell gets copies of its parent's file descriptors. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: eval doesn't close file descriptor?
On Tue, Feb 12, 2013 at 04:22:08PM -0500, Matei David wrote: Generally speaking, it took me quite some time to figure out how to properly create a double pipe without using any intermediate files or fifos. The concept is very easy: in - tee - two parallel, independent pipes - join - out. A first complication is the limited pipe capacity, and the possibility of one side getting stuck if the other stops pulling in data. I then wrote a cat-like program which doesn't block on stdout, but keeps reading stdin, buffering in as much data as needed. I used it at the very end of either pipe. But then I had to create the actual processes, and then I stumbled upon all these issues with coproc and file descriptors. You leave the wrong one open and the thing gets stuck... I wish there was a howto on this subject. I really like to see these threads that poke at the edges of things in bash, providing a great learning opportunity for lurkers like me. But that said, perhaps what you're trying to so might be handled easily by socat; from http://www.dest-unreach.org/socat/doc/README: socat is a relay for bidirectional data transfer between two independent data channels. Each of these data channels may be a file, pipe, device..., a socket..., a file descriptor (stdin etc.) ... Just a thought. Ken
Re: [Parameter Expansion] bug in ${variable% *}
On 12.02.2013 18:50, Dashing wrote:
On Tue, 12 Feb 2013 18:02:05 +0100 Chet Ramey
I've fixed the problem, and the fix will be in the next release.
Thank you, Chet!
On Tue, 12 Feb 2013 10:13:46 +0100 Bernd Eggink
superfluous blank in the ${rest: -1} expression
Bernd, this blank makes a big difference:
${rest: -1} last character of $rest
${rest:-1} $rest or 1 if $rest is empty
You're right, of course. Sigh. Fortunately I've not yet fallen into that
pithole...
Bernd
--
www.sudrala.de
Re: eval doesn't close file descriptor?
On 2/12/13 11:40 AM, Pierre Gaston wrote:
On Tue, Feb 12, 2013 at 6:07 PM, Matei David matei.da...@gmail.com wrote:
Ok, but I see the same behaviour when eval runs in a subshell:
$ bash -c 'llfd () { echo pid:$BASHPID 2; ls -l /proc/$BASHPID/fd/
2; }; x=3; eval exec $x/dev/null; llfd; echo | eval llfd $x-'
[same output, fd 10 open, pointing to /dev/null, even though it's a
subshell]
eval runs in a subshell, but it's the same thing inside this subshell.
eg you could have: echo | { eval llfd $x-; echo blah 3; }
Bash could optimize this once it realizes there's only one command, but
it's probably not that simple to implement.
The basic flow is like this for any builtin command or shell function that
has a redirection (let's choose 'llfd 3-').
1. The redirection is performed in the current shell, noting that it
should be `undoable'. That takes three steps:
1a. In this case, since fd 3 is in use, we dup it (to fd 10) and mark fd
10 as close-on-exec. We add a separate redirection to an internal
list that basically says close fd 10. Then we add another
redirection to the front of the same internal list that says dup fd
10 back to fd 3. Let's call this list redirection_undo_list. We
will use it to restore the original state after the builtin or
function completes.
1b. Take the first redirection from step 1a and add it to a separate
internal list that will clean up internal redirections in the case
that exec causes the redirections to be preserved, and not undone.
Let's call this list exec_redirection_undo_list.
1c. Perform the redirection. Here, that means close fd 3.
[perform step 1 for each redirection associated with the command]
2. If we're running the exec builtin, throw away the list from 1a. If
we're not running the exec builtin, throw away the list from 1b. Save
a handle to the list we didn't discard.
3. Run the function or builtin.
4. Take the list saved in step 2 and perform the redirections to
restore the previous state. Here, that means we dup fd 10 back to fd
3, then close fd 10.
If you look at the steps, it should be clear why fd 10 is still open when
llfd executes.
Bash `cheats' when running builtins or shell functions in pipelines or
other subshells. It knows it's already going to be in a child process
when it performs the redirections, so it doesn't bother setting up the
structures to undo them.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
