BASH {VARNAME}>&- redirection (was: Re: Konsole is not bash)

[Max Nikulin]

On 23/08/2023 02:59, Greg Wooledge wrote:

bar() {
 local fd
 foo {fd}>&1 1>&2 2>&${fd} {fd}>&-
}

Running this appears to gives the correct results -- outputs go to the
right places -- but it leaves the temporary FD open.  The final
redirection to close it doesn't work.


At first glance it is documented and effect of {fd}>&- is limited to the 
function call


info "(bash) Redirections"

If {VARNAME} is supplied, the redirection persists
beyond the scope of the command, allowing the shell programmer to manage
the file descriptor's lifetime manually.  The 'varredir_close' shell
option manages this behavior (*note The Shopt Builtin::).


info "(bash) The Shopt Builtin"

 'varredir_close'
  If set, the shell automatically closes file descriptors
  assigned using the '{varname}' redirection syntax (*note
  Redirections::) instead of leaving them open when the command
  completes.


However the redirection does not persist if an external executable is 
called instead of a BASH function. Perhaps it should be discussed with 
BASH developers.


foo() {
sh -c 'echo "internal $1" /proc/self/fd/*' foo "$1"
}
bar() {
local fd
sh -c "echo 'external before' /proc/self/fd/*"
sh -c "echo 'external closed' /proc/self/fd/*" {fd}>&1 1>&2 
2>&${fd} {fd}>&-

sh -c "echo 'external after ' /proc/self/fd/*"
echo "fd: $fd"
sh -c "echo 'external before' /proc/self/fd/*"
sh -c "echo 'external open  ' /proc/self/fd/*" {fd}>&1 1>&2 2>&${fd}
sh -c "echo 'external after ' /proc/self/fd/*"
echo "fd: $fd"
echo
sh -c "echo 'internal before' /proc/self/fd/*"
foo "closed" {fd}>&1 1>&2 2>&${fd} {fd}>&-
sh -c "echo 'internal after ' /proc/self/fd/*"
echo "fd: $fd"
if [ -n "$fd" ]; then exec {fd}>&-; fd=; fi
sh -c "echo 'internal before' /proc/self/fd/*"
foo "open  " {fd}>&1 1>&2 2>&${fd}
sh -c "echo 'internal after ' /proc/self/fd/*"
echo "fd: $fd"
if [ -n "$fd" ]; then exec {fd}>&-; fd=; fi
}
bar

Pay attention to {fd} that is is 10, other may be ignored, in particular 
/proc/self/fd opened by readdir is 3, some inotify is 20 below.


external before /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/2 
/proc/self/fd/20 /proc/self/fd/3
external closed /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/2 
/proc/self/fd/20 /proc/self/fd/3
external after  /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/2 
/proc/self/fd/20 /proc/self/fd/3

fd:
external before /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/2 
/proc/self/fd/20 /proc/self/fd/3
external open   /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/10 
/proc/self/fd/2 /proc/self/fd/20 /proc/self/fd/3
external after  /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/2 
/proc/self/fd/20 /proc/self/fd/3

fd:

internal before /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/2 
/proc/self/fd/20 /proc/self/fd/3
internal closed /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/2 
/proc/self/fd/20 /proc/self/fd/3
internal after  /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/10 
/proc/self/fd/2 /proc/self/fd/20 /proc/self/fd/3

fd: 10
internal before /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/2 
/proc/self/fd/20 /proc/self/fd/3
internal open   /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/10 
/proc/self/fd/2 /proc/self/fd/20 /proc/self/fd/3
internal after  /proc/self/fd/0 /proc/self/fd/1 /proc/self/fd/10 
/proc/self/fd/2 /proc/self/fd/20 /proc/self/fd/3

fd: 10

GNU bash, version 5.2.15(1)-release (x86_64-pc-linux-gnu)

Greg Wooledge

bar() {
 local fd rc
 foo {fd}>&1 1>&2 2>&${fd}
 rc=$?
 exec {fd}>&-
 return "$rc"
}


I have not tested if "trap" is more robust in the presence of "set -e -o 
pipefail". I know, there are enough pitfalls with this options.



At this point, the "pure sh" version looks a whole lot simpler,
doesn't it?


You mentioned the limitation: a spare file descriptor must be known.

Re: Konsole is not bash

[Greg Wooledge]

See, this is why I hate using bash extensions.  So many weird corner
cases and surprises

As it turns out, the answer I gave yesterday was only partially correct.
The "pure sh" solution is fine, but I offered this bash alternative:

cmd {fd}>&1 1>&2 2>&$fd {fd}>&-

This doesn't actually work as expected.  It fails to close the FD in the
final redirection.

What does it actually do?  Hell if I know.  I literally can't figure it
out.

Consider this function:

foo() { echo out; echo err >&2; }

Simple enough, right?  It writes "out" to stdout, and "err" to stderr.  We
can use this to see what's going where, while we play games with
redirections.

Next, we write a simple wrapper, which should in theory swap stdout and
stderr:

bar() { 
local fd
foo {fd}>&1 1>&2 2>&${fd} {fd}>&-
}

Running this appears to gives the correct results -- outputs go to the
right places -- but it leaves the temporary FD open.  The final
redirection to close it doesn't work.

But *this* one works:

bar() { 
local fd
foo {fd}>&1 1>&2 2>&${fd}
exec {fd}>&-
}

Why?  Again, I do not know.  My theories all fail to account for all of
the observed results.  And, of course, if we wanted to preserve the exit
status of foo, the second wrapper script doesn't do that.  It'll need
another variable to hold that:

bar() { 
local fd rc
foo {fd}>&1 1>&2 2>&${fd}
rc=$?
exec {fd}>&-
return "$rc"
}

At this point, the "pure sh" version looks a whole lot simpler,
doesn't it?

THIS is why I have a wiki devoted to bash and its problems.

Re: Konsole is not bash

[Greg Wooledge]

On Mon, Aug 21, 2023 at 09:19:00AM +0700, Max Nikulin wrote:
> On 21/08/2023 01:48, Greg Wooledge wrote:
> > Some shell features do change over time, but the significant
> > ordering of redirections has remained stable ever since the original
> > Bourne shell.
> 
> An exercise that relies on order of redirections and thus demonstrates its
> importance:
> 
> Swap stderr and stdout of a shell command.

Hmm.  I thought  might have
it, but it doesn't.  Some examples are pretty close, but none are an
exact match.

cmd 3>&1 1>&2 2>&3 3>&-

Obviously this relies on FD 3 not being in use at that point.  In pure
sh, you'd need to have knowledge of *some* FD number which is available.
In bash, you can use {somevar}>&1 to let the shell pick an FD for you,
but that's an extension.

cmd {fd}>&1 1>&2 2>&$fd {fd}>&-

Re: Konsole is not bash

[Max Nikulin]

On 21/08/2023 01:05, Felix Miata wrote:


In the most recent versions of Konsole I've started (5.27.x), the default 
profile
has inexplicably been changed from /bin/bash to /bin/sh.


Is it konsole or plasma version? May it happen that /bin/sh was just 
saved to your konsole config files or it is your shell specified in 
/etc/passwd?

Re: Konsole is not bash

[Max Nikulin]

On 21/08/2023 01:48, Greg Wooledge wrote:

Some shell features do change over time, but the significant
ordering of redirections has remained stable ever since the original
Bourne shell.


An exercise that relies on order of redirections and thus demonstrates 
its importance:


Swap stderr and stdout of a shell command.

Re: Konsole is not bash

[gene heskett]

On 8/20/23 14:23, Greg Wooledge wrote:

On Sun, Aug 20, 2023 at 02:05:49PM -0400, Felix Miata wrote:

Bob Weber composed on 2023-08-20 11:04 (UTC-0400):


gene heskett wrote:



I cannot make bashes redirection (cmd 2>&1 >tmp/cmd.log) work in Konsole. What
terminal actually uses bash for the heavy lifting?



In konsole its in the settings for the profile you are using.  Mine just says
bash not /usr/bin/bash.  If a profile uses ssh that will be there also.  Its
under "Settings/Edit current profile"  or  "Settings/Manage profiles".


In the most recent versions of Konsole I've started (5.27.x), the default 
profile
has inexplicably been changed from /bin/bash to /bin/sh.


The redirection syntax Gene is using is the same in all Bourne family
shells (anything that /bin/sh may legitimately point to, including
dash and bash).

Bash has a few extensions, of course, but none of them are in Gene's
original message.

If there's a legitimate concern that you might be in the wrong shell,
this command will tell you which shell is currently active:

 ps -p $$


Which does indeed confirm its bash.  Thanks Greg.


That works in all Bourne family and csh family shells, and on all Unix
systems (BSD- or System V-derived ps command syntax).

Avoid things like "echo $SHELL" because that may give misleading results.
The SHELL variable does *not* necessarily match the currently running
shell.

.


Cheers, Gene Heskett.
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page 

Re: Konsole is not bash

[John Hasler]

Gene writes:
> And that order of arguments is not mentioned in the bash scripting
> manual

It isn't an argument.  It's an instruction to the shell.  See the
REDIRECTION section of the bash man page.
-- 
John Hasler 
j...@sugarbit.com
Elmwood, WI USA

Re: Konsole is not bash

[Greg Wooledge]

On Sun, Aug 20, 2023 at 02:27:38PM -0400, gene heskett wrote:
> I should have prefaced cmd with a $ sign but forgot to add it in my post,
> cmd was meant to be whatever I wanted to trace, in this case digiKam which
> is the current 8.1.0 AppImage, probably updated by now as the do that about
> monthly but not yet.  Sorry for the confusion.  Greg sorted me out, and my
> several year old dead tree 500 page tome on bash scripting is out of date.

The way redirections work has not changed, ever.  It has always been
this way.  Some shell features do change over time, but the significant
ordering of redirections has remained stable ever since the original
Bourne shell.

Whatever reference material you're using isn't "out of date".  It's simply
incomplete (or perhaps confusing, or perhaps just plain wrong) as a
teaching text.

The lack of good books on shell scripting is one of the major reasons
why I've had to put so much effort into the Bash FAQ and other documents
on my wiki.  There are a lot of incorrect or incomplete sources out
there, and virtually *every* shell script in existence has massive bugs,
which don't trigger only because you don't have sufficiently evil
filenames to trigger them.  Sadly, that means people usually learn from
bad examples, and perpetuate the bugs that they see, without realizing
that they're doing it wrong.

Re: Konsole is not bash

[gene heskett]

On 8/20/23 10:52, Cindy Sue Causey wrote:

On 8/20/23, gene heskett  wrote:

I cannot make bashes redirection (cmd 2>&1 >tmp/cmd.log) work in
Konsole. What terminal actually uses bash for the heavy lifting?



Well, I started out attempting to play along in xfce4-terminal and received:

bash: tmp/cmd.log: No such file or directory

Next I tried konsole which surprised me when it was already installed.
That's when I noticed there's no "/" in front of tmp so I added it.
Was then advised that "cmd" is not installed. I took that as a sign of
progress.

That makes me now ask: What error message are you receiving IF you are
receiving error feedback? Mine's now acting like it would perform
properly if "cmd" was installed.

Cindy :)

N.B. Now remember why konsole is installed. It's still majorly doing
all kinds of hinky BAD things with text input. It's still throwing in
extra spacing and lunging text all over the place. YUCK.


My mistake Cindy. And I don't recall ever seeing it do that here. I like 
it because it can do tabs, so I have ssh sessions open to all my other 
machines all on one workspace. From the only comfy chair for that sort 
of thing in the whole house. ;o)>


I should have prefaced cmd with a $ sign but forgot to add it in my 
post, cmd was meant to be whatever I wanted to trace, in this case 
digiKam which is the current 8.1.0 AppImage, probably updated by now as 
the do that about monthly but not yet.  Sorry for the confusion.  Greg 
sorted me out, and my several year old dead tree 500 page tome on bash 
scripting is out of date.


Cheers, Gene Heskett.
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page 

Re: Konsole is not bash

[Greg Wooledge]

On Sun, Aug 20, 2023 at 02:05:49PM -0400, Felix Miata wrote:
> Bob Weber composed on 2023-08-20 11:04 (UTC-0400):
> 
> > gene heskett wrote:
> 
> >> I cannot make bashes redirection (cmd 2>&1 >tmp/cmd.log) work in Konsole. 
> >> What 
> >> terminal actually uses bash for the heavy lifting?
> 
> > In konsole its in the settings for the profile you are using.  Mine just 
> > says 
> > bash not /usr/bin/bash.  If a profile uses ssh that will be there also.  
> > Its 
> > under "Settings/Edit current profile"  or  "Settings/Manage profiles".
> 
> In the most recent versions of Konsole I've started (5.27.x), the default 
> profile
> has inexplicably been changed from /bin/bash to /bin/sh.

The redirection syntax Gene is using is the same in all Bourne family
shells (anything that /bin/sh may legitimately point to, including
dash and bash).

Bash has a few extensions, of course, but none of them are in Gene's
original message.

If there's a legitimate concern that you might be in the wrong shell,
this command will tell you which shell is currently active:

ps -p $$

That works in all Bourne family and csh family shells, and on all Unix
systems (BSD- or System V-derived ps command syntax).

Avoid things like "echo $SHELL" because that may give misleading results.
The SHELL variable does *not* necessarily match the currently running
shell.

Re: Konsole is not bash

[gene heskett]

On 8/20/23 10:36, Greg Wooledge wrote:

On Sun, Aug 20, 2023 at 10:28:44AM -0400, gene heskett wrote:

I cannot make bashes redirection (cmd 2>&1 >tmp/cmd.log) work in Konsole.
What terminal actually uses bash for the heavy lifting?


The terminal is irrelevant.  This is entirely done in the shell.

Your redirections are backwards.  If you want both stdout and stderr
in the same file, you need 2>&1 to be *last*.  Also, you *might* want
/tmp/cmd.log to be an absolute path.

 cmd >/tmp/cmd.log 2>&1

If tmp is a local directory relative to your working directory, then
ignore that change.  But the 2>&1 being last matters here.

it is local to /home/me.  And that order of arguments is not mentioned 
in the bash scripting manual I printed several years ago. About 500 
pages of it.


However it just now worked, I now have the log Maik on the digikam list 
requested, and my dead tree bash script docs are out of date.  Many 
thanks, take care & stay well, Greg.



See also .

.


Cheers, Gene Heskett.
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page 

Re: Konsole is not bash

[Felix Miata]

Bob Weber composed on 2023-08-20 11:04 (UTC-0400):

> gene heskett wrote:

>> I cannot make bashes redirection (cmd 2>&1 >tmp/cmd.log) work in Konsole. 
>> What 
>> terminal actually uses bash for the heavy lifting?

> In konsole its in the settings for the profile you are using.  Mine just says 
> bash not /usr/bin/bash.  If a profile uses ssh that will be there also.  Its 
> under "Settings/Edit current profile"  or  "Settings/Manage profiles".

In the most recent versions of Konsole I've started (5.27.x), the default 
profile
has inexplicably been changed from /bin/bash to /bin/sh.
-- 
Evolution as taught in public schools is, like religion,
based on faith, not based on science.

 Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!

Felix Miata

Re: Konsole is not bash

[Bob Weber]

On 8/20/23 10:28, gene heskett wrote:
I cannot make bashes redirection (cmd 2>&1 >tmp/cmd.log) work in Konsole. What 
terminal actually uses bash for the heavy lifting?


Cheers, Gene Heskett.


In konsole its in the settings for the profile you are using.  Mine just says 
bash not /usr/bin/bash.  If a profile uses ssh that will be there also.  Its 
under "Settings/Edit current profile"  or  "Settings/Manage profiles".


--


*...Bob*

Re: Konsole is not bash

[Cindy Sue Causey]

On 8/20/23, gene heskett  wrote:
> I cannot make bashes redirection (cmd 2>&1 >tmp/cmd.log) work in
> Konsole. What terminal actually uses bash for the heavy lifting?


Well, I started out attempting to play along in xfce4-terminal and received:

bash: tmp/cmd.log: No such file or directory

Next I tried konsole which surprised me when it was already installed.
That's when I noticed there's no "/" in front of tmp so I added it.
Was then advised that "cmd" is not installed. I took that as a sign of
progress.

That makes me now ask: What error message are you receiving IF you are
receiving error feedback? Mine's now acting like it would perform
properly if "cmd" was installed.

Cindy :)

N.B. Now remember why konsole is installed. It's still majorly doing
all kinds of hinky BAD things with text input. It's still throwing in
extra spacing and lunging text all over the place. YUCK.
-- 
Talking Rock, Pickens County, Georgia, USA
* runs with birdseed *

Re: Konsole is not bash

[Greg Wooledge]

On Sun, Aug 20, 2023 at 10:28:44AM -0400, gene heskett wrote:
> I cannot make bashes redirection (cmd 2>&1 >tmp/cmd.log) work in Konsole.
> What terminal actually uses bash for the heavy lifting?

The terminal is irrelevant.  This is entirely done in the shell.

Your redirections are backwards.  If you want both stdout and stderr
in the same file, you need 2>&1 to be *last*.  Also, you *might* want
/tmp/cmd.log to be an absolute path.

cmd >/tmp/cmd.log 2>&1

If tmp is a local directory relative to your working directory, then
ignore that change.  But the 2>&1 being last matters here.

See also .