Re: Syntax error with "command . file" (was: [1003.1(2016/18)/Issue7+TC2 0001629]: Shell vs. read(2) errors on the script)
On 12/03/2023 19:10, Robert Elz wrote:
Date:Fri, 10 Mar 2023 23:40:18 +
From:"Harald van Dijk via austin-group-l at The Open Group"
Message-ID:
| Based on past experiences, I am assuming the e-mail this is a reply to
| was meant to be sent to the list and I am quoting it in full and
| replying on the list for that reason.
Thanks, and yes, it was - my MUA absolutely believes in the one true
meaning of Reply-To (where the author of the message to which the reply
is being sent requests that replies be sent -- to addresses in that field,
and no others). I need to manually override it when I choose to ignore
that request and send to different addresses (which is allowed, but in
general, done only with proper consideration of why). This list always
directs that all replies go only to the author of the message, and never
to the list itself. Irritating...
I know. Even more frustrating is the reasoning we have been given as to
the reasoning for it ("to better handle DMARC email authentication for
messages" -- no, DMARC absolutely does not require that).
| Sourcing arbitrary script fragments and having assurance that they do
| not exit the shell is not reasonable, as the arbitrary script fragment
| could contain an 'exit' command.
Of course, deliberate exits aren't the issue, only accidental ones.
| Beyond shell options and variable assignments not persisting in the
| parent shell, are there any other issues you see with running them in a
| subshell?
The whole point of many . scripts is to alter the shell's environment,
if they were just arbitrary commands, not intended to affect the current
shell, they'd just be sh scripts, and run the normal way. The very act
of using the '.' command more or less means "must run in the current shell).
"(. file)" is silly, "file" would accomplish the same thing (if executable,
otherwise "sh < file" after finding the path to file) in a more obvious way.
This does not result in the same behaviour. (. file) can be used to
inherit unexported shell variables, shell functions, aliases, etc. from
the parent shell that would not be available to an explicitly launched
sh subprocess.
I agree that it is likely that a . script would be sourced in order to
change the current shell environment, but I do not agree that it is the
only legitimate use of it, and I had assumed based on your "without
risking the shell exiting" that you were considering one of those other
uses.
Apart from options and variables, . files often define functions, change
the umask and perhaps ulimit, and may alter the current directory, set exit
(or other) traps, ... anything in fact.
Sure, they are in the same category as options and variables.
As an example, consider what you might put in your .profile or $ENV
file - those are run in more or less the same way as a '.' file (just
without the PATH search to locate the file). XRAT C.2.5.3 says almost
exactly that about ENV.
Indeed. Worth explicitly pointing out here is that they are only used by
interactive shells, so in this comparison, only the behaviour of the '.'
command in interactive shells is relevant.
(Strangely though, even though .profile is
mentioned several times as a place where things can be set, it doesn't
appear in the standard (as something that shells process) at all - which
is kind of odd really, since it is considerably older then ENV, and as best
I can tell, supported by everything. The closest that we get is a mention
in XRAT that "some shells" run it at startup of a login shell. Which are
the other shells? That is, the ones that don't run .profile? And I don't
mean in situations like bash, which prefers .bash_profile if it exists.
bash appears to disables the reading of .profile in POSIX mode entirely.
yash only ever appears to source .yash_profile, it does not fall back to
.profile if no .yash_profile exists.
I doubt that you'd want those scripts run in a subshell environment,
I also doubt that you want the shell to exit if there's an error in
one of them. How would you ever be able to log in (and start a shell)
if it exited before you ever had a chance to run a command? If you can't
log in, because your shell won't start, how would you ever fix the problem?
As best I can tell (I have done very limited testing of this) shells tend
to simply abort processing one of those scripts upon encountering an error
(like a syntax error, etc - not executing "exit" - that should exit) and
just go on to the next step of initialising the shell. They don't just
exit because there's a syntax error - most shells report the error (not all),
but I couldn't find one which exits.
This is indeed a problem. My own testing supports your conclusion,
shells are in agreement that a syntax error here does not result in the
shell terminating, and does result in the remainder of the envfile being
ignored. An inval
Re: [1003.1(2016/18)/Issue7+TC2 0001640]: The rationale given for retaining "true" is nonsense.
2023-03-12 14:04:55 -0400, Lawrence Velázquez via austin-group-l at The Open
Group:
[...]
> > What happens (exit status etc) if there's a write error while writing that
> > output?
>
> % /opt/local/libexec/gnubin/true --version >&-; echo "$?"
> true: write error: Bad file descriptor
> 1
>
> Interestingly, the documentation anticipates this concern:
> https://www.gnu.org/software/coreutils/manual/html_node/true-invocation.html
[...]
In any case a command can exit with a non-zero status for all
sorts of pathological cases like when:
- it's not invoked as meant to (here as far as POSIX is
concerned if given any argument other than --).
- it fails to allocate memory
- it runs out of stack size
- the dynamic linker can't find some dependencies, or is told to
load the wrong ones (LD_LIBRARY_PATH, LD_PRELOAD, LD_DEBUG...)
- it's killed (including from reaching some limits, writing to
a closed pipe/socket...)
- fails to decode its arguments as text. Here, - being part of
the portable character set and should be single byte and
invariant across locales (or is it?), there's no reason why a
true that doesn't implement extensions over the standard want
to decode its args as text but one that does might.
- for the above, the LC_* variable and locale processing could
trigger errors.
- and likely plenty other reasons I'm not thinking of why it may
fail. Consider that some implementations like busybox' are
toybox' are quite complex. For instance busybox' true will do
something else if passed a argv[0] that is not true or doesn't
end in /true.
But the takeaway here is that:
true "$var"
is not POSIX and *in practice not portable* as at least one
implementation (GNU) doesn't ignore its arguments (and
toybox/busybox don't ignore argv[0]), so true can't be used in
place of : for things like:
: "${QUERYSTRING-$1}"
or:
: "$(( i -= 1 ))"
printf '' or command : could be used if the point is to avoid
the specialness of ":". Or redefine true as:
true() { : ; }
Or:
true() case for in esac
or even:
true() { command true; }
--
Stephane
Re: [1003.1(2016/18)/Issue7+TC2 0001640]: The rationale given for retaining "true" is nonsense.
Thanks, I think that's all we needed to know about what it does. kre
Re: Syntax error with "command . file" (was: [1003.1(2016/18)/Issue7+TC2 0001629]: Shell vs. read(2) errors on the script)
Date:Fri, 10 Mar 2023 23:40:18 + From:"Harald van Dijk via austin-group-l at The Open Group" Message-ID: | Based on past experiences, I am assuming the e-mail this is a reply to | was meant to be sent to the list and I am quoting it in full and | replying on the list for that reason. Thanks, and yes, it was - my MUA absolutely believes in the one true meaning of Reply-To (where the author of the message to which the reply is being sent requests that replies be sent -- to addresses in that field, and no others). I need to manually override it when I choose to ignore that request and send to different addresses (which is allowed, but in general, done only with proper consideration of why). This list always directs that all replies go only to the author of the message, and never to the list itself. Irritating... | Sourcing arbitrary script fragments and having assurance that they do | not exit the shell is not reasonable, as the arbitrary script fragment | could contain an 'exit' command. Of course, deliberate exits aren't the issue, only accidental ones. | Beyond shell options and variable assignments not persisting in the | parent shell, are there any other issues you see with running them in a | subshell? The whole point of many . scripts is to alter the shell's environment, if they were just arbitrary commands, not intended to affect the current shell, they'd just be sh scripts, and run the normal way. The very act of using the '.' command more or less means "must run in the current shell). "(. file)" is silly, "file" would accomplish the same thing (if executable, otherwise "sh < file" after finding the path to file) in a more obvious way. Apart from options and variables, . files often define functions, change the umask and perhaps ulimit, and may alter the current directory, set exit (or other) traps, ... anything in fact. As an example, consider what you might put in your .profile or $ENV file - those are run in more or less the same way as a '.' file (just without the PATH search to locate the file). XRAT C.2.5.3 says almost exactly that about ENV. (Strangely though, even though .profile is mentioned several times as a place where things can be set, it doesn't appear in the standard (as something that shells process) at all - which is kind of odd really, since it is considerably older then ENV, and as best I can tell, supported by everything. The closest that we get is a mention in XRAT that "some shells" run it at startup of a login shell. Which are the other shells? That is, the ones that don't run .profile? And I don't mean in situations like bash, which prefers .bash_profile if it exists. I doubt that you'd want those scripts run in a subshell environment, I also doubt that you want the shell to exit if there's an error in one of them. How would you ever be able to log in (and start a shell) if it exited before you ever had a chance to run a command? If you can't log in, because your shell won't start, how would you ever fix the problem? As best I can tell (I have done very limited testing of this) shells tend to simply abort processing one of those scripts upon encountering an error (like a syntax error, etc - not executing "exit" - that should exit) and just go on to the next step of initialising the shell. They don't just exit because there's a syntax error - most shells report the error (not all), but I couldn't find one which exits. | You have left out bash 4 here. For the same reason I didn't include ancient versions of all the other shells either. That's obsolete, not going to change in the future, and has been replaced. [And because I happen not to have a binary of it at the minute - I could make one, I do have sources, just don't really see the need.] | I do not expect bosh to have a large user base (even if it will be wider | than mine), but as I am sure J�rg would have pointed out, the shell has | historical significance in that it is a descendant of the Bourne shell | from which POSIX shell language is also derived. So is/was ksh88, and then ksh93 ... they were just modified more. | (Although I wouldn't be | opposed to a change to POSIX to *allow* something different.) As I hinted in the note in bugid:1629 which spawned this discussion (bugnote:6200) I expect this part might need to move to "may exit" rather than "shall not exit" (away from "shall exit" which it is now, in the cases in question, not all) for a release cycle (or two) - but then again given the number, and popularity, of the shells which already don't exit in these circumstances, perhaps that won't be needed. That should be discussed further. The reason that read errors are different in this regard (at least in the main script, not in . files -- not sure it is possible to have an equivalent to a read error in "eval" - perhaps an EILSEQ (bad char encoding) in the string might count? -- and that
Re: [1003.1(2016/18)/Issue7+TC2 0001640]: The rationale given for retaining "true" is nonsense.
On Sun, Mar 12, 2023, at 1:35 PM, Robert Elz via austin-group-l at The Open Group wrote: > With the GNU version, what would be more interesting to know, is what > it does when run as > true --nonsense > true -- > true '--:) (-;' > (and similar). What's the exit status, is there any output, and if > so, to stdout or stderr? % /opt/local/libexec/gnubin/true --nonsense; echo "$?" 0 % /opt/local/libexec/gnubin/true --; echo "$?" 0 % /opt/local/libexec/gnubin/true '--:) (-;'; echo "$?" 0 > I'm also assuming that the --version and --help (to be meaningful "accepted" > rather than just ignored - everyone's true allows and ignores those, and > any other args given) actually produce some output. stdout or stderr? They print messages to stdout. > What happens (exit status etc) if there's a write error while writing that > output? % /opt/local/libexec/gnubin/true --version >&-; echo "$?" true: write error: Bad file descriptor 1 Interestingly, the documentation anticipates this concern: https://www.gnu.org/software/coreutils/manual/html_node/true-invocation.html -- vq
Re: [1003.1(2016/18)/Issue7+TC2 0001640]: The rationale given for retaining "true" is nonsense.
Date:Sun, 12 Mar 2023 16:54:34 + From:Austin Group Bug Tracker Message-ID: <0a945390fc5d0c6c366071bcd2d29...@austingroupbugs.net> | A NOTE has been added to this issue. I don't think this discussion needs to be in notes, or not unless something relevant to the actual issue itself is revealed. | GNU true accepts some --version, --help options. That is perhaps not surprising - weird though, as if true was going to need multiple versions to get it right, or add features, or that anyone needs help writing "true" ...But OK, and apart from one potential issue (later). | I don't have access to ksh93 just now but I'd expect its true to supports | those as well as --author --man --usage and many more in that vein like | most of its builtins do. Not that I can see, it appears to ignore any operands to true, just as (almost) everyone else's (and all sane) versions do. With the GNU version, what would be more interesting to know, is what it does when run as true --nonsense true -- true '--:) (-;' (and similar). What's the exit status, is there any output, and if so, to stdout or stderr? I'm also assuming that the --version and --help (to be meaningful "accepted" rather than just ignored - everyone's true allows and ignores those, and any other args given) actually produce some output. stdout or stderr? What happens (exit status etc) if there's a write error while writing that output? kre
[1003.1(2016/18)/Issue7+TC2 0001640]: The rationale given for retaining "true" is nonsense.
A NOTE has been added to this issue. == https://austingroupbugs.net/view.php?id=1640 == Reported By:kre Assigned To: == Project:1003.1(2016/18)/Issue7+TC2 Issue ID: 1640 Category: Shell and Utilities Type: Error Severity: Objection Priority: normal Status: New Name: Robert Elz Organization: User Reference: Section:XCU 3 / true Page Number:3318 Line Number:111745 - 111748 Interp Status: --- Final Accepted Text: == Date Submitted: 2023-03-12 07:00 UTC Last Modified: 2023-03-12 16:54 UTC == Summary:The rationale given for retaining "true" is nonsense. == -- (0006202) stephane (reporter) - 2023-03-12 16:54 https://austingroupbugs.net/view.php?id=1640#c6202 -- GNU true accepts some --version, --help options. I don't have access to ksh93 just now but I'd expect its true to supports those as well as --author --man --usage and many more in that vein like most of its builtins do. Issue History Date ModifiedUsername FieldChange == 2023-03-12 07:00 kreNew Issue 2023-03-12 07:00 kreName => Robert Elz 2023-03-12 07:00 kreSection => XCU 3 / true 2023-03-12 07:00 krePage Number => 3318 2023-03-12 07:00 kreLine Number => 111745 - 111748 2023-03-12 16:54 stephane Note Added: 0006202 ==
