Re: [1003.1(2016)/Issue7+TC2 0001341]: The resolution of bugid:1208 as amended by bugnote:4830 is incorrect
The Solaris posix_spawnp() system call implementation does has an action to change chdir and it will actually search the PATH in the kernel. While the kernel has the ability the execute the file actions after the exec as we are still in control, the native syscall posix_spawn implementation uses the exact order as a libc implementation is forced to obey. The PATH being searched is not necessarily the same PATH in the child process but that of the parent. Casper
Re: [1003.1(2016)/Issue7+TC2 0001341]: The resolution of bugid:1208 as amended by bugnote:4830 is incorrect
Date:Wed, 6 May 2020 13:20:40 +0100 From:Geoff Clare Message-ID: <20200506122040.GA24339@lt2.masqnet> | Why so averse to what is obviously the simplest and cleanest solution? Simple, perhaps (certainly for the libc code, not so much for the kernel) clean, not even close. | If you don't want to do the string processing in the kernel, then | you can have the userland function create an array of pathnames to | try loading. That's fractionally better, but not a lot. | That approach won't work because it will repeat the list of file actions, | some of which might not be idempotent. The only possibility I can see is open(O_CREAT|O_EXCL), or (O_CREATE, 0444) - I'd be willing to simply allow those ones to fail (tell people not to use those combinations with posix_spawnp() - they aren't very likely cases) if that is the sole drawback. open(O_CREAT) is the only thing that alters anything that isn't local to the new (child) process image that I can see (there's no unlink file action I don't believe - having that would cause problems), and everything done in that vanishes as soon as the posix_spawn() fails. The alternative seems to be either extreme ugliness, or not allowing the chdir file actions at all with posix_spawnp() - and I know which of those I'd prefer (especially as nothing uses it anyway...) | Also, how would you distinguish between a path related error caused | by the attempted image load from one caused by one of the file actions? You don't - you err on the side of caution, if it might be a path error, treat it that way. If it is a file action error, the effect will be that we get the same error once for each prefix in PATH (the error won't magically go away). That's a little ineffecient, but it's only for an unlikely case (an error case), so doesn't really matter. kre
Re: [1003.1(2016)/Issue7+TC2 0001340]: PATH specification has an ambiguity.
Date:Wed, 06 May 2020 20:07:00 +0700
From:Robert Elz
Message-ID: <18112.1588770...@jinx.noi.kre.to>
| OK, will do that (though what that will be is already there...)
Now done. See note 4866. The final text I didn't include as it
would be identical to what is in note 4862.
The one real change is to replace (in a slightly different place)
the words
known only by a filename
with
If the pathname being sought contains no slash ('/') characters,
and hence is a filename,
which is just a more long winded way of saying what the original was
trying to convey.That is, the list of prefixes in PATH is used only
when the pathname is "known only by a filename" - which the reader is
expected to know means "contains no '/' character" as that is what the
definition of "filename" in XBD 2 says it means.
The rest of the change is just a rearrangement of what is there now
to make it appear in (IMO) a more logical order. That is, the most
important thing is that we're taking each prefix in turn, applying
that to the filename, and seeing if it works... The details of how
to merge the prefix and filename in the various cases, is the less
important part, and so comes after we know why we care about those
details.
I also included a (not very good) suggestion of what to do to fix the
"only pathnames starying with a '/' are resolved" (apparent) wording
in the current text.
kre
[1003.1(2016)/Issue7+TC2 0001340]: PATH specification has an ambiguity.
A NOTE has been added to this issue.
==
https://www.austingroupbugs.net/view.php?id=1340
==
Reported By:shware_systems
Assigned To:
==
Project:1003.1(2016)/Issue7+TC2
Issue ID: 1340
Category: Base Definitions and Headers
Type: Clarification Requested
Severity: Objection
Priority: normal
Status: New
Name: Mark Ziegast
Organization: SHware Systems Dev.
User Reference:
Section:XBD 8.3
Page Number:178
Line Number:5854-7
Interp Status: ---
Final Accepted Text:
==
Date Submitted: 2020-05-04 16:28 UTC
Last Modified: 2020-05-06 14:39 UTC
==
Summary:PATH specification has an ambiguity.
==
--
(0004866) kre (reporter) - 2020-05-06 14:39
https://www.austingroupbugs.net/view.php?id=1340#c4866
--
Here are the editing instructions to get to text shown in
https://www.austingroupbugs.net/view.php?id=1340#c4862
as requested in https://www.austingroupbugs.net/view.php?id=1340#c4864
Al of this applies to the description of the environment variable PATH
in XBD 8.3, on page 178 (2016 edition).
In lines 5843-4 delete the words "known only by a filename".
In line 5844, after the sentence that ends "...separated by a
(':')."
insert the words (beginning a new sentence)
If the pathname being sought contains no slash ('/') characters,
and hence is a filename,
and then follow that by moving the sentence that starts on line 5851
The list shall be searched...
(up to the end of that sentence, on line 5854)
...permissions is found.
While moving that sentence, change the capital 'T' in the leading "The"
to lower case, as it is no longer the beginning of a sentence, just a
claise.
That's it, the rest of the text is unchanged (since in
https://www.austingroupbugs.net/view.php?id=1340#c4862 I did
not
attempt to fix the "resolve the path" issue.
What I might do for that is to replace the sentence that runs from line
5855-7, viz:
If the pathname begins with a , the specified path is
resolved (see Section 4.13, on page 111).
and replace it with
In each case, either the result of appending the filename to each
prefix
from PATH when the pathname (filename) contains no slash characters,
or the pathname when it does contain one or more slash characters,
is resolved (see Section 4.13, on page 111).
but I am less happy with that wording than the earlier part.
Someone please suggest something better.
Issue History
Date ModifiedUsername FieldChange
==
2020-05-04 16:28 shware_systems New Issue
2020-05-04 16:28 shware_systems Name => Mark Ziegast
2020-05-04 16:28 shware_systems Organization => SHware Systems Dev.
2020-05-04 16:28 shware_systems Section => XBD 8.3
2020-05-04 16:28 shware_systems Page Number => 178
2020-05-04 16:28 shware_systems Line Number => 5854-7
2020-05-04 19:31 kreNote Added: 0004846
2020-05-04 19:33 kreNote Added: 0004847
2020-05-04 20:05 shware_systems Note Added: 0004848
2020-05-05 03:00 kreNote Added: 0004849
2020-05-05 03:16 shware_systems Note Added: 0004850
2020-05-05 04:39 kreNote Added: 0004851
2020-05-05 05:03 kreNote Edited: 0004851
2020-05-05 05:04 kreNote Edited: 0004851
2020-05-05 05:11 kreNote Edited: 0004851
2020-05-05 05:13 kreNote Edited: 0004851
2020-05-05 08:40 geoffclare Note Added: 0004854
2020-05-05 08:46 geoffclare Note Added: 0004855
2020-05-05 08:47 geoffclare Note Edited: 0004855
2020-05-05 16:19 shware_systems Note Added: 0004861
2020-05-05
Re: [1003.1(2016)/Issue7+TC2 0001340]: PATH specification has an ambiguity.
Date:Wed, 6 May 2020 11:49:13 +0100 From:Geoff Clare Message-ID: <20200506104913.GA21901@lt2.masqnet> | There have certainly been bug resolutions in the past that have | included "move xxx to after yyy" type instructions. They are rare | because it's unusual to move something without also changing it. OK, I had not seen one. | If you are worried that specifying the changes piecemeal will make | it harder to review the end result, you could add: | | The end result of the above changes is: OK, will do that (though what that will be is already there...) | I think the standard only uses "command name" in the context of how | the shell parses commands. The amended fc -e description in the | standard should refer to editor as a "pathname". I wasn't suggesting using that text literally, just for the kind of think that might be done. kre ps: I am not here expecting that the wording I suggest for PATH should be adopted unchanged (though as I am personally adding so few words this one probably has a better chance than most) - fixing any wording I produce is always OK. Getting the conceptual ideas right is what matters most to me, not how that's expressed.
Re: [1003.1(2016)/Issue7+TC2 0001341]: The resolution of bugid:1208 as amended by bugnote:4830 is incorrect
Robert Elz wrote, on 06 May 2020: > > | On such a system, presumably posix_spawnp() would be a C > | library function > > Yes, so it is. > > | that calls a posix_spawnp system call passing it the value > | of PATH as an extra argument. > > but no, no thanks, we will not be putting PATH searches into the kernel. Why so averse to what is obviously the simplest and cleanest solution? The best place to try the different path prefixes is at the place where you actually use the filename, i.e. when you load the new process image from a file. If you don't want to do the string processing in the kernel, then you can have the userland function create an array of pathnames to try loading. (Then posix_spawn() could use the same system call and just pass it an array containing one pathname.) > I have an idea for an implementation method for posix_spawnp() which > might work, and allow even the fchdir "file action" (it isn't really, > chdir is a process action, it just happens to affect path resolution) > to work as well. > > But since we don't have (does anyone) the chdir file actions in the > kernel, those (for just posix_spwan) need to come first, before I can > test whether the technique I think might work is actually implementable. > > That will depend a lot on whether our posix_spawn actually does the > "return error" technique, or the "exit 127" technique on errors - if > it is the latter, then I still see no hope, but if posix_spawn() > doing a return(0) actually means that everything worked, it might be > possible (by simply doing the loop around posix_spawn() calls, modifying > the pathname according to PATH, until one works, or a non path related > error is returned). That approach won't work because it will repeat the list of file actions, some of which might not be idempotent. Also, how would you distinguish between a path related error caused by the attempted image load from one caused by one of the file actions? -- Geoff Clare The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
Re: [1003.1(2016)/Issue7+TC2 0001340]: PATH specification has an ambiguity.
Robert Elz wrote, on 06 May 2020: > > | - it would be helpful if you could propose the changes you want in smaller > | chunks so there is less unchanged text to compare.) > > I can do that, and will do ... but I am not sure in which form to > do that. What I see normally is stuff like > > "at line NNN change xxx to yyy" > or > "at line NNN after xxx add yyy" > or similar. > > I can do it that way, but then it won't be obvious (without > doing a manual compare) that the text is actually not changing, > just being rearranged. > > I could instead do > > "move the sentence at line NNN beginning (or containing, >or "that is") xxx to after yyy on line MMM" > > which makes it easier to see that the text is not changing, just > being moved around - but much harder to visualise what the changes > and end result will be (without actually doing it). There have certainly been bug resolutions in the past that have included "move xxx to after yyy" type instructions. They are rare because it's unusual to move something without also changing it. If you are worried that specifying the changes piecemeal will make it harder to review the end result, you could add: The end result of the above changes is: followed by the full new description. > > | The problems with fc -e and xargs could either be addressed here or in a > | separate bug. > > If it helps the NetBSD man page for sh, in the section on fc, says of -e > > -e editor >Use the editor named by editor to edit the commands. The >editor string is a command name, subject to search via the >PATH variable. > I think the standard only uses "command name" in the context of how the shell parses commands. The amended fc -e description in the standard should refer to editor as a "pathname". -- Geoff Clare The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
Re: [1003.1(2016)/Issue7+TC2 0001341]: The resolution of bugid:1208 as amended by bugnote:4830 is incorrect
Date:Wed, 6 May 2020 08:47:56 + From:Austin Group Bug Tracker Message-ID: | The new specified behaviour is what naturally happens if posix_spawn() and | posix_spawnp() are implemented using fork and exec, as in the XRAT B.3.3 | example implementation of posix_spawn() as amended by bug 1208. (This can | be turned into a posix_spawnp() implementation by doing a PATH loop around | the execve() call, if path does not contain any slashes.) Sure, but an implementation like that is either a proof of concept, or useful only to implementations whose sole reason for implementing posix_spawn is because POSIX says it has to be available. No-one would actually use a posix_spawm implementation done that way, as its interface is far more difficult that simply doing the fork() exec() (with intervening close/sup/... calls). posix_spawn is useful in two cases... one is where fork() and exec() aren't available (and consequently its implementation cannot use them) and second (and forgotten in the rationale, though this is actually probably the far more common justification for its use) so that a BIG process can cheaply run a child process. Even vfork() has to move the memory mappings around (fork() needs to duplicate everything) and for big processes, that's a significant cost - if there's a genuine need to fork() that is tolerable, but when the fork() is followed almost immediately by an exec(), so all that work then has to be torn down (more work), it starts getting really annoying. An implementation of posix_spawn() that uses fork()/exec() is useless to overcome this problem. That is, IRL, any implementation of posix_spawn, to be useful, must be a kernel interface - a userland implementation, however possible is useless. | An implementation that chooses to make posix_spawn() a system call needs to | ensure the end result is the same as for a typical fork+exec | implementation. Yes. But just because something can be done with a "typical" (not really) fork+exec implementation (which actually means anything at all) doesn't mean it is rational (or necessarily even possible) to do it in a kernel implementation, and if the kernel implementation cannot, or will not, do it, then it is pointless having (an application that needs the functionality which is in a userland posix_spawn implementation can just use fork/exec itself - which is what posix_spawn would be doing). | On such a system, presumably posix_spawnp() would be a C | library function Yes, so it is. | that calls a posix_spawnp system call passing it the value | of PATH as an extra argument. but no, no thanks, we will not be putting PATH searches into the kernel. I have an idea for an implementation method for posix_spawnp() which might work, and allow even the fchdir "file action" (it isn't really, chdir is a process action, it just happens to affect path resolution) to work as well. But since we don't have (does anyone) the chdir file actions in the kernel, those (for just posix_spwan) need to come first, before I can test whether the technique I think might work is actually implementable. That will depend a lot on whether our posix_spawn actually does the "return error" technique, or the "exit 127" technique on errors - if it is the latter, then I still see no hope, but if posix_spawn() doing a return(0) actually means that everything worked, it might be possible (by simply doing the loop around posix_spawn() calls, modifying the pathname according to PATH, until one works, or a non path related error is returned). I don't know - I have never used it, and don't know of any NetBSD code that does (except perhaps some tests). kre
Re: [1003.1(2016)/Issue7+TC2 0001340]: PATH specification has an ambiguity.
Date:Wed, 6 May 2020 08:18:17 + From:Austin Group Bug Tracker Message-ID: <8de7a2b24ecdb11ff3e05b4279ae5...@austingroupbugs.net> | (I didn't compare the rest of | your new PATH description with the current one to see what else you changed Almost all the words I used were simple cut & paste from what is there now - the existing text is actually all correct (modulo the missing resolve a path containing a slash but not beginning with one bit). It is just hard to read to know that it is all correct... Moving the text around and adding just a few words helps, I think. | - it would be helpful if you could propose the changes you want in smaller | chunks so there is less unchanged text to compare.) I can do that, and will do ... but I am not sure in which form to do that. What I see normally is stuff like "at line NNN change xxx to yyy" or "at line NNN after xxx add yyy" or similar. I can do it that way, but then it won't be obvious (without doing a manual compare) that the text is actually not changing, just being rearranged. I could instead do "move the sentence at line NNN beginning (or containing, or "that is") xxx to after yyy on line MMM" which makes it easier to see that the text is not changing, just being moved around - but much harder to visualise what the changes and end result will be (without actually doing it). | The problems with fc -e and xargs could either be addressed here or in a | separate bug. If it helps the NetBSD man page for sh, in the section on fc, says of -e -e editor Use the editor named by editor to edit the commands. The editor string is a command name, subject to search via the PATH variable. (Then it goes on to what happens if -e is omitted, which I am not sure really belongs in the description of the -e option, but that's not material here). [Aside; in this e-mail I cannot (easily) reproduce the font/colour changes that occur (where possible) in the man page, but the first use of "editor" (in the paragraph) is just a word, the 2nd refers to the arg of the "-e" option, as does the 3rd (more generically).] For xargs we're no better than the standard it appears, the sole reference to PATH searches (such as it is) in its man page is SEE ALSO execvp(3) ... That is the sole reference to execvp in the man page, or for that matter, to any of the exec*() family of functions, the string "exec" otherwise occurs only as part of the word "executed". Someone industrious enough could deduce from its presence, for no apparent reason, that that reference implies that execvp() is used to run the utility, and go check its man page, where info on PATH searching can be found. I do not recommend copying this method for POSIX! kre
Minutes of the 4th May 2020 Teleconference
hi All Attached are the minutes from the Monday meeting regards Andrew - Minutes of the 4th May 2020 Teleconference Austin-1025 Page 1 of 1 Submitted by Andrew Josey, The Open Group. 6th May 2020 Attendees: Nick Stoughton, USENIX, ISO/IEC JTC 1/SC 22 OR Don Cragun, IEEE PASC OR Andrew Josey, The Open Group Geoff Clare, The Open Group Eric Ackermann, HPI, University of Potsdam Joerg Schilling, FOKUS Fraunhofer Eric Blake, Red Hat, Open Group OR Mark Ziegast, SHware Systems Dev. Richard Hansen * General news The etherpad was back for the meeting. * Outstanding actions (Please note that this section has been flushed to shorten the minutes - to locate the previous set of outstanding actions, look to the minutes from 13th June 2019 and earlier) Bug 1254: "asynchronous list" description uses "command" instead of "AND-OR list" OPEN https://austingroupbugs.net/view.php?id=1254 Action: Joerg to investigate how his shell behaves. Bug 700 - Nick to raise this issue with the C committee Bug 713 - Nick to raise with the C committee. Bug 739 - Nick to raise with the C committee. Bug 1216 - Eric to ask if The Open Group is willing to sponsor this interface, referencing bug note 4478. * Current Business Bug 0001208: calling chdir as part of posix_spawn Accepted as Marked https://www.austingroupbugs.net/view.php?id=1208 We revisited bug 1208. Mark took an action to file a new bug report about PATH - which he has completed (see bug 1340). Gettext draft. We continued this discussion and will continue on the next call. The gettext draft and discussion is at https://posix.rhansen.org/p/gettext_draft Next Steps -- May 7 2020 (Thursday) This call will be for 90 minutes. May 11 2020 (Monday) This call will be for 60 minutes. Calls are anchored on US time. (8am Pacific) Please check the calendar invites for dial in details. Bugs are at: https://austingroupbugs.net An etherpad is usually up for the meeting, with a URL using the date format as below: https://posix.rhansen.org/p/20xx-mm-dd username=posix password=2115756# Andrew JoseyThe Open Group Austin Group Chair Email: a.jo...@opengroup.org Apex Plaza, Forbury Road,Reading,Berks.RG1 1AX,England To learn how we maintain your privacy, please review The Open Group Privacy Statement at http://www.opengroup.org/privacy. To unsubscribe/opt-out from this mailing list login to The Open Group collaboration portal at https://collaboration.opengroup.org/operational/portal.php?action=unsub=2481
[1003.1(2016)/Issue7+TC2 0001341]: The resolution of bugid:1208 as amended by bugnote:4830 is incorrect
A NOTE has been added to this issue. == https://austingroupbugs.net/view.php?id=1341 == Reported By:kre Assigned To: == Project:1003.1(2016)/Issue7+TC2 Issue ID: 1341 Category: System Interfaces Type: Error Severity: Objection Priority: normal Status: New Name: Robert Elz Organization: User Reference: Section:posix_spawn Page Number:1452 ff. Line Number:48227 ff. Interp Status: --- Final Accepted Text: == Date Submitted: 2020-05-05 04:53 UTC Last Modified: 2020-05-06 08:47 UTC == Summary:The resolution of https://austingroupbugs.net/view.php?id=1208 as amended by https://austingroupbugs.net/view.php?id=1208#c4830 is incorrect == -- (0004865) geoffclare (manager) - 2020-05-06 08:47 https://austingroupbugs.net/view.php?id=1341#c4865 -- Re https://austingroupbugs.net/view.php?id=1341#c4858 The new specified behaviour is what naturally happens if posix_spawn() and posix_spawnp() are implemented using fork and exec, as in the XRAT B.3.3 example implementation of posix_spawn() as amended by bug 1208. (This can be turned into a posix_spawnp() implementation by doing a PATH loop around the execve() call, if path does not contain any slashes.) An implementation that chooses to make posix_spawn() a system call needs to ensure the end result is the same as for a typical fork+exec implementation. On such a system, presumably posix_spawnp() would be a C library function that calls a posix_spawnp system call passing it the value of PATH as an extra argument. Issue History Date ModifiedUsername FieldChange == 2020-05-05 04:53 kreNew Issue 2020-05-05 04:53 kreName => Robert Elz 2020-05-05 04:53 kreSection => posix_spawn 2020-05-05 04:53 krePage Number => 1452 ff. 2020-05-05 04:53 kreLine Number => 48227 ff. 2020-05-05 05:07 kreNote Added: 0004852 2020-05-05 05:09 kreNote Deleted: 0004852 2020-05-05 08:18 geoffclare Note Added: 0004853 2020-05-05 15:28 kreNote Added: 0004858 2020-05-05 15:30 kreNote Edited: 0004858 2020-05-05 17:36 kreNote Edited: 0004858 2020-05-06 08:47 geoffclare Note Added: 0004865 ==
[1003.1(2016)/Issue7+TC2 0001340]: PATH specification has an ambiguity.
A NOTE has been added to this issue.
==
https://austingroupbugs.net/view.php?id=1340
==
Reported By:shware_systems
Assigned To:
==
Project:1003.1(2016)/Issue7+TC2
Issue ID: 1340
Category: Base Definitions and Headers
Type: Clarification Requested
Severity: Objection
Priority: normal
Status: New
Name: Mark Ziegast
Organization: SHware Systems Dev.
User Reference:
Section:XBD 8.3
Page Number:178
Line Number:5854-7
Interp Status: ---
Final Accepted Text:
==
Date Submitted: 2020-05-04 16:28 UTC
Last Modified: 2020-05-06 08:18 UTC
==
Summary:PATH specification has an ambiguity.
==
--
(0004864) geoffclare (manager) - 2020-05-06 08:18
https://austingroupbugs.net/view.php?id=1340#c4864
--
Re https://austingroupbugs.net/view.php?id=1340#c4862
Seems I didn't look hard enough for the places that require a PATH search.
I have now found a place that doesn't say PATH is not searched if there is
a slash in the search string: find -exec (and -ok), where the requirement
that a PATH search is done is only made via the PATH entry in ENVIRONMENT
VARIABLES - it's not in the description of -exec. There is also a slight
problem with fc -e and xargs because they both state that the utility to be
invoked is specified by its name, implying that portable applications can't
use fc -e or xargs to invoke a utility by means of a pathname that includes
a slash. Once those are fixed to allow a pathname, they would also need to
specify the no-slash rule for PATH searching, unless it is covered by the
description of PATH.
So I am now in agreement that the description of PATH should handle this.
Your suggestion of "If the pathname being sought contains no slash ('/')
characters, and hence is a filename, the list of prefixes shall be searched
..." is okay with me as a way to do that. (I didn't compare the rest of
your new PATH description with the current one to see what else you changed
- it would be helpful if you could propose the changes you want in smaller
chunks so there is less unchanged text to compare.)
The problems with fc -e and xargs could either be addressed here or in a
separate bug.
Issue History
Date ModifiedUsername FieldChange
==
2020-05-04 16:28 shware_systems New Issue
2020-05-04 16:28 shware_systems Name => Mark Ziegast
2020-05-04 16:28 shware_systems Organization => SHware Systems Dev.
2020-05-04 16:28 shware_systems Section => XBD 8.3
2020-05-04 16:28 shware_systems Page Number => 178
2020-05-04 16:28 shware_systems Line Number => 5854-7
2020-05-04 19:31 kreNote Added: 0004846
2020-05-04 19:33 kreNote Added: 0004847
2020-05-04 20:05 shware_systems Note Added: 0004848
2020-05-05 03:00 kreNote Added: 0004849
2020-05-05 03:16 shware_systems Note Added: 0004850
2020-05-05 04:39 kreNote Added: 0004851
2020-05-05 05:03 kreNote Edited: 0004851
2020-05-05 05:04 kreNote Edited: 0004851
2020-05-05 05:11 kreNote Edited: 0004851
2020-05-05 05:13 kreNote Edited: 0004851
2020-05-05 08:40 geoffclare Note Added: 0004854
2020-05-05 08:46 geoffclare Note Added: 0004855
2020-05-05 08:47 geoffclare Note Edited: 0004855
2020-05-05 16:19 shware_systems Note Added: 0004861
2020-05-05 16:21 kreNote Added: 0004862
2020-05-06 08:18 geoffclare Note Added: 0004864
==
