A NOTE has been added to this issue. ====================================================================== https://austingroupbugs.net/view.php?id=1801 ====================================================================== Reported By: mohd_akram Assigned To: ====================================================================== Project: Issue 8 drafts Issue ID: 1801 Category: Shell and Utilities Type: Enhancement Request Severity: Editorial Priority: normal Status: Resolved Name: Mohamed Akram Organization: User Reference: Section: xargs Page Number: 3600-3601 Line Number: 123162, 123252 Final Accepted Text: See https://austingroupbugs.net/view.php?id=1801#c6657. Resolution: Accepted As Marked Fixed in Version: ====================================================================== Date Submitted: 2024-01-25 21:39 UTC Last Modified: 2024-02-25 06:26 UTC ====================================================================== Summary: xargs: add -P option ====================================================================== Relationships ID Summary ---------------------------------------------------------------------- related to 0001811 xargs: add -P option to FUTURE DIRECTIO... ======================================================================
---------------------------------------------------------------------- (0006675) kre (reporter) - 2024-02-25 06:26 https://austingroupbugs.net/view.php?id=1801#c6675 ---------------------------------------------------------------------- Re https://austingroupbugs.net/view.php?id=1801#c6670 - thanks for investigating the various implementations, I don't have access to almost any versions of xargs to test against. wrt: I don't exactly get what you mean by "run at the same time". What I meant was, as I understand how most implementations work, there's no guarantee when -P is used, that more than one invocation of utility will ever run at once. Consider searching a whole (huge) source tree for some particular rare operation, piping the found file names to xargs, with some (quick) operation to be performed upon each discovered file. It is quite possible (even likely) that the first invocation of the utility might complete before xargs has collected sufficient args to begin execution of the next one. But as worded, the proposed resolution says to "execute ... concurrently", which might be read as forbidding that behaviour. But no, by "at the same time" I didn't mean to require some new variant of exec() (and fork()) (or posix_spawn) which could begin execution of all of the invocations at the same instant - rather just that it could be read as requiring the implementation to collect sufficient args to (if there are enough of them available) be able to make the arg lists for maxprocs invocations, and once all of those args are collected then run a fork()/exec() loop to start all maxprocs (or less, if there are insufficient args) invocations of the utility, one after another, so they are all running in parallel, rather than consecutively which might happen otherwise. If someone were to implement xargs (with -P) by allocating the first arg to invocvation 1, the 2nd arg to invocation 2, ... until the maxprocs'th arg to invocation maxprocs (and then the next arg to invocation 1 again, and so on) then this concurrent execution would be a natural outcome - and that feature could be interpreted as being a requirement of the specification as written, because of the "execute concurrently" wording. Note that in all of this, I am not suggesting any specific way that xargs should work, the way that current implementations of -P work is almost certainly just fine (and where they differ, the standard can just allow different behaviours, or pick one, or one of a few) - my complaint is entirely with the extraordinarily sloppy proposed (even approved!) wording of the change. Word the thing properly, leave nothing (that anyone can think of anyway) as implicitly unspecified, and it all should be good. I don't think I'd change the wording like you suggested in the final paragraph of the note however, instead I'd say something more along the lines of "When -P is used, when a utility is invoked, xargs shall not wait for that utility to complete before continuing to collect more args and invoke the utility again, provided that no more than maxprocs invocations of the utility are invoked, after than more should not be invoked until one of the earlier ones terminates" (which as written here is horrid, it needs wordsmithing, but it gets the point across). Don't forget to make it clear whether once maxprocs invocations have started, and xargs is waiting for one of them to finish, is it allowed to be collecting more args from stdin so it is ready to start a new invocation of utility as soon as one of the running ones finishes? Or must it just wait, and only start collecting args again after one of the existing invocations completes. It is clear that without -P it isn't allowed to keep collecting args, when it invokes the utility, it is required to just wait for that to exit before doing anything else (otherwise the exit(255) behaviour cannot be implemented). The spec also needs to be clearer what "at most maxprocs" means - read literally any implementation could implement -P by simply ignoring it, running just 1 invocation of utility at a time, since all that is required is not to run more than maxprocs, which is required to be a positive (integer I assume, though the proposed spec doesn't say that - is "xargs -P 3.7 ..." meaningful, or even "xargs -P seven" ?) and so 1 is certainly "at most maxprocs" for any positive integral value. I rather suspect the "at most" is so that the implementation isn't required to run more invocations of the utility than are required for the number of args collected. That is xargs -P 5 echo < /dev/null should still run "echo" once (no -r is given), and isn't expected to run echo 5 times in this usage. All this needs to be clarified. Issue History Date Modified Username Field Change ====================================================================== 2024-01-25 21:39 mohd_akram New Issue 2024-01-25 21:39 mohd_akram Name => Mohamed Akram 2024-01-25 21:39 mohd_akram Section => xargs 2024-01-25 21:39 mohd_akram Page Number => 3600-3601 2024-01-25 21:39 mohd_akram Line Number => 123162, 123252 2024-02-15 16:47 geoffclare Relationship added related to 0001811 2024-02-15 16:52 Don Cragun Note Added: 0006657 2024-02-15 16:53 Don Cragun Status New => Resolved 2024-02-15 16:53 Don Cragun Resolution Open => Accepted As Marked 2024-02-15 16:55 Don Cragun Note Edited: 0006657 2024-02-15 16:55 Don Cragun Tag Attached: issue9 2024-02-15 16:59 Don Cragun Note Edited: 0006657 2024-02-15 17:01 Don Cragun Final Accepted Text => See https://austingroupbugs.net/view.php?id=1801#c6657. 2024-02-16 11:31 kre Note Added: 0006660 2024-02-21 00:20 gabravier Note Added: 0006670 2024-02-21 16:49 gabravier Note Added: 0006672 2024-02-25 06:26 kre Note Added: 0006675 ======================================================================
