A NOTE has been added to this issue. ====================================================================== https://austingroupbugs.net/view.php?id=1436 ====================================================================== Reported By: steffen Assigned To: ====================================================================== Project: 1003.1(2016/18)/Issue7+TC2 Issue ID: 1436 Category: Shell and Utilities Type: Enhancement Request Severity: Editorial Priority: normal Status: Under Review Name: steffen Organization: User Reference: Section: Vol. 3: Shell and Utilities, Issue 7, make Page Number: 2969 Line Number: 98473 Interp Status: --- Final Accepted Text: https://austingroupbugs.net/view.php?id=1436#c5327 ====================================================================== Date Submitted: 2020-12-15 21:00 UTC Last Modified: 2021-09-09 15:35 UTC ====================================================================== Summary: make: add "-j max_jobs" option to support simultaneous rule processing ====================================================================== Relationships ID Summary ---------------------------------------------------------------------- related to 0001437 make: (document .NOTPARALLEL and .WAIT ... ======================================================================
---------------------------------------------------------------------- (0005496) nick (manager) - 2021-09-09 15:35 https://austingroupbugs.net/view.php?id=1436#c5496 ---------------------------------------------------------------------- Updated changes after resolving https://austingroupbugs.net/view.php?id=1437 After page 2969 line 98472, make OPTIONS, add: <blockquote><b>-j</b> <i>maxjobs</i> Set the maximum number of targets that can be updated concurrently. If this option is specified multiple times, the last value of <i>maxjobs</i> specified shall take precedence. If this option is not specified, or if <i>maxjobs</i> is 1, only one target shall be updated at a time (no parallelization). If the value of <i>maxjobs</i> is non-positive, the behavior is unspecified. When <i>maxjobs</i> is greater than 1, <i>make</i> shall create a pool of up to <i>maxjobs</i> - 1 tokens. (Note that implementations are not required to create a pool of exactly <i>maxjobs</i> - 1 tokens. For example, an implementation could limit the pool size based on the number of processors available.) If the size of the token pool would be 0, <i>make</i> need not implement a token pool. When all of the following are true:<ul> <li>there is a target with commands that is not up-to-date</li> <li>the target's prerequisites (if any) are up-to-date</li> <li><i>make</i> is not waiting to bring the target up-to-date (see <b>.WAIT</b>)</li> <li><i>make</i> is currently bringing a different target with commands up-to-date</li> <li><i>make</i> is not currently bringing <i>maxjobs</i> targets up-to-date in parallel</li> <li>the special target <b>.NOTPARALLEL</b> is not specified</li> <li>the token pool is not empty</li> </ul>then <i>make</i> may attempt to remove one token from the pool. If a token is successfully removed, it shall attempt to bring this target up-to-date in parallel, and after this processing completes shall return the token to the pool. When <i>make</i> is bringing a target without commands up-to-date, it need not remove a token from the pool. If a rule invokes a sub-<i>make</i> either via the <i>MAKE</i> macro or via a command line that begins with '+', the sub-<i>make</i> is the same implementation as the <i>make</i> that invoked the sub-<i>make</i>, and the <b>-j</b> option is passed to the sub-<i>make</i> via the <i>MAKEFLAGS</i> environment variable with the same <i>maxjobs</i> value and is not overridden by a <i>maxjobs</i> value from another source (even if it has the same value), the sub-<i>make</i> shall use the same token pool as its invoking <i>make</i> rather than create a new token pool. Otherwise, it is unspecified whether the sub-<i>make</i> uses the same token pool as its invoking <i>make</i> or creates a new token pool. If a rule executes multiple sub-<i>make</i> processes asynchronously the behavior is unspecified.</blockquote> On page 2969 line 98476 change: <blockquote>However, lines with a <plus-sign> (<tt>'+'</tt>) prefix shall be executed.</blockquote> to: <blockquote>However, lines with a <plus-sign> (<tt>'+'</tt>) prefix and lines that expand the <i>MAKE</i> macro shall be executed.</blockquote> On page 2986 line 99178, delete the bullet item: <blockquote>Specifying that command lines containing the strings "${MAKE}" and "$(MAKE)" are executed when the <b>−n</b> option is specified (GNU and System V).</blockquote> On page 2987, delete lines 99229 - 99235 (the paragraph beginning "Early proposals ..."). On page 2990, after line 99387 add the following new paragraphs: <blockquote>The standard specifies a way for portable applications to request parallel updating of targets with commands by using the <b>-j</b> <i>maxjobs</i> option. This feature is described in terms of a token pool initially containing <b>up to</b> <i>maxjobs<i> - 1 tokens. Note that this is not intended to prescribe a particular implementation design; the usual "as if" rule applies. Implementations are permitted to silently limit the pool size for a few reasons, including: Implementations that do not support parallelism can support the <b>-j</b> option by simply ignoring the option (other than passing it to sub-<i>make</i> invocations via the <i>MAKEFLAGS</i> environment variable). In effect, such an implementation silently restricts the size of the token pool to zero (and therefore need not create a token pool). Some historical implementations dynamically limit the token pool size based on current system load to avoid overloading the system. Implementations may want to limit the token pool size based on the number of processors available. Implementations may want to limit the token pool size based on resource limits. Limiting the pool size does not change the value of <i>maxjobs</i> that is passed to sub-<i>make</i> invocations via the <i>MAKEFLAGS</i> environment variable. When a different <i>maxjobs</i> value is passed to a sub-<i>make</i>, some historical <i>make</i> implementations created a separate pool of tokens while other historical <i>make</i> implementations continued to obtain tokens from the invoking <i>make</i> but limited the number of tokens held at a time to the new value of <i>maxjobs</i> - 1. Both behaviors are believed to have merit in different situations: The former gives a sub-<i>make</i> complete control the amount of parallelism while the latter allows the user to control overall system load. The standard permits either behavior. The standard calls for a token pool of size <i>maxjobs</i> - 1, and for removal from that pool only for the second and subsequent tasks in a set of parallel tasks. This design was chosen because this is effectively what existing implementations do, and also because the token consumed by a parallel task that invokes a sub-<i>make</i> is effectively lent to the sub-<i>make</i>. Lending the token to the sub-<i>make</i> has the following advantages:<ul> <li>It prevents the sub-<i>make</i> from being completely idle due to token starvation, allowing it to always make some progress regardless of how many tokens other sub-<i>make</i> invocations have consumed.</li> <li>It prevents token pool exhaustion caused by a long chain of sub-<i>make</i> invocations. If the token consumed by the invoking rule was not effectively lent to the sub-<i>make</i>, then the pool would be exhausted by a chain of sub-<i>make</i> invocations that is <i>maxjobs</i> long. Such a chain would never accomplish any work, and would thus never complete.</li> </ul> When a rule invokes multiple sub-<i>make</i> processes asynchronously (for example by using an asynchronous list in the shell), some implementations allow each sub-<i>make</i> to execute at least one rule even though this would cause the total number of parallel rule executions across all <i>make</i> instances to exceed <i>maxjobs</i> (after discounting the rules that execute sub-<i>make</i> processes). This behavior may not be ideal, but it is easier to implement and is unlikely to cause problems in practice because applications typically do not have any rules that invoke multiple sub-<i>make</i> processes asynchronously. For this reason the behavior is unspecified if a rule executes multiple sub-<i>make</i> processes asynchronously. When multiple sub-<i>make</i> processes are running in parallel there is no requirement placed on the ordering of output from these processes. Some implementations of <i>make</i> attempt to serialize output from each sub-<make>; others make no such attempt. If diagnostic messages from failed commands are intermixed, the usual way to deal with this is to repeat the <i>make</i> without <b>-j</b> (or with <b>-j</b> 1) so that intermixing will not occur. </blockquote> Issue History Date Modified Username Field Change ====================================================================== 2020-12-15 21:00 steffen New Issue 2020-12-15 21:00 steffen Name => steffen 2020-12-15 21:00 steffen Section => Vol. 3: Shell and Utilities, Issue 7, make 2020-12-15 21:00 steffen Page Number => 2969 2020-12-15 21:00 steffen Line Number => 98473 2021-03-22 21:27 psmith Note Added: 0005299 2021-04-22 15:34 rhansen Note Added: 0005327 2021-04-22 15:36 rhansen Interp Status => --- 2021-04-22 15:36 rhansen Final Accepted Text => https://austingroupbugs.net/view.php?id=1436#c5327 2021-04-22 15:36 rhansen Status New => Resolved 2021-04-22 15:36 rhansen Resolution Open => Accepted As Marked 2021-04-22 15:36 rhansen Tag Attached: issue8 2021-04-22 18:36 psmith Note Added: 0005328 2021-04-23 17:40 psmith Note Added: 0005329 2021-04-23 17:47 psmith Note Added: 0005330 2021-04-23 18:09 psmith Note Edited: 0005330 2021-04-23 18:28 psmith Note Added: 0005331 2021-04-29 16:59 rhansen Note Added: 0005345 2021-04-29 17:00 rhansen Note Edited: 0005345 2021-04-29 17:00 rhansen Note Edited: 0005345 2021-04-29 17:03 rhansen Note Added: 0005346 2021-04-29 17:18 rhansen Note Edited: 0005346 2021-04-29 17:30 rhansen Note Added: 0005347 2021-04-29 17:34 rhansen Note Edited: 0005347 2021-04-29 20:19 psmith Note Added: 0005348 2021-05-14 14:12 psmith Note Added: 0005353 2021-05-14 14:30 geoffclare Note Added: 0005354 2021-05-14 14:30 geoffclare Status Resolved => Under Review 2021-05-14 14:30 geoffclare Resolution Accepted As Marked => Reopened 2021-05-14 14:35 psmith Note Added: 0005355 2021-05-20 17:08 rhansen Note Added: 0005362 2021-05-22 19:19 psmith Note Added: 0005364 2021-08-19 16:59 rhansen Relationship added related to 0001437 2021-09-09 15:35 nick Note Added: 0005496 ======================================================================
