A NOTE has been added to this issue. ====================================================================== http://austingroupbugs.net/view.php?id=789 ====================================================================== Reported By: weeks Assigned To: ajosey ====================================================================== Project: 1003.1(2008)/Issue 7 Issue ID: 789 Category: Shell and Utilities Type: Enhancement Request Severity: Comment Priority: normal Status: Under Review Name: Nathan Weeks Organization: USDA-ARS User Reference: Section: 2.9.2 Page Number: 2341 Line Number: 74464-74467 Interp Status: --- Final Accepted Text: ====================================================================== Date Submitted: 2013-11-08 16:02 UTC Last Modified: 2018-09-04 20:36 UTC ====================================================================== Summary: Add set -o pipefail ======================================================================
---------------------------------------------------------------------- (0004102) kre (reporter) - 2018-09-04 20:36 http://austingroupbugs.net/view.php?id=789#c4102 ---------------------------------------------------------------------- What has not been considered here is the operation of pipefail in an asynchronous pipeline. In particular, at what instant is the setting of the pipefail option tested. I see three possibilities (which all amount to the same thing for foreground pipelines, so it does not matter there) 1) the pipefail option setting when the pipeline is created (and/or perhaps the setting for each command in the pipeline when it is started). This allows set -o pipefail my | pipe | line & P=$! set +o pipefail # and then later wait $P # and here $? is set to the status of the pipeline, using the pipefail option, even though it is not enabled when the wait is done. 2) the status when the script waits for the status of the pipeline (when the status is made available to the script). This allows my | pipe | line & P=$! # and later set -o pipefail; wait $P; X=$?; set +o pipefail and X is the status from the pipeline, with the pipefail option set (regardless of its value when the pipeline was created, or at any intervening time). 3) The status of the pipefail option is tested as the shell detects that each process of the pipeline is done (has exited), so in set +o pipefail (sleep 1; exit 1) | (sleep 5; exit 5) | (sleep 9; exit 0) & P=$? sleep 3 sleep 3 set -o pipefail set -o pipefail sleep 4 set +o pipefail sleep 4 set -o pipefail wait $P wait $P $? == 0 $? == 5 In the sequence in the left column, pipefail is not set when the last process exits, so the status of the pipeline is the status of the rightmost process (0). In the right column, pipefail is enabled then and so the status of the pipeline is the status of the rightmost process with non-zero status to exit while the pipefail option was set, so 5 in this case (the middle of the 3 processes). If that process had done exit 0 instead of exit 5, the status of the pipeline would be 0, as the pipefail option was not set when the first process exited, hence its status is irrelevant. This 3rd option is, of course, essentially unusable, but is the easiest to implement - it allows the shell to immediately decide what to do with the exit status of each command in the pipeline as it exits, it does not need to retain all the status info until the whole pipeline is finished, which it would need to do with the 2nd option. The first option is the most useful I believe, but I also suspect not implemented anywhere - it allows the shell to collect the status as exch process in the pipeline exits, and also provides predictable behaviour to the application. Pity that no-one (that I know of) implements that. I added pipefail to the NetBSD sh (ash derivative) a while ago, it is in the NetBSD 8 release shell - that implementation uses scheme 2 above for dealing with async pipelines (because being different, and implementing (1) did not seem useful, when no-one else does, and (3), while simple, is useless.) We have no arrays (and I hope, never will) - there is currently no way to determine which process in the pipeline produced the status that is returned when the pipefail option is set. My impression is that most applications which would want to use it are content to simply discover "they all worked" and are prepared to write the script in a way to avoid inadvertent SIGPIPE/EPIPE exit issues (eg: using "sed -e '2,$d'" or "sed -n -e 1p" rather than "head -n1" or "sed -e 1q"). Alternatively the script can insert another process whose purpose is to insulate an earlier process from EPIPE by always reading until EOF, and writing to its stdout, (just like cat with no args) but ignoring any write errors (or at least, ignoring EPIPE and also ignoring SIGPIPE ... and always doing exit 0.) In general I support the pipefail option. I do not think it is required that everything that could possibly be useful as an adjunct be defined at the same time - if making the status of the processes in a pipe available is useful, and arrays are not to be the solution, then something else should be implemented and made available before anyone worries about specifying how to do it. Similarly, we do not need to invent ways to deal with pipes closing - let the applications work on it, and if they need shell assistance, ask for what they actually need. But the operation of the option needs to be fully defined (even if that includes "is unspecified" for some things.) Issue History Date Modified Username Field Change ====================================================================== 2013-11-08 16:02 weeks New Issue 2013-11-08 16:02 weeks Status New => Under Review 2013-11-08 16:02 weeks Assigned To => ajosey 2013-11-08 16:02 weeks Name => Nathan Weeks 2013-11-08 16:02 weeks Organization => USDA-ARS 2013-11-08 16:02 weeks Section => 2.9.2 2013-11-08 16:02 weeks Page Number => 0 2013-11-08 16:02 weeks Line Number => 0 2013-11-08 16:06 weeks File Added: pipeline_exit_status.txt 2013-11-08 16:07 weeks Desired Action Updated 2013-11-08 16:15 eblake Note Added: 0001976 2013-11-08 16:50 Don Cragun Interp Status => --- 2013-11-08 16:50 Don Cragun Desired Action Updated 2013-11-08 16:57 Don Cragun Page Number 0 => 2341 2013-11-08 16:57 Don Cragun Line Number 0 => 74464-74467 2013-11-08 16:57 Don Cragun Note Added: 0001977 2013-11-08 16:58 Don Cragun File Deleted: pipeline_exit_status.txt 2013-11-08 21:05 shware_systems Note Added: 0001978 2013-11-12 15:42 weeks Note Added: 0001982 2013-11-12 15:51 eblake Note Added: 0001983 2013-11-12 16:08 shware_systems Note Edited: 0001978 2013-12-11 22:40 weeks Note Added: 0002067 2014-01-16 23:15 shware_systems Note Added: 0002107 2014-05-28 15:47 weeks Note Added: 0002253 2015-05-29 21:14 stephane Note Added: 0002685 2015-05-31 18:27 stephane Note Added: 0002689 2015-05-31 19:27 stephane Note Edited: 0002689 2017-12-27 22:54 jilles Note Added: 0003904 2017-12-28 09:43 stephane Note Added: 0003905 2017-12-29 12:58 stephane Note Edited: 0003905 2018-09-04 20:36 kre Note Added: 0004102 ======================================================================
