[ https://issues.apache.org/jira/browse/SPARK-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180643#comment-15180643 ]
Marcelo Vanzin commented on SPARK-13670: ---------------------------------------- Here's one that works regardless of the output of Main. It would be nice to have someone try this on MacOS. {code} # The launcher library will print arguments separated by a NULL character, to allow arguments with # characters that would be otherwise interpreted by the shell. Read that in a while loop, populating # an array that will be used to exec the final command. # # The exit code of the launcher is appended to the output, so the parent shell removes it from the # command array and checks the value to see if the launcher succeeded. build_command() { "$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@" printf "%d\0" $? } CMD=() while IFS= read -d '' -r ARG; do CMD+=("$ARG") done < <(build_command "$@") COUNT=${#CMD[@]} LAST=$((COUNT - 1)) EC=${CMD[$LAST]} if [ $EC != 0 ]; then exit $EC fi CMD=("${CMD[@]:0:$LAST}") exec "${CMD[@]}" {code} > spark-class doesn't bubble up error from launcher command > --------------------------------------------------------- > > Key: SPARK-13670 > URL: https://issues.apache.org/jira/browse/SPARK-13670 > Project: Spark > Issue Type: Bug > Components: Spark Submit > Affects Versions: 2.0.0 > Reporter: Mark Grover > Priority: Minor > > There's a particular snippet in spark-class > [here|https://github.com/apache/spark/blob/master/bin/spark-class#L86] that > runs the spark-launcher code in a subshell. > {code} > # The launcher library will print arguments separated by a NULL character, to > allow arguments with > # characters that would be otherwise interpreted by the shell. Read that in a > while loop, populating > # an array that will be used to exec the final command. > CMD=() > while IFS= read -d '' -r ARG; do > CMD+=("$ARG") > done < <("$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main > "$@") > {code} > The problem is that the if the launcher Main fails, this code still still > returns success and continues, even though the top level script is marked > {{set -e}}. This is because the launcher.Main is run within a subshell. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org