[ https://issues.apache.org/jira/browse/SPARK-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180427#comment-15180427 ]
Marcelo Vanzin commented on SPARK-13670: ---------------------------------------- After some fun playing with arcane bash syntax, here's something that worked for me: {code} run_command() { CMD=() while IFS='' read -d '' -r ARG; do echo "line: $ARG" CMD+=("$ARG") done if [ ${#CMD[@]} -gt 0 ]; then exec "${CMD[@]}" fi } set -o pipefail "$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@" | run_command {code} Example: {noformat} $ ./bin/spark-shell NOTE: SPARK_PREPEND_CLASSES is set, placing locally compiled Spark classes ahead of assembly. Exception in thread "main" java.lang.IllegalArgumentException: Testing, testing, testing... at org.apache.spark.launcher.Main.main(Main.java:93) $ echo $? 1 {noformat} > spark-class doesn't bubble up error from launcher command > --------------------------------------------------------- > > Key: SPARK-13670 > URL: https://issues.apache.org/jira/browse/SPARK-13670 > Project: Spark > Issue Type: Bug > Components: Spark Submit > Affects Versions: 2.0.0 > Reporter: Mark Grover > Priority: Minor > > There's a particular snippet in spark-class > [here|https://github.com/apache/spark/blob/master/bin/spark-class#L86] that > runs the spark-launcher code in a subshell. > {code} > # The launcher library will print arguments separated by a NULL character, to > allow arguments with > # characters that would be otherwise interpreted by the shell. Read that in a > while loop, populating > # an array that will be used to exec the final command. > CMD=() > while IFS= read -d '' -r ARG; do > CMD+=("$ARG") > done < <("$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main > "$@") > {code} > The problem is that the if the launcher Main fails, this code still still > returns success and continues, even though the top level script is marked > {{set -e}}. This is because the launcher.Main is run within a subshell. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org