[
https://issues.apache.org/jira/browse/SPARK-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180643#comment-15180643
]
Marcelo Vanzin commented on SPARK-13670:
----------------------------------------
Here's one that works regardless of the output of Main. It would be nice to
have someone try this on MacOS.
{code}
# The launcher library will print arguments separated by a NULL character, to
allow arguments with
# characters that would be otherwise interpreted by the shell. Read that in a
while loop, populating
# an array that will be used to exec the final command.
#
# The exit code of the launcher is appended to the output, so the parent shell
removes it from the
# command array and checks the value to see if the launcher succeeded.
build_command() {
"$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@"
printf "%d\0" $?
}
CMD=()
while IFS= read -d '' -r ARG; do
CMD+=("$ARG")
done < <(build_command "$@")
COUNT=${#CMD[@]}
LAST=$((COUNT - 1))
EC=${CMD[$LAST]}
if [ $EC != 0 ]; then
exit $EC
fi
CMD=("${CMD[@]:0:$LAST}")
exec "${CMD[@]}"
{code}
> spark-class doesn't bubble up error from launcher command
> ---------------------------------------------------------
>
> Key: SPARK-13670
> URL: https://issues.apache.org/jira/browse/SPARK-13670
> Project: Spark
> Issue Type: Bug
> Components: Spark Submit
> Affects Versions: 2.0.0
> Reporter: Mark Grover
> Priority: Minor
>
> There's a particular snippet in spark-class
> [here|https://github.com/apache/spark/blob/master/bin/spark-class#L86] that
> runs the spark-launcher code in a subshell.
> {code}
> # The launcher library will print arguments separated by a NULL character, to
> allow arguments with
> # characters that would be otherwise interpreted by the shell. Read that in a
> while loop, populating
> # an array that will be used to exec the final command.
> CMD=()
> while IFS= read -d '' -r ARG; do
> CMD+=("$ARG")
> done < <("$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main
> "$@")
> {code}
> The problem is that the if the launcher Main fails, this code still still
> returns success and continues, even though the top level script is marked
> {{set -e}}. This is because the launcher.Main is run within a subshell.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
