[
https://issues.apache.org/jira/browse/SPARK-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180535#comment-15180535
]
Marcelo Vanzin commented on SPARK-13670:
----------------------------------------
bq. And, it's guaranteed that Main would never return that output?
I'm reasonably sure it won't. I just tried spark-shell, pyspark and sparkR and
they all work (since they all generate multiple command line arguments).
But if worried about that, you could always append the exit code to the output
of {{build_command}} by always executing the printf. Then just remove the exit
code in the parent shell, something like:
{code}
COUNT=${#CMD[@]}
EC=${CMD[$COUNT]}
CMD="${CMD[@]:0:$((COUNT -1))}"
{code}
Not 100% sure about the syntax, but seems to work in a quick test.
> spark-class doesn't bubble up error from launcher command
> ---------------------------------------------------------
>
> Key: SPARK-13670
> URL: https://issues.apache.org/jira/browse/SPARK-13670
> Project: Spark
> Issue Type: Bug
> Components: Spark Submit
> Affects Versions: 2.0.0
> Reporter: Mark Grover
> Priority: Minor
>
> There's a particular snippet in spark-class
> [here|https://github.com/apache/spark/blob/master/bin/spark-class#L86] that
> runs the spark-launcher code in a subshell.
> {code}
> # The launcher library will print arguments separated by a NULL character, to
> allow arguments with
> # characters that would be otherwise interpreted by the shell. Read that in a
> while loop, populating
> # an array that will be used to exec the final command.
> CMD=()
> while IFS= read -d '' -r ARG; do
> CMD+=("$ARG")
> done < <("$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main
> "$@")
> {code}
> The problem is that the if the launcher Main fails, this code still still
> returns success and continues, even though the top level script is marked
> {{set -e}}. This is because the launcher.Main is run within a subshell.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
