[
https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16802887#comment-16802887
]
Maxime Nannan commented on SPARK-26365:
---------------------------------------
I had a similar problem and I've created a docker image if you want to
reproduce it.
Docker image has been created with the following script
{code:java}
curl
http://mirror.easyname.ch/apache/spark/spark-2.4.0/spark-2.4.0-bin-hadoop2.7.tgz
| tar -xz
echo "raise RuntimeError()" > test.py
cp test.py spark-2.4.0-bin-hadoop2.7/python/lib
cd spark-2.4.0-bin-hadoop2.7
./bin/docker-image-tool.sh -t issue_exit_code -r mnannanph/spark build
./bin/docker-image-tool.sh -t issue_exit_code -r mnannanph/spark push{code}
The docker image is public on dockerhub
[https://cloud.docker.com/repository/docker/mnannanph/spark-py] and can be
pulled by running
{noformat}
docker pull mnannanph/spark-py:issue_exit_code{noformat}
If you run the following command on a kubernetes cluster that can run spark on
kubernetes
{code:java}
MASTER_URL= # Fill with your kubernetes master ip
bin/spark-submit \
--master ${MASTER_URL} \
--deploy-mode cluster \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--name spark-test \
--conf spark.executor.instances=1 \
--conf spark.kubernetes.container.image=mnannanph/spark-py:issue_exit_code\
/opt/spark/python/lib/test.py{code}
You will obtain an exit code 0 whereas the spark-submit has failed.
The script executed raises an error so the exit code should be 1.
test.py is just a one-line python script with this command
{code:java}
raise RuntimeError()
{code}
Find below one part of driver pod logs:
{code:java}
2019-03-27 14:38:43 WARN NativeCodeLoader:62 - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Traceback (most recent call last):
File "/opt/spark/python/lib/test.py", line 1, in <module>
raise RuntimeError()
RuntimeError
2019-03-27 14:38:44 INFO ShutdownHookManager:54 - Shutdown hook called
2019-03-27 14:38:44 INFO ShutdownHookManager:54 - Deleting directory
/tmp/spark-9d85097e-c848-4c29-8e67-a0d0c0cdeb71
{code}
Hope this will help.
> spark-submit for k8s cluster doesn't propagate exit code
> --------------------------------------------------------
>
> Key: SPARK-26365
> URL: https://issues.apache.org/jira/browse/SPARK-26365
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes, Spark Submit
> Affects Versions: 2.3.2, 2.4.0
> Reporter: Oscar Bonilla
> Priority: Minor
>
> When launching apps using spark-submit in a kubernetes cluster, if the Spark
> applications fails (returns exit code = 1 for example), spark-submit will
> still exit gracefully and return exit code = 0.
> This is problematic, since there's no way to know if there's been a problem
> with the Spark application.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
