[
https://issues.apache.org/jira/browse/SPARK-54067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18034226#comment-18034226
]
Dongjoon Hyun commented on SPARK-54067:
---------------------------------------
I collected this as a subtask of SPARK-51727 in order to improve the visibility.
> Improve `SparkSubmit` to invoke `exitFn` with the root cause instead of
> `SparkUserAppException`
> -----------------------------------------------------------------------------------------------
>
> Key: SPARK-54067
> URL: https://issues.apache.org/jira/browse/SPARK-54067
> Project: Spark
> Issue Type: Sub-task
> Components: Declarative Pipelines
> Affects Versions: 4.1.0
> Reporter: Sanford Ryza
> Assignee: Sanford Ryza
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> I hit this when I ran a pipeline that had no flows:
> ```
> org.apache.spark.SparkUserAppException: User application exited with 1
> at org.apache.spark.deploy.PythonRunner$.main(PythonRunner.scala:127)
> at org.apache.spark.deploy.PythonRunner.main(PythonRunner.scala)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:569)
> at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1028)
> at
> org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:226)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:95)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1166)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1175)
> at org.apache.spark.deploy.SparkPipelines$.main(SparkPipelines.scala:42)
> at org.apache.spark.deploy.SparkPipelines.main(SparkPipelines.scala)
> ```
> This is not information that's relevant to the user.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
