[jira] [Commented] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code
[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438174#comment-17438174 ] Naresh commented on SPARK-26365: Yes. Its not fixed in 3.x yet. I am using spark 3.2 and still see the issue > spark-submit for k8s cluster doesn't propagate exit code > > > Key: SPARK-26365 > URL: https://issues.apache.org/jira/browse/SPARK-26365 > Project: Spark > Issue Type: Bug > Components: Kubernetes, Spark Core, Spark Submit >Affects Versions: 2.3.2, 2.4.0, 3.0.0, 3.1.0 >Reporter: Oscar Bonilla >Priority: Major > Attachments: spark-2.4.5-raise-exception-k8s-failure.patch, > spark-3.0.0-raise-exception-k8s-failure.patch > > > When launching apps using spark-submit in a kubernetes cluster, if the Spark > applications fails (returns exit code = 1 for example), spark-submit will > still exit gracefully and return exit code = 0. > This is problematic, since there's no way to know if there's been a problem > with the Spark application. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code
[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17436144#comment-17436144 ] Naresh edited comment on SPARK-26365 at 10/29/21, 7:41 PM: --- [~oscar.bonilla] Any plans to prioritize issue?? This will definitely block the spark usage with K8s was (Author: gangishetty): [~oscar.bonilla] Any plans to prioritize issue?? This will definitely lock the spark usage with K8s > spark-submit for k8s cluster doesn't propagate exit code > > > Key: SPARK-26365 > URL: https://issues.apache.org/jira/browse/SPARK-26365 > Project: Spark > Issue Type: Bug > Components: Kubernetes, Spark Core, Spark Submit >Affects Versions: 2.3.2, 2.4.0 >Reporter: Oscar Bonilla >Priority: Minor > Attachments: spark-2.4.5-raise-exception-k8s-failure.patch, > spark-3.0.0-raise-exception-k8s-failure.patch > > > When launching apps using spark-submit in a kubernetes cluster, if the Spark > applications fails (returns exit code = 1 for example), spark-submit will > still exit gracefully and return exit code = 0. > This is problematic, since there's no way to know if there's been a problem > with the Spark application. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code
[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17436144#comment-17436144 ] Naresh commented on SPARK-26365: [~oscar.bonilla] Any plans to prioritize issue?? This will definitely lock the spark usage with K8s > spark-submit for k8s cluster doesn't propagate exit code > > > Key: SPARK-26365 > URL: https://issues.apache.org/jira/browse/SPARK-26365 > Project: Spark > Issue Type: Bug > Components: Kubernetes, Spark Core, Spark Submit >Affects Versions: 2.3.2, 2.4.0 >Reporter: Oscar Bonilla >Priority: Minor > Attachments: spark-2.4.5-raise-exception-k8s-failure.patch, > spark-3.0.0-raise-exception-k8s-failure.patch > > > When launching apps using spark-submit in a kubernetes cluster, if the Spark > applications fails (returns exit code = 1 for example), spark-submit will > still exit gracefully and return exit code = 0. > This is problematic, since there's no way to know if there's been a problem > with the Spark application. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-10436) spark-submit overwrites spark.files defaults with the job script filename
[ https://issues.apache.org/jira/browse/SPARK-10436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923175#comment-15923175 ] Naresh edited comment on SPARK-10436 at 3/13/17 11:30 PM: -- This issue got fixed in spark 2.0.0 version. file:https://github.com/apache/spark/blob/branch-2.0/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala line 176:files = Option(files).orElse(sparkProperties.get("spark.files")).orNull Fixes this issue. I have manually tested the versions (1.4,1.5,1.6) - it fixed in 2.0 was (Author: nareshgundla): This issue got fixed in spark 2.0.0 version. file:https://github.com/apache/spark/blob/branch-2.0/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala line 176:files = Option(files).orElse(sparkProperties.get("spark.files")).orNull Fixes this issue. > spark-submit overwrites spark.files defaults with the job script filename > - > > Key: SPARK-10436 > URL: https://issues.apache.org/jira/browse/SPARK-10436 > Project: Spark > Issue Type: Bug > Components: Spark Submit >Affects Versions: 1.4.0 > Environment: Ubuntu, Spark 1.4.0 Standalone >Reporter: axel dahl >Priority: Minor > Labels: easyfix, feature > > In my spark-defaults.conf I have configured a set of libararies to be > uploaded to my Spark 1.4.0 Standalone cluster. The entry appears as: > spark.files libarary.zip,file1.py,file2.py > When I execute spark-submit -v test.py > I see that spark-submit reads the defaults correctly, but that it overwrites > the "spark.files" default entry and replaces it with the name if the job > script, i.e. "test.py". > This behavior doesn't seem intuitive. test.py, should be added to the spark > working folder, but it should not overwrite the "spark.files" defaults. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-10436) spark-submit overwrites spark.files defaults with the job script filename
[ https://issues.apache.org/jira/browse/SPARK-10436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923175#comment-15923175 ] Naresh commented on SPARK-10436: This issue got fixed in spark 2.0.0 version. file:https://github.com/apache/spark/blob/branch-2.0/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala line 176:files = Option(files).orElse(sparkProperties.get("spark.files")).orNull Fixes this issue. > spark-submit overwrites spark.files defaults with the job script filename > - > > Key: SPARK-10436 > URL: https://issues.apache.org/jira/browse/SPARK-10436 > Project: Spark > Issue Type: Bug > Components: Spark Submit >Affects Versions: 1.4.0 > Environment: Ubuntu, Spark 1.4.0 Standalone >Reporter: axel dahl >Priority: Minor > Labels: easyfix, feature > > In my spark-defaults.conf I have configured a set of libararies to be > uploaded to my Spark 1.4.0 Standalone cluster. The entry appears as: > spark.files libarary.zip,file1.py,file2.py > When I execute spark-submit -v test.py > I see that spark-submit reads the defaults correctly, but that it overwrites > the "spark.files" default entry and replaces it with the name if the job > script, i.e. "test.py". > This behavior doesn't seem intuitive. test.py, should be added to the spark > working folder, but it should not overwrite the "spark.files" defaults. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org