[jira] [Commented] (SPARK-16095) Yarn cluster mode should return consistent result for command line and SparkLauncher
[ https://issues.apache.org/jira/browse/SPARK-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354509#comment-15354509 ] Peng Zhang commented on SPARK-16095: [~tgraves] [~sowen] I have added a test case for explaining this issue, please take a look. > Yarn cluster mode should return consistent result for command line and > SparkLauncher > > > Key: SPARK-16095 > URL: https://issues.apache.org/jira/browse/SPARK-16095 > Project: Spark > Issue Type: Bug > Components: YARN >Affects Versions: 2.0.0 >Reporter: Peng Zhang > > For an application with YarnApplicationState.FINISHED and > FinalApplicationStatus.FAILED, invoking spark-submit from command line will > got Exception, submit with SparkLauncher will got state with FINISHED which > means app succeeded. > Also because the above fact, in test YarnClusterSuite, assert with false > condition will not fail the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16095) Yarn cluster mode should return consistent result for command line and SparkLauncher
[ https://issues.apache.org/jira/browse/SPARK-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354425#comment-15354425 ] Apache Spark commented on SPARK-16095: -- User 'renozhang' has created a pull request for this issue: https://github.com/apache/spark/pull/13962 > Yarn cluster mode should return consistent result for command line and > SparkLauncher > > > Key: SPARK-16095 > URL: https://issues.apache.org/jira/browse/SPARK-16095 > Project: Spark > Issue Type: Bug > Components: YARN >Affects Versions: 2.0.0 >Reporter: Peng Zhang > > For an application with YarnApplicationState.FINISHED and > FinalApplicationStatus.FAILED, invoking spark-submit from command line will > got Exception, submit with SparkLauncher will got state with FINISHED which > means app succeeded. > Also because the above fact, in test YarnClusterSuite, assert with false > condition will not fail the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16095) Yarn cluster mode should return consistent result for command line and SparkLauncher
[
https://issues.apache.org/jira/browse/SPARK-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15343760#comment-15343760
]
Peng Zhang commented on SPARK-16095:
I want to make a unit test to explain this issue, but found YarnClusterSuite
doesn't run into block * if (conf.get("spark.master") == "yarn-cluster") *.
And I filed SPARK-16125 to fix it first.
> Yarn cluster mode should return consistent result for command line and
> SparkLauncher
>
>
> Key: SPARK-16095
> URL: https://issues.apache.org/jira/browse/SPARK-16095
> Project: Spark
> Issue Type: Bug
> Components: YARN
>Affects Versions: 2.0.0
>Reporter: Peng Zhang
>
> For an application with YarnApplicationState.FINISHED and
> FinalApplicationStatus.FAILED, invoking spark-submit from command line will
> got Exception, submit with SparkLauncher will got state with FINISHED which
> means app succeeded.
> Also because the above fact, in test YarnClusterSuite, assert with false
> condition will not fail the test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16095) Yarn cluster mode should return consistent result for command line and SparkLauncher
[ https://issues.apache.org/jira/browse/SPARK-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341903#comment-15341903 ] Thomas Graves commented on SPARK-16095: --- FINISHED does not mean success, finished means it completed in some fashion on yarn, the FinalApplicationStatus is what tells you failed or success of the application itself. This is a YARN thing, state is what yarn things its in, accepted, running, finished, sometimes failed if it things went really bad, but the actual application status is the FinalApplicationStatus and that is what you should be looking at. > Yarn cluster mode should return consistent result for command line and > SparkLauncher > > > Key: SPARK-16095 > URL: https://issues.apache.org/jira/browse/SPARK-16095 > Project: Spark > Issue Type: Bug > Components: YARN >Affects Versions: 2.0.0 >Reporter: Peng Zhang > > For an application with YarnApplicationState.FINISHED and > FinalApplicationStatus.FAILED, invoking spark-submit from command line will > got Exception, submit with SparkLauncher will got state with FINISHED which > means app succeeded. > Also because the above fact, in test YarnClusterSuite, assert with false > condition will not fail the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16095) Yarn cluster mode should return consistent result for command line and SparkLauncher
[ https://issues.apache.org/jira/browse/SPARK-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341695#comment-15341695 ] Sean Owen commented on SPARK-16095: --- Is this description correct? you're saying they both result in state FINISHED. What exception are you seeing? what part of the test do you expect should behave differently? this needs more info. > Yarn cluster mode should return consistent result for command line and > SparkLauncher > > > Key: SPARK-16095 > URL: https://issues.apache.org/jira/browse/SPARK-16095 > Project: Spark > Issue Type: Bug > Components: YARN >Affects Versions: 2.0.0 >Reporter: Peng Zhang > > For an application with YarnApplicationState.FINISHED and > FinalApplicationStatus.FAILED, invoking spark-submit from command line will > got Exception, submit with SparkLauncher will got state with FINISHED which > means app succeeded. > Also because the above fact, in test YarnClusterSuite, assert with false > condition will not fail the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
