[
https://issues.apache.org/jira/browse/OOZIE-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15686084#comment-15686084
]
Peter Cseh edited comment on OOZIE-2730 at 11/22/16 8:41 AM:
-------------------------------------------------------------
If you omit the <mode> tag, the mode should be defined in the <master> tag.
These are both valid definitions and have the same meaning:
{quote}
<master>yarn</master>
<mode>cluster</mode>
{quote}
{quote}
<master>yarn-cluster</master>
{quote}
But defining the master in spark-opts is not supported. Oozie should give an
error message when that happens.
Edit: I've mixed up the tags.
was (Author: gezapeti):
If you omit the <mode> tag, the mode should be defined in the <master> tag.
These are both valid definitions and have the same meaning:
{quote}
<mode>yarn</mode>
<master>cluster</master>
{quote}
{quote}
<mode>yarn-cluster</mode>
{quote}
But defining the master in spark-opts is not supported. Oozie should give an
error message when that happens.
> Spark submission fails, when spark-mode is specified through spark-opts
> instead of mode tag in action
> -----------------------------------------------------------------------------------------------------
>
> Key: OOZIE-2730
> URL: https://issues.apache.org/jira/browse/OOZIE-2730
> Project: Oozie
> Issue Type: Bug
> Components: action
> Affects Versions: 4.1.0
> Environment: cloudera 4.8.0, oozie 4.1.0, java 8, java version
> "1.8.0_101"/ Java(TM) SE Runtime Environment (build 1.8.0_101-b13)/ Java
> HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode), Linux host
> 4.2.0-42-generic #49~14.04.1-Ubuntu SMP Wed Jun 29 20:22:11 UTC 2016 x86_64
> x86_64 x86_64 GNU/Linux
> Reporter: Rinat Sharipov
> Attachments: failed_job.png, job_configuration.xml, job_definition.xml
>
>
> Hi !
> I'm trying to submit spark job through oozie spark action and specify
> spark-mode in spark-options instead of mode tag (xsd schema specifies that
> it's optional). But submission fails with further message "Error: Could not
> find or load main class org.apache.spark.deploy.yarn.ApplicationMaster"
> I've attached further files:
> - job_configuration.xml - properties, that is used to run a spark job
> - job_definition.xml - spark action config
> - failed_job.png - is an example of failed oozie job from oozie UI
> I think, that this behaviour is unexpected, because xsd schema specifies,
> that <mode> tag is optional
> Also, it's not very clear, what will happen, if I specify different values in
> spark-action mode tag and spark-opts (--deploy-mode), which option will have
> the highest priority ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
