[
https://issues.apache.org/jira/browse/SPARK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168491#comment-14168491
]
Chester commented on SPARK-3445:
--------------------------------
What does Deprecate YARN-alpha implies ?
1) No longer active develop and maintained for sure.
2) When we submit PR, should we make sure they compile for Yarn-alpha ?
I am writing a PR that requires Spark Yarn Client API changes. should I worries
that yarn-alpha Client API changes as well ?
Currently in master branch, the yarn-alpha doesn't even compile, for example
sbt/sbt -Pyarn-alpha -Dhadoop.version=2.0.6-alpha
[error]
/Users/chester/projects/alpine/spark/yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:125:
type mismatch;
[error] found : Object
[error] required: String
[error] Option(report.getClientToken).getOrElse("")
[error] ^
[error] one error found
[error] (streaming-flume-sink/compile:compile) Compilation failed
[error] (yarn-alpha/compile:compile) Compilation failed
[error] Total time: 9 s, completed Oct 11, 2014 7:57:30 PM
This is certainly a bug as the stable version would be
Option(report.getClientToken).map(_toString).getOrElse("")
Should this be fixed ?
Also, I got the following while doing
sbt/sbt -Pyarn-alpha -Dhadoop.version=2.0.6-alpha
test:compile
...
[error] 6 errors found
[error] (streaming-flume-sink/compile:compile) Compilation failed
[error] Total time: 3 s, completed Oct 11, 2014 8:02:57 PM
All the errors are pom.xml dependency related. I don't get the same error when
I do compile yarn-stable version.
Advice ?
> Deprecate and later remove YARN alpha support
> ---------------------------------------------
>
> Key: SPARK-3445
> URL: https://issues.apache.org/jira/browse/SPARK-3445
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Reporter: Patrick Wendell
>
> This will depend a bit on both user demand and the commitment level of
> maintainers, but I'd like to propose the following timeline for yarn-alpha
> support.
> Spark 1.2: Deprecate YARN-alpha
> Spark 1.3: Remove YARN-alpha (i.e. require YARN-stable)
> Since YARN-alpha is clearly identified as an alpha API, it seems reasonable
> to drop support for it in a minor release. However, it does depend a bit
> whether anyone uses this outside of Yahoo!, and that I'm not sure of. In the
> past this API has been used and maintained by Yahoo, but they'll be migrating
> soon to the stable API's.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
