[ https://issues.apache.org/jira/browse/SPARK-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14005578#comment-14005578 ]
Tathagata Das edited comment on SPARK-1898 at 5/22/14 3:34 AM: --------------------------------------------------------------- To get an idea of whether this affects currently available YARN distribution or not, I ran Scala Spark Pi example and Spark shell against YARN 2.4 (which I believe is one of the latest) using a HDP2.1 VM. Seemed to work fine. So this probably works with existing released versions of YARN. This change is a little scary, as its hard to know the implications of this change with all existing versions of YARN. This change should be accepted after a little bit of testing with various versions of YARN, so that we are sure that we dont break compatibility. I am not sure it is a good idea to cut a new release because of this change in Spark 1.0, since Spark 1.0 probably works fine with currently released versions of YARN. was (Author: tdas): To get an idea of whether this affects currently available YARN distribution or not, I ran Scala Spark Pi example and Spark shell against YARN 2.4 (which I believe is one of the latest) using a HDP2.1 VM. Seemed to work fine. So this probably works with existing released versions of YARN. This change is a little scary, as its hard to know the implications of this change with all existing versions of YARN. This change should be accepted after a little bit of testing with various versions of YARN, so that we are sure that we dont break compatibility. I am not sure it is a good idea to accept this change in Spark 1.0, since Spark 1.0 probably works fine with currently released versions of YARN. > In deploy.yarn.Client, use YarnClient rather than YarnClientImpl > ---------------------------------------------------------------- > > Key: SPARK-1898 > URL: https://issues.apache.org/jira/browse/SPARK-1898 > Project: Spark > Issue Type: Bug > Affects Versions: 1.0.0 > Reporter: Colin Patrick McCabe > > In {{deploy.yarn.Client}}, we should use {{YarnClient}} rather than > {{YarnClientImpl}}. The latter is annotated as {{Private}} and {{Unstable}} > in Hadoop, which means it could change in an incompatible way at any time. -- This message was sent by Atlassian JIRA (v6.2#6252)