In
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
, at line 86:
// Workaround until hadoop moves to something which has
// https://issues.apache.org/jira/browse/HADOOP-8406 - fixed in
(2.0.2-alpha but no 0.23 line)
//
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(conf)
Looks like the above comment can be removed since hadoop 2.2 has HADOOP-8406
Cheers
On Fri, Jan 3, 2014 at 2:39 PM, Patrick Wendell <[email protected]> wrote:
> Just to give a bit more color - these exist because the yarn API
> changed and we'd like to support both for a while. Rather than
> introduce shims we just factored out the common code and maintain
> isolated builds for either one.
>
> - Patrick
>
> On Fri, Jan 3, 2014 at 2:34 PM, Patrick Wendell <[email protected]>
> wrote:
> > Yes
> >
> > On Fri, Jan 3, 2014 at 2:33 PM, Ted Yu <[email protected]> wrote:
> >> Hi,
> >> There're two copies of classes w.r.t. YARN:
> >>
> >> -rw-r--r-- 1 tyu staff 9710 Jan 3 10:14
> >>
> /Users/tyu/spark//yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/WorkerLauncher.scala
> >> -rw-r--r-- 1 tyu staff 8189 Jan 3 10:14
> >>
> /Users/tyu/spark//yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/WorkerLauncher.scala
> >>
> >> The class under yarn/stable is for YARN 2.2, right ?
> >>
> >> Cheers
>
