Steve, that's correct, but the problem only shows up when different versions of the YARN jars are included on the classpath.
-Sandy On Tue, Mar 24, 2015 at 6:29 AM, Steve Loughran <ste...@hortonworks.com> wrote: > > > On 24 Mar 2015, at 02:10, Marcelo Vanzin <van...@cloudera.com> wrote: > > > > This happens most probably because the Spark 1.3 you have downloaded > > is built against an older version of the Hadoop libraries than those > > used by CDH, and those libraries cannot parse the container IDs > > generated by CDH. > > > This sounds suspiciously like the changes in YARN for HA (the epoch > number) isn't being parsed by older versions of the YARN client libs. This > is effectively a regression in the YARN code -its creating container IDs > that can't be easily parsed by old apps. It may be possible to fix that > spark-side by having its own parser for the YARN container/app environment > variable > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
