Given that the error is java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
...this usually means there is a Hadoop version problem. But in particular it's https://issues.apache.org/jira/browse/SPARK-3039 which affects assembly using the Hive code. There's a workaround there but I'm not sure what the resolution is. You may have luck building for hadoop.version=2.5.0-cdh5.2.1, if your cluster in CDH. The packaging probably harmonizes this correctly even with the Hive profile, but I have not tested this myself. On Tue, Dec 16, 2014 at 4:19 PM, manasdebashiskar <manasdebashis...@gmail.com> wrote: > Hi All, > I saw some helps online about forcing avro-mapred to hadoop2 using > classifiers. > > Now my configuration is thus > val avro = "org.apache.avro" % "avro-mapred" % V.avro classifier > "hadoop2" > > How ever I still get java.lang.IncompatibleClassChangeError. I think I am > not building spark correctly. Clearly the following steps is missing > something avro related. > > /( mvn -Pyarn -Dhadoop.version=2.6.0 -Dyarn.version=2.6.0 -Phadoop-2.3 > -Phive -DskipTests clean package)/ > > > *Can someone please help me build spark1.2 for either CDH5.2 or HDP2.2 + > Hive + Avro * > > Thanks > > > > > > ----- > Manas Kar > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-1-2-Avro-does-not-work-in-HDP2-2-tp20667p20721.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
