Thanks for the explanation.
To be clear, I meant to speak for any hadoop 2 releases before 2.2, which
have profiles in Spark. I referred to CDH4, since that¹s the only Hadoop
2.0/2.1 version Spark ships a prebuilt package for.
I understand the hesitation of making a code change if Spark doesn¹t p
True, although a number of other little issues make me, personally,
not want to continue down this road:
- There are already a lot of build profiles to try to cover Hadoop versions
- I don't think it's quite right to have vendor-specific builds in
Spark to begin with
- We should be moving to only
Hi all,
Related to https://issues.apache.org/jira/browse/SPARK-3039, the default CDH4
build, which is built with "mvn -Dhadoop.version=2.0.0-mr1-cdh4.2.0 -DskipTests
clean package”, pulls in avro-mapred hadoop1, as opposed to avro-mapred
hadoop2. This ends up in the same error as mentioned in t
