It could be that CDH6 has the integration but somehow I am getting the
following very frequently while building Spark2.4.4 with Hadoop 3.2.0 and
running spark tests.
caused by: java.lang.IllegalArgumentException: Unrecognized Hadoop major
version number: 3.2.0
at
org.apache.hadoop.hive.shims.S
Hi,
Spark 2.x is already part of Cloudera CDH6 who is based on Hadoop 3.x so
they support officially Spark2+Hadoop3
So for sure, there is tests and development done from this side. In other
part, I don't know the status for Hadoop 3.2.
Regards,
On Tue, Nov 26, 2019 at 1:46 AM Alfredo Marquez
wr
Thank you Ismael! That's what I was looking for. I can take this to our
platform team.
Alfredo
On Mon, Nov 25, 2019, 3:32 PM Ismaël Mejía wrote:
> Not officially. Apache Spark only announced support for Hadoop 3.x
> starting with the upcoming Spark 3.
> There is a preview release of Spark 3 wit
Not officially. Apache Spark only announced support for Hadoop 3.x starting
with the upcoming Spark 3.
There is a preview release of Spark 3 with support for Hadoop 3.2 that you
can try now:
https://archive.apache.org/dist/spark/spark-3.0.0-preview/spark-3.0.0-preview-bin-hadoop3.2.tgz
Enjoy!
O
I also would like know the answer to this question.
Thanks,
Alfredo
On Tue, Nov 19, 2019, 8:24 AM bsikander wrote:
> Hi,
> Are Spark 2.4.4 and Hadoop 3.2.0 compatible?
> I tried to search the mailing list but couldn't find anything relevant.
>
>
>
>
>
> --
> Sent from: http://apache-spark-user
