Hi,
Spark 2.x is already part of Cloudera CDH6 who is based on Hadoop 3.x so
they support officially Spark2+Hadoop3
So for sure, there is tests and development done from this side. In other
part, I don't know the status for Hadoop 3.2.
Regards,
On Tue, Nov 26, 2019 at 1:46 AM Alfredo Marquez
Thank you Ismael! That's what I was looking for. I can take this to our
platform team.
Alfredo
On Mon, Nov 25, 2019, 3:32 PM Ismaël Mejía wrote:
> Not officially. Apache Spark only announced support for Hadoop 3.x
> starting with the upcoming Spark 3.
> There is a preview release of Spark 3
Not officially. Apache Spark only announced support for Hadoop 3.x starting
with the upcoming Spark 3.
There is a preview release of Spark 3 with support for Hadoop 3.2 that you
can try now:
https://archive.apache.org/dist/spark/spark-3.0.0-preview/spark-3.0.0-preview-bin-hadoop3.2.tgz
Enjoy!
Thanks for your answer, my use case is friend recommandation for 200
million profils.
Le lun. 25 nov. 2019 à 14:10, Jörn Franke a écrit :
> I think it depends what you want do. Interactive big data graph analytics
> are probably better of in Janusgraph or similar.
> Batch processing (once-off)
I think it depends what you want do. Interactive big data graph analytics are
probably better of in Janusgraph or similar.
Batch processing (once-off) can be still fine in graphx - you have though to
carefully design the process.
> Am 25.11.2019 um 20:04 schrieb mahzad kalantari :
>
>
> Hi
Hi all
My question is about GraphX, I 'm looking for user feedbacks on the
performance.
I read this paper written by Facebook team that says Graphx has very poor
performance.
https://engineering.fb.com/core-data/a-comparison-of-state-of-the-art-graph-processing-systems/
Has anyone already
Hi all,
I will give you some informations about ARM CI of Spark:
Our team and community are working on build/test Spark master on ARM64
server, after find and fix some issues[1], we have integrated two ARM
testing jobs[2] to community CI(AMPLAB Jenkins),
they run as daily job and have been
I found _sqlContext is null , how to resolve it ?
2019-11-25
lk_spark
发件人："lk_spark"
发送时间：2019-11-25 16:00
主题：how spark structrued stream write to kudu
收件人："user.spark"
抄送：
hi,all:
I'm using spark 2.4.4 to readstream data from kafka and want to write to
kudu 1.7.0 , my code like
hi,all:
I'm using spark 2.4.4 to readstream data from kafka and want to write to
kudu 1.7.0 , my code like below :
val kuduContext = new KuduContext("master:7051", spark.sparkContext)
val console = cnew.select("*").as[CstoreNew]
.writeStream
