[
https://issues.apache.org/jira/browse/SPARK-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060145#comment-14060145
]
Nicholas Chammas commented on SPARK-524:
----------------------------------------
+1 for cleanup of issues that likely have no further action.
> spark integration issue with Cloudera hadoop
> --------------------------------------------
>
> Key: SPARK-524
> URL: https://issues.apache.org/jira/browse/SPARK-524
> Project: Spark
> Issue Type: Bug
> Reporter: openreserach
>
> Hi,
> 1. I am using single EC2 instance with pre-built mesos (ami-0fcb7966) (Same
> issue if I build mesos from source code in locall VM)
> 2. Follow instruction on
> https://github.com/mesos/spark/wiki/Running-spark-on-mesos with some tweaks.
> 3. I install Cloudera cdhu5 by yum (not using pre-built hadoop due to lack of
> document)
> 4. ./spartk-shell.sh
> import spark._
> val sc = new SparkContext("localhost:5050","passwd")
> val ec2 = sc.textFile("hdfs://localhost:8020/tmp/passwd")
> IF I keep val HADOOP_VERSION = "0.20.205.0" in project/SparkBuild.scala
> at val file = sc.textFile("hdfs://localhost:8020/tmp/passwd")
> I am getting error
> Protocol org.apache.hadoop.hdfs.protocol.ClientProtocol version mismatch.
> (client = 61, server = 63)
> IF I set val HADOOP_VERSION = "0.20.2-cdh3u5" or val HADOOP_VERSION =
> "0.20.2-cdh3u3"
> I am getting error at ec2.count()
> ERROR spark.SimpleJob: Task 0:0 failed more than 4 times; aborting job
> like the one reported at
> http://mail-archives.apache.org/mod_mbox/incubator-mesos-dev/201108.mbox/%3cbd25ae7a-c9dc-4020-ad40-41c66dcaa...@eecs.berkeley.edu%3E
> Please let me know if you cannot replicate this error, and give more
> instruction on how Spark integrate with Cloudera Hadoop
> Thanks
> -QH
--
This message was sent by Atlassian JIRA
(v6.2#6252)
