Hi there The following is my steps. And got the same exception with Daniel's. Another question: how can I build a tgz file like the pre-build file I download from official website?
1. download trunk from git. 2. add following lines in pom.xml + <profile> + <id>hadoop-2.6</id> + <properties> + <hadoop.version>2.6.0</hadoop.version> + <protobuf.version>2.5.0</protobuf.version> + <jets3t.version>0.9.0</jets3t.version> + </properties> + </profile> 3. run "mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package" 4. in $SPARK_HOME, run following command ./bin/spark-submit --master yarn-cluster --class org.apache.spark.examples.SparkPi lib/spark-examples*.jar 10 Kyle 2014-12-18 2:24 GMT+08:00 Daniel Haviv <[email protected]>: > > Thanks for your replies. > I was building spark from trunk. > > Daniel > > On 17 בדצמ׳ 2014, at 19:49, Nicholas Chammas <[email protected]> > wrote: > > Thanks for the correction, Sean. Do the docs need to be updated on this > point, or is it safer for now just to note 2.4 specifically? > > On Wed Dec 17 2014 at 5:54:53 AM Sean Owen <[email protected]> wrote: > >> Spark works fine with 2.4 *and later*. The docs don't mean to imply >> 2.4 is the last supported version. >> >> On Wed, Dec 17, 2014 at 10:19 AM, Nicholas Chammas >> <[email protected]> wrote: >> > Spark 1.3 does not exist. Spark 1.2 hasn't been released just yet. Which >> > version of Spark did you mean? >> > >> > Also, from what I can see in the docs, I believe the latest version of >> > Hadoop that Spark supports is 2.4, not 2.6. >> > >> > Nick >> > >> > On Wed Dec 17 2014 at 2:09:56 AM Kyle Lin <[email protected]> >> wrote: >> >> >> >> >> >> I also got the same problem...... >> >> >> >> 2014-12-09 22:58 GMT+08:00 Daniel Haviv <[email protected]>: >> >>> >> >>> Hi, >> >>> I've built spark 1.3 with hadoop 2.6 but when I startup the >> spark-shell I >> >>> get the following exception: >> >>> >> >>> 14/12/09 06:54:24 INFO server.AbstractConnector: Started >> >>> [email protected]:4040 >> >>> 14/12/09 06:54:24 INFO util.Utils: Successfully started service >> 'SparkUI' >> >>> on port 4040. >> >>> 14/12/09 06:54:24 INFO ui.SparkUI: Started SparkUI at >> http://hdname:4040 >> >>> 14/12/09 06:54:25 INFO impl.TimelineClientImpl: Timeline service >> address: >> >>> http://0.0.0.0:8188/ws/v1/timeline/ >> >>> java.lang.NoClassDefFoundError: >> >>> org/codehaus/jackson/map/deser/std/StdDeserializer >> >>> at java.lang.ClassLoader.defineClass1(Native Method) >> >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) >> >>> at >> >>> java.security.SecureClassLoader.defineClass(SecureClassLoade >> r.java:142) >> >>> >> >>> Any idea why ? >> >>> >> >>> Thanks, >> >>> Daniel >> >>> >> >>> >> > >> >
