Re: Unable to load native-hadoop library problem
This seems unrelated to not being able to load native-hadoop library. Is it failing to connect to ResourceManager? Have you verified that there is an RM process listening on port 8032 at the specified IP? On Tue, May 6, 2014 at 6:25 PM, Sophia wrote: > Hi,everyone, > [root@CHBM220 spark-0.9.1]# > > SPARK_JAR=.assembly/target/scala-2.10/spark-assembly_2.10-0.9.1-hadoop2.2.0.jar > ./bin/spark-class org.apache.spark.deploy.yarn.Client --jar > examples/target/scala-2.10/spark-examples_2.10-assembly-0.9.1.jar --class > org.apache.spark.examples.SparkPi --args yarn-standalone --num-workers 3 > --master-memory 2g --worker-memory 2g --worker-cores 1 > 14/05/07 09:05:14 WARN NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > 14/05/07 09:05:14 INFO RMProxy: Connecting to ResourceManager at > CHBM220/192.168.10.220:8032 > Then it stopped,my hadoop_conf_dir has been configued well,what should I do > to? > Wish you happy everyday. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Unable-to-load-native-hadoop-library-problem-tp5469.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
Re: Unable to load native-hadoop library problem
Hello Sophia You are only providing the Spark jar here (nevertheless, a spark jar that contains hadoop libraries in it, but that is not sufficient). Where is your hadoop installed? (Most probably: /usr/lib/hadoop/*) So you need to add that to your class path (by using -cp) I guess. Let me know if that works shivani On Tue, May 6, 2014 at 6:25 PM, Sophia wrote: > Hi,everyone, > [root@CHBM220 spark-0.9.1]# > > SPARK_JAR=.assembly/target/scala-2.10/spark-assembly_2.10-0.9.1-hadoop2.2.0.jar > ./bin/spark-class org.apache.spark.deploy.yarn.Client --jar > examples/target/scala-2.10/spark-examples_2.10-assembly-0.9.1.jar --class > org.apache.spark.examples.SparkPi --args yarn-standalone --num-workers 3 > --master-memory 2g --worker-memory 2g --worker-cores 1 > 14/05/07 09:05:14 WARN NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > 14/05/07 09:05:14 INFO RMProxy: Connecting to ResourceManager at > CHBM220/192.168.10.220:8032 > Then it stopped,my hadoop_conf_dir has been configued well,what should I do > to? > Wish you happy everyday. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Unable-to-load-native-hadoop-library-problem-tp5469.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > -- Software Engineer Analytics Engineering Team@ Box Mountain View, CA
Unable to load native-hadoop library problem
Hi,everyone, [root@CHBM220 spark-0.9.1]# SPARK_JAR=.assembly/target/scala-2.10/spark-assembly_2.10-0.9.1-hadoop2.2.0.jar ./bin/spark-class org.apache.spark.deploy.yarn.Client --jar examples/target/scala-2.10/spark-examples_2.10-assembly-0.9.1.jar --class org.apache.spark.examples.SparkPi --args yarn-standalone --num-workers 3 --master-memory 2g --worker-memory 2g --worker-cores 1 14/05/07 09:05:14 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 14/05/07 09:05:14 INFO RMProxy: Connecting to ResourceManager at CHBM220/192.168.10.220:8032 Then it stopped,my hadoop_conf_dir has been configued well,what should I do to? Wish you happy everyday. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Unable-to-load-native-hadoop-library-problem-tp5469.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
