Re: Spark Job History Server

[patcharee]

I turned it on. But it failed to start. In the log,

Spark assembly has been built with Hive, including Datanucleus jars on 
classpath
Spark Command: /usr/lib/jvm/java-1.7.0-openjdk.x86_64/bin/java -cp 
:/root/spark-1.3.0-bin-hadoop2.4/sbin/../conf:/root/spark-1.3.0-bin-hadoop2.4/lib/spark-assembly-1.3.0-hadoop2.4.0.jar:/root/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-rdbms-3.2.9.jar:/root/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-core-3.2.10.jar:/root/spark-1.3.0-bin-hadoop2.4/lib/datanucleus-api-jdo-3.2.6.jar:/etc/hadoop/conf 
-XX:MaxPermSize=128m -Dspark.akka.logLifecycleEvents=true -Xms512m 
-Xmx512m org.apache.spark.deploy.history.HistoryServer
========================================
15/03/18 10:23:46 WARN NativeCodeLoader: Unable to load native-hadoop 
library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.ClassNotFoundException: 
org.apache.spark.deploy.yarn.history.YarnHistoryProvider
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:191)
        at 
org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:183)
        at 
org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala)

Patcharee

On 18. mars 2015 11:35, Akhil Das wrote:
You can simply turn it on using:
|./sbin/start-history-server.sh|

​Read more here <http://spark.apache.org/docs/1.3.0/monitoring.html>.​


Thanks
Best Regards

On Wed, Mar 18, 2015 at 4:00 PM, patcharee <patcharee.thong...@uni.no 
<mailto:patcharee.thong...@uni.no>> wrote:

    Hi,

    I am using spark 1.3. I would like to use Spark Job History
    Server. I added the following line into conf/spark-defaults.conf

    spark.yarn.services
    org.apache.spark.deploy.yarn.history.YarnHistoryService
    spark.history.provider
    org.apache.spark.deploy.yarn.history.YarnHistoryProvider
    spark.yarn.historyServer.address sandbox.hortonworks.com:19888
    <http://sandbox.hortonworks.com:19888>

    But got Exception in thread "main"
    java.lang.ClassNotFoundException:
    org.apache.spark.deploy.yarn.history.YarnHistoryProvider

    What class is really needed? How to fix it?

    Br,
    Patcharee

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
    <mailto:user-unsubscr...@spark.apache.org>
    For additional commands, e-mail: user-h...@spark.apache.org
    <mailto:user-h...@spark.apache.org>