Hi,
I've downloaded the 3.2.2-without Hadoop Spark distribution in
order to test it in a without Hadoop context.
I tested the version with Hadoop and it worked well.
When I wanted to start the master, I encountered this basic error
: Caused by: java.lang.NoClassDefFoundError: org/apache/log4j/spi/Filter
$ ./start-master.sh
starting org.apache.spark.deploy.master.Master, logging to
/projets/kfk/home/kfkusrm1/spark-3.2.2-bin-without-hadoop/logs/spark-kfkusrm1-org.apache.spark.deploy.master.Master-1-pcld1313.angers.cnp.fr.out
failed to launch: nice -n 0
/projets/kfk/home/kfkusrm1/spark-3.2.2-bin-without-hadoop/bin/spark-class
org.apache.spark.deploy.master.Master --host pcld1313.angers.cnp.fr --port 7077
--webui-port 8080
Spark Command: /soft/java/jdk-11.0.11+9/bin/java -cp
/projets/kfk/home/kfkusrm1/spark-3.2.2-bin-without-hadoop/conf/:/projets/kfk/home/kfkusrm1/spark-3.2.2-bin-without-hadoop/jars/*
-Xmx1g org.apache.spark.deploy.master.Master --host pcld1313.angers.cnp.fr
--port 7077 --webui-port 8080
========================================
Error: Unable to initialize main class org.apache.spark.deploy.master.Master
Caused by: java.lang.NoClassDefFoundError: org/apache/log4j/spi/Filter
full log in
/projets/kfk/home/kfkusrm1/spark-3.2.2-bin-without-hadoop/logs/spark-kfkusrm1-org.apache.spark.deploy.master.Master-1-pcld1313.angers.cnp.fr.out
I didn't find any librairies slf4j in the distribution « without hadoop » !
But we can find it in the distribution with Hadoop :
slf4j-log4j12-1.7.30.jar
slf4j-api-1.7.30.jar
Is it a problem in the distribution available at
https://www.apache.org/dyn/closer.lua/spark/spark-3.2.2/spark-3.2.2-bin-without-hadoop.tgz
?
Thanks for your help.
Regards
Grégory
[cid:[email protected]]
Grégory FLORANCE
Service Architecture - YU2
Direction de l'expérience client, des services numériques et de la donnée
CNP Assurances
1 place François Mitterrand, 49100 ANGERS
Tél : 02 41 96 39 64 ou 06 08 02 63 45
[email protected]<mailto:[email protected]>
Pensez à l'environnement, n'imprimez ce courriel que si nécessaire
Ce message (et toutes ses pièces jointes éventuelles) est confidentiel et
établi a l'intention exclusive de ses destinataires.
Toute utilisation de ce message non conforme a sa destination, toute diffusion
ou toute publication, totale ou partielle, est
interdite, sauf autorisation expresse.
L'internet ne permettant pas d'assurer l’intégrité de ce message, CNP
Assurances et ses filiales déclinent toute responsabilité
au titre de ce message, s'il a été altéré, déformé ou falsifié.
*****************************************************************************
*****************************************************************************
This message and any attachments (the "message") are confidential and intended
solely for the addressees.
Any unauthorised use or dissemination is prohibited.
E-mails are susceptible to alteration.
Neither CNP Assurances nor any of its subsidiaries or affiliates shall be
liable for the message if altered, changed or falsified.
