Hello all,
I am trying to run JavaKMeansExample of the spark example project. I am
getting the classnotfound exception error:
*Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/spark/internal/Logging*
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at
jcom.dfki.spark.kmeans.KMeansSpark.JavaKMeansExample.main(JavaKMeansExample.java:43)
*Caused by: java.lang.ClassNotFoundException:
org.apache.spark.internal.Logging*
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
I have added all the logging related dependencies as below:
<dependency> <groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId> <version>${slf4j.version}</version>
</dependency> <dependency> <groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId> <version>${slf4j.version}</version>
<scope>${hadoop.deps.scope}</scope> </dependency> <dependency>
<groupId>org.slf4j</groupId> <artifactId>jul-to-slf4j</artifactId>
<version>${slf4j.version}</version> </dependency> <dependency>
<groupId>org.slf4j</groupId> <artifactId>jcl-over-slf4j</artifactId>
<version>${slf4j.version}</version> <!-- <scope>runtime</scope> --> <!--
more correct, but scalac 2.10.3 doesn't like it --> </dependency>
<dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId>
<version>${log4j.version}</version> </dependency> <dependency>
<groupId>commons-logging</groupId> <artifactId>commons-logging</artifactId>
<version>1.2</version> </dependency> What depedencies could I be missing,
any idea? Regards, Subash Basnet
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.apache.spark</groupId>
<artifactId>gettingStarted</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>gettingStarted</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<maven.version>3.3.3</maven.version>
<slf4j.version>1.7.16</slf4j.version>
<log4j.version>1.2.17</log4j.version>
<commons.math3.version>3.4.1</commons.math3.version>
<!-- managed up from 3.2.1 for SPARK-11652 -->
<commons.collections.version>3.2.2</commons.collections.version>
<scala.version>2.11.8</scala.version>
<scala.binary.version>2.11</scala.binary.version>
<commons-codec.version>1.10</commons-codec.version>
<commons-io.version>2.4</commons-io.version>
<!-- org.apache.commons/commons-lang/ -->
<PermGen>64m</PermGen>
<MaxPermGen>512m</MaxPermGen>
<CodeCacheSize>512m</CodeCacheSize>
</properties>
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
<scope>${hadoop.deps.scope}</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4j.version}</version>
<scope>${hadoop.deps.scope}</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jul-to-slf4j</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>${slf4j.version}</version>
<!-- <scope>runtime</scope> --> <!-- more correct, but scalac 2.10.3 doesn't like it -->
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>${log4j.version}</version>
<scope>${hadoop.deps.scope}</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.6.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/commons-logging/commons-logging -->
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.10</artifactId>
<version>2.0.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-mllib_2.10 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.10</artifactId>
<version>2.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.1.3</version>
</dependency>
</dependencies>
</project>
---------------------------------------------------------------------
To unsubscribe e-mail: [email protected]