[ https://issues.apache.org/jira/browse/IGNITE-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753948#comment-15753948 ]
Anton Vinogradov commented on IGNITE-3710: ------------------------------------------ Alexey, we have 160 transitive dependencies at spark module according to maven report. I found that addition of {noformat} <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-catalyst_2.11</artifactId> <version>${spark.version}</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-client</artifactId> <version>${hadoop.version}</version> </dependency> {noformat} starts fixing of issues. I'll continue to detect required libs, but main question is how it worked before (for spark 1.x) without any transitive dependencies. > Upgrade ignite-spark module to Spark 2.0 > ---------------------------------------- > > Key: IGNITE-3710 > URL: https://issues.apache.org/jira/browse/IGNITE-3710 > Project: Ignite > Issue Type: Improvement > Components: Ignite RDD > Affects Versions: 1.7 > Reporter: Valentin Kulichenko > Assignee: Anton Vinogradov > Priority: Critical > Attachments: Ignite_Tests_Ignite_RDD_495.log > > > Currently {{ignite-spark}} depends on Spark 1.5.2 and fails with 2.0 with > this exception: > {noformat} > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/spark/Logging > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:763) > at > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) > at java.net.URLClassLoader.access$100(URLClassLoader.java:73) > at java.net.URLClassLoader$1.run(URLClassLoader.java:368) > at java.net.URLClassLoader$1.run(URLClassLoader.java:362) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:361) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at > org.apache.ignite.spark.JavaIgniteContext.<init>(JavaIgniteContext.scala:42) > at client.SparkIgniteClient.main(SparkIgniteClient.java:75) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > at > com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) > Caused by: java.lang.ClassNotFoundException: org.apache.spark.Logging > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > {noformat} > Need to investigate if we can upgrade without breaking compatibility with old > versions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)