Re: Spark LOCAL mode and external jar (extraClassPath)

2018-04-13 Thread jb44
Haoyuan - As I mentioned below, I've been through the documentation already. It has not helped me to resolve the issue. Here is what I have tried so far: - setting extraClassPath as explained below - adding fs.alluxio.impl through sparkconf - adding spark.sql.hive.metastore.sharedPrefixes

Spark LOCAL mode and external jar (extraClassPath)

2018-04-12 Thread jb44
I'm running spark in LOCAL mode and trying to get it to talk to alluxio. I'm getting the error: java.lang.ClassNotFoundException: Class alluxio.hadoop.FileSystem not found The cause of this error is apparently that Spark cannot find the alluxio client jar in its classpath. I have looked at the