Re: Classpath problem trying to use DataFrames

Thanks for the response. I lost access to my cluster over the weekend, so I had to wait until today to check. All of the correct Hive jars are in classpath.txt. Also, this error seems to be happening in the driver rather than the executors. It's running in yarn-client mode, so it should use

??????Classpath problem trying to use DataFrames

topher.br...@oracle.com>, user <user@spark.apache.org> ????: Re: Classpath problem trying to use DataFrames Do you have all your hive jars listed in the classpath.txt / SPARK_DIST_CLASSPATH env., specifically the hive-exec jar? Is the location of that jar also the same on all the distribut

Re: Classpath problem trying to use DataFrames

Do you have all your hive jars listed in the classpath.txt / SPARK_DIST_CLASSPATH env., specifically the hive-exec jar? Is the location of that jar also the same on all the distributed hosts? Passing an explicit executor classpath string may also help overcome this (replace HIVE_BASE_DIR to the

Classpath problem trying to use DataFrames

I'm trying to run a basic "Hello world" type example using DataFrames with Hive in yarn-client mode. My code is: JavaSparkContext sc = new JavaSparkContext("yarn-client", "Test app")) HiveContext sqlContext = new HiveContext(sc.sc()); sqlContext.sql("SELECT * FROM my_table").count(); The