RE: Executing hive query from Spark code

2015-03-02 Thread Cheng, Hao
, March 3, 2015 2:19 PM To: Felix C; Ted Yu; nitinkak001 Cc: user Subject: RE: Executing hive query from Spark code I am not so sure how Spark SQL compiled in CDH, but if didn’t specify the –Phive and –Phive-thriftserver flags during the build, most likely it will not work if just by providing the

RE: Executing hive query from Spark code

2015-03-02 Thread Cheng, Hao
curious with that, any hint will be appreciated. From: Felix C [mailto:felixcheun...@hotmail.com] Sent: Tuesday, March 3, 2015 12:59 PM To: Ted Yu; nitinkak001 Cc: user Subject: Re: Executing hive query from Spark code It should work in CDH without having to recompile. http

Re: Executing hive query from Spark code

2015-03-02 Thread Felix C
It should work in CDH without having to recompile. http://eradiating.wordpress.com/2015/02/22/getting-hivecontext-to-work-in-cdh/ --- Original Message --- From: "Ted Yu" Sent: March 2, 2015 1:35 PM To: "nitinkak001" Cc: "user" Subject: Re: Executing hive

Re: Executing hive query from Spark code

2015-03-02 Thread Ted Yu
Here is snippet of dependency tree for spark-hive module: [INFO] org.apache.spark:spark-hive_2.10:jar:1.3.0-SNAPSHOT ... [INFO] +- org.spark-project.hive:hive-metastore:jar:0.13.1a:compile [INFO] | +- org.spark-project.hive:hive-shims:jar:0.13.1a:compile [INFO] | | +- org.spark-project.hive.shi