Re: upgrade of Hadoop cluster from 5.1.2 to 5.2.1.

2015-05-20 Thread Anupam sinha
Hello everyone, i am a member of nested group which has select priviledges. Still not able to access Hive StoreOps Database, Please suggest on this. Thank you. On Thu, May 21, 2015 at 9:30 AM, Anupam sinha wrote: > I have upgrade of Hadoop cluster from 5.1.2 to 5.2.1. > Now i am una

upgrade of Hadoop cluster from 5.1.2 to 5.2.1.

2015-05-20 Thread Anupam sinha
I have upgrade of Hadoop cluster from 5.1.2 to 5.2.1. Now i am unable to read the hive tables, before the upgrade i am able to access the Hive StoreOps Database. Please suggest any changes i need to do Here is the output i receive, java.io.IOException: org.apache.hadoop.security.AccessControlEx

ORC Table creation failure

2015-05-20 Thread vivek veeramani
Hi , I'm trying to create a subset table from a master table which is stored in ORC format. The query I'm using is : CREATE TABLE tab_20150510 STORED AS ORC AS SELECT * FROM master_tab WHERE col_id_60 <> '2015-05-20'; The master table is a table with 60 columns with all string data type but 1 c

Create ORC Table on Tez Failed

2015-05-20 Thread Vijay Bhoomireddy
Hi, We are trying to create a Hive ORC Table with Tez. DDL command for the same is create table orc_table ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' STORED AS ORC AS select * from select_table; However, it fails with the following error message: M

Re: Hive on Spark VS Spark SQL

2015-05-20 Thread Alexander Pivovarov
Thank you Xuefu! Excellent explanation and comparison! We should put it to Hive on Spark wiki. https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark On Wed, May 20, 2015 at 10:45 AM, Xuefu Zhang wrote: > I have been working on HIve on Spark, and knows a little about SparkSQL. > Here a

Re: Hive on Spark VS Spark SQL

2015-05-20 Thread Xuefu Zhang
I have been working on HIve on Spark, and knows a little about SparkSQL. Here are a few factors to be considered: 1. SparkSQL is similar to Shark (discontinued) in that it clones Hive's front end (parser and semantic analyzer) and metastore, and inject in between a laryer where Hive's operator tre

Re: Hive on Spark VS Spark SQL

2015-05-20 Thread Edward Capriolo
What about outer lateral view? On Wed, May 20, 2015 at 11:28 AM, matshyeq wrote: > From my experience SparkSQL is still way faster than tez. > Also, SparkSQL (even 1.2.1 which I'm on) supports *lateral view* > > On Wed, May 20, 2015 at 3:41 PM, Edward Capriolo > wrote: > >> Beyond window querie

Re: Hive on Spark VS Spark SQL

2015-05-20 Thread matshyeq
>From my experience SparkSQL is still way faster than tez. Also, SparkSQL (even 1.2.1 which I'm on) supports *lateral view* On Wed, May 20, 2015 at 3:41 PM, Edward Capriolo wrote: > Beyond window queries, hive still has concepts like cube or lateral view > that many "better than hive" systems do

Re: Hive on Spark VS Spark SQL

2015-05-20 Thread Edward Capriolo
Beyond window queries, hive still has concepts like cube or lateral view that many "better than hive" systems don't have. Also now many people went around broadcasting SparkSQL/SparkSQL was/is better/faster than hive but now that tez has "whooped" them in a benchmark they are very quite. http://w

RE: Hive on Spark VS Spark SQL

2015-05-20 Thread Dragga, Christopher
While I've not experimented with the most recent versions of SparkSQL, earlier releases could not cope with intermediate result sets that exceeded the available memory; Hive handles this sort of situation much more gracefully. If you have a smallish cluster and large data, this could pose a pro

Re: Hive on Spark VS Spark SQL

2015-05-20 Thread Uli Bethke
Interesting question and one that I have asked myself. If you are already heavily invested in the Hive ecosystem in terms of code and skills I would look at Hive on Spark as my engine. In theory swapping out engines (MR, TEZ, Spark) should be easy. Even though the devil is in the detail. SparkS