Re: Using Spark SQL / Hive on AWS EMR

2016-03-03 Thread Suresh Thalamati
the error ('Invalid method name: ‘alter_table_with_cascade’’) you are seeing may be related to mismatch of hive versions. Error looks similar to one reported in https://issues.apache.org/jira/browse/SPARK-12496 > On Mar 3, 2016, at 7:43 AM, Gourav Sengupta wrote: > > Hi, > > Why are you

Re: Using Spark SQL / Hive on AWS EMR

2016-03-03 Thread Gourav Sengupta
Hi, Why are you trying to load data into HIVE and then access it via hiveContext? (by the way hiveContext tables are not visible in the sqlContext). Please read the data directly into a SPARK dataframe and then register it as a temp table to run queries on it. Regards, Gourav On Thu, Mar 3, 20

Using Spark SQL / Hive on AWS EMR

2016-03-03 Thread Afshartous, Nick
Hi, On AWS EMR 4.2 / Spark 1.5.2, I tried the example here https://spark.apache.org/docs/1.5.0/sql-programming-guide.html#hive-tables to load data from a file into a Hive table. scala> val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc) scala> sqlContext.sql("CREATE TABLE