Re: Using Spark SQL / Hive on AWS EMR

2016-03-03 Thread Gourav Sengupta
Hi, Why are you trying to load data into HIVE and then access it via hiveContext? (by the way hiveContext tables are not visible in the sqlContext). Please read the data directly into a SPARK dataframe and then register it as a temp table to run queries on it. Regards, Gourav On Thu, Mar 3,

Using Spark SQL / Hive on AWS EMR

2016-03-03 Thread Afshartous, Nick
Hi, On AWS EMR 4.2 / Spark 1.5.2, I tried the example here https://spark.apache.org/docs/1.5.0/sql-programming-guide.html#hive-tables to load data from a file into a Hive table. scala> val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc) scala> sqlContext.sql("CREATE