LATERAL VIEW explode issue

2015-05-20 Thread kiran mavatoor
Hi, When I use LATERAL VIEW explode on the registered temp table in spark shell, it works.  But when I use the same in spark-submit (as jar file) it is not working. its giving error -  failure: ``union'' expected but identifier VIEW found sql statement i am using is SELECT id,mapKey FROM

Re: LATERAL VIEW explode issue

2015-05-20 Thread kiran mavatoor
but this looks like parser error...Which would make me guess different  context or different spark versio on the cluster you are submitting to... Sent on the new Sprint Network from my Samsung Galaxy S®4. Original message From: kiran mavatoor Date:05/20/2015 5:57 AM (GMT-05:00) To: User

example code for current date in spark sql

2015-05-05 Thread kiran mavatoor
Hi, In Hive , I am using unix_timestamp() as 'update_on' to insert current date in 'update_on' column of the table. Now I am converting it into spark sql. Please suggest example code to insert current date and time into column of the table using spark sql.  CheersKiran.

spark sql LEFT OUTER JOIN java.lang.ClassCastException

2015-04-27 Thread kiran mavatoor
Hi There, I am using spark sql left out join query.  The sql query is  scala val test = sqlContext.sql(SELECT e.departmentID FROM employee e LEFT OUTER JOIN department d ON d.departmentId = e.departmentId).toDF() In the spark 1.3.1 its working fine, but the latest pull is give the below error