Using Hivecontext solved it.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/spark-sql-join-sql-fails-after-sqlCtx-cacheTable-tp16893p21807.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
the compiler. Shall I replay
your session? I can re-run each line except the last one.
Thanks
Tridib
Date: Tue, 21 Oct 2014 09:39:49 -0700
Subject: Re: spark sql: join sql fails after sqlCtx.cacheTable()
From: ri...@infoobjects.com
To: tridib.sama...@live.com
CC: u...@spark.incubator.apache.org
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
val personPath = /hdd/spark/person.json
val person = sqlContext.jsonFile(personPath)
person.printSchema()
person.registerTempTable(person)
val addressPath = /hdd/spark/address.json
val address = sqlContext.jsonFile(addressPath)
Hi Tridib,
I changed SQLContext to HiveContext and it started working. These are steps
I used.
val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
val person = sqlContext.jsonFile(json/person.json)
person.printSchema()
person.registerTempTable(person)
val address =
Hmm... I thought HiveContext will only worki if Hive is present. I am curious
to know when to use HiveContext and when to use SqlContext.
Thanks Regards
Tridib
--
View this message in context:
Hmm... I thought HiveContext will only worki if Hive is present. I am
curious
to know when to use HiveContext and when to use SqlContext.
http://spark.apache.org/docs/latest/sql-programming-guide.html#getting-started
TLDR; Always use HiveContext if your application does not have a dependency
Thank for pointing that out.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/spark-sql-join-sql-fails-after-sqlCtx-cacheTable-tp16893p16933.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
