SQLContext is in driver side, and I don’t think you can use it in executors. How to provide lookup functionality in executors really depends on how you would use them.
Thanks. Zhan Zhang On Dec 22, 2015, at 4:44 PM, SRK <swethakasire...@gmail.com> wrote: > Hi, > > Can SQL Context be used inside mapPartitions? My requirement is to register > a set of data from hdfs as a temp table and to be able to lookup from inside > MapPartitions based on a key. If it is not supported, is there a different > way of doing this? > > Thanks! > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Can-SqlContext-be-used-inside-mapPartitions-tp25771.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
