I think you can use foreachpartition instead of foreachrdd
Sathish
On Sun, Jan 3, 2016 at 5:51 AM SRK <swethakasire...@gmail.com> wrote:
> Hi,
>
> Can a tempTable registered in sqlContext be used to query inside forEachRDD
> as shown below?
> My requirement is that I have a set of data in the form of parquet inside
> hdfs and I need to register the data
> as a tempTable using sqlContext and query it inside forEachRDD as shown
> below.
>
> sqlContext.registerTempTable("tempTable")
>
> messages.foreachRDD { rdd =>
> val message:RDD[String] = rdd.map { y => y._2 }
>
> sqlContext.sql("SELECT time,To FROM tempTable")
> }
>
> Thanks!
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Can-a-tempTable-registered-by-sqlContext-be-used-inside-a-forEachRDD-tp25862.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>
