Re: How to query a hive table from inside a map in Spark

[Alex Kozlov]

While this is possible via jdbc calls, it is not the best practice: you
should probably use variable broadcasting

instead.

On Sun, Feb 14, 2016 at 8:40 PM, SRK  wrote:

> Hi,
>
> Is it possible to query a hive table which has data stored in the form of a
> parquet file from inside map/partitions in Spark? My requirement is that I
> have a User table in Hive/hdfs and for each record inside a sessions RDD, I
> should be able to query the User table and if the User table already has a
> record for that userId, query the record and do further processing.
>
>
> Thanks!
>
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-query-a-hive-table-from-inside-a-map-in-Spark-tp26224.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>


-- 
Alex Kozlov
(408) 507-4987
(650) 887-2135 efax
ale...@gmail.com

How to query a hive table from inside a map in Spark

[SRK]

Hi,

Is it possible to query a hive table which has data stored in the form of a
parquet file from inside map/partitions in Spark? My requirement is that I
have a User table in Hive/hdfs and for each record inside a sessions RDD, I
should be able to query the User table and if the User table already has a
record for that userId, query the record and do further processing.


Thanks!







--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/How-to-query-a-hive-table-from-inside-a-map-in-Spark-tp26224.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org