;denny.g@gmail.com<mailto:denny.g@gmail.com>>,
"user@spark.apache.org<mailto:user@spark.apache.org>"
<user@spark.apache.org<mailto:user@spark.apache.org>>
Subject: Re: How do I convert a data frame to broadcast variable?
Hi Nishit,
Yes the JDBC connector
:48 PM
To: Denny Lee; user@spark.apache.org
Subject: Re: How do I convert a data frame to broadcast variable?
Thanks Denny! That does help. I will give that a shot.
Question: If I am going this route, I am wondering how can I only read few
columns of a table (not whole table) from JDBC as data fram
ark.apache.org<mailto:user@spark.apache.org>>
Subject: Re: How do I convert a data frame to broadcast variable?
If you're able to read the data in as a DataFrame, perhaps you can use a
BroadcastHashJoin so that way you can join to that table presuming its small
enough to dis
If you're able to read the data in as a DataFrame, perhaps you can use a
BroadcastHashJoin so that way you can join to that table presuming its
small enough to distributed? Here's a handy guide on a BroadcastHashJoin:
I have a lookup table in HANA database. I want to create a spark broadcast
variable for it.
What would be the suggested approach? Should I read it as an data frame and
convert data frame into broadcast variable?
Thanks,
Nishit
