On 12/17/14 1:43 PM, Jerry Raj wrote:

Hi,
I'm using the Scala DSL for Spark SQL, but I'm not able to do joins. I have two tables (backed by Parquet files) and I need to do a join across them using a common field (user_id). This works fine using standard SQL but not using the language-integrated DSL neither

t1.join(t2, on = 't1.user_id == t2.user_id)

Two issues this line:

1. use |===| instead of |==|
2. Add a single quote before |t2|


nor

t1.join(t2, on = Some('t1.user_id == t2.user_id))

work, or even compile. I could not find any examples of how to perform a join using the DSL. Any pointers will be appreciated :)

Thanks
-Jerry

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org


Reply via email to