[
https://issues.apache.org/jira/browse/SPARK-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Armbrust resolved SPARK-3753.
-------------------------------------
Resolution: Cannot Reproduce
I can't reproduce this in Spark 1.5. Please reopen if you have additional
details.
> Spark hive join results in empty with shared hive context
> ---------------------------------------------------------
>
> Key: SPARK-3753
> URL: https://issues.apache.org/jira/browse/SPARK-3753
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.1.0
> Reporter: Hector Yee
> Priority: Minor
>
> When I have two hive tables and do a join with the same hive context I get
> the empty set
> e.g.
> val hc = new HiveContext(sc)
> val table1 = hc.sql("SELECT * from t1")
> val table2 = hc.sql("SELECT * from t2")
> val intersect = table1.join(table2).take(10)
> // empty set
> but this works if I do
> val hc1 = new HiveContext(sc)
> val table1 = hc1.sql("SELECT * from t1")
> val hc2 = new HiveContext(sc)
> val table2 = hc2.sql("SELECT * from t2")
> val intersect = table1.join(table2).take(10)
> I am not sure if take is propagating up the take to table1 and table2 and
> then doing the intersect (in the case of large tables that means no results)
> or if it is some other problem with hive context.
> Doing the join in one SQL query also seems to result in the empty set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
