Liang-Chi Hsieh created SPARK-13854:
---------------------------------------
Summary: Add constraints to outer join
Key: SPARK-13854
URL: https://issues.apache.org/jira/browse/SPARK-13854
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Liang-Chi Hsieh
Currently, for left outer join we only keep left side constraint. For right
outer join, we only keep right side constraints. For full outer join, the
constraints are empty.
In fact, the constraints are less than the actual constraints for the join
operator.
For example, for left outer join, besides the constraints from left side, the
constraints of right side should be inherited with a bit modification.
Consider a join as following:
{code}
val tr1 = LocalRelation('a.int, 'b.int, 'c.int).subquery('tr1)
val tr2 = LocalRelation('a.int, 'd.int, 'e.int).subquery('tr2)
tr1.where('a.attr > 10)
.join(tr2.where('d.attr < 100), LeftOuter, Some("tr1.a".attr ===
"tr2.a".attr))
{code}
The constraints are not only "a" > 10, "a" is not null. It should also include
("d" is null || "d" < 100).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
