Liang-Chi Hsieh created SPARK-13854: ---------------------------------------
Summary: Add constraints to outer join Key: SPARK-13854 URL: https://issues.apache.org/jira/browse/SPARK-13854 Project: Spark Issue Type: Improvement Components: SQL Reporter: Liang-Chi Hsieh Currently, for left outer join we only keep left side constraint. For right outer join, we only keep right side constraints. For full outer join, the constraints are empty. In fact, the constraints are less than the actual constraints for the join operator. For example, for left outer join, besides the constraints from left side, the constraints of right side should be inherited with a bit modification. Consider a join as following: {code} val tr1 = LocalRelation('a.int, 'b.int, 'c.int).subquery('tr1) val tr2 = LocalRelation('a.int, 'd.int, 'e.int).subquery('tr2) tr1.where('a.attr > 10) .join(tr2.where('d.attr < 100), LeftOuter, Some("tr1.a".attr === "tr2.a".attr)) {code} The constraints are not only "a" > 10, "a" is not null. It should also include ("d" is null || "d" < 100). -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org