Spencer created SPARK-27935: ------------------------------- Summary: Introduce leftOuterJoinWith and fullOuterJoinWith Key: SPARK-27935 URL: https://issues.apache.org/jira/browse/SPARK-27935 Project: Spark Issue Type: New Feature Components: SQL Affects Versions: 3.0.0 Reporter: Spencer
Currently, calling `Dataset[A].joinWith(Dataset[B], col, "left_outer")` or `Dataset[A].joinWith(Dataset[B], col, "full_outer")` require users to do null checks on the resulting `Dataset[(A, B)]` To make the expected result types of outer joins more explicit, I propose a couple of new joinWith functions: {noformat} def leftOuterJoinWith[U](other: Dataset[U], condition: Column): Dataset[(T, Option[U])] def fullOuterJoinWith[U](other: Dataset[U], condition: Column): Dataset[(Option[T], Option[U])]{noformat} The return type of `fullOuterJoinWith` is imperfect, since `(None, None)` is an invalid case, but still an improvement on the present interface. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org