Rex Xiong created SPARK-31660: --------------------------------- Summary: Dataset.joinWith supports JoinType object as input parameter Key: SPARK-31660 URL: https://issues.apache.org/jira/browse/SPARK-31660 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 2.4.5 Reporter: Rex Xiong
Current Dataset.joinWith API accepts String type joinType, it doesn't support JoinType object. I prefer JoinType object (like enum) than String, less chance to have typo and has better readability {code:scala} def joinWith[U](other: Dataset[U], condition: Column, joinType: String): Dataset[(T, U)] = {{code} https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala If I pass LeftOuter.sql to joinType, it will throw exception, since there's a white space in LeftOuter.sql {code:scala} case object LeftOuter extends JoinType { override def sql: String = "LEFT OUTER" } {code} While the constructor of JoinType only removes underscore, doesn't handle white spaces, {code:scala} object JoinType { def apply(typ: String): JoinType = typ.toLowerCase(Locale.ROOT).replace("_", "") match { case "inner" => Inner case "outer" | "full" | "fullouter" => FullOuter case "leftouter" | "left" => LeftOuter case "rightouter" | "right" => RightOuter case "leftsemi" | "semi" => LeftSemi case "leftanti" | "anti" => LeftAnti case "cross" => Cross case _ => val supported = Seq( "inner", "outer", "full", "fullouter", "full_outer", "leftouter", "left", "left_outer", "rightouter", "right", "right_outer", "leftsemi", "left_semi", "semi", "leftanti", "left_anti", "anti", "cross") throw new IllegalArgumentException(s"Unsupported join type '$typ'. " + "Supported join types include: " + supported.mkString("'", "', '", "'") + ".") } }{code} https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/joinTypes.scala I suggest we either add another set of APIs which provide JoinType instead of String, or change JoinType.apply to remove white space as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org