[GitHub] spark pull request #22591: [SPARK-25571][SQL] Add withColumnsRenamed method ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22591 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22591: [SPARK-25571][SQL] Add withColumnsRenamed method ...
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/22591#discussion_r221431453 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2300,6 +2300,60 @@ class Dataset[T] private[sql]( } } + /** + * (Scala-specific) Returns a new Dataset with renamed columns. + * This is a no-op if schema doesn't contain any columns in map. + * + * {{{ + * ds.withColumnsRenamed( + * "exist_column1" -> "new_column1", + * "exist_column2" -> "new_column2" + * ) + * }}} + * + * @group untypedrel + * @since 3.0.0 + */ + @scala.annotation.varargs + def withColumnsRenamed(columnMap: (String, String), columnMaps: (String, String)*): DataFrame = { +withColumnsRenamed((columnMap +: columnMaps).toMap) + } + + /** + * (Scala-specific) Returns a new Dataset with renamed columns. + * This is a no-op if schema doesn't contain any columns in map. + * + * {{{ + * ds.withColumnsRenamed(Map( + * "exist_column1" -> "new_column1", + * "exist_column2" -> "new_column2" + * )) + * }}} + * + * @group untypedrel + * @since 3.0.0 + */ + def withColumnsRenamed(columnMap: Map[String, String]): DataFrame = { --- End diff -- These changes seem to be duplicated: https://github.com/apache/spark/pull/22428/files#diff-7a46f10c3cedbf013cf255564d9483cdR2316 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22591: [SPARK-25571][SQL] Add withColumnsRenamed method ...
GitHub user cryeo opened a pull request: https://github.com/apache/spark/pull/22591 [SPARK-25571][SQL] Add withColumnsRenamed method to Dataset ## What changes were proposed in this pull request? Add `withColumnsRenamed` method to rename multiple columns at a time as follows. ```scala // before ds.withColumnRenamed("first_name", "firstName") .withColumnRenamed("last_name", "lastName") .withColumnRenamed("postal_code", "postalCode") // after ds.withColumnsRenamed( "first_name" -> "firstName", "last_name" -> "lastName", "postal_code" -> "postalCode" ) // or ds.withColumnsRenamed(Map( "first_name" -> "firstName", "last_name" -> "lastName", "postal_code" -> "postalCode" )) ``` ## How was this patch tested? This patch is tested by unit test. You can merge this pull request into a Git repository by running: $ git pull https://github.com/cryeo/spark SPARK-25571 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22591.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22591 commit af9dabe3919ae3811c258e5dbf224fb7bde225be Author: Chaerim YEO Date: 2018-09-29T13:57:16Z [SPARK-25571][SQL] Add withColumnsRenamed method to Dataset --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org