[ https://issues.apache.org/jira/browse/SPARK-35173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492320#comment-17492320 ]
Apache Spark commented on SPARK-35173: -------------------------------------- User 'HyukjinKwon' has created a pull request for this issue: https://github.com/apache/spark/pull/35518 > Support columns batch adding in PySpark.dataframe > ------------------------------------------------- > > Key: SPARK-35173 > URL: https://issues.apache.org/jira/browse/SPARK-35173 > Project: Spark > Issue Type: Improvement > Components: PySpark, SQL > Affects Versions: 3.2.0 > Reporter: Yikun Jiang > Assignee: Yikun Jiang > Priority: Major > Fix For: 3.3.0 > > > Now, the pyspark can only use withColumn to do column adding a column or > replacing the existing column that has the same name. The scala withColumn > can adding columns at one pass. [1] > > Before this added, the user can only use withColumn again and again like: > > {code:java} > self.df.withColumn("key1", col("key1")).withColumn("key2", > col("key2")).withColumn("key3", col("key3")){code} > > After the support, you user can use the with_columns complete batch > operations: > > {code:java} > self.df.withColumn(["key1", "key2", "key3"], [col("key1"), col("key2"), > col("key3")]){code} > > [1] > [https://github.com/apache/spark/blob/b5241c97b17a1139a4ff719bfce7f68aef094d95/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L2402] -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org