Mihaly Toth created SPARK-23465:
-----------------------------------
Summary: Dataset.withAllColumnsRenamed should map all column names
to a new one
Key: SPARK-23465
URL: https://issues.apache.org/jira/browse/SPARK-23465
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.3.1
Reporter: Mihaly Toth
Currently one can only rename a column only one by one using
{{withColumnRenamed()}} function. When one would like to rename all or most of
the columns it would be easier to specify an algorithm for mapping from the old
to the new name (like prefixing) than iterating over all the fields.
Example usage is joining to a Dataset with the same or similar schema (special
case is self joining) where the names are the same or overlapping. Such a
joined Dataset would fail at {{saveAsTable()}}
With the new function usage would be easy like that:
{code:java}
ds.withAllColumnsRenamed("prefix" + _)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
