EnricoMi commented on code in PR #36150:
URL: https://github.com/apache/spark/pull/36150#discussion_r892575789
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala:
##########
@@ -1227,6 +1227,49 @@ case class Pivot(
override protected def withNewChildInternal(newChild: LogicalPlan): Pivot =
copy(child = newChild)
}
+/**
+ * A constructor for creating a melt, which will later be converted to a
[[Expand]]
+ * during the query analysis.
+ *
+ * An empty values array will be replaced during analysis with all resolved
outputs of child except
+ * the ids. This expansion allows to easily melt all non-id columns.
+ *
+ * The type of the value column is derived from all value columns during
analysis once all values
+ * are resolved. All values' types have to be compatible, otherwise the result
value column cannot
+ * be assigned the individual values and an AnalysisException is thrown.
+ *
+ * @see `org.apache.spark.sql.catalyst.analysis.TypeCoercionBase.MeltCoercion`
+ *
+ * @param ids Id columns
+ * @param values Value columns to melt
+ * @param variableColumnName Name of the variable column
+ * @param valueColumnName Name of the value column
+ * @param valueType Type of value column once known
+ * @param child Child operator
+ */
+case class Melt(
+ ids: Seq[NamedExpression],
+ values: Seq[NamedExpression],
Review Comment:
With auto-generated you mean names like these: `Alias(expr,
toPrettySQL(expr))()`?
If the user is not happy with the name of their expression, then `alias` /
`name` / `as` is there:
```
val melted = ds.melt(
Array(($"id" * 10).as("primary"), $"str1".as("secondary")),
Array(($"int1" + $"long1").as("sum"), length($"str2").as("len")),
variableColumnName = "var",
valueColumnName = "val")
```
Are there other methods on `Dataset` that have a similar requirement?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
