[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-07-15 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r921912420 ## core/src/main/resources/error/error-classes.json: ## @@ -256,6 +256,18 @@ "Key does not exist. Use `try_element_at` to tolerate non-existent key and return

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-07-14 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r921059099 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala: ## @@ -422,6 +426,15 @@ trait CheckAnalysis extends PredicateHelper with Loo

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-07-14 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r921057828 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala: ## @@ -422,6 +426,15 @@ trait CheckAnalysis extends PredicateHelper with Loo

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-07-14 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r921057828 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala: ## @@ -422,6 +426,15 @@ trait CheckAnalysis extends PredicateHelper with Loo

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-07-14 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r921053543 ## sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala: ## @@ -2012,7 +2012,97 @@ class Dataset[T] private[sql]( @scala.annotation.varargs def agg(expr: Co

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-07-14 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r921052382 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala: ## @@ -55,6 +55,10 @@ trait CheckAnalysis extends PredicateHelper with Looku

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-07-14 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r921037613 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala: ## @@ -1382,6 +1417,12 @@ class Analyzer(override val catalogManager: CatalogMana

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-07-14 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r921028076 ## core/src/main/resources/error/error-classes.json: ## @@ -256,6 +256,18 @@ "Key does not exist. Use `try_element_at` to tolerate non-existent key and return

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-07-12 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r919026261 ## core/src/main/resources/error/error-classes.json: ## @@ -256,6 +256,18 @@ "Key does not exist. Use `try_element_at` to tolerate non-existent key and return

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-13 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r893842370 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala: ## @@ -1382,6 +1417,12 @@ class Analyzer(override val catalogManager: CatalogMana

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-08 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r892098440 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala: ## @@ -1227,6 +1227,49 @@ case class Pivot( override protecte

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-08 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r892098440 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala: ## @@ -1227,6 +1227,49 @@ case class Pivot( override protecte

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-08 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r892593552 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala: ## @@ -736,6 +737,24 @@ abstract class TypeCoercionBase { } } + /**

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-08 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r892575789 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala: ## @@ -1227,6 +1227,49 @@ case class Pivot( override protecte

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-08 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r892098440 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala: ## @@ -1227,6 +1227,49 @@ case class Pivot( override protecte

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-08 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r892428863 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala: ## @@ -1227,6 +1227,49 @@ case class Pivot( override protecte

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-08 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r892376424 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala: ## @@ -736,6 +737,24 @@ abstract class TypeCoercionBase { } } + /**

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-08 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r892374424 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala: ## @@ -736,6 +737,24 @@ abstract class TypeCoercionBase { } } + /**

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-08 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r892338094 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala: ## @@ -736,6 +737,24 @@ abstract class TypeCoercionBase { } } + /**

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-08 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r892101500 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala: ## @@ -806,6 +825,7 @@ abstract class TypeCoercionBase { object TypeCoercion

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-08 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r892098440 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala: ## @@ -1227,6 +1227,49 @@ case class Pivot( override protecte

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-03 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r889077442 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-03 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r889074533 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-03 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r889072078 ## sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala: ## @@ -2012,6 +2012,152 @@ class Dataset[T] private[sql]( @scala.annotation.varargs def agg(expr: C

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-03 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r889068185 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-06-03 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r889066692 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-26 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r882526293 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-26 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r882526293 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-26 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r882526293 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-26 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r882411766 ## sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala: ## @@ -2012,6 +2012,152 @@ class Dataset[T] private[sql]( @scala.annotation.varargs def agg(expr: C

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r882143957 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r882077001 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r881793259 ## sql/core/src/test/scala/org/apache/spark/sql/MeltSuite.scala: ## @@ -0,0 +1,297 @@ +/* Review Comment: When I run above example, I get this error. Looks like res

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r881802721 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r881802721 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r881793259 ## sql/core/src/test/scala/org/apache/spark/sql/MeltSuite.scala: ## @@ -0,0 +1,297 @@ +/* Review Comment: When I run above example, I get this error. Looks like res

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r881780193 ## sql/core/src/test/scala/org/apache/spark/sql/MeltSuite.scala: ## @@ -0,0 +1,297 @@ +/* Review Comment: Partially! Pivot has an aggregation part, which aggregates

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r881726314 ## sql/core/src/test/scala/org/apache/spark/sql/MeltSuite.scala: ## @@ -0,0 +1,297 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * con

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r881700066 ## sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala: ## @@ -2012,6 +2012,152 @@ class Dataset[T] private[sql]( @scala.annotation.varargs def agg(expr: C

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r881699551 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r881699194 ## sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala: ## @@ -2012,6 +2012,152 @@ class Dataset[T] private[sql]( @scala.annotation.varargs def agg(expr: C

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r881698995 ## sql/core/src/main/scala/org/apache/spark/sql/Melt.scala: ## @@ -0,0 +1,125 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [spark] EnricoMi commented on a diff in pull request #36150: [SPARK-38864][SQL] Add melt / unpivot to Dataset

2022-05-25 Thread GitBox
EnricoMi commented on code in PR #36150: URL: https://github.com/apache/spark/pull/36150#discussion_r881695872 ## sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala: ## @@ -2012,6 +2012,152 @@ class Dataset[T] private[sql]( @scala.annotation.varargs def agg(expr: C