Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-08 Thread via GitHub
stefankandic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1518057279 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -183,6 +185,57 @@ class CollationSuite extends DatasourceV2SQLBase { } } +

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-08 Thread via GitHub
stefankandic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1518057279 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -183,6 +185,57 @@ class CollationSuite extends DatasourceV2SQLBase { } } +

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-08 Thread via GitHub
dbatomic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1517982724 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -183,6 +185,57 @@ class CollationSuite extends DatasourceV2SQLBase { } } +

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-08 Thread via GitHub
dbatomic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1517357872 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -183,6 +185,57 @@ class CollationSuite extends DatasourceV2SQLBase { } } +

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-07 Thread via GitHub
HyukjinKwon commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1517309692 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -183,6 +185,57 @@ class CollationSuite extends DatasourceV2SQLBase { } } +

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-07 Thread via GitHub
LuciferYang commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1517243024 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -183,6 +185,57 @@ class CollationSuite extends DatasourceV2SQLBase { } } +

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-07 Thread via GitHub
LuciferYang commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1517243024 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -183,6 +185,57 @@ class CollationSuite extends DatasourceV2SQLBase { } } +

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-07 Thread via GitHub
LuciferYang commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1517243024 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -183,6 +185,57 @@ class CollationSuite extends DatasourceV2SQLBase { } } +

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-03 Thread via GitHub
cloud-fan closed pull request #45290: [SPARK-46834][SQL][Collations] Support for aggregates URL: https://github.com/apache/spark/pull/45290 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-03 Thread via GitHub
cloud-fan commented on PR #45290: URL: https://github.com/apache/spark/pull/45290#issuecomment-1975551130 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-01 Thread via GitHub
dbatomic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1509062803 ## sql/api/src/main/scala/org/apache/spark/sql/types/StringType.scala: ## @@ -33,6 +33,14 @@ class StringType private(val collationId: Int) extends AtomicType with

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-01 Thread via GitHub
cloud-fan commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1508938251 ## sql/api/src/main/scala/org/apache/spark/sql/types/StringType.scala: ## @@ -33,6 +33,14 @@ class StringType private(val collationId: Int) extends AtomicType with

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-03-01 Thread via GitHub
dbatomic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1508848081 ## sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashMapGenerator.scala: ## @@ -173,7 +173,10 @@ abstract class HashMapGenerator(

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-29 Thread via GitHub
cloud-fan commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1507924464 ## sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashMapGenerator.scala: ## @@ -173,7 +173,10 @@ abstract class HashMapGenerator(

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-29 Thread via GitHub
cloud-fan commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1507922548 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/UnsafeRowUtils.scala: ## @@ -197,4 +197,21 @@ object UnsafeRowUtils { s"rowSizeInBytes:

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-29 Thread via GitHub
cloud-fan commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1507922150 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/UnsafeRowUtils.scala: ## @@ -197,4 +197,21 @@ object UnsafeRowUtils { s"rowSizeInBytes:

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-29 Thread via GitHub
cloud-fan commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1507920309 ## sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java: ## @@ -30,6 +30,7 @@ import

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-29 Thread via GitHub
dbatomic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1507736504 ## sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java: ## @@ -95,6 +96,20 @@ public static boolean isMutable(DataType dt) {

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-29 Thread via GitHub
dbatomic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1507732131 ## sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashMapGenerator.scala: ## @@ -173,7 +173,10 @@ abstract class HashMapGenerator(

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-28 Thread via GitHub
dbatomic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1506259546 ## sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java: ## @@ -95,6 +96,20 @@ public static boolean isMutable(DataType dt) {

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-28 Thread via GitHub
dbatomic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1506258459 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/MergeScalarSubqueries.scala: ## @@ -353,9 +353,20 @@ object MergeScalarSubqueries extends

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-28 Thread via GitHub
cloud-fan commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1506223036 ## sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java: ## @@ -95,6 +96,20 @@ public static boolean isMutable(DataType dt) {

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-28 Thread via GitHub
cloud-fan commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1506221249 ## sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashMapGenerator.scala: ## @@ -173,7 +173,10 @@ abstract class HashMapGenerator(

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-28 Thread via GitHub
dbatomic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1506214267 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/MergeScalarSubqueries.scala: ## @@ -353,9 +353,20 @@ object MergeScalarSubqueries extends

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-28 Thread via GitHub
dbatomic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1506202232 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -183,6 +186,57 @@ class CollationSuite extends DatasourceV2SQLBase { } } +

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-28 Thread via GitHub
dbatomic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1505922202 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/hash.scala: ## @@ -405,11 +405,21 @@ abstract class HashExpression[E] extends Expression {

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-27 Thread via GitHub
cloud-fan commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1505328033 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/MergeScalarSubqueries.scala: ## @@ -353,9 +353,20 @@ object MergeScalarSubqueries extends

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-27 Thread via GitHub
cloud-fan commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1505326604 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/hash.scala: ## @@ -405,11 +405,21 @@ abstract class HashExpression[E] extends Expression {

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-27 Thread via GitHub
stefankandic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1504687213 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/hash.scala: ## @@ -405,11 +405,21 @@ abstract class HashExpression[E] extends Expression

Re: [PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-27 Thread via GitHub
stefankandic commented on code in PR #45290: URL: https://github.com/apache/spark/pull/45290#discussion_r1504680132 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -183,6 +186,57 @@ class CollationSuite extends DatasourceV2SQLBase { } } +

[PR] [SPARK-46834][SQL][Collations] Support for aggregates [spark]

2024-02-27 Thread via GitHub
dbatomic opened a new pull request, #45290: URL: https://github.com/apache/spark/pull/45290 ### What changes were proposed in this pull request? This PR is part of collation effort. For details please refer to umbrella JIRA ticket with linked designed document. This PR