Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

MaxGekk commented on PR #45064: URL: https://github.com/apache/spark/pull/45064#issuecomment-1944153947 +1, LGTM. Merging to master. Thank you, @dbatomic @stefankandic and @cloud-fan @mkaravel for review. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

MaxGekk closed pull request #45064: [SPARK-46832][SQL] Introducing Collate and Collation expressions URL: https://github.com/apache/spark/pull/45064 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

stefankandic commented on PR #45064: URL: https://github.com/apache/spark/pull/45064#issuecomment-1943448975 @MaxGekk since @dbatomic is off today I added the changes you requested. Let me know if there is anything else that you think we should do 😄 -- This is an automated message from t

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

stefankandic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1489222736 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -0,0 +1,98 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

MaxGekk commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1489031135 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -0,0 +1,98 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

MaxGekk commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1488560783 ## common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java: ## @@ -1410,6 +1422,13 @@ public boolean equals(final Object other) { } } + /**

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

dbatomic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1488536286 ## common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java: ## @@ -1410,6 +1422,13 @@ public boolean equals(final Object other) { } } + /*

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

dbatomic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1488515575 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

dbatomic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1488515363 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

MaxGekk commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1487518940 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

MaxGekk commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1487489877 ## common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java: ## @@ -1410,6 +1422,13 @@ public boolean equals(final Object other) { } } + /**

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

dbatomic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1486959602 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala: ## @@ -520,6 +520,8 @@ object FunctionRegistry { expression[Ascii]("

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

dbatomic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1486678526 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

dbatomic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1486676778 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollationExpressionSuite.scala: ## @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

MaxGekk commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1485603308 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala: ## @@ -520,6 +520,8 @@ object FunctionRegistry { expression[Ascii]("a

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

dbatomic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1484781108 ## sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala: ## @@ -270,8 +273,8 @@ private[columnar] final class StringColumnStats extends C

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

stefankandic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1484248038 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

stefankandic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1484248038 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

stefankandic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1484246345 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

stefankandic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1484246345 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

dbatomic commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1483558418 ## sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala: ## @@ -270,8 +273,8 @@ private[columnar] final class StringColumnStats extends C

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

mkaravel commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1483412350 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

mkaravel commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1483409046 ## sql/api/src/main/scala/org/apache/spark/sql/types/StringType.scala: ## @@ -26,7 +27,27 @@ import org.apache.spark.annotation.Stable * @param collationId The id of

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

mkaravel commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1483407187 ## sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4: ## @@ -989,6 +989,7 @@ primaryExpression | CASE whenClause+ (ELSE elseExpression

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

cloud-fan commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1482891642 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala: ## @@ -650,7 +651,7 @@ class CodegenContext extends Logging {

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

cloud-fan commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1482890623 ## common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java: ## @@ -1410,6 +1422,13 @@ public boolean equals(final Object other) { } } + /

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

cloud-fan commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1482888220 ## common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java: ## @@ -46,7 +47,7 @@ * * Note: This is not designed for general use cases, should n

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

cloud-fan commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1482874569 ## sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala: ## @@ -270,8 +273,8 @@ private[columnar] final class StringColumnStats extends

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

cloud-fan commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1482872108 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

cloud-fan commented on code in PR #45064: URL: https://github.com/apache/spark/pull/45064#discussion_r1482867744 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala: ## @@ -650,7 +651,7 @@ class CodegenContext extends Logging {

[PR] [SPARK-46832][SQL] Introducing Collate and Collation expressions [spark]

dbatomic opened a new pull request, #45064: URL: https://github.com/apache/spark/pull/45064 ### What changes were proposed in this pull request? This PR adds E2E support for `collate` and `collation` expressions. Following changes were made to get us there: 1) Set the rig