Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-03-04 Thread via GitHub
MaxGekk commented on PR #45285: URL: https://github.com/apache/spark/pull/45285#issuecomment-1977197356 @mihailom-db The test failure is related to your changes: ``` [info] Cause: org.scalatest.exceptions.TestFailedException: Function 'collate', Expression class

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-03-04 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1510854919 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -592,3 +593,18 @@ case class QualifyLocationWithWarehouse(catalog:

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-03-04 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1510853996 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -592,3 +593,18 @@ case class QualifyLocationWithWarehouse(catalog:

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-03-04 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1510851292 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -592,3 +593,18 @@ case class QualifyLocationWithWarehouse(catalog:

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-03-01 Thread via GitHub
dbatomic commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1509146815 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -40,6 +41,11 @@ import org.apache.spark.sql.types._

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-03-01 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1508931880 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -592,3 +593,28 @@ case class QualifyLocationWithWarehouse(catalog:

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-03-01 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1508930731 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -592,3 +593,28 @@ case class QualifyLocationWithWarehouse(catalog:

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-03-01 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1508927759 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -592,3 +593,18 @@ case class QualifyLocationWithWarehouse(catalog:

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1508556060 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -592,3 +593,18 @@ case class QualifyLocationWithWarehouse(catalog:

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507274792 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -67,7 +73,13 @@ object CollateExpressionBuilder extends

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507269159 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -67,7 +73,13 @@ object CollateExpressionBuilder extends

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507252815 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -67,7 +73,13 @@ object CollateExpressionBuilder extends

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507250569 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -67,7 +73,13 @@ object CollateExpressionBuilder extends

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507247915 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -67,7 +73,13 @@ object CollateExpressionBuilder extends

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507245967 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -67,7 +73,13 @@ object CollateExpressionBuilder extends

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507245924 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -592,3 +593,18 @@ case class QualifyLocationWithWarehouse(catalog:

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507244351 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -67,7 +73,13 @@ object CollateExpressionBuilder extends

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507243652 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala: ## @@ -3198,7 +3198,13 @@ class AstBuilder extends DataTypeAstBuilder with

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
mihailom-db commented on PR #45285: URL: https://github.com/apache/spark/pull/45285#issuecomment-1970706405 Current SQLConf.COLLATION_ENABLED cannot be used to prevent errors of invalid collation name in TypeContext. How should I proceed with this? In my view, if customer sends a query

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-29 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507242125 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -67,7 +73,13 @@ object CollateExpressionBuilder extends

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507153207 ## sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala: ## @@ -964,6 +964,52 @@ class QueryCompilationErrorsSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507151474 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -592,3 +593,18 @@ case class QualifyLocationWithWarehouse(catalog:

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1507150465 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala: ## @@ -3198,7 +3198,11 @@ class AstBuilder extends DataTypeAstBuilder with

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
MaxGekk commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1506256531 ## sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala: ## @@ -964,6 +964,52 @@ class QueryCompilationErrorsSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1506081331 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -753,6 +753,14 @@ object SQLConf { .checkValue(_ > 0, "The initial number of

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1506078176 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -32,6 +33,11 @@ import org.apache.spark.sql.types.StringType class CollationSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1506075900 ## sql/core/src/main/scala/org/apache/spark/sql/internal/BaseSessionStateBuilder.scala: ## @@ -223,6 +223,7 @@ abstract class BaseSessionStateBuilder(

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1506074914 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -592,3 +593,18 @@ case class QualifyLocationWithWarehouse(catalog:

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1506070480 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala: ## @@ -3198,7 +3198,11 @@ class AstBuilder extends DataTypeAstBuilder with

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1506068842 ## common/utils/src/main/resources/error/error-classes.json: ## @@ -3854,6 +3854,11 @@ "Catalog does not support ." ] }, +

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1505984306 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -40,6 +41,12 @@ import org.apache.spark.sql.types._

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1505706359 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -32,6 +33,11 @@ import org.apache.spark.sql.types.StringType class CollationSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1505618055 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -32,6 +33,11 @@ import org.apache.spark.sql.types.StringType class CollationSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1505618055 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -32,6 +33,11 @@ import org.apache.spark.sql.types.StringType class CollationSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1505536348 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -32,6 +33,11 @@ import org.apache.spark.sql.types.StringType class CollationSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1505536348 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -32,6 +33,11 @@ import org.apache.spark.sql.types.StringType class CollationSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1505527557 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -32,6 +33,11 @@ import org.apache.spark.sql.types.StringType class CollationSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-28 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1505514617 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -32,6 +33,11 @@ import org.apache.spark.sql.types.StringType class CollationSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-27 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1505476279 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -32,6 +33,11 @@ import org.apache.spark.sql.types.StringType class CollationSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-27 Thread via GitHub
mihailom-db commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1505436616 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -32,6 +33,11 @@ import org.apache.spark.sql.types.StringType class CollationSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-27 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1505370756 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -90,6 +100,9 @@ case class Collate(child: Expression,

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-27 Thread via GitHub
MaxGekk commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r150462 ## sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala: ## @@ -297,6 +297,14 @@ private[sql] object QueryCompilationErrors extends

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-27 Thread via GitHub
mkaravel commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1504599200 ## common/utils/src/main/resources/error/error-classes.json: ## @@ -3854,6 +3854,11 @@ "Catalog does not support ." ] }, +

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-27 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1504415553 ## sql/core/src/test/scala/org/apache/spark/sql/errors/QueryCompilationErrorsSuite.scala: ## @@ -964,6 +964,35 @@ class QueryCompilationErrorsSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-27 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1504413598 ## sql/core/src/test/scala/org/apache/spark/sql/CollationSuite.scala: ## @@ -32,6 +33,11 @@ import org.apache.spark.sql.types.StringType class CollationSuite

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-27 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1504410342 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala: ## @@ -67,6 +74,9 @@ object CollateExpressionBuilder extends

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-27 Thread via GitHub
cloud-fan commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1504406480 ## common/utils/src/main/resources/error/error-classes.json: ## @@ -3854,6 +3854,11 @@ "Catalog does not support ." ] }, +

Re: [PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-27 Thread via GitHub
MaxGekk commented on code in PR #45285: URL: https://github.com/apache/spark/pull/45285#discussion_r1504206127 ## common/utils/src/main/resources/error/error-classes.json: ## @@ -3854,6 +3854,11 @@ "Catalog does not support ." ] }, +

[PR] [SPARK-47102][SQL][COLLATION] Add COLLATION_ENABLED config flag [spark]

2024-02-27 Thread via GitHub
mihailom-db opened a new pull request, #45285: URL: https://github.com/apache/spark/pull/45285 ### What changes were proposed in this pull request? This PR adds `COLLATION_ENABLED` config to `SQLConf` and introduces new error class `COLLATION_SUPPORT_NOT_ENABLED` to appropriately report