[PR] [WIP][SQL] UTF8 string validation [spark]

2024-06-03 Thread via GitHub
uros-db opened a new pull request, #46845: URL: https://github.com/apache/spark/pull/46845 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was

Re: [PR] [WIP][SQL] UTF8 string validation [spark]

2024-06-06 Thread via GitHub
mkaravel commented on code in PR #46845: URL: https://github.com/apache/spark/pull/46845#discussion_r1630715164 ## common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java: ## @@ -1436,6 +1436,48 @@ public byte toByteExact() { throw new NumberFormatExceptio

Re: [PR] [WIP][SQL] UTF8 string validation [spark]

2024-06-07 Thread via GitHub
uros-db commented on code in PR #46845: URL: https://github.com/apache/spark/pull/46845#discussion_r1631152669 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala: ## @@ -2050,4 +2051,93 @@ class StringExpressionsSuite extends Sp

Re: [PR] [WIP][SQL] UTF8 string validation [spark]

2024-06-07 Thread via GitHub
uros-db commented on code in PR #46845: URL: https://github.com/apache/spark/pull/46845#discussion_r1631165016 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala: ## @@ -686,6 +686,205 @@ case class EndsWith(left: Expression, right:

Re: [PR] [WIP][SQL] UTF8 string validation [spark]

2024-06-07 Thread via GitHub
uros-db commented on code in PR #46845: URL: https://github.com/apache/spark/pull/46845#discussion_r1631166079 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala: ## @@ -686,6 +686,205 @@ case class EndsWith(left: Expression, right:

Re: [PR] [WIP][SQL] UTF8 string validation [spark]

2024-06-07 Thread via GitHub
uros-db commented on code in PR #46845: URL: https://github.com/apache/spark/pull/46845#discussion_r1631166863 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala: ## @@ -686,6 +686,205 @@ case class EndsWith(left: Expression, right:

Re: [PR] [WIP][SQL] UTF8 string validation [spark]

2024-06-07 Thread via GitHub
uros-db commented on code in PR #46845: URL: https://github.com/apache/spark/pull/46845#discussion_r1631168863 ## sql/core/src/test/resources/sql-tests/inputs/string-functions.sql: ## @@ -276,3 +276,16 @@ select luhn_check(6017); select luhn_check(6018);