andygrove commented on PR #2925:
URL:
https://github.com/apache/datafusion-comet/pull/2925#issuecomment-3663382328
@coderfender, the functionality looks good overall. There are likely some
edge cases we do not support that we need to handle or document. I have only
found a few so far, and here is a suggested test to add.
```scala
test("cast StringType to DecimalType non-ASCII characters") {
// TODO fix for Spark 4.0.0
assume(!isSpark40Plus)
val values = Seq(
"\uff11\uff12\uff13", // fullwidth digits １２３
"123\u0000", // null byte at end
"12\u00003", // null byte in middle
"\u0000123", // null byte at start
null).toDF("a")
Seq(true, false).foreach(k =>
castTest(values, DataTypes.createDecimalType(10, 2), testAnsi = k))
}
```
```
!== Correct Answer - 5 == == Spark Answer - 5 ==
struct<a:string,a:decimal(10,2)> struct<a:string,a:decimal(10,2)>
[null,null] [null,null]
![ 123,123.00] [ 123,null]
[12 3,null] [12 3,null]
![123 ,123.00] [123 ,null]
![１２３,123.00] [１２３,null]
```
I think it is fine if we do not support everything, but we should mark the
cast as `Incompatible` and add documentation explaining the current limitations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
