coderfender commented on issue #286: URL: https://github.com/apache/datafusion-comet/issues/286#issuecomment-3347608664
> ### What is the problem the feature request solves? > Comet currently delegates to DataFusion for many cast operations, and the behavior is not guaranteed to match Spark. This epic is to track fully implementing Spark-compatible `cast` and `try_cast` operations in Comet, with support for ANSI mode. > > For each item in this list to be considered complete, we should have scala tests demonstrating that `cast` and `try_cast` produce the same results as Spark, both with ANSI mode enabled and disabled, using fuzz testing to find edge cases. We can update this list with links to issues as we make progress. > > For cast operations that we cannot easily support with full compatibility, we should either fall back to Spark or provide a configuration that the user can enable to allow the operation to run in Comet. We should also provide documentation explaining any differences in behavior compared to Spark. > > * * [ ] Cast from string to another type > > * [x] Boolean - [feat: Support ANSI mode in CAST from String to Bool #290](https://github.com/apache/datafusion-comet/pull/290)* [x] Integral Types (byte, short, int, long) - [feat: Implement Spark-compatible CAST from string to integral types #307](https://github.com/apache/datafusion-comet/pull/307)* [ ] Floating-point (float, double) - [Implement Spark-compatible CAST from String to Floating Point #326](https://github.com/apache/datafusion-comet/issues/326)* [ ] Decimal - [Implement Spark-compatible CAST from String to Decimal #325](https://github.com/apache/datafusion-comet/issues/325)* [x] Date - [Implement Spark-compatible CAST from String to Date #327](https://github.com/apache/datafusion-comet/issues/327)* [ ] Timestamp - [Implement Spark-compatible CAST from String to Timestamp #328](https://github.com/apache/datafusion-comet/issues/328) and [Cast string to timestamp remaining work #376](https://github.com/apache/datafusion-comet/issues/376)* [ ] Cast to string from primitive types > > * [x] Boolean - [chore: Add more cast tests and improve test framework #351](https://github.com/apache/datafusion-comet/pull/351)* [x] Integral Types (byte, short, int, long) - [chore: Add more cast tests and improve test framework #351](https://github.com/apache/datafusion-comet/pull/351)* [x] Floating-point (float, double) - [Implement Spark-compatible CAST float/double to string #312](https://github.com/apache/datafusion-comet/issues/312)* [ ] Decimal - seems to correct but needs tests to confirm, also should fall back to Spark if `spark.sql.legacy.allowNegativeScaleOfDecimal` is true and scale is negative* [ ] Date - seems correct but needs tests to confirm* [ ] Timestamp - seems correct but needs tests to confirm* [ ] Cast between numeric types > > * [x] Integral to Integral - [Implement Spark-compatible CAST between integer types #311](https://github.com/apache/datafusion-comet/issues/311)* [x] Integral to Boolean - [chore: Add more cast tests and improve test framework #351](https://github.com/apache/datafusion-comet/pull/351)* [ ] Integral to Decimal - [Implement Spark-compatible cast from integral types to decimal #2049](https://github.com/apache/datafusion-comet/issues/2049)* [x] Integral to Floating-point - [chore: Add more cast tests and improve test framework #351](https://github.com/apache/datafusion-comet/pull/351)* [x] Floating-point to Boolean - [chore: Add more cast tests and improve test framework #351](https://github.com/apache/datafusion-comet/pull/351)* [x] Floating-point to Decimal - [Implement Spark-compatible CAST from floating-point to decimal #371](https://github.com/apache/datafusion-comet/issues/371)* [x] Floating-point to Integral - [Implement Spark-compatible CAST from float/doubl e to integer types #350](https://github.com/apache/datafusion-comet/issues/350)* [ ] Decimal to Boolean - fails because arrow does not support this cast* [x] Decimal to Integral - same issues as [Implement Spark-compatible CAST from float/double to integer types #350](https://github.com/apache/datafusion-comet/issues/350)* [ ] Decimal to Floating-point - seems correct but needs tests to confirm* [x] [Implement Spark-compatible cast between decimals with different precision and scale #375](https://github.com/apache/datafusion-comet/issues/375)* [ ] Cast between temporal types > > * [ ] Date to boolean/int/float/decimal - results are incorrect* [ ] Date to Timestamp / TimestampNTZ* [ ] Timestamp to boolean/int/float/decimal - [Implement Spark-compatible CAST from timestamp to numeric types #352](https://github.com/apache/datafusion-comet/issues/352)* [ ] Timestamp to Date* [ ] Other > > * [x] [Implement Spark-compatible cast to/from binary type #377](https://github.com/apache/datafusion-comet/issues/377)* [ ] [Implement Spark-compatible CAST to/from TimestampNTZType #378](https://github.com/apache/datafusion-comet/issues/378) > > In addition to the above tasks, we also need to do the following: > > * [x] Implement a mechanism where we can selectively fall back to Spark for specific cast operations ([feat: Disable cast string to timestamp by default #337](https://github.com/apache/datafusion-comet/pull/337) )[ ] Write documentation that explains any differences between Comet and Spark[x] [Add support for TryCast expression in Spark 3.2 and 3.3 #374](https://github.com/apache/datafusion-comet/issues/374) Raised issue : https://github.com/apache/datafusion-comet/issues/2489 to track Decimal to Boolean cast op -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
