theirix opened a new issue, #18889: URL: https://github.com/apache/datafusion/issues/18889
### Is your feature request related to a problem or challenge? Math UDFs in the [datafusion-functions crate](https://github.com/apache/datafusion/tree/main/datafusion/functions) support integer and floating types, but not Decimals. This epic is dedicated to adding and improving decimal support for UDFs. It is a follow-up epic to adding Decimal support to the DataFusion core #3523. So far, it is implemented for `log`, `power` binary UDFs; `round`, `ceil` unary UDF. Turns out, more code should be moved into helper functions to make UDFs leaner and abstracted from details (e.g., scalar vs. array cases, casting, etc). ### Describe the solution you'd like There are the following primary directions: 1) Adding support for well-known `Decimal128` and `Decimal256` to existing functions 2) Adding support for new `Decimal32` and `Decimal64`, which are not yet fully supported 3) Refining coercion rules to work with mixtures of floats/decimals 4) Ensuring it would work properly with the new `parse_float_as_decimal` flag, forcing floats to be decimals after SQL parsing 5) Improving tests to validate correct behaviour for floats/decimals and corner cases 6) Moving some core support to the Arrow libraries I welcome thoughts and discussions about these directions. ### Describe alternatives you've considered The approach of coercing decimals to floats could work, but it loses precision and data and doesn't match the behaviour of existing SQL engines (Postgres, Spark). Decimals should be first-class citizens. ### Additional context **Related tickets:** Function support: - https://github.com/apache/datafusion/issues/17054 - https://github.com/apache/datafusion/issues/18031 - https://github.com/apache/datafusion/issues/17555 - https://github.com/apache/datafusion/issues/1545 - https://github.com/apache/datafusion/issues/7689 - https://github.com/apache/datafusion/issues/3572 Core support: - https://github.com/apache/datafusion/issues/14612 - https://github.com/apache/datafusion/issues/18092 - https://github.com/apache/datafusion/issues/14760 - https://github.com/apache/datafusion/issues/14763 - https://github.com/apache/datafusion/issues/17489 - https://github.com/apache/datafusion/issues/17747 Coercion and type issues: - https://github.com/apache/datafusion/issues/14272 - https://github.com/apache/datafusion/issues/8795 - https://github.com/apache/datafusion/issues/16667 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
