alamb commented on issue #5398:
URL:
https://github.com/apache/arrow-datafusion/issues/5398#issuecomment-1891040303
> Question on implementation detail while implementing this: should this
throw an Err if the timestamp cannot be parsed given either the default
behaviour or with any provided formats? Current behaviour seems to be be to
throw an Err which mirrors postgresql behaviour, whereas with spark for example
the behaviour is to return null for any unparseable string.
I think we should keep the existing behavior
> A thought I did have would be to change the behaviour via flag that would
be documented as part of the api and user guide.
I agree a flag (or maybe another function) could be appropriate
This high level observation is that different SQL implementations have
different semantics (Decimal handling is another thing that spark seems to do
differently). Given DataFusion's built in functions have only one version, we
can't mirror both systems.
My long term hope / plan is to pull as many functions out of the core as
possible (e.g https://github.com/apache/arrow-datafusion/issues/8045 ) so that
people can more easily customize the behavior. For example, we could have a
`to_tmestamp` that mirrors postgres and a `to_timestamp` that mirrors spark,
and people could choose which one to use for a particular SessionContext
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
