stevomitric opened a new pull request, #59577: URL: https://github.com/apache/spark/pull/59577
### What changes were proposed in this pull request? This PR adds an optional `precision` argument to `to_timestamp`, `to_timestamp_ntz`, `to_timestamp_ltz` and `try_to_timestamp`: `to_timestamp(timestamp_str, fmt, precision)`. A precision in [7, 9] parses up to 9 fractional digits into `TIMESTAMP_NTZ(p)` / `TIMESTAMP_LTZ(p)` and truncates the digits beyond `p`. Precision 6 keeps the microsecond type. The argument is validated like `current_timestamp(p)` and requires `spark.sql.timestampNanosTypes.enabled`. Without a precision, a nanosecond-precision input now keeps its precision instead of being narrowed to microseconds, e.g. `to_timestamp(TIMESTAMP_NTZ '2020-01-01 00:00:00.123456789')` returns `TIMESTAMP_LTZ(9)`. `GetTimestamp` now produces nanosecond results in both the interpreted and codegen paths, using the nanosecond-aware `TimestampFormatter`. `to_timestamp` is now registered through an expression builder, like the other three functions. ### Why are the changes needed? Part of SPARK-56822. These functions are the SQL way to parse a string with a custom format, and today they always return microseconds. A nanosecond input is also silently truncated, even under ANSI mode. ### Does this PR introduce _any_ user-facing change? Yes, and only when `spark.sql.timestampNanosTypes.enabled` is on. The three-argument form is new, and a nanosecond-precision input keeps its precision. The one- and two-argument forms behave as before for all other inputs. ### How was this patch tested? - New `DateExpressionsSuite` cases for `GetTimestamp` with nanosecond result types, covering both error modes, interpreted and codegen. - New `TimestampNanosFunctionsSuiteBase` tests, which run with ANSI on and off. - New golden cases in `timestamp-ntz-nanos.sql` / `timestamp-ltz-nanos.sql`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Isaac -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
