peterxcli commented on code in PR #5682:
URL: https://github.com/apache/datafusion-comet/pull/5682#discussion_r3942133878
##########
native/spark-expr/src/conversion_funcs/string.rs:
##########
@@ -1470,7 +1474,13 @@ fn timestamp_parser<T: TimeZone>(
if !has_direct_match {
if let Some((stripped, suffix_tz)) = extract_offset_suffix(value) {
- return timestamp_parser_with_tz(stripped, eval_mode, &suffix_tz);
+ let stripped = stripped.trim_end();
Review Comment:
Fixed in 89e5ff5cc. The Spark 4 raw-leading-whitespace guard now checks the
T prefix before suffix extraction, rather than requiring a base-pattern match
that a timezone suffix can hide. Spark 3.x keeps its existing trim-first path.
Added the exact TAB-prefixed column case, a fractional-seconds case, and a
no-leading-whitespace control through the Parquet-backed cast helper. Each is
compared with Spark in Legacy, TRY, and ANSI modes for TIMESTAMP and
TIMESTAMP_NTZ. Extended the native regression for Legacy/TRY/ANSI rejection and
Spark 3.x acceptance.
The native regression failed on the reported input before the fix.
Afterward: all 662 spark-expr unit tests and all 8 selected Spark 4.1.3
timestamp-cast tests passed. Native build, Rust formatting, Spotless, and diff
whitespace checks passed. Spark 3.x compatibility was checked through the
native version flag; I did not run a separate Spark 3.5 JVM suite.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
