pearu commented on PR #54661: URL: https://github.com/apache/arrow/pull/54661#issuecomment-6447528568
@kou @pitrou @ianmcook Thanks, I'll follow the `date_parsers` proposal. The plan: - A new sub-issue of #41488 and a new PR from main for `ConvertOptions::date_parsers` (C++, pyarrow, docs; R and GLib as follow-ups), as proposed: same type as `timestamp_parsers`, replaces the built-in ISO-8601 parser when non-empty, parsed timestamps are floored to the day, zone offsets are rejected, and it is also used for type inference. `timestamp_parsers` stays as is. - `time_parsers` follows in a second sub-issue and PR, depending on the feedback here. - This PR will be closed when the new one is ready; #54662 and the issues #54650 and #54651 will be closed once the new PRs have landed. Two behaviours differ from what @ianmcook asked for in #50146, so please object if you see a problem: - `ISO8601()` in `date_parsers` rejects fractional seconds, as it does for `timestamp[s]` (#50146 accepts and discards them). - In `time_parsers`, a date-only ISO-8601 value gives `00:00:00`: the user gets what the option specifies. The retry in seconds for dates outside the time unit's range stays. Since type inference uses `date_parsers`: with `ISO8601()` in it, ISO date-time columns are inferred as dates (dates are tried before timestamps; DuckDB does the same), and without it, ISO dates are inferred as timestamps. Both will be documented. Values are whitespace-trimmed, as with the built-in date parser. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
