parthchandra opened a new issue, #3776: URL: https://github.com/apache/datafusion-comet/issues/3776
While working on https://github.com/apache/datafusion-comet/pull/3730, it was discovered that we are not testing various date time formats because the tests in Spark's DateTimeUtilsSuite are never being called. The test suite is in `sql/catalyst` and does not extend either `SQLTestUtils` or `CometTestBase`. The tests in this subproject are not part of Comet CI. Also, `IgnoreComet` is not available in this subproject and it may not be possible to disable the tests selectively. The following tests are relevant - | Test | Format coverage | |---|---| | `string to timestamp` | `YYYY-MM-DD`, `YYYY-MM-DDTHH:mm:ss`, space-vs-T separator, time-only (`HH:mm:ss`, `T18:12:15`), timezone suffixes (`Z`, `UTC`, `GMT±hh:mm`, `±hhmm`, `±h:m`, named tz like `Europe/Moscow`), milliseconds/microseconds, nanosecond truncation, whitespace/control char permutations, ~20 invalid formats that must return `None` | | `SPARK-35780: support full range of timestamp string` | Negative years (`-1969-…`), 5-digit years (`99999-…`), zero-padded years, `Long.MaxValue`/`Long.MinValue` boundary timestamps, integer overflow in individual fields | | `SPARK-37326: stringToTimestampWithoutTimeZone with allowTimeZone` | `TIMESTAMP_NTZ` with `allowTimeZone=true/false` and a `+hh:mm` suffix | | `SPARK-15379: special invalid date string` | Feb 29 on non-leap year, Apr 31 — must be rejected by both `toDate` and `stringToTimestamp` | | `trailing characters while converting string to timestamp` | Trailing garbage after a valid ISO timestamp (e.g. `2019-10-31T10:59:23Z:::`) must return `None` | | `SPARK-29012: special timestamp values` | Keyword strings: `Epoch`, `NOW`, `Yesterday`, `Today`, `tomorrow` (with optional trailing timezone) | | `SPARK-35979: special timestamp without time zone values` | Same keyword strings for `TIMESTAMP_NTZ` via `convertSpecialTimestampNTZ` | It might be easier to copy the format-related tests from DateTimeUtilsSuite into a new or existing Comet test class (e.g. CometDateTimeUtilsSuite), as opposed to trying to enable catalyst tests in Comet CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
