Max Gekk created SPARK-56965:
--------------------------------
Summary: Add SQL parser support for nanosecond-capable
TIMESTAMP_NTZ(p) and TIMESTAMP_LTZ(p) (p in 7-9)
Key: SPARK-56965
URL: https://issues.apache.org/jira/browse/SPARK-56965
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.2.0
Reporter: Max Gekk
h3. Summary
Extend the SQL data type parser so fractional-second precision p in [7, 9] is
supported in DDL and CAST(... AS ...) type strings, aligned with SPARK-56822
(SPIP scope: nanosecond-capable band only). Parser-only: maps SQL text to
TimestampNTZNanosType(p) / TimestampLTZNanosType(p). No UnsafeRow, casts,
Parquet, expressions, or SQLConf in this issue.
Follow the TIME(p) pattern in DataTypeAstBuilder / DataTypeParserSuite, limited
to p = 7, 8, 9.
h3. Scope
* DataTypeAstBuilder (and grammar if needed) for:
** TIMESTAMP_NTZ(p), TIMESTAMP_LTZ(p) with p in {7, 8, 9}
** Aliases: TIMESTAMP(p) WITHOUT TIME ZONE, TIMESTAMP(p) WITH LOCAL TIME ZONE
(same p range)
* Map to TimestampNTZNanosType(p) / TimestampLTZNanosType(p) (SPARK-56876)
* Reject p < 7 or p > 9 with UNSUPPORTED_TIMESTAMP_NTZ_PRECISION /
UNSUPPORTED_TIMESTAMP_LTZ_PRECISION
* DataTypeParserSuite tests
h3. Out of scope
* p in [0, 6] (microsecond types stay unparameterized timestamp / timestamp_ntz
/ timestamp_ltz)
* SQLConf / preview flags
* UnsafeRow / PhysicalDataType / TypeOps
* Cast, coercion, expressions, Parquet, Connect, PySpark
* End-to-end query execution
h3. Acceptance criteria
* parseDataType("TIMESTAMP_NTZ(9)") -> TimestampNTZNanosType(9) (and 7, 8)
* parseDataType("TIMESTAMP_LTZ(9)") -> TimestampLTZNanosType(9)
* parseDataType("TIMESTAMP(8) WITHOUT TIME ZONE") -> TimestampNTZNanosType(8)
* parseDataType("TIMESTAMP(7) WITH LOCAL TIME ZONE") -> TimestampLTZNanosType(7)
* parseDataType("TIMESTAMP_NTZ(6)") / "(10)" ->
UNSUPPORTED_TIMESTAMP_*_PRECISION
* Unparameterized timestamp, timestamp_ntz, timestamp_ltz unchanged
* DataTypeParserSuite covers aliases for p in [7, 9]
h3. References
* Parent: SPARK-56822
* API types: SPARK-56876 / https://github.com/apache/spark/pull/55952
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
