aleksandar-trajkovic-db opened a new pull request, #59140: URL: https://github.com/apache/spark/pull/59140
### What changes were proposed in this pull request? SPARK-58876 decides per column, through column-metadata markers, whether Oracle `TimestampNTZType` values are read and written as wall-clock `LocalDateTime`. This PR makes it a per-read/write decision: - Adds `JdbcDialect.timestampNTZAsWallClock(options: JDBCOptions)`. `OracleDialect` returns true unless the legacy mapping is on, and `AggregatedDialect` returns true if any wrapped dialect does. - `JDBCOptions` snapshots `spark.sql.legacy.oracle.timestampNTZMapping.enabled` when it is built. `JDBCTableCatalog.loadTable` pins that snapshot into the table options, so V2 scans and writes, which rebuild `JDBCOptions` later in the query, keep it. - Removes the markers and the unreleased `JdbcDialect.updateExtraColumnMetaForWrite`. ### Why are the changes needed? The decision depends only on the dialect and the legacy flag, not on the column, so it belongs with the read/write options rather than in each column's metadata. This also drops a dialect hook whose only job was stamping that metadata for writes. ### Does this PR introduce _any_ user-facing change? Only relative to the unreleased SPARK-58876: Oracle reads apply the wall-clock conversion to every `TimestampNTZType` column, not just the `DATE`/`TIMESTAMP` columns `OracleDialect` maps (e.g. a column that `customSchema` forces to `TIMESTAMP_NTZ`). `JdbcDialect` gains `timestampNTZAsWallClock` and loses `updateExtraColumnMetaForWrite`. ### How was this patch tested? Updated the SPARK-58876 tests in `JDBCSuite` and `OracleIntegrationSuite`, including reads planned before a legacy-flag flip, and added a `JDBCTableCatalogSuite` test that a V2 scan keeps the flag value pinned by `loadTable`. Ran `JDBCSuite`, `JDBCTableCatalogSuite`, and `OracleIntegrationSuite` (Docker) locally. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Anthropic Claude Opus 5.5) This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
