stevomitric opened a new pull request, #57634:
URL: https://github.com/apache/spark/pull/57634
### What changes were proposed in this pull request?
This is a test-only change under the nanosecond-timestamp SPIP
(SPARK-56822). It adds end-to-end coverage for operations over the
nanosecond-precision timestamp types `TIMESTAMP_NTZ(p)` / `TIMESTAMP_LTZ(p)`
(`p` in `[7, 9]`) that ride on the cross-precision widening (SPARK-57454) and
string/nanos coercion (SPARK-57811) but had no dedicated tests:
- Map-key lookup fast path: nanos-key sections added to the
threshold-parameterized `GetMapValue` (`ComplexTypeSuite`) and `ElementAt`
(`CollectionExpressionsSuite`) blocks, exercising both the hash (threshold 0)
and linear (threshold Int.MaxValue) paths in interpreted and codegen modes.
- Mixed-precision joins: `TimestampNanosJoinSuiteBase` gains `nanos(p1) JOIN
nanos(p2)` and `micro JOIN nanos` cases across every forced join strategy and
codegen mode, asserting the join key widens to the common nanos type and still
distinguishes the sub-microsecond remainder.
- New `TimestampNanosRenderingSuiteBase` for `show()` string rendering and
`collect()` external round-trip at `p = 7/8/9`.
- Golden SQL sections appended to `timestamp-{ntz,ltz}-nanos.sql` for
INTERSECT/EXCEPT, BETWEEN, if/nvl/ifnull, IN (subquery), explode, element_at,
struct-field extraction, and map lookup by string key and by nanosecond key.
The map-key hash fast path was flagged as a possible latent issue (its hash
arm lists only microsecond timestamps). It is in fact already correct: nanos
keys are physical `TimestampNanosVal` objects that hash and compare through the
object `hashCode()`/ `equals()` fall-through arm, consistently on the driver
and codegen sides. The new tests lock this in; no production change was
required.
### Why are the changes needed?
To establish and guard the behavior of nanosecond-precision timestamps
across joins, set operations, conditionals, subqueries, complex-type access,
and rendering.
### Does this PR introduce any user-facing change?
No. Test-only.
### How was this patch tested?
New and updated tests:
- `catalyst/testOnly *.ComplexTypeSuite *.CollectionExpressionsSuite`
- `sql/testOnly *.TimestampNanosJoin*Suite *.TimestampNanosRendering*Suite`
- `sql/testOnly *.SQLQueryTestSuite -- -z nanos`
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
