hhr293 commented on PR #58424:
URL: https://github.com/apache/spark/pull/58424#issuecomment-5694976841
Hi, @cloud-fan Thanks for the review! I've addressed all four points:
1. TIME → TIMESTAMP clock-dependent cast: Removed the hand-rolled type
enumeration and delegated the check directly to Cast.needsTimeZone, which
already recurses into array/map/struct and covers all the timestamp/time/date
cast pairs, so there's no separate copy of Spark's cast rules to keep in sync.
2. ANSI test outcome: Updated the ansi=true/ansi=false branches to each
assert the specific outcome type they're expected to produce (success rows vs.
the specific error class), instead
of accepting whichever shape both sides happened to agree on. While
auditing this pattern I found the same gap in a third test (the NULL equi-key
case) that wasn't flagged but had the identical issue, so I fixed that one too.
3. sameResult snapshot identity: Added a leaf-level check in
isSameBaseRelation — using collectLeaves() (not collectFirst, since a self-join
side can hold more than one Parquet leaf) to gather all leaves on both sides,
then comparing FileIndex instances with eq rather than relying on
sameResult/equals alone. This guards against InMemoryFileIndex.equals only
comparing root paths, which could otherwise let two independently-resolved
scans of the same path (e.g. before/after an append) be treated as the same
snapshot.
4. Fixed the stale class-header comment and the
DIVIDE_BY_ZERO/REMAINDER_BY_ZERO nit.
In addtion, CI failed due to Kafka streaming test
(KafkaRealTimeModeAggregationSuite, an eventually-timeout in an unrelated
module) that isn't touched by this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
