Rajesh Vakkalagadda created SPARK-59346:
-------------------------------------------
Summary: Add tests for ProjectingInternalRow
Key: SPARK-59346
URL: https://issues.apache.org/jira/browse/SPARK-59346
Project: Spark
Issue Type: Test
Components: SQL
Affects Versions: 4.4.0
Reporter: Rajesh Vakkalagadda
Description:
ProjectingInternalRow
(sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ProjectingInternalRow.scala)
has no dedicated test suite. It backs the DSv2 row-level write path
(ReplaceDataProjections / WriteDeltaProjections, used by MERGE/UPDATE/DELETE on
V2 tables) by re-exposing an underlying InternalRow through a colOrdinals index
remapping, via ~18 near-identical one-line accessor delegations (getBoolean
through getVariant, plus getStruct/getArray/getMap/get). That repetition is
exactly the shape of code where a copy-paste slip — reading the wrong source
ordinal, or delegating to a similarly-named sibling accessor (e.g.
getTimestampNTZNanos calling into the LTZ path) — would silently return the
wrong value during a write, rather than fail loudly. Nothing today would catch
that.
h3. Proposed tests
New suite: {{{}ProjectingInternalRowSuite{}}}, under
{{{}sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/{}}}.
- Every accessor (boolean, byte, short, int, long, float, double, decimal,
UTF8String, binary, binary view, interval, timestamp NTZ nanos, timestamp LTZ
nanos,
variant, struct, array, map, and the generic {{{}get(ordinal, dataType){}}}),
plus {{{}isNullAt{}}}, correctly reads through a non-identity {{colOrdinals}}
permutation
to the right source column — not the identity mapping, and not a
similarly-named neighboring accessor.
- {{project()}} lets a single {{ProjectingInternalRow}} instance be reused
across different underlying rows (including rows of a different shape) without
leaking
stale state from whichever row was projected previously.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
