felipepessoto commented on PR #12215:
URL: https://github.com/apache/gluten/pull/12215#issuecomment-5325656272
Answering the question about row ID `9223372036854775807`: it was never
written. It is not a row index at all, and the defect is upstream of the
deletion-vector aggregator.
**The chain, and how each step was checked.**
The scan emits a batch whose row-index child is empty. With Velox's operator
output validation turned on:
```
Output validation failed for [operator: TableScan, plan node ID: 0]:
Child vector has size 0 less than parent and parent has no nulls 1.
```
Downstream, that empty child gets wrapped in a dictionary of size 1, so the
aggregator's input is a dictionary addressing row 0 of a base that has no rows:
```
Delta bitmap aggregator input column:
type=BIGINT encoding=DICTIONARY size=1 selected=1/1 decodedBase=FLAT
baseSize=0 valuesBytes=0 valuesCapacityRows=0
badValues=[(row=0 baseIndex=0 value=-5163232936757035008
hex=0xb858830000000000 neighbours=<empty values buffer>)]
```
Reading it reads past the end of a zero-length buffer, so the value is
whatever heap memory is there. That is why it differs on every occurrence.
`BaseVector::wrapInDictionary` does not validate indexes against the base's
size, so the malformed vector is built silently and only misbehaves when read.
**This now reproduces deterministically**, which it did not before:
```
spark/testOnly *MergeIntoExtendedSyntaxSQLPathBasedDVsPredPushOnSuite \
-- -z "only update - isPartitioned: true"
```
with `debug.validate_output_from_operators=true` in the Velox query config.
It fails on every run (~65 s, single fork). The same test on the non-DV suite
(`MergeIntoExtendedSyntaxSQLPathBasedSuite`) passes with zero validation
failures, so this is specific to the path that requests `_metadata.row_index`,
not general scan noise.
**Why it looked so rare.** The bounds check only rejects negatives and
values above `kMaxRepresentableValue`. Linux user-space pointers are around
1.4e14 -- positive, and well inside Delta's range -- so they pass.
Instrumenting the malformed read instead of the value, a single test pass
showed 176 out-of-bounds reads: 144 returned pointer-magnitude values, 27
returned `0`, 18 returned small values, and all were accepted. The abort we
have been chasing is the rare tail.
That means a garbage row index is usually added to the deletion vector
rather than raising. Pointer-magnitude values address rows that do not exist; a
`0` marks row 0 of that file deleted. The suites do pass at baseline, so
something is evidently limiting the impact and I have not worked out what --
but I would not assume this is only a flaky crash.
The fix does not belong in the aggregator; it is the first reader of a
column that was already malformed when the scan produced it. #12783 has the
diagnostic that produced the output above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
