eldenmoon opened a new pull request, #67955:
URL: https://github.com/apache/doris/pull/67955
### What problem does this PR solve?
Issue Number: None
Related PR: #60847
Problem Summary:
On a stored VARIANT column, `element_at` with an integer index returns NULL
when it is applied to an array that was itself extracted with `element_at`, for
example `element_at(element_at(v, 'items'), 1)` or `v['items'][1]` on a row
holding `{"items": [2, 3, 4]}`. The same expressions on a constant VARIANT
value return the element.
`VariantSubPathPruning` pushes an `element_at` chain on a VARIANT slot down
to the scan as a storage sub-column path. #60847 made it accept integer
literals as path segments too, so the query above reads the sub-column
`items.1` (`subColPath=[items, 1]` in `EXPLAIN VERBOSE`). Storage sub-paths
address object keys only, so the scan looks for an object key `1` under
`items`, finds none, and returns NULL instead of indexing the array. With `SET
disable_nereids_rules = 'VARIANT_SUB_PATH_PRUNING'` the same query returns the
right value.
This PR stops the storage sub-path at the last string key. An integer index
is no longer added to the path, so the scan reads `items`
(`subColPath=[items]`) and `element_at` applies the index to the extracted
value at runtime, with the same 1-based and negative-index semantics as for a
constant value. String keys are still pushed down.
### Release note
Fix `element_at` with an integer index (for example `element_at(v['items'],
1)`) returning NULL for an array inside a VARIANT column.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [x] Unit Test
- [x] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- `./run-fe-ut.sh --run
org.apache.doris.nereids.rules.rewrite.PruneNestedColumnTest`: 63 tests passed,
including the new `testVariantIntegerIndexStopsSubColumnPath`.
- `./run-regression-test.sh --run -d variant_p0 -s
regression_test_variant_element_at`: passed. The new cases cover indexes 1, -1,
0 and out of range, a nested array, an integer index on an object, a missing
path, a string key that looks like a number, and constant VARIANT values; the
expected output was reviewed row by row.
- Manual: on a single-node cluster built from master,
`element_at(element_at(v, 'items'), 1)` returned NULL with `subColPath=[items,
1]` before the change; after it, indexes 1, -1 and `['items'][2]` return 2, 4
and 3 with `subColPath=[items]`.
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
