HappenLee opened a new pull request, #68246:
URL: https://github.com/apache/doris/pull/68246
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
With a zero `space_expand_rate`, `topn` and `topn_array` can return a value
in one-stage aggregation but an empty result in two-stage aggregation. The BE
computes a zero candidate capacity, so serializing the partial state discards
every candidate. For example, `topn(s, 1, 0)` over `['a', 'b', 'a']` should
return `{"a":2}` in both aggregation phases.
Treat non-positive expansion rates as unlimited candidate capacity in the
shared BE state used by `topn`, `topn_array`, and `topn_weighted`. Retain all
intermediate candidates and exact counts through serialization and merging;
`top_num` still limits the final result. Positive expansion rates and the
default rate of 50 keep their existing behavior.
Add coverage for zero, negative and minimum INT rates, multiple
serialization/merge stages, global winners outside each local top N, weighted
values, empty/NULL input, and state reuse. Update the existing aggregate-state
parameter tests: non-positive rates describe compatible unlimited states, while
unlimited and finite-capacity states are incompatible. The empty-state
regression cases now use NULL input.
### Release note
`topn`, `topn_array`, and `topn_weighted` now interpret a non-positive
`space_expand_rate` as unlimited intermediate candidate retention. The result
remains limited by `top_num`. Retaining all distinct candidates can increase
intermediate-state memory and network traffic.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [x] Manual test
- [ ] No need to test or manual test.
Validation before rebasing onto current master:
- FE and BE ASAN builds passed.
- 10 TOPN BE unit tests passed.
- Both `topn` and `topn_unlimited` regression suites passed; expected output
was generated by the regression runner.
- SQL checks returned `{"a":2}` and `["a"]` for non-positive rates in both
aggregation phases; EXPLAIN confirmed partial serialization followed by
merge/finalization.
- clang-format 16, build hygiene, and clang-tidy passed.
Validation on the PR branch:
- Updated the reset/reuse test and existing aggregate-state tests for
current master.
- clang-format 16, build hygiene, and clang-tidy passed.
- The ASAN BE unit test rebuild is in progress; final results will be added
here.
- The newly adjusted `test_agg_state_parameters` regression suite has not
been rerun locally.
- Behavior changed:
- [ ] No.
- [x] Yes. Non-positive expansion rates explicitly mean unlimited
candidate retention.
- Does this need documentation?
- [ ] No.
- [x] Yes. The SQL function docs should describe unlimited candidate
retention for non-positive expansion rates; a documentation PR is not included.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
