avamingli opened a new pull request, #1261:
URL: https://github.com/apache/cloudberry/pull/1261
PostgreSQL's parallel processing cannot handle window functions. In
contrast, our distributed environment enables parallel execution of window
functions across multiple processes on multiple segments. For example:
```sql
sum(a) over(partition by b order by c)
```
The window function can be processed by redistributing data based on column
b to ensure all rows with the same b value are processed by the same worker,
significantly improving efficiency.
Even without PARTITION BY clauses, we can still enable parallelism by
allowing partial_path for window functions and subpaths, with parallel scanning
of underlying tables for data filtering.
Exclude CASE WHEN expressions in window functions (as they complicate
parallelization and make it difficult to guarantee correct data ordering)
Example non-parallel execution plan:
```sql
SELECT sum(salary) OVER w, rank() OVER w FROM empsalary WINDOW w AS
(PARTITION BY depname ORDER BY salary DESC);
QUERY PLAN
----------------------------------------------
Gather Motion 3:1 (slice1; segments: 3)
-> WindowAgg
Partition By: depname
Order By: salary
-> Sort
Sort Key: depname, salary DESC
-> Seq Scan on empsalary
```
```sql
Parallel execution plan (4-parallel):
SELECT sum(salary) OVER w, rank() OVER w FROM empsalary WINDOW w AS
(PARTITION BY depname ORDER BY salary DESC);
QUERY PLAN
---------------------------------------------------------------------
Gather Motion 12:1 (slice1; segments: 12)
-> WindowAgg
Partition By: depname
Order By: salary
-> Sort
Sort Key: depname, salary DESC
-> Redistribute Motion 12:12 (slice2; segments: 12)
Hash Key: depname
Hash Module: 3
-> Parallel Seq Scan on empsalary
```
For window function execution plans that can be parallelized, performance
typically scales in positive correlation with the degree of parallelism.
In complex queries containing window functions, parallel processing may
sometimes be inhibited due to cost considerations or other constraints.
However, our approach still provides valuable parallelization opportunities for
window function subpaths, delivering measurable query efficiency improvements.
We have observed significant performance gains in TPC-DS benchmarks through
this partial parallelization capability.
TPC-DS queries via parallel execution plans (50G AOCS, 4 workers):
| Query | Before(ms) | After(ms) | Saved(ms) | Gain | Plan Change |
|-------|-----------:|----------:|----------:|------:|-----------------|
| q12 | 10,439.08 | 4,613.52 | 5,825.56 | 55.8% | serial→parallel |
| q20 | 21,487.08 | 8,723.74 | 12,763.34 | 59.4% | serial→parallel |
| q44 | 33,816.75 | 22,515.03 | 11,301.72 | 33.4% | better parallel |
| q49 | 60,039.45 | 28,603.51 | 31,435.95 | 52.4% | serial→parallel |
| q98 | 40,114.21 | 17,052.78 | 23,061.43 | 57.5% | serial→parallel |
changes:
- Enabled parallel plans for q12/q20/q49/q98 (prev. serial)
- Optimized parallel plan for q44
- Avg gain: 52% (best: q20 59.4%, saved 12.7s)
Authored-by: Zhang Mingli [email protected]
<!-- Thank you for your contribution to Apache Cloudberry (Incubating)! -->
Fixes #ISSUE_Number
### What does this PR do?
<!-- Brief overview of the changes, including any major features or fixes -->
### Type of Change
- [ ] Bug fix (non-breaking change)
- [ ] New feature (non-breaking change)
- [ ] Breaking change (fix or feature with breaking changes)
- [ ] Documentation update
### Breaking Changes
<!-- Remove if not applicable. If yes, explain impact and migration path -->
### Test Plan
<!-- How did you test these changes? -->
- [ ] Unit tests added/updated
- [ ] Integration tests added/updated
- [ ] Passed `make installcheck`
- [ ] Passed `make -C src/test installcheck-cbdb-parallel`
### Impact
<!-- Remove sections that don't apply -->
**Performance:**
<!-- Any performance implications? -->
**User-facing changes:**
<!-- Any changes visible to users? -->
**Dependencies:**
<!-- New dependencies or version changes? -->
### Checklist
- [ ] Followed [contribution
guide](https://cloudberry.apache.org/contribute/code)
- [ ] Added/updated documentation
- [ ] Reviewed code for security implications
- [ ] Requested review from [cloudberry
committers](https://github.com/orgs/apache/teams/cloudberry-committers)
### Additional Context
<!-- Any other information that would help reviewers? Remove if none -->
### CI Skip Instructions
<!--
To skip CI builds, add the appropriate CI skip identifier to your PR title.
The identifier must:
- Be in square brackets []
- Include the word "ci" and either "skip" or "no"
- Only use for documentation-only changes or when absolutely necessary
-->
---
<!-- Join our community:
- Mailing list:
[[email protected]](https://lists.apache.org/[email protected])
(subscribe: [email protected])
- Discussions: https://github.com/apache/cloudberry/discussions -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
