bhollis-dbx opened a new pull request, #58873: URL: https://github.com/apache/spark/pull/58873
### What changes were proposed in this pull request? Use `AttributeMap` when restoring attributes from CTE references to CTE definitions in `PushdownPredicatesAndPruneColumnsForCTEDef`. This replaces a linear semantic search per attribute with expression-ID lookup. ### Why are the changes needed? The current implementation linearly searches the CTE attribute mapping for every restored attribute, making planning quadratic in CTE width. The cost reaches tens of milliseconds per rule application for wide CTEs and is repeated for CTE references and optimizer iterations. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Ran `PushdownPredicatesForCTEDefStalenessSuite`. All 7 tests passed. Local benchmark output for the full rule: | Columns | Before | After | Change | |---:|---:|---:|---:| | 1 | 2.38 us | 2.48 us | 4% slower | | 2 | 2.85 us | 2.92 us | 2% slower | | 5 | 5.02 us | 4.72 us | 6% faster | | 10 | 8.15 us | 7.21 us | 12% faster | | 20 | 14.78 us | 12.41 us | 16% faster | | 50 | 41.36 us | 28.96 us | 30% faster | | 100 | 100.11 us | 54.71 us | 45% faster | | 500 | 1.65 ms | 0.27 ms | 6.1x faster | | 1,000 | 6.53 ms | 0.58 ms | 11.2x faster | | 2,000 | 26.54 ms | 1.33 ms | 19.9x faster | ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
