dwsmith1983 opened a new pull request, #25655: URL: https://github.com/apache/datafusion/pull/25655
## Which issue does this PR close? - Related to #15775 (speeding up the logical optimizer). ## Rationale for this change Every time a `Projection` is built, DataFusion resolves each projection expression back to an input column so it can project the input's functional dependencies. That resolution formats every expression to a string, allocates every input field name, and scans the names linearly per expression. When the input has no functional dependencies, which is the default for every table provider, the result is thrown away. On wide schemas this dominates planning: for `SELECT * FROM t1000` it is about a million string comparisons per projection rebuild, and projections are rebuilt several times per query. ## What changes are included in this PR? - `calc_func_dependencies_for_project` returns an empty set immediately when the input schema has no functional dependencies, and otherwise resolves names through a hash map built once (first index wins, matching the previous linear scan). - `aggregate_functional_dependencies` skips its per-dependence loop when the input carries none. The block that reports the GROUP BY output as a `Single` dependency still runs, so grouped plans keep that dependency. - `add_group_by_exprs_from_dependencies` returns the group expressions untouched when the schema has no dependencies, without formatting their names. Planning benchmarks (`sql_planner`), interleaved against the base commit, noise floor about 2%: | benchmark | change | |---|---| | logical_select_all_from_1000 | -25% | | physical_select_all_from_1000 | -12% | | physical_plan_clickbench_all | -7% | | logical_wide_aggregate_100_exprs | -3% | | TPC-H planning | unchanged | ## What is the testing strategy for this PR? Behavior is pinned by new unit tests in `datafusion/expr`: `projection_with_alias_preserves_pk`, `projection_over_unconstrained_table_has_no_dependencies`, `projection_duplicate_flattened_name_uses_first_input_index`, `aggregate_group_by_on_primary_key_reports_single_dependency`, `aggregate_group_by_without_constraints_still_reports_single_dependency`, and `plan_builder_aggregate_with_implicit_group_by_exprs_no_constraints`. The `functional_dependencies` and `group_by` sqllogictest files pass unchanged. ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
