mattsu2020 opened a new pull request, #26769:
URL: https://github.com/apache/datafusion/pull/26769
## Which issue does this PR close?
- Closes #25449.
## Rationale for this change
A user alias such as `__datafusion_extracted_1` can currently be mistaken
for an optimizer-generated extraction. This lets an ordinary column name change
how optimizer rules treat the expression. Plans built through the public API or
restored from serialization have the same ambiguity.
## What changes are included in this PR?
- Add `AliasOrigin::{User, CommonSubexpr, LeafExtraction}`. Mark generated
definitions explicitly and use the origin to recognize leaf extractions.
- Preserve origins through expression traversal, coercion, name restoration,
and projection merging. Include the origin in alias equality, ordering, and
hashing.
- Add an optional protobuf origin field and regenerate the Rust/JSON models.
Missing origins default to `User`; names are never used to infer the origin.
Older readers can discard the new field.
- Keep displayed alias names and the existing collision-prevention scan.
Origins do not change name-based column resolution, so a fresh alias generator
still needs to advance past existing names, including aliases inside subqueries.
- Box alias field metadata to retain the existing `Expr` and `LogicalPlan`
sizes of 112 and 176 bytes.
- Update extraction fixtures to set their origins explicitly and update two
SQL plan expectations where user aliases are now handled as user aliases.
SQL cannot encode the marker. The unparser continues to emit ordinary
aliases, and reparsing gives them `User` origins. This PR tests leaf-extraction
SQL round trips; it retains the existing SQL reservation of `__common_expr`
names.
## What is the testing strategy for this PR?
Regression coverage includes:
- User aliases with the extraction prefix, and extraction markers with
arbitrary names.
- Origin preservation during expression rewrites and protobuf binary/JSON
round trips.
- Backward-compatible decoding of aliases without an origin.
- Re-optimizing a serialized optimized plan with a fresh context.
- Unparsing, replanning, and executing a leaf-extraction plan containing a
user alias named `__datafusion_extracted_1`.
Validation passed:
- `cargo fmt --all` and the final formatting check.
- `cargo clippy --all-targets --all-features -- -D warnings`.
- The workspace extended test command below: 12,990 Rust tests passed, 8
ignored, and all 542 SQL test files passed.
- A separate run of `projection_pushdown.slt`.
```bash
RUST_BACKTRACE=1 cargo test --profile ci \
--exclude datafusion-examples --exclude datafusion-benchmarks --exclude
datafusion-cli \
--workspace --lib --tests --bins \
--features
avro,json,backtrace,extended_tests,recursive_protection,parquet_encryption
```
Compared the five existing `optimize_projections` benchmark cases against
Apache main at `350ca76505c9513d16e7dd9910bcd9bde2c911ed`, using the `ci`
profile and sequential measurements without concurrent builds/tests. Point
estimates ranged from -5.2% to +1.0%. These are unoptimized-profile
measurements.
## Are there any user-facing changes?
User aliases beginning with `__datafusion_extracted` are no longer
identified as optimizer-generated solely from their names. Output column names
remain intact.
Public API changes:
- `Alias` gains an `origin` field and `with_origin()`; existing alias
constructors default to `User`.
- `Alias.metadata` changes from `Option<FieldMetadata>` to
`Option<Box<FieldMetadata>>`. The `with_metadata()` argument type remains
unchanged.
- `SavedName::Saved` gains an `origin` field.
The protobuf addition is optional, and SQL reserved-name validation is
unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
