kosiew opened a new pull request, #27361:
URL: https://github.com/apache/datafusion/pull/27361
## Which issue does this PR close?
Related to #26580.
## Rationale for this change
Rust SQL planning currently supports lowercase folding or preservation of
unquoted identifiers, but not uppercase folding. Identifier binding, relation
discovery, and DDL naming also use different normalization paths, which can
produce inconsistent catalog keys.
This change allows Rust callers to select a consistent uppercase policy
without rewriting SQL or introducing a session configuration setting. It
preserves existing behavior for callers that do not explicitly select a policy.
## What changes are included in this PR?
- Introduce `IdentNormalization::{Lower, Upper, Preserve}` with ASCII-only
case folding and preservation of quoted identifiers.
- Add an optional policy to `ParserOptions`, including a fluent builder and
effective-policy accessor. Retain existing boolean APIs and compatibility
wrappers.
- Apply the selected policy consistently to SQL identifier binding, relation
discovery, CTEs, DDL names, aliases, lambda parameters, UPDATE targets,
structured fields, and partition columns.
- Add policy-aware table-reference conversion and relation-discovery APIs.
- Preserve existing function-registry lookup, scalar-variable, and
configuration-key behavior.
- Update `ParserOptions` construction sites and document the public API
migration in the 57.0.0 upgrade guide.
No session configuration setting is introduced.
## What is the testing strategy for this PR?
The patch adds the following unit and integration tests:
- `ident_normalizer_policy`: ASCII folding, quoted identifiers, empty names,
legacy compatibility, and policy precedence.
- `resolve_table_references_uppercase_policy`: Exact physical-table and CTE
discovery, quoted and multipart names, shadowing, and recursive references.
- `parse_ident_normalization_uppercase`: Table and column binding, aliases,
CTEs, qualified wildcards, builtin functions, scalar variables, and
configuration keys.
- `ident_normalization_partitions`: COPY and external-table partition
identifiers, including quoted forms.
- `ident_normalization_pipe_alias`: Alias normalization across pipe-query
stages.
- `ident_normalization_update`: Exact UPDATE target, projection, and
table-scan names.
- `ident_normalization_struct_fields`: Structured-field declarations and
references.
- `ident_normalization_expression_alias`: Direct expression-planning aliases.
- `create_schema_uppercase_policy`: Catalog, schema, table, and view naming,
including legacy behavior.
- `ident_normalization_scalar_variables`: Preservation of legacy
scalar-variable normalization.
- `lateral_table_function_lookup_preserves_legacy_name_normalization`:
Independent table-function registry lookup.
- `lambda_ident_normalization_uppercase`: Lambda binding, quoted
distinctions, duplicate detection, and legacy compatibility.
The tests use distinguishable `FOO` and `foo` schemas to verify actual
binding behavior.
Test execution results, code coverage, and planning benchmark results have
not been provided.
## Are there any user-facing changes?
Yes. Rust SQL API consumers can explicitly select uppercase identifier
folding through `ParserOptions::with_ident_normalization()` and use matching
policy-aware relation-discovery APIs.
Adding the public `ParserOptions.ident_normalization` field is a
**source-breaking API change** for callers using complete struct literals. Such
callers must add `ident_normalization: None` or use a default-based
construction. This is documented in the 57.0.0 upgrade guide and requires the
`api change` label.
Explicit `Preserve` also differs from the legacy disabled-normalization flag
for certain historically lowercase-only paths.
Folding remains ASCII-only. Quoted identifiers retain their spelling. The
existing limitation involving dots inside quoted schema-name components in
schema DDL remains unchanged.
## LLM-generated code disclosure
This PR includes LLM-generated code and comments. All LLM-generated content
has been manually reviewed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
