adriangb opened a new issue, #25669:
URL: https://github.com/apache/datafusion/issues/25669
# Describe the bug
A `UNION ALL` whose arms select an untyped placeholder under an alias
(`SELECT $1 AS a UNION ALL SELECT $2 AS a`) cannot be analyzed or optimized
before the placeholders are bound. The error names the alias as a missing
column:
```
Schema error: No field named a.
```
This affects two public paths:
1. `PREPARE p AS SELECT $1 AS a UNION ALL SELECT $2 AS a` fails in the
optimizer rule `optimize_unions`.
2. `SessionState::optimize` on the plan from
`SessionState::create_logical_plan` (placeholders still unbound) fails in the
analyzer rule `type_coercion`. Optimizing first and binding with
`LogicalPlan::replace_params_with_values` afterwards is therefore impossible
for this shape.
Binding first (`DataFrame::with_param_values`, then execute) works. A single
arm without `UNION` works. The same `UNION` without the alias (`SELECT $1 UNION
ALL SELECT $2`) works. Declaring the placeholder type works: `CAST($1 AS
VARCHAR) AS a`, or `PREPARE p(VARCHAR, VARCHAR) AS ...`.
`EXPLAIN SELECT $1 AS a UNION ALL SELECT $2 AS a` shows the analyzer failure:
```
| logical_plan after type_coercion | Schema error: No field
named a. |
```
# To Reproduce
## datafusion-cli
```sql
PREPARE p AS SELECT $1 AS a UNION ALL SELECT $2 AS a;
```
Actual:
```
Optimizer rule 'optimize_unions' failed
caused by
Schema error: No field named a.
```
The same happens in a CTE that is joined to a table, which is the realistic
shape (a small list of key pairs supplied as parameters):
```sql
CREATE TABLE t (k1 VARCHAR, k2 VARCHAR) AS VALUES ('a0', 'b0'), ('a1',
'b1'), ('a2', 'b2');
PREPARE q AS
WITH keys AS (
SELECT $1 AS k1, $2 AS k2
UNION ALL
SELECT $3 AS k1, $4 AS k2
)
SELECT t.k1 FROM t JOIN keys ON t.k1 = keys.k1 AND t.k2 = keys.k2;
```
```
Optimizer rule 'optimize_unions' failed
caused by
Schema error: No field named k1.
```
## Rust (optimize, then bind)
```rust
use datafusion::common::{ParamValues, ScalarValue};
use datafusion::error::Result;
use datafusion::prelude::*;
#[tokio::main]
async fn main() -> Result<()> {
let ctx = SessionContext::new();
let state = ctx.state();
let plan = state
.create_logical_plan("SELECT $1 AS a UNION ALL SELECT $2 AS a")
.await?;
// Fails here:
// type_coercion
// caused by
// Schema error: No field named a.
let optimized = state.optimize(&plan)?;
let bound = optimized.replace_params_with_values(&ParamValues::List(vec![
ScalarValue::from("x").into(),
ScalarValue::from("y").into(),
]))?;
let _ = state.optimize(&bound)?;
Ok(())
}
```
`Debug` output of the error:
```
Context("type_coercion", SchemaError(FieldNotFound { field: Column {
relation: None, name: "a" }, valid_fields: [] }, Some("")))
```
Named placeholders (`SELECT $x AS a UNION ALL SELECT $y AS a`) fail the same
way.
# Expected behavior
The plan analyzes and optimizes with the placeholders unbound, as it does
for the same query without `UNION` and for the same query with `CAST($1 AS
VARCHAR)`. After binding, `SELECT $1 AS a UNION ALL SELECT $2 AS a` with `('x',
'y')` returns the two rows `x` and `y`.
# Additional context
The unoptimized plan is:
```
Union [a:Null;N]
Projection: $1 AS a [a:Null;N]
EmptyRelation: rows=1 []
Projection: $2 AS a [a:Null;N]
EmptyRelation: rows=1 []
```
Both failing rules (`TypeCoercionRewriter::coerce_union` in the analyzer and
`OptimizeUnions` in the optimizer) call `coerce_plan_expr_for_schema` on each
`UNION` arm. For a `Projection` arm, that function calls
`coerce_exprs_for_schema(expr, input.schema(), union_schema)`, which calls
`expr.get_type(input.schema())` on each projection expression.
`Expr::get_type` in `datafusion/expr/src/expr_schema.rs` has a special case
for an alias of an untyped placeholder:
```rust
Expr::Alias(Alias { expr, name, .. }) => match &**expr {
Expr::Placeholder(Placeholder { field, .. }) => match &field {
None => schema.data_type(&Column::from_name(name)).cloned(),
Some(field) => Ok(field.data_type().clone()),
},
_ => expr.get_type(schema),
},
```
When the placeholder has no type, it looks up the alias name (`a`) as a
column in the schema that it receives. Here that schema is the projection's
input (`EmptyRelation`, no fields), so the lookup fails. The special case
appears to assume that the schema is the projection's own output schema. It
came from #4701 (prepared statement parameter type inference).
`Expr::to_field` does not have this special case. For the same expression it
returns a `Null` field, so `Projection::try_new` builds the schema `a:Null`
without error. `get_type` and `to_field` therefore disagree for
`Alias(Placeholder { field: None })`.
Possible fixes:
- In the `Alias(Placeholder { field: None })` branch of `get_type`, return
`DataType::Null` (the same result as a bare untyped placeholder and as
`to_field`) when the alias name is not in the schema, or remove the branch.
- Or make `coerce_exprs_for_schema` resolve the current type of a projection
expression from the projection's output schema (the field at the same index),
not from the input schema.
Related, but different:
- #18102 and #18522: `PREPARE myplan AS SELECT $1 AS one, $2 AS two` failed
with "No field named one" in an older release. That case now works. The `UNION`
case still fails.
- #8819: optimizing a `LogicalPlan` that contains placeholders must not fail.
# Version
- `apache/datafusion` `main` at commit
`7570366fd929daf9ced744bb8397686b50565b18` (2026-09-23, workspace version
55.1.0), both with `datafusion-cli` built from that commit and with the Rust
program above
- Also reproduced with released `datafusion-cli` 54.0.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
