paleolimbot opened a new issue, #18102:
URL: https://github.com/apache/datafusion/issues/18102
### Describe the bug
When trying to bind parameters and execute queries like `"SELECT a, b FROM
(VALUES ($1, $2)) AS t(a, b)"` and `"SELECT $1, $2"`, I get mismatched schemas
and/or an error when I try to execute the query.
I think this is because the SubqueryAlias, Values, and/or EmptyRelation both
cache their schema and this is not updated when updating parameter values.
(There may be other logical plan items that cache their schemas too)
### To Reproduce
For SubqueryAlias:
```rust
let df = ctx.sql("SELECT a, b FROM (VALUES ($1, $2)) AS t(a, b)").await?;
let df_with_params_replaced = df.with_param_values(vec![
ScalarValue::UInt32(Some(1)),
ScalarValue::Utf8(Some("foofy".to_string())),
])?;
dbg!(df_with_params_replaced.clone().schema().as_arrow());
dbg!(df_with_params_replaced.collect().await?[0].schema());
#> Error: ArrowError(InvalidArgumentError("column types must match schema
types, expected Null but found UInt32 at column index 0"), Some(""))
```
For EmptyRelation (mismatched schemas but execution works):
```rust
let df = ctx.sql("SELECT $1, $2").await?;
let df_with_params_replaced = df.with_param_values(vec![
ScalarValue::UInt32(Some(1)),
ScalarValue::Utf8(Some("foofy".to_string())),
])?;
dbg!(df_with_params_replaced.clone().schema().as_arrow());
dbg!(df_with_params_replaced.collect().await?[0].schema());
```
<details>
```
[datafusion/core/tests/sql/select.rs:333:5]
df_with_params_replaced.clone().schema().as_arrow() = Schema {
fields: [
Field {
name: "$1",
data_type: Null,
nullable: true,
dict_id: 0,
dict_is_ordered: false,
metadata: {},
},
Field {
name: "$2",
data_type: Null,
nullable: true,
dict_id: 0,
dict_is_ordered: false,
metadata: {},
},
],
metadata: {},
}
[datafusion/core/tests/sql/select.rs:334:5]
df_with_params_replaced.collect().await?[0].schema() = Schema {
fields: [
Field {
name: "$1",
data_type: UInt32,
nullable: false,
dict_id: 0,
dict_is_ordered: false,
metadata: {},
},
Field {
name: "$2",
data_type: Utf8,
nullable: false,
dict_id: 0,
dict_is_ordered: false,
metadata: {},
},
],
metadata: {},
}
```
</details>
### Expected behavior
I expected the schema and generated batches to match and the queries to
execute without error.
### Additional context
Encountered whist writing tests for
https://github.com/apache/datafusion/pull/17986
SubqueryAlias:
https://github.com/apache/datafusion/blob/7c3b0d0a68d89ba0ac079c7d9adaa3d52ece1c39/datafusion/expr/src/logical_plan/plan.rs#L2223-L2234
Values:
https://github.com/apache/datafusion/blob/7c3b0d0a68d89ba0ac079c7d9adaa3d52ece1c39/datafusion/expr/src/logical_plan/plan.rs#L2106-L2115
EmptyRelation:
https://github.com/apache/datafusion/blob/7c3b0d0a68d89ba0ac079c7d9adaa3d52ece1c39/datafusion/expr/src/logical_plan/plan.rs#L2049-L2058
Replacing Parameters:
https://github.com/apache/datafusion/blob/7c3b0d0a68d89ba0ac079c7d9adaa3d52ece1c39/datafusion/expr/src/logical_plan/plan.rs#L1448-L1478
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
