adriangb opened a new issue, #27477:
URL: https://github.com/apache/datafusion/issues/27477
### Describe the bug
`Unparser::expr_to_sql` and `plan_to_sql` render a non-finite `Float16`,
`Float32`, or `Float64` literal as a bare SQL number token: `inf`, `-inf`, or
`NaN`. The SQL parser reads that token as an identifier. So the unparsed SQL
fails to plan with `Schema error: No field named inf.` (or `nan`, after
identifier normalization).
The cause is in `datafusion/sql/src/unparser/expr.rs`, in `scalar_to_sql`.
The `Float32` and `Float64` arms branch on `f.fract()`, which is `NaN` for a
non-finite value. So they take the `format!("{f}")` arm and wrap the result in
`ast::Value::Number`. The `Float16` arm uses `f.to_string()` and has the same
problem.
A query does not need a literal `inf` to hit this. Constant folding turns
`'inf'::double`, `'NaN'::double`, and `1e999` into non-finite `Float64`
literals before the unparse.
### To Reproduce
`Cargo.toml`:
```toml
[dependencies]
datafusion = { version = "=55.2.0", default-features = false, features =
["sql"] }
tokio = { version = "1", features = ["rt", "macros"] }
```
`src/main.rs`:
```rust
use datafusion::arrow::datatypes::{DataType, Field, Schema};
use datafusion::prelude::*;
use datafusion::datasource::empty::EmptyTable;
use datafusion::sql::unparser::{Unparser, dialect::PostgreSqlDialect};
use std::sync::Arc;
#[tokio::main(flavor = "current_thread")]
async fn main() {
for v in [f64::INFINITY, f64::NEG_INFINITY, f64::NAN] {
println!("f64 {v:?} -> {}",
Unparser::default().expr_to_sql(&lit(v)).unwrap());
}
for v in [f32::INFINITY, f32::NAN] {
println!("f32 {v:?} -> {}",
Unparser::default().expr_to_sql(&lit(v)).unwrap());
}
let ctx = SessionContext::new();
let schema = Arc::new(Schema::new(vec![Field::new("x",
DataType::Float64, true)]));
ctx.register_table("t", Arc::new(EmptyTable::new(schema))).unwrap();
for q in ["SELECT * FROM t WHERE x < 'inf'::double", "SELECT * FROM t
WHERE x <> 'NaN'::double", "SELECT * FROM t WHERE x < 1e999"] {
let plan = ctx.state().create_logical_plan(q).await.unwrap();
let plan = ctx.state().optimize(&plan).unwrap();
let sql = Unparser::new(&PostgreSqlDialect
{}).plan_to_sql(&plan).unwrap().to_string();
println!("\nin: {q}\nout: {sql}");
match ctx.sql(&sql).await { Ok(_) => println!("reparse: ok"), Err(e)
=> println!("reparse: {e}") }
}
}
```
Output with DataFusion 55.2.0:
```
warning: linker stderr: ld: __eh_frame section too large (max 16MB) to
encode dwarf unwind offsets in compact unwind table, performance of exception
handling might be affected
|
= note: `#[warn(linker_messages)]` on by default
f64 inf -> inf
f64 -inf -> -inf
f64 NaN -> NaN
f32 inf -> inf
f32 NaN -> NaN
in: SELECT * FROM t WHERE x < 'inf'::double
out: SELECT "t"."x" FROM "t" WHERE ("t"."x" < inf)
reparse: Schema error: No field named inf.
Valid fields are t.x.
in: SELECT * FROM t WHERE x <> 'NaN'::double
out: SELECT "t"."x" FROM "t" WHERE ("t"."x" <> NaN)
reparse: Schema error: No field named nan.
Valid fields are t.x.
in: SELECT * FROM t WHERE x < 1e999
out: SELECT "t"."x" FROM "t" WHERE ("t"."x" < inf)
reparse: Schema error: No field named inf.
Valid fields are t.x.
```
### Expected behavior
The unparser renders a non-finite float as a cast of a string literal, for
example `CAST('Infinity' AS DOUBLE)`. The SQL then plans again and keeps the
same value.
### Additional context
`main` has the same code. The `Dialect` trait has no hook for float
literals, so a custom dialect cannot work around this. I plan to open a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]