sdf-jkl opened a new issue, #12214:
URL: https://github.com/apache/arrow-rs/issues/12214
## Describe the bug
When `variant_get` cannot convert a value to a string or binary type, the
error names the Rust builder type instead of the Arrow data type. Other targets
name the data type, such as `Int32`.
This affects `Utf8`, `LargeUtf8`, `Utf8View`, `Binary`, `LargeBinary`, and
`BinaryView`. Reported downstream in datafusion-contrib/datafusion-variant#173,
where `SELECT parse_json('{"status":200}'):status::VARCHAR` shows this message
to SQL users.
## To Reproduce
Reproduced with `arrow`, `parquet-variant`, and `parquet-variant-compute`
`60.1.0`:
```rust
use std::sync::Arc;
use arrow::array::{ArrayRef, StringArray};
use arrow::compute::CastOptions;
use arrow::datatypes::{DataType, Field};
use parquet_variant::VariantPath;
use parquet_variant_compute::{GetOptions, json_to_variant, variant_get};
fn get_error(json: &str, data_type: DataType) -> String {
let input: ArrayRef = Arc::new(StringArray::from(vec![json]));
let variant = ArrayRef::from(json_to_variant(&input).unwrap());
let options =
GetOptions::new_with_path(VariantPath::try_from("status").unwrap())
.with_as_type(Some(Arc::new(Field::new("result", data_type, true))))
.with_cast_options(CastOptions {
safe: false,
..Default::default()
});
variant_get(&variant, options).unwrap_err().to_string()
}
fn main() {
println!("{}", get_error(r#"{"status":"n/a"}"#, DataType::Int32));
println!("{}", get_error(r#"{"status":200}"#, DataType::Utf8));
println!("{}", get_error(r#"{"status":200}"#, DataType::Utf8View));
println!("{}", get_error(r#"{"status":200}"#, DataType::Binary));
}
```
Output:
```text
Cast error: Failed to extract primitive of type Int32 from variant
ShortString(ShortString("n/a")) at path VariantPath([])
Cast error: Failed to extract primitive of type
arrow_array::builder::generic_bytes_builder::GenericByteBuilder<arrow_array::types::GenericStringType<i32>>
from variant Int16(200) at path VariantPath([])
Cast error: Failed to extract primitive of type
arrow_array::builder::generic_bytes_view_builder::GenericByteViewBuilder<arrow_array::types::StringViewType>
from variant Int16(200) at path VariantPath([])
Cast error: Failed to extract primitive of type
arrow_array::builder::generic_bytes_builder::GenericByteBuilder<arrow_array::types::GenericBinaryType<i32>>
from variant Int16(200) at path VariantPath([])
```
## Expected behavior
The error names the requested Arrow type, as it does for `Int32`. For
example, `Utf8`, `Utf8View`, and `Binary` for the last three lines above.
## Additional context
The
[string](https://github.com/apache/arrow-rs/blob/b2b7d4705191a200ac7e395596c0946786d6f69c/parquet-variant-compute/src/variant_to_arrow.rs#L1266-L1271)
and
[binary](https://github.com/apache/arrow-rs/blob/b2b7d4705191a200ac7e395596c0946786d6f69c/parquet-variant-compute/src/variant_to_arrow.rs#L1303-L1308)
arms of `define_variant_to_primitive_builder!` pass `B::type_name()` into the
message, while the [other
arms](https://github.com/apache/arrow-rs/blob/b2b7d4705191a200ac7e395596c0946786d6f69c/parquet-variant-compute/src/variant_to_arrow.rs#L1280-L1285)
pass an Arrow `DataType`.
[`StringLikeArrayBuilder::type_name`](https://github.com/apache/arrow-rs/blob/b2b7d4705191a200ac7e395596c0946786d6f69c/arrow-array/src/builder/generic_bytes_builder.rs#L400-L402)
and
[`BinaryLikeArrayBuilder::type_name`](https://github.com/apache/arrow-rs/blob/b2b7d4705191a200ac7e395596c0946786d6f69c/arrow-array/src/builder/generic_bytes_builder.rs#L444-L446)
are documented as human-readable, but every i
mplementation returns `std::any::type_name`. These two arms are their only
callers. The code is unchanged on `main` at `b2b7d47051`.
#10651 tracks the other parts of this message: the hardcoded
`VariantPath([])` and the `Debug` formatting of the value. Whether a numeric
variant should convert to a string at all is a separate question.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
