Jefffrey commented on code in PR #18104:
URL: https://github.com/apache/datafusion/pull/18104#discussion_r2442715208
##########
datafusion/functions-nested/src/set_ops.rs:
##########
@@ -290,10 +290,14 @@ impl ScalarUDFImpl for ArrayDistinct {
fn return_type(&self, arg_types: &[DataType]) -> Result<DataType> {
match &arg_types[0] {
- List(field) => Ok(DataType::new_list(field.data_type().clone(),
true)),
- LargeList(field) => {
- Ok(DataType::new_large_list(field.data_type().clone(), true))
- }
+ List(field) => Ok(DataType::new_list(
+ field.data_type().clone(),
+ field.is_nullable(),
+ )),
+ LargeList(field) => Ok(DataType::new_large_list(
+ field.data_type().clone(),
+ field.is_nullable(),
+ )),
Review Comment:
Actually would this just be as simple as:
```rust
fn return_type(&self, arg_types: &[DataType]) -> Result<DataType> {
Ok(arg_types[0].clone())
}
```
?
##########
datafusion/core/tests/dataframe/mod.rs:
##########
@@ -6479,3 +6479,87 @@ async fn
test_duplicate_state_fields_for_dfschema_construct() -> Result<()> {
Ok(())
}
+
+#[tokio::test]
+async fn array_distinct_on_list_with_inner_nullability_causing_type_mismatch(
Review Comment:
We could also place this test in `set_ops.rs` itself to get closer to the
function, for example `array_has`:
https://github.com/apache/datafusion/blob/b98cad616ad9c69df9a425fc7473b799ffc258ee/datafusion/functions-nested/src/array_has.rs#L753-L791
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
