ziting-openai commented on code in PR #5262:
URL: https://github.com/apache/datafusion-comet/pull/5262#discussion_r3930137552
##########
native/Cargo.toml:
##########
@@ -35,18 +35,18 @@ license = "Apache-2.0"
edition = "2021"
# Comet uses the same minimum Rust version as DataFusion
-rust-version = "1.88"
+rust-version = "1.94.0"
[workspace.dependencies]
-arrow = { version = "58.4.0", features = ["prettyprint", "ffi", "chrono-tz"] }
-arrow-select = { version = "58.4.0" }
+arrow = { version = "59.2.0", features = ["prettyprint", "ffi", "chrono-tz"] }
+arrow-select = { version = "59.2.0" }
async-trait = { version = "0.1" }
bytes = { version = "1.11.1" }
-parquet = { version = "58.4.0", default-features = false, features =
["experimental"] }
-datafusion = { version = "54.1.0", default-features = false, features =
["unicode_expressions", "crypto_expressions", "nested_expressions", "parquet"] }
-datafusion-datasource = { version = "54.1.0" }
-datafusion-physical-expr-adapter = { version = "54.1.0" }
-datafusion-spark = { version = "54.1.0", features = ["core"] }
+parquet = { version = "59.2.0", default-features = false, features =
["experimental"] }
+datafusion = { version = "55.0.0", default-features = false, features =
["unicode_expressions", "crypto_expressions", "nested_expressions", "parquet"] }
Review Comment:
The compilation fixes now pass native Clippy and `cargo check --benches` at
`598f5e979bdaf82b929596dfcae6ec116bae265e`, but one DF55 test migration
remains: [the same Rust CI
job](https://github.com/apache/datafusion-comet/actions/runs/33824701130/job/100875289278)
fails `test_collect_agg_absorbs_nested_nullability_drift` at `planner.rs:6962`
because declared and produced types now agree for `collect_set`.
DF55's `ScalarValue::new_list` casts non-empty values to the requested
`data_type`; DF54.1 returned the runtime type. This is the upstream fix
explicitly anticipated by the test's comment, so the negative "expected drift"
assertion is obsolete, not evidence of a production correctness regression.
Please adapt that sentinel to the corrected behavior while retaining positive
state-schema coverage for both collect aggregates. This is remaining work on
the existing DataFusion migration P2; the old compiler errors are fixed.
Keep the `collect_list` coercion: that accumulator still concatenates
runtime arrays without the declared-type cast. Only the `collect_set` negative
sentinel is obsolete; do not invert the assertion for both functions or remove
the shared protection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
