HyukjinKwon opened a new pull request, #230: URL: https://github.com/apache/spark-connect-rust/pull/230
### What changes were proposed in this pull request? This PR removes the type inference of `createDataFrame` across rows from `dev/parity/known-differences.md`. The entry was tracked by SPARK-60908, which made the inference match PySpark Connect 4.3.0. ### Why are the changes needed? The entry described the behavior before SPARK-60908. On a local Spark 4.3.0 server, `createDataFrame` now infers, and fails for, the cases that the entry listed as official PySpark 4.3.0 does: - other atomic types merge into a string type - nested values null in every row raise `CANNOT_DETERMINE_TYPE` - other conflicts raise `CANNOT_MERGE_TYPE` - a `Decimal` infers `decimal(38,18)` ### Does this PR introduce _any_ user-facing change? No. It changes a maintainer note. ### How was this patch tested? On a local Spark 4.3.0 server, 7 cases of the inference give the same schemas and errors as official PySpark 4.3.0. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
