kosiew commented on PR #20840: URL: https://github.com/apache/datafusion/pull/20840#issuecomment-4133082383
@alamb > these tests are "unit test" level as they seem to be exercising the Rust API functions Is it possible to implement "end user API" tests -- specifically, .slt tests? It isn't clear to me why we can't create these cases using SQL (or DataFrame) The main reason I wrote them this way is fixture creation: the cases need multiple Parquet files with intentionally different nested physical schemas (List<Struct<...>> / LargeList<Struct<...>>, additive nullable fields, extra fields, and incompatible variants). SQL/DataFrame is a good fit for the read/query side, but it doesn’t naturally create those mismatched Parquet fixtures in a self-contained way. In Rust I can generate the files inline and keep the test focused on the exact evolution shapes without checking in binary fixtures. That said, I agree there is value in an end-user API test. I can look at splitting this into: - .slt coverage for the happy-path query behavior (SELECT * and projected nested fields) using checked-in fixtures or generated fixtures if we have a pattern for that - Rust tests retained for the fixture-heavy/error-path cases -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
