namanjain24-sudo commented on PR #25090: URL: https://github.com/apache/datafusion/pull/25090#issuecomment-5774683321
Thanks for flagging the bigger picture, and I agree the systems-level interop test is what we actually want long term — reasoning from the spec text and even a hand-built substrait-java repro (like the one above) only proves what I happened to think to check. For this specific PR: the substrait-java repro shows it's a real, currently-reachable failure for anyone reading DataFusion-produced Substrait with that engine, and the fix is the same pattern already established for scalar functions in #15831/#20597, so I'd still like to land it as an incremental correctness fix rather than hold it for the broader test. On the automated interop testing itself — I hadn't found an existing round-trip harness against substrait-java or another consumer when I went looking for #25100/#25190/#25090's repros; if there isn't one, would it be useful for me to look into building one (e.g., something that writes a DataFusion-produced plan, feeds it to substrait-java or another external consumer, and checks it reads back / matches the expected explain plan)? Given I've already been building these repros by hand for a few PRs, I have a reasonable head start on knowing what would need covering. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
