malinjawi commented on PR #12131: URL: https://github.com/apache/gluten/pull/12131#issuecomment-4590856099
> Hi @malinjawi, would you help split the PR? > > I recommend to first raise a PR to add a utility to extract DV info from Delta scan to essential DV info, something like [this API](https://github.com/apache/gluten/pull/10740/changes#diff-79019208b21aca99fd6e93c9ab3554c6c54f82aad5111e100cb21886ba1b7c7eR29) in the PoC. I feel that part needs to be reviewed first. Thanks @zhztheplayer, that makes sense. I split the original PR into smaller pieces as suggested: - #12197 adds the focused Delta DV scan-info extraction utility first. - It extracts the essential per-file DV scan info from `PartitionedFile` metadata. - It covers Delta 3.3 and Delta 4.0. - It is utility-only: no Substrait changes, no native/C++ changes, and no scan offload behavior changes. - #12198 is the follow-up handoff PR stacked after #12197. - It consumes the utility from #12197. - It adds the JVM/Substrait/Velox scan handoff path. I’ll keep #12198 as the follow-up and would appreciate review on #12197 first so we can settle the utility/API shape before continuing with the full handoff. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
