zhztheplayer commented on issue #10377: URL: https://github.com/apache/incubator-gluten/issues/10377#issuecomment-3269967418
> For dv reader, we implemented something similar to iceberg, didn't touch the delta repository. It utilizes a class inherited SplitReader.cpp and extract dv info from the query plan in Gluten. And yes, it's not small change. I could imagine it's non-trivial change to Velox reader, so was inclined to keep the original Delta design where the reads of DVs and data are in separated layers. > It would be better as it would be hard to merge those changes to open source delta repo. I think it's possible not to modify Delta's code by just replacing the format object in the scan to Gluten's version. CH backend does seem to have a practice like that already (not 100% sure). See if we can combine the ideas together then come up with a solid solution on this. Perhaps, a separated DV layer + Gluten reader with format object replaced. We can rely on least code change in both Delta and Velox then. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
