malinjawi commented on PR #12024: URL: https://github.com/apache/gluten/pull/12024#issuecomment-4429309999
Thanks for catching this. Yes, the current main docs are too broad/misleading if read as native Velox support. The current merged state is: - PR #12024 adds native offload only for plain Delta `OPTIMIZE` bin-packing compaction. - Liquid/clustered-table `OPTIMIZE` is explicitly excluded in `OffloadDeltaCommand.shouldOffloadOptimize` through `!isClusteredOptimize(optimize)`, and the code comment says liquid clustering continues through Delta original command path. - `ClusteredTableClusteringSuite` gives correctness coverage for clustered-table OPTIMIZE, but that is fallback behavior, not native liquid clustering offload. So I would describe Liquid as: ordinary Delta scans/writes on those tables can still follow the normal Delta offload rules when the final plan validates, but the Liquid-specific clustering/OPTIMIZE operation itself falls back to Delta/Spark today. It should not be documented as a blanket `Yes`. I updated the draft docs PR here to make that explicit: #12050. It now marks Liquid clustering as `Fallback` and separately marks plain OPTIMIZE compaction as `ExperimentalOffload`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
