imback82 commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-657297080
Thanks @c21! > Re POC - I feel overall approach looks good to me. But IMO I think we should do the coalesce/divide in physical plan rule, but not logical plan rule. Yes, I agree. (The POC was done before this PR, which also started with logical plan but changed to physical plan rule per suggestion) > Also I think we probably can leave vectorization code path aside now, as it introduces too much change to handle vectorization when doing filter for it. Without supporting `ColumnarBatch`, you cannot enable the wholestage codegen. Is that desirable? > Let me know if you are still on it, or I can help with this feature as well. Thanks. I will clean up the POC and create a PR with some benchmark numbers. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
