andygrove commented on PR #6125: URL: https://github.com/apache/datafusion-comet/pull/6125#issuecomment-7017205669
Thank you @comphead, @viirya, @sunchao, @parthchandra, @rich7420 and @dwsmith1983 for the thorough reviews. They found real problems: the swallowed non-refusal errors, a consumer registered per reader, a decode estimate that ignores the file, and blocking in the planning step. I'm closing this PR rather than iterating on it. Since #6128, #6247 and #6261 landed, the design has to be reworked around them, and that is more than a rebase. I've written up the approach, the measurements and every review finding in #7688, along with a suggested plan, so the next attempt starts from there. The prototype branch is `parquet-scan-memory-accounting` on my fork. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
