ahmed-mez commented on issue #18907: URL: https://github.com/apache/datafusion/issues/18907#issuecomment-3570720445
Thanks for the pointer @alamb ! We're already using separate thread pools for CPU and IO as described in the docs. The issue we're seeing is within the CPU thread pool itself. Is the recommendation to use multiple separate CPU thread pools for different query priorities/types? Because it is also something we're exploring (cc @geoffreyclaude). The idea behind chunked emission in https://github.com/apache/datafusion/pull/18906 is to make aggregate computations more scheduler-friendly. Do you expect potential problems with that approach? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
