tustvold commented on issue #5882: URL: https://github.com/apache/arrow-rs/issues/5882#issuecomment-2288643623
You should avoid doing _any_ non-trivial CPU-bound work on the tokio threadpool that you use for IO. The way I've seen this done successfully is running DF in one tokio threadpool, and then spawning IO from it into a different one. There was some work in the past to make this easier, see https://github.com/apache/arrow-rs/pull/4040, but I never got it over the line. I'll file a ticket -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
