lidavidm commented on pull request #9656: URL: https://github.com/apache/arrow/pull/9656#issuecomment-812099547
Here's a flamegraph of this PR + ARROW-7001 + converting file_ipc.cc to use the generator: https://share.firefox.dev/3sHSJap (You can right-click the top pane and hide all of jemalloc's threads.) It seems the I/O threads mostly sit idle, while CPU threads spend most of their time doing work, but do spend a nontrivial amount of time on what looks like overhead from shared_ptrs and Futures. (And changing ReadableFile to not use a separate thread doesn't have any apparent impact on things either way for datasets.) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
