zuston commented on issue #2569: URL: https://github.com/apache/uniffle/issues/2569#issuecomment-3142984036
Based on my observation of many Spark jobs, decompression operations contribute to the majority of the time spent in the reader phase. However, the current shuffle read fetchWaitTime metric only captures the time spent on data fetching and does not reflect the actual read duration, even though this behavior is consistent with Spark’s current design. Given this, I believe we should introduce clearer and more comprehensive metrics to capture the different stages of the reader phase, like shuffle write phase has did. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
