[GitHub] [spark] shardulm94 commented on pull request #33446: [SPARK-36215][SHUFFLE] Add logging for slow fetches to diagnose external shuffle service issues

shardulm94 commented on pull request #33446: URL: https://github.com/apache/spark/pull/33446#issuecomment-1008103255 @mridulm I have rebased to master and also addressed comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] shardulm94 commented on pull request #33446: [SPARK-36215][SHUFFLE] Add logging for slow fetches to diagnose external shuffle service issues

shardulm94 commented on pull request #33446: URL: https://github.com/apache/spark/pull/33446#issuecomment-887304237 I am not completely sure how showing the 5%ile would work. 1) We will need a reasonable number of shuffles on an executor before we can even calculate 5%ile. We cannot dedu

[GitHub] [spark] shardulm94 commented on pull request #33446: [SPARK-36215][SHUFFLE] Add logging for slow fetches to diagnose external shuffle service issues

shardulm94 commented on pull request #33446: URL: https://github.com/apache/spark/pull/33446#issuecomment-884554606 > Just wondering if we should log this for the push-merged meta request as well? Code [here](https://github.com/apache/spark/blob/c0d84e6cf1046b7944796038414ef21fe9c7e3b5/cor

[GitHub] [spark] shardulm94 commented on pull request #33446: [SPARK-36215][SHUFFLE] Add logging for slow fetches to diagnose external shuffle service issues

shardulm94 commented on pull request #33446: URL: https://github.com/apache/spark/pull/33446#issuecomment-883573002 cc: @xkrogen Who initially worked on this change at LinkedIn Also @Ngone51 @mridulm @otterc @dongjoon-hyun who have recently worked on or reviewed patches to this file