[GitHub] [spark] thejdeep commented on a diff in pull request #36165: [SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

2022-11-28 Thread GitBox
thejdeep commented on code in PR #36165: URL: https://github.com/apache/spark/pull/36165#discussion_r1034008608 ## project/MimaExcludes.scala: ## @@ -87,6 +87,11 @@ object MimaExcludes { // [SPARK-36511][MINOR][SQL] Remove ColumnIOUtil

[GitHub] [spark] thejdeep commented on a diff in pull request #36165: [SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

2022-11-22 Thread GitBox
thejdeep commented on code in PR #36165: URL: https://github.com/apache/spark/pull/36165#discussion_r1029584122 ## core/src/main/scala/org/apache/spark/executor/ShuffleReadMetrics.scala: ## @@ -146,6 +268,16 @@ private[spark] class TempShuffleReadMetrics extends

[GitHub] [spark] thejdeep commented on a diff in pull request #36165: [SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

2022-11-01 Thread GitBox
thejdeep commented on code in PR #36165: URL: https://github.com/apache/spark/pull/36165#discussion_r1010567174 ## core/src/main/scala/org/apache/spark/executor/Executor.scala: ## @@ -654,6 +654,27 @@ private[spark] class Executor(

[GitHub] [spark] thejdeep commented on a diff in pull request #36165: [SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

2022-11-01 Thread GitBox
thejdeep commented on code in PR #36165: URL: https://github.com/apache/spark/pull/36165#discussion_r1010566832 ## core/src/main/scala/org/apache/spark/status/storeTypes.scala: ## @@ -138,6 +138,16 @@ private[spark] object TaskIndexNames { final val SHUFFLE_WRITE_RECORDS =

[GitHub] [spark] thejdeep commented on a diff in pull request #36165: [SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

2022-11-01 Thread GitBox
thejdeep commented on code in PR #36165: URL: https://github.com/apache/spark/pull/36165#discussion_r1010566209 ## core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala: ## @@ -2623,48 +2677,118 @@ private[spark] object JsonProtocolSuite extends Assertions {

[GitHub] [spark] thejdeep commented on a diff in pull request #36165: [SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

2022-10-28 Thread GitBox
thejdeep commented on code in PR #36165: URL: https://github.com/apache/spark/pull/36165#discussion_r1008470301 ## core/src/main/scala/org/apache/spark/status/storeTypes.scala: ## @@ -233,6 +243,38 @@ private[spark] class TaskDataWrapper( val shuffleLocalBytesRead: Long,