[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1125602608 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2468,4 +2468,15 @@ package object config { .version("3.4.0") .booleanConf

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1122750718 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,31 +1328,74 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1122750441 ## core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala: ## @@ -399,7 +426,14 @@ private[storage] class BlockInfoManager extends Logging { try {

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1118028518 ## core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala: ## @@ -2266,6 +2270,160 @@ class BlockManagerSuite extends SparkFunSuite with Matchers with

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1118028223 ## core/src/main/scala/org/apache/spark/storage/BlockManagerStorageEndpoint.scala: ## @@ -81,6 +81,8 @@ class BlockManagerStorageEndpoint( case

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1118028179 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -210,6 +220,65 @@ class BlockManagerMasterEndpoint( case

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1118028046 ## core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala: ## @@ -525,6 +562,7 @@ private[storage] class BlockInfoManager extends Logging {

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1118028028 ## core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala: ## @@ -150,6 +150,12 @@ private[storage] class BlockInfoManager extends Logging { */

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1118028001 ## core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala: ## @@ -180,6 +186,27 @@ private[storage] class BlockInfoManager extends Logging { //

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1118027972 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2468,4 +2468,15 @@ package object config { .version("3.4.0") .booleanConf

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1118013777 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -728,7 +800,22 @@ class BlockManagerMasterEndpoint( } if

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1118005755 ## core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala: ## @@ -399,7 +426,14 @@ private[storage] class BlockInfoManager extends Logging { try {

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1113188852 ## core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala: ## @@ -502,7 +536,10 @@ private[storage] class BlockInfoManager extends Logging { throw

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1113157159 ## core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala: ## @@ -399,7 +426,14 @@ private[storage] class BlockInfoManager extends Logging { try {

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1112123963 ## core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala: ## @@ -150,6 +150,12 @@ private[storage] class BlockInfoManager extends Logging { */

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1112123512 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -77,6 +78,11 @@ class BlockManagerMasterEndpoint( // Mapping from block id to

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1112123108 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -210,6 +219,65 @@ class BlockManagerMasterEndpoint( case

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1112122792 ## core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala: ## @@ -2266,6 +2270,150 @@ class BlockManagerSuite extends SparkFunSuite with Matchers with

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1112113795 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1424,6 +1470,28 @@ private[spark] class BlockManager( blockStoreUpdater.save() } +

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r227309 ## core/src/main/scala/org/apache/spark/internal/config/package.scala: ## @@ -2468,4 +2468,15 @@ package object config { .version("3.4.0") .booleanConf

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r226606 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,14 +1325,47 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r226580 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1424,6 +1457,16 @@ private[spark] class BlockManager( blockStoreUpdater.save() } +

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r226512 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -77,6 +77,11 @@ class BlockManagerMasterEndpoint( // Mapping from block id to

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r226349 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,31 +1328,71 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r226310 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -210,6 +219,51 @@ class BlockManagerMasterEndpoint( case

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1110773664 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,31 +1328,71 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1107261503 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1424,6 +1457,16 @@ private[spark] class BlockManager( blockStoreUpdater.save() } +

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1107228885 ## core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala: ## @@ -1787,6 +1792,12 @@ private[spark] class DAGScheduler( case _: ExceptionFailure | _:

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1107226569 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,31 +1328,71 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1107215072 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -77,6 +77,11 @@ class BlockManagerMasterEndpoint( // Mapping from block id to

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1103799711 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1424,6 +1457,16 @@ private[spark] class BlockManager( blockStoreUpdater.save() } +

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1103794439 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -77,6 +77,11 @@ class BlockManagerMasterEndpoint( // Mapping from block id to

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1100333545 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -77,6 +77,11 @@ class BlockManagerMasterEndpoint( // Mapping from block id to

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1100268004 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1424,6 +1457,16 @@ private[spark] class BlockManager( blockStoreUpdater.save() } +

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1096527082 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,14 +1325,47 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1096527017 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,14 +1325,47 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1096526943 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,14 +1325,47 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1096526714 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1424,6 +1457,16 @@ private[spark] class BlockManager( blockStoreUpdater.save() } +

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1096520993 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -77,6 +77,11 @@ class BlockManagerMasterEndpoint( // Mapping from block id to

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1090773361 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,14 +1325,47 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1090491731 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -77,6 +77,11 @@ class BlockManagerMasterEndpoint( // Mapping from block id to

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1090483687 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1424,6 +1457,16 @@ private[spark] class BlockManager( blockStoreUpdater.save() } +

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1090483687 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1424,6 +1457,16 @@ private[spark] class BlockManager( blockStoreUpdater.save() } +

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1090482420 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,14 +1325,47 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1090480637 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,14 +1325,47 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1090479561 ## core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala: ## @@ -180,6 +186,14 @@ private[storage] class BlockInfoManager extends Logging { //

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1090478629 ## core/src/test/scala/org/apache/spark/AccumulatorSuite.scala: ## @@ -89,6 +89,38 @@ class AccumulatorSuite extends SparkFunSuite with Matchers with LocalSparkContex

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1090477930 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,14 +1325,47 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1070632750 ## core/src/main/scala/org/apache/spark/storage/BlockManager.scala: ## @@ -1325,6 +1325,64 @@ private[spark] class BlockManager(

[GitHub] [spark] ivoson commented on a diff in pull request #39459: [SPARK-41497][CORE] Fixing accumulator undercount in the case of the retry task with rdd cache

ivoson commented on code in PR #39459: URL: https://github.com/apache/spark/pull/39459#discussion_r1070632692 ## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ## @@ -76,6 +76,8 @@ class BlockManagerMasterEndpoint( // Mapping from block id