[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-30 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532915261 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -471,6 +471,16 @@ package object config { "cache blo

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-30 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532889890 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -471,6 +471,16 @@ package object config { "cache blo

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-30 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532866941 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala ## @@ -627,7 +627,16 @@ private[spark] class BlockManager( override

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-30 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532866604 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala ## @@ -627,7 +627,16 @@ private[spark] class BlockManager( override

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-30 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532864965 ## File path: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala ## @@ -295,8 +295,9 @@ final class ShuffleBlockFetcher

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-30 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532863142 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala ## @@ -627,7 +627,16 @@ private[spark] class BlockManager( override

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-30 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532857177 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -471,6 +471,16 @@ package object config { "cache blo

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-30 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532856954 ## File path: core/pom.xml ## @@ -461,6 +461,47 @@ test + + org.apache.hadoop + hadoop-aws + ${hadoop.version} +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-29 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532255184 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -471,6 +471,15 @@ package object config { "cache blo

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-29 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532255169 ## File path: core/src/main/scala/org/apache/spark/storage/FallbackStorage.scala ## @@ -0,0 +1,171 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-28 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532122517 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala ## @@ -627,7 +627,16 @@ private[spark] class BlockManager( override

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-28 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532122517 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala ## @@ -627,7 +627,16 @@ private[spark] class BlockManager( override

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-28 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r532122517 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala ## @@ -627,7 +627,16 @@ private[spark] class BlockManager( override

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-26 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r531310069 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala ## @@ -627,7 +627,16 @@ private[spark] class BlockManager( override

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-26 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r531306631 ## File path: core/src/main/scala/org/apache/spark/storage/FallbackStorage.scala ## @@ -0,0 +1,171 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-26 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r531305162 ## File path: core/src/main/scala/org/apache/spark/storage/FallbackStorage.scala ## @@ -0,0 +1,171 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-26 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r531305162 ## File path: core/src/main/scala/org/apache/spark/storage/FallbackStorage.scala ## @@ -0,0 +1,171 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-26 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r531303923 ## File path: core/src/main/scala/org/apache/spark/storage/FallbackStorage.scala ## @@ -0,0 +1,171 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-26 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r531303923 ## File path: core/src/main/scala/org/apache/spark/storage/FallbackStorage.scala ## @@ -0,0 +1,171 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-26 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r531303180 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManagerDecommissioner.scala ## @@ -114,6 +115,8 @@ private[storage] class BlockMana

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-24 Thread GitBox
dongjoon-hyun commented on a change in pull request #30492: URL: https://github.com/apache/spark/pull/30492#discussion_r530065239 ## File path: core/src/main/scala/org/apache/spark/storage/FallbackStorage.scala ## @@ -0,0 +1,170 @@ +/* + * Licensed to the Apache Software Founda