Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21451#discussion_r198652991
--- Diff:
core/src/main/scala/org/apache/spark/network/BlockDataManager.scala ---
@@ -43,6 +44,17 @@ trait BlockDataManager {
level:
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21451#discussion_r198657121
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -1341,12 +1390,16 @@ private[spark] class BlockManager(
try {
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21451#discussion_r198652931
--- Diff:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/protocol/UploadBlockStream.java
---
@@ -0,0 +1,89 @@
+/*
+ *
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21451#discussion_r198656665
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -659,6 +701,11 @@ private[spark] class BlockManager(
* Get block
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21451#discussion_r198653427
--- Diff:
core/src/main/scala/org/apache/spark/network/netty/NettyBlockRpcServer.scala ---
@@ -73,10 +73,34 @@ class NettyBlockRpcServer(
}
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21451#discussion_r198656910
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -723,7 +770,9 @@ private[spark] class BlockManager(
}
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/21451#discussion_r192796600
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/server/RpcHandler.java
---
@@ -38,15 +38,24 @@
*
* This method will
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/21451#discussion_r192279111
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/server/RpcHandler.java
---
@@ -38,15 +38,24 @@
*
* This method will
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/21451#discussion_r192136490
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/server/RpcHandler.java
---
@@ -38,15 +38,24 @@
*
* This method will
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/21451#discussion_r191628993
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/server/RpcHandler.java
---
@@ -38,15 +38,24 @@
*
* This method will
GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/21451
[SPARK-24296][CORE][WIP] Replicate large blocks as a stream.
When replicating large cached RDD blocks, it can be helpful to replicate
them as a stream, to avoid using large amounts of memory
11 matches
Mail list logo
