Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/23058#discussion_r237284719
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -718,13 +718,9 @@ private[spark] class BlockManager(
}
/**
- * Get block from remote block managers as serialized bytes.
+ * Get block from remote block managers as a ManagedBuffer.
*/
- def getRemoteBytes(blockId: BlockId): Option[ChunkedByteBuffer] = {
- // TODO SPARK-25905 if we change this method to return the
ManagedBuffer, then getRemoteValues
- // could just use the inputStream on the temp file, rather than
reading the file into memory.
- // Until then, replication can cause the process to use too much
memory and get killed
- // even though we've read the data to disk.
+ def getRemoteManagedBuffer(blockId: BlockId): Option[ManagedBuffer] = {
--- End diff --
Can this be private?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
