[ https://issues.apache.org/jira/browse/SPARK-14186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Tianyi updated SPARK-14186: ------------------------------ Component/s: (was: Spark Core) Block Manager > Size awareness during spill and shuffle > --------------------------------------- > > Key: SPARK-14186 > URL: https://issues.apache.org/jira/browse/SPARK-14186 > Project: Spark > Issue Type: Improvement > Components: Block Manager, Shuffle > Affects Versions: 1.6.0 > Reporter: He Tianyi > Priority: Minor > > Currently during spilling of a collection of record, sorter calls > {{createTempShuffleBlock}} for allocating a local block. > This call provides no size information about required block. Without > knowledge of estimated size, this makes customized allocation strategy hard > to implement . > On the other hand, theoretically both {{PartitionedPairBuffer}} and > {{PartitionedAppendOnlyMap}} are able to track its estimated size. > In this case, it is convenient to extract size information and pass it as > parameter to {{createTempShuffleBlock}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org