zhoukang created SPARK-23129: -------------------------------- Summary: Lazy init DiskMapIterator#deserializeStream to reduce memory usage when ExternalAppendOnlyMap spill too much times Key: SPARK-23129 URL: https://issues.apache.org/jira/browse/SPARK-23129 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 2.1.0 Reporter: zhoukang
Currently,the deserializeStream in ExternalAppendOnlyMap#DiskMapIterator init when DiskMapIterator instance created.This will cause memory use overhead when ExternalAppendOnlyMap spill too much times. We can avoid this by making deserializeStream init when it is used the first time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org