zhoukang created SPARK-23129:
--------------------------------
Summary: Lazy init DiskMapIterator#deserializeStream to reduce
memory usage when ExternalAppendOnlyMap spill too much times
Key: SPARK-23129
URL: https://issues.apache.org/jira/browse/SPARK-23129
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.1.0
Reporter: zhoukang
Currently,the deserializeStream in ExternalAppendOnlyMap#DiskMapIterator init
when DiskMapIterator instance created.This will cause memory use overhead when
ExternalAppendOnlyMap spill too much times.
We can avoid this by making deserializeStream init when it is used the first
time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
