[
https://issues.apache.org/jira/browse/SPARK-19839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897975#comment-15897975
]
Zhan Zhang commented on SPARK-19839:
------------------------------------
When BytesToBytesMap spills, its longArray should be released. Otherwise, it
may not released until the task complete. This array may take a significant
amount of memory, which cannot be used by later operator, such as
UnsafeShuffleExternalSorter, resulting in more frequent spill in sorter. This
patch release the array as destructive iterator will not use this array anymore.
> Fix memory leak in BytesToBytesMap
> ----------------------------------
>
> Key: SPARK-19839
> URL: https://issues.apache.org/jira/browse/SPARK-19839
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.1.0
> Reporter: Zhan Zhang
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
