jin xing created SPARK-21236:
--------------------------------
Summary: Make the threshold of using HighlyCompressedStatus
configurable.
Key: SPARK-21236
URL: https://issues.apache.org/jira/browse/SPARK-21236
Project: Spark
Issue Type: Bug
Components: Shuffle
Affects Versions: 2.1.1
Reporter: jin xing
Priority: Minor
Currently the threshold of using {{HighlyCompressedMapStatus}} is hardcoded
2000.
We could make this configurable. Thus users having enough memory on driver can
configure the threshold to be larger thus to save the size of blocks more
accurately in {{CompressedMapStatus}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
