Repository: spark Updated Branches: refs/heads/branch-1.6 fe71cabd4 -> 897599601
[SPARK-13482][MINOR][CONFIGURATION] Make consistency of the configuraiton named in TransportConf. `spark.storage.memoryMapThreshold` has two kind of the value, one is 2*1024*1024 as integer and the other one is '2m' as string. "2m" is recommanded in document but it will go wrong if the code goes into `TransportConf#memoryMapBytes`. [Jira](https://issues.apache.org/jira/browse/SPARK-13482) Author: huangzhaowei <carlmartin...@gmail.com> Closes #11360 from SaintBacchus/SPARK-13482. (cherry picked from commit 264533b553be806b6c45457201952e83c028ec78) Signed-off-by: Reynold Xin <r...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/89759960 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/89759960 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/89759960 Branch: refs/heads/branch-1.6 Commit: 897599601a5ca0f95fd70f16e89df58b9b17705c Parents: fe71cab Author: huangzhaowei <carlmartin...@gmail.com> Authored: Wed Feb 24 23:52:17 2016 -0800 Committer: Reynold Xin <r...@databricks.com> Committed: Wed Feb 24 23:52:23 2016 -0800 ---------------------------------------------------------------------- .../main/java/org/apache/spark/network/util/TransportConf.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/89759960/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java ---------------------------------------------------------------------- diff --git a/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java b/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java index 115135d..9f030da 100644 --- a/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java +++ b/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java @@ -132,7 +132,8 @@ public class TransportConf { * memory mapping has high overhead for blocks close to or below the page size of the OS. */ public int memoryMapBytes() { - return conf.getInt("spark.storage.memoryMapThreshold", 2 * 1024 * 1024); + return Ints.checkedCast(JavaUtils.byteStringAsBytes( + conf.get("spark.storage.memoryMapThreshold", "2m"))); } /** --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
