[GitHub] spark pull request #23244: [SPARK-26289][CORE]cleanup enablePerfMetrics para...

2018-12-06 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/23244


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #23244: [SPARK-26289][CORE]cleanup enablePerfMetrics para...

2018-12-06 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/23244#discussion_r239675382
  
--- Diff: 
core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java ---
@@ -209,23 +205,14 @@ public BytesToBytesMap(
   TaskMemoryManager taskMemoryManager,
--- End diff --

OK all tests, then we are fine


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #23244: [SPARK-26289][CORE]cleanup enablePerfMetrics para...

2018-12-06 Thread heary-cao 
Github user heary-cao commented on a diff in the pull request:

https://github.com/apache/spark/pull/23244#discussion_r239671696
  
--- Diff: 
core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java ---
@@ -209,23 +205,14 @@ public BytesToBytesMap(
   TaskMemoryManager taskMemoryManager,
--- End diff --

review the current master branch, this constructor is called by 
`HashedRelationSuite , AggregateBenchmark, UnsafeKVExternalSorterSuite, 
AbstractBytesToBytesMapSuite`. thanks.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #23244: [SPARK-26289][CORE]cleanup enablePerfMetrics para...

2018-12-06 Thread cloud-fan 
Github user cloud-fan commented on a diff in the pull request:

https://github.com/apache/spark/pull/23244#discussion_r239451274
  
--- Diff: 
core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java ---
@@ -209,23 +205,14 @@ public BytesToBytesMap(
   TaskMemoryManager taskMemoryManager,
--- End diff --

If this constructor is called, the `enablePerfMetrics` will be false. Where 
do we use this constructor?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #23244: [SPARK-26289][CORE]cleanup enablePerfMetrics para...

2018-12-06 Thread heary-cao 
GitHub user heary-cao opened a pull request:

https://github.com/apache/spark/pull/23244

[SPARK-26289][CORE]cleanup enablePerfMetrics parameter from BytesToBytesMap

## What changes were proposed in this pull request?

`enablePerfMetrics `was originally designed in `BytesToBytesMap `to control 
`getNumHashCollisions  getTimeSpentResizingNs  getAverageProbesPerLookup`.

However, as the Spark version gradual progress.  this parameter is only 
used for `getAverageProbesPerLookup ` and always given to true when using 
`BytesToBytesMap`.

 it is also dangerous to determine whether `getAverageProbesPerLookup 
`opens and throws an `IllegalStateException `exception. 
So this pr will be remove `enablePerfMetrics `parameter from 
`BytesToBytesMap`. thanks.

## How was this patch tested?

the existed test cases.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/heary-cao/spark enablePerfMetrics

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/23244.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #23244


commit 16d30eca343b401a4402953652d1e8e8e26fb0bb
Author: caoxuewen 
Date:   2018-12-06T07:54:25Z

cleanup enablePerfMetrics parameter from BytesToBytesMap




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org