[
https://issues.apache.org/jira/browse/SPARK-52418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie resolved SPARK-52418.
------------------------------
Fix Version/s: 4.1.0
Resolution: Fixed
Issue resolved by pull request 51116
[https://github.com/apache/spark/pull/51116]
> Performance gap in PercentileHeap when using Java 21 compared to using Java
> 17.
> -------------------------------------------------------------------------------
>
> Key: SPARK-52418
> URL: https://issues.apache.org/jira/browse/SPARK-52418
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 4.0.0, 4.1.0
> Reporter: Yang Jie
> Assignee: Yang Jie
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> refer to the test results of {{PercentileHeapBenchmark}}
>
> build/sbt clean "core/Test/runMain
> org.apache.spark.util.collection.PercentileHeapBenchmark"
>
> Java 17
>
>
> {code:java}
> ================================================================================================
> PercentileHeap Operations
> ================================================================================================
> OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.11.0-1015-azure
> AMD EPYC 7763 64-Core Processor
> PercentileHeap Operations - Input Size: 10000: Best Time(ms) Avg Time(ms)
> Stdev(ms) Rate(M/s) Per Row(ns) Relative
> -----------------------------------------------------------------------------------------------------------------------------
> Percentile: 0.5 1 1
> 0 10.0 100.4 1.0X
> Percentile: 0.9 1 1
> 0 18.0 55.6 1.8X
> Percentile: 0.95 0 0
> 0 23.2 43.2 2.3X
> Percentile: 0.99 0 0
> 0 31.1 32.1 3.1X
> OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.11.0-1015-azure
> AMD EPYC 7763 64-Core Processor
> PercentileHeap Operations - Input Size: 50000: Best Time(ms) Avg Time(ms)
> Stdev(ms) Rate(M/s) Per Row(ns) Relative
> -----------------------------------------------------------------------------------------------------------------------------
> Percentile: 0.5 5 6
> 0 9.2 108.2 1.0X
> Percentile: 0.9 3 3
> 0 15.7 63.8 1.7X
> Percentile: 0.95 2 2
> 0 20.4 49.0 2.2X
> Percentile: 0.99 2 2
> 0 29.4 34.0 3.2X
> OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.11.0-1015-azure
> AMD EPYC 7763 64-Core Processor
> PercentileHeap Operations - Input Size: 100000: Best Time(ms) Avg Time(ms)
> Stdev(ms) Rate(M/s) Per Row(ns) Relative
> ------------------------------------------------------------------------------------------------------------------------------
> Percentile: 0.5 12 13
> 0 8.2 122.5 1.0X
> Percentile: 0.9 6 7
> 0 15.6 64.3 1.9X
> Percentile: 0.95 5 5
> 0 20.9 47.7 2.6X
> Percentile: 0.99 4 4
> 0 27.9 35.8 3.4X
> OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.11.0-1015-azure
> AMD EPYC 7763 64-Core Processor
> PercentileHeap Operations - Input Size: 200000: Best Time(ms) Avg Time(ms)
> Stdev(ms) Rate(M/s) Per Row(ns) Relative
> ------------------------------------------------------------------------------------------------------------------------------
> Percentile: 0.5 26 26
> 1 7.7 129.7 1.0X
> Percentile: 0.9 13 14
> 1 14.9 67.3 1.9X
> Percentile: 0.95 10 10
> 0 19.6 51.0 2.5X
> Percentile: 0.99 7 7
> 0 28.4 35.2 3.7X
> {code}
>
>
> Java 21
>
>
> {code:java}
> ================================================================================================
> PercentileHeap Operations
> ================================================================================================
> OpenJDK 64-Bit Server VM 21.0.7+6-LTS on Linux 6.11.0-1015-azure
> AMD EPYC 7763 64-Core Processor
> PercentileHeap Operations - Input Size: 10000: Best Time(ms) Avg Time(ms)
> Stdev(ms) Rate(M/s) Per Row(ns) Relative
> -----------------------------------------------------------------------------------------------------------------------------
> Percentile: 0.5 59 59
> 0 0.2 5916.3 1.0X
> Percentile: 0.9 59 59
> 0 0.2 5871.2 1.0X
> Percentile: 0.95 59 59
> 0 0.2 5866.8 1.0X
> Percentile: 0.99 59 59
> 1 0.2 5861.6 1.0X
> OpenJDK 64-Bit Server VM 21.0.7+6-LTS on Linux 6.11.0-1015-azure
> AMD EPYC 7763 64-Core Processor
> PercentileHeap Operations - Input Size: 50000: Best Time(ms) Avg Time(ms)
> Stdev(ms) Rate(M/s) Per Row(ns) Relative
> -----------------------------------------------------------------------------------------------------------------------------
> Percentile: 0.5 1465 1466
> 1 0.0 29294.4 1.0X
> Percentile: 0.9 1459 1461
> 3 0.0 29170.8 1.0X
> Percentile: 0.95 1456 1458
> 2 0.0 29127.4 1.0X
> Percentile: 0.99 1455 1458
> 2 0.0 29106.2 1.0X
> OpenJDK 64-Bit Server VM 21.0.7+6-LTS on Linux 6.11.0-1015-azure
> AMD EPYC 7763 64-Core Processor
> PercentileHeap Operations - Input Size: 100000: Best Time(ms) Avg Time(ms)
> Stdev(ms) Rate(M/s) Per Row(ns) Relative
> ------------------------------------------------------------------------------------------------------------------------------
> Percentile: 0.5 5834 5836
> 2 0.0 58336.8 1.0X
> Percentile: 0.9 5830 5838
> 9 0.0 58295.2 1.0X
> Percentile: 0.95 5830 5832
> 2 0.0 58302.2 1.0X
> Percentile: 0.99 5819 5822
> 3 0.0 58190.4 1.0X
> OpenJDK 64-Bit Server VM 21.0.7+6-LTS on Linux 6.11.0-1015-azure
> AMD EPYC 7763 64-Core Processor
> PercentileHeap Operations - Input Size: 200000: Best Time(ms) Avg Time(ms)
> Stdev(ms) Rate(M/s) Per Row(ns) Relative
> ------------------------------------------------------------------------------------------------------------------------------
> Percentile: 0.5 23300 23326
> 31 0.0 116497.8 1.0X
> Percentile: 0.9 23303 23336
> 31 0.0 116515.2 1.0X
> Percentile: 0.95 23297 23328
> 35 0.0 116483.0 1.0X
> Percentile: 0.99 23276 23292
> 15 0.0 116381.6 1.0X {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
