----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66190/ -----------------------------------------------------------
Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, and Stephan Erb. Repository: aurora Description ------- This benchmark was using the deprecated `production` flag when building the tasks for the cluster. `PendingTaskProcessor` depends on `tier` instead, so this benchmark ended up not testing the correct codepath. Removed references to `production` and added `tier` instead. Additionally, removed some unused options. Diffs ----- src/jmh/java/org/apache/aurora/benchmark/BenchmarkSettings.java ddab2eccb22a93ecb67481f399707d2d82df5db2 src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 1f9a5764b502f939f0345ff99fb0fc2830b4c2f0 src/jmh/java/org/apache/aurora/benchmark/Tasks.java 60c62bbf3061650a5dd8654045dc8189293d0190 Diff: https://reviews.apache.org/r/66190/diff/1/ Testing ------- Old: ``` # Run complete. Total time: 00:08:32 Benchmark (numPendingTasks) Mode Cnt Score Error Units SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 10 57.670 ± 20.451 ops/s SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate 1 thrpt 10 595.374 ± 211.805 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate.norm 1 thrpt 10 10830342.916 ± 380.919 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space 1 thrpt 10 593.530 ± 222.002 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space.norm 1 thrpt 10 10717947.102 ± 1280229.296 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space 1 thrpt 10 0.305 ± 1.264 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space.norm 1 thrpt 10 13552.434 ± 61403.918 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.count 1 thrpt 10 60.000 counts SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.time 1 thrpt 10 202.000 ms SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·stack 1 thrpt NaN --- SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 10 52.161 ± 8.526 ops/s SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate 10 thrpt 10 550.771 ± 89.939 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate.norm 10 thrpt 10 11074211.352 ± 318.376 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space 10 thrpt 10 550.125 ± 107.470 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space.norm 10 thrpt 10 11073792.311 ± 1621636.993 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space 10 thrpt 10 0.038 ± 0.049 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space.norm 10 thrpt 10 737.753 ± 919.460 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.count 10 thrpt 10 55.000 counts SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.time 10 thrpt 10 155.000 ms SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·stack 10 thrpt NaN --- SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 10 55.574 ± 7.414 ops/s SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate 100 thrpt 10 588.906 ± 78.568 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate.norm 100 thrpt 10 11113945.746 ± 360.266 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space 100 thrpt 10 581.912 ± 91.360 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space.norm 100 thrpt 10 10980432.961 ± 933636.090 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space 100 thrpt 10 0.037 ± 0.025 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space.norm 100 thrpt 10 710.276 ± 502.196 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.count 100 thrpt 10 59.000 counts SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.time 100 thrpt 10 155.000 ms SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·stack 100 thrpt NaN --- SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 10 53.541 ± 8.977 ops/s SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate 1000 thrpt 10 587.639 ± 98.529 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate.norm 1000 thrpt 10 11509965.960 ± 256.384 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space 1000 thrpt 10 590.747 ± 102.626 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space.norm 1000 thrpt 10 11581787.397 ± 919456.716 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space 1000 thrpt 10 0.060 ± 0.072 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space.norm 1000 thrpt 10 1206.676 ± 1392.806 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.count 1000 thrpt 10 59.000 counts SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.time 1000 thrpt 10 161.000 ms SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·stack 1000 thrpt NaN --- ``` New: ``` # Run complete. Total time: 00:08:32 Benchmark (numPendingTasks) Mode Cnt Score Error Units SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 10 9.742 ± 1.220 ops/s SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate 1 thrpt 10 1134.480 ± 142.355 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate.norm 1 thrpt 10 122145964.287 ± 162.619 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space 1 thrpt 10 1137.705 ± 130.036 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space.norm 1 thrpt 10 122666774.722 ± 7826221.674 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space 1 thrpt 10 0.038 ± 0.058 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space.norm 1 thrpt 10 4437.495 ± 7332.908 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.count 1 thrpt 10 113.000 counts SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.time 1 thrpt 10 278.000 ms SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·stack 1 thrpt NaN --- SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 10 9.004 ± 1.477 ops/s SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate 10 thrpt 10 1048.718 ± 172.117 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate.norm 10 thrpt 10 122149834.627 ± 2.709 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space 10 thrpt 10 1057.001 ± 189.010 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space.norm 10 thrpt 10 123140186.200 ± 9676263.385 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space 10 thrpt 10 0.016 ± 0.009 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space.norm 10 thrpt 10 1798.961 ± 960.508 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.count 10 thrpt 10 105.000 counts SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.time 10 thrpt 10 298.000 ms SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·stack 10 thrpt NaN --- SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 10 9.674 ± 1.712 ops/s SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate 100 thrpt 10 1127.256 ± 199.510 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate.norm 100 thrpt 10 122190185.861 ± 694.357 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space 100 thrpt 10 1130.353 ± 202.566 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space.norm 100 thrpt 10 122611030.168 ± 7904770.698 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space 100 thrpt 10 0.285 ± 1.196 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space.norm 100 thrpt 10 32082.849 ± 135251.004 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.count 100 thrpt 10 114.000 counts SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.time 100 thrpt 10 471.000 ms SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·stack 100 thrpt NaN --- SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 10 9.391 ± 1.679 ops/s SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate 1000 thrpt 10 1097.729 ± 196.175 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.alloc.rate.norm 1000 thrpt 10 122587540.024 ± 6076.493 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space 1000 thrpt 10 1098.710 ± 200.125 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Eden_Space.norm 1000 thrpt 10 122749450.285 ± 8291123.010 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space 1000 thrpt 10 0.276 ± 1.249 MB/sec SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.churn.PS_Survivor_Space.norm 1000 thrpt 10 43334.762 ± 199709.049 B/op SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.count 1000 thrpt 10 111.000 counts SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·gc.time 1000 thrpt 10 519.000 ms SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark:·stack 1000 thrpt NaN --- ``` Benchmark is slower as it is doing more (correct) work. Thanks, Jordan Ly
