-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55357/#review160974
-----------------------------------------------------------
Master (d4ebb56) is red with this patch.
./build-support/jenkins/build.sh
at
org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
at
org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:72)
at
org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
at
org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
at
org.gradle.launcher.daemon.server.exec.HintGCAfterBuild.execute(HintGCAfterBuild.java:44)
at
org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
at
org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:50)
at
org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:293)
at
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
at
org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
:pmdTest
:testI0109 21:45:25.788 [ShutdownHook, SchedulerMain] Stopping scheduler
services.
org.apache.aurora.scheduler.preemptor.PendingTaskProcessorTest >
testMultipleTaskGroups FAILED
java.lang.AssertionError at PendingTaskProcessorTest.java:212
1070 tests completed, 1 failed, 2 skipped
:test FAILED
:jacocoTestReport
Coverage report generated:
file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/jacoco/test/html/index.html
:analyzeReport
Instruction coverage is 0.8884190293128303, but must be greater than 0.89
Branch coverage is 0.8018160284247927, but must be greater than 0.835
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at:
> file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/tests/test/index.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output.
BUILD FAILED
Total time: 4 mins 6.749 secs
I will refresh this build result if you post a review containing "@ReviewBot
retry"
- Aurora ReviewBot
On Jan. 9, 2017, 9:40 p.m., Mehrdad Nurolahzade wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55357/
> -----------------------------------------------------------
>
> (Updated Jan. 9, 2017, 9:40 p.m.)
>
>
> Review request for Aurora, David McLaughlin, Joshua Cohen, Stephan Erb, and
> Zameer Manji.
>
>
> Bugs: AURORA-1867
> https://issues.apache.org/jira/browse/AURORA-1867
>
>
> Repository: aurora
>
>
> Description
> -------
>
> To be fair, PendingTaskProcessor interleaves tasks from different groups.
> However, this fairness comes at the price of increasing reservation time.
> Even if reservations are being made for the same task group, the processor
> would still restart iterating through slaves for each task instance. This
> results in reevaluating all slaves already rejected in a previous search
> before it finds a new viable candidate.
>
> This patch improves `PendingTaskProcessor` performance by reducing slave
> search/evaluation time, at the cost of reduced fairness.
> `PendingTaskProcessor` now does reservation for a configurable maximum of _N_
> candidates per task group in each iteration over the list of slaves.
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java
> fa37236e68657b539b182519b9d46d96d5b0953a
>
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessor.java
> f59f3fd8959b1ba3726b55a2943fb9228a049ac5
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorMetrics.java
> 67822cafbe89f4798b4ea6da3856663cc4872798
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java
> 23d1c120657d5cb9d294a80c63e8a04512d361ca
>
> src/test/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorTest.java
> d11ae5883f2a00dca4c4b36f0ab58ea95c7ecb2e
>
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java
> 67b6d69e3ddd1028dfe9ff451b171cd888674920
>
> Diff: https://reviews.apache.org/r/55357/diff/
>
>
> Testing
> -------
>
> As is, the cluster setup in our existing preemption benchmark does not
> reflect the improvements resulting from this patch. Currently, all existing
> victims can be preempted, therefore all `PendingTaskProcessor` has to is look
> at the next slave.
>
> ```
> BEFORE
> Benchmark
> (numPendingTasks) Mode Cnt Score Error Units
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark
> 1 thrpt 10 75.386 ± 2.984 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark
> 10 thrpt 10 74.584 ± 2.598 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark
> 100 thrpt 10 79.731 ± 2.182 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark
> 1000 thrpt 10 66.386 ± 1.833 ops/s
>
> AFTER
> Benchmark
> (numPendingTasks) Mode Cnt Score Error Units
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark
> 1 thrpt 10 78.266 ± 3.290 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark
> 10 thrpt 10 76.743 ± 2.073 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark
> 100 thrpt 10 75.343 ± 1.943 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark
> 1000 thrpt 10 68.284 ± 2.413 ops/s
> ```
>
> I need to further imprpve the cluster setup for this benchmark to reflect the
> improvements in the patch. A more representative cluster setup would be one
> in which only a subset of potential victims pass
> `PreemptionVictimFilter.filterPreemptionVictims()` test.
>
>
> Thanks,
>
> Mehrdad Nurolahzade
>
>
