Review Request 32208: Reduce loglevel for insufficient GC resources to fine
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32208/ --- Review request for Aurora. Repository: aurora Description --- Reduce loglevel for insufficient GC resources to fine Diffs - src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 31aa2bbaab3d97875493ad75c4d2c7c82ac7fa58 Diff: https://reviews.apache.org/r/32208/diff/ Testing --- ./gradlew -Pq build Thanks, Stephan Erb
Re: Review Request 32208: Reduce loglevel for insufficient GC resources to fine
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32208/#review76918 --- Master (a419863) is red with this patch. ./build-support/jenkins/build.sh src.test.python.apache.aurora.client.factory . SUCCESS src.test.python.apache.aurora.client.hooks.hooked_api . SUCCESS src.test.python.apache.aurora.client.hooks.non_hooked_api . SUCCESS src.test.python.apache.aurora.common.test_aurora_job_key . SUCCESS src.test.python.apache.aurora.common.test_cluster . SUCCESS src.test.python.apache.aurora.common.test_cluster_option . SUCCESS src.test.python.apache.aurora.common.test_clusters . SUCCESS src.test.python.apache.aurora.common.test_http_signaler . SUCCESS src.test.python.apache.aurora.common.test_pex_version . SUCCESS src.test.python.apache.aurora.common.test_shellify . SUCCESS src.test.python.apache.aurora.common.test_transport . SUCCESS src.test.python.apache.aurora.config.test_base . SUCCESS src.test.python.apache.aurora.config.test_constraint_parsing . SUCCESS src.test.python.apache.aurora.config.test_loader . SUCCESS src.test.python.apache.aurora.config.test_thrift . SUCCESS src.test.python.apache.aurora.executor.common.announcer . SUCCESS src.test.python.apache.aurora.executor.common.directory_sandbox . SUCCESS src.test.python.apache.aurora.executor.common.executor_detector . SUCCESS src.test.python.apache.aurora.executor.common.executor_timeout . SUCCESS src.test.python.apache.aurora.executor.common.health_checker . SUCCESS src.test.python.apache.aurora.executor.common.kill_manager . SUCCESS src.test.python.apache.aurora.executor.common.path_detector . SUCCESS src.test.python.apache.aurora.executor.common.status_checker . SUCCESS src.test.python.apache.aurora.executor.common.task_info . SUCCESS src.test.python.apache.aurora.executor.executor_base . SUCCESS src.test.python.apache.aurora.executor.executor_vars . SUCCESS src.test.python.apache.aurora.executor.gc_executor . FAILURE src.test.python.apache.aurora.executor.status_manager . SUCCESS src.test.python.apache.aurora.executor.thermos_task_runner . SUCCESS src.test.python.apache.thermos.cli.commands.commands . SUCCESS src.test.python.apache.thermos.cli.common . SUCCESS src.test.python.apache.thermos.cli.main . SUCCESS src.test.python.apache.thermos.common.test_pathspec . SUCCESS src.test.python.apache.thermos.core.test_runner_integration . SUCCESS src.test.python.apache.thermos.monitoring.test_disk . SUCCESS FAILURE [31m FAILURE[0m I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 18, 2015, 6:28 p.m., Stephan Erb wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32208/ --- (Updated March 18, 2015, 6:28 p.m.) Review request for Aurora. Repository: aurora
Re: Review Request 32220: Making preemptor asynchronous. Part 2 - async handling.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32220/ --- (Updated March 19, 2015, 12:29 a.m.) Review request for Aurora, Bill Farner and Zameer Manji. Changes --- Dropping PreemptionSlotCache visibility. Bugs: AURORA-1158 https://issues.apache.org/jira/browse/AURORA-1158 Repository: aurora Description --- This diff makes preemption asynchronous wrt the scheduling loop. New flow works as follows: - TaskScheduler is unable to schedule a task and calls into PreemptorImpl.attemptPreemptionFor() to acquire a reservation. - PreemptorImpl keeps track of found preemption slots in PreemptionSlotCache, fires an async slot search request and replies back with empty result. - A search is finished and a slot (if found) is added into internal slot cache. - TaskScheduler calls into attemptPreemptionFor(), finds a preemption slot, validates a slot is still valid and preempts tasks. A reservation for a slave is created. This is still an intermediate milestone on the way to a fully independent background preemptor. Benchmark refactoring will be addressed in a separate diff. Diffs (updated) - src/main/java/org/apache/aurora/scheduler/async/OfferManager.java 7d2cb46aa86dd4c3c6d53848725eed1542307ebd src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptionSlotCache.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptionSlotFinder.java e748b42eaa8f54e0cf1a0a883da4aceff3d7a3b8 src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 1808b71546423dfe80ccb1902e8cebd545674a27 src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorMetrics.java 801a6d7f3cb0e9987e2029fd8c4c89015e8d3b65 src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorModule.java dbfebf99bc6028faf433a69db4308a239ff61290 src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptionSlotCacheTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImplTest.java cfbc1a039262d92481ded2733d50ac51293a5b91 src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptorSlotFinderTest.java e329358f70028f52a807cd987378cbc002af36a9 Diff: https://reviews.apache.org/r/32220/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 32221: Remove excessively low timeout in SIGTERM swallowing test.
On March 18, 2015, 11:01 p.m., Aurora ReviewBot wrote: Master (6396410) is red with this patch. ./build-support/jenkins/build.sh src.test.python.apache.aurora.client.cli.plugins . SUCCESS src.test.python.apache.aurora.client.cli.quota . SUCCESS src.test.python.apache.aurora.client.cli.sla . SUCCESS src.test.python.apache.aurora.client.cli.supdate . SUCCESS src.test.python.apache.aurora.client.cli.task . SUCCESS src.test.python.apache.aurora.client.cli.update . SUCCESS src.test.python.apache.aurora.client.cli.version . SUCCESS src.test.python.apache.aurora.client.config . SUCCESS src.test.python.apache.aurora.client.factory . SUCCESS src.test.python.apache.aurora.client.hooks.hooked_api . SUCCESS src.test.python.apache.aurora.client.hooks.non_hooked_api . SUCCESS src.test.python.apache.aurora.common.test_aurora_job_key . SUCCESS src.test.python.apache.aurora.common.test_cluster . SUCCESS src.test.python.apache.aurora.common.test_cluster_option . SUCCESS src.test.python.apache.aurora.common.test_clusters . SUCCESS src.test.python.apache.aurora.common.test_http_signaler . SUCCESS src.test.python.apache.aurora.common.test_pex_version . SUCCESS src.test.python.apache.aurora.common.test_shellify . SUCCESS src.test.python.apache.aurora.common.test_transport . SUCCESS src.test.python.apache.aurora.config.test_base . SUCCESS src.test.python.apache.aurora.config.test_constraint_parsing . SUCCESS src.test.python.apache.aurora.config.test_loader . SUCCESS src.test.python.apache.aurora.config.test_thrift . SUCCESS src.test.python.apache.aurora.executor.common.path_detector . SUCCESS src.test.python.apache.aurora.executor.common.task_info . SUCCESS src.test.python.apache.aurora.executor.executor_base . SUCCESS src.test.python.apache.aurora.executor.executor_vars . SUCCESS src.test.python.apache.aurora.executor.status_manager . SUCCESS src.test.python.apache.aurora.executor.thermos_task_runner . FAILURE src.test.python.apache.thermos.cli.commands.commands . SUCCESS src.test.python.apache.thermos.cli.common . SUCCESS src.test.python.apache.thermos.cli.main . SUCCESS src.test.python.apache.thermos.common.test_pathspec . SUCCESS src.test.python.apache.thermos.core.test_runner_integration . SUCCESS src.test.python.apache.thermos.monitoring.test_disk . SUCCESS FAILURE [31m FAILURE[0m I will refresh this build result if you post a review containing @ReviewBot retry Brian Wickman wrote: welp my kingdom for reviewbot to print out stderr logs - Brian --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/#review76965 --- On March 18, 2015, 10:44 p.m., Brian Wickman wrote:
Review Request 32225: Adding preemptor jmh benchmark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32225/ --- Review request for Aurora and Bill Farner. Repository: aurora Description --- Adding a preemptor slot search perf benchmark. Will not apply cleanly, diffed against https://reviews.apache.org/r/32220/ Diffs - src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java ad49effdaf700bb9d5715aa5bdd1a5d0b276f83f src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorModule.java dbfebf99bc6028faf433a69db4308a239ff61290 src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptorModuleTest.java c9d10e4cec44045806ec2d75d8c158dc40d7de98 Diff: https://reviews.apache.org/r/32225/diff/ Testing --- ./gradlew jmh Thanks, Maxim Khutornenko
Re: Review Request 32225: Adding preemptor jmh benchmark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32225/#review76987 --- This patch does not apply cleanly on master (6396410), do you need to rebase? I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 19, 2015, 12:23 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32225/ --- (Updated March 19, 2015, 12:23 a.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-1158 https://issues.apache.org/jira/browse/AURORA-1158 Repository: aurora Description --- Adding a preemptor slot search perf benchmark. Will not apply cleanly, diffed against https://reviews.apache.org/r/32220/ Diffs - src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java ad49effdaf700bb9d5715aa5bdd1a5d0b276f83f src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorModule.java dbfebf99bc6028faf433a69db4308a239ff61290 src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptorModuleTest.java c9d10e4cec44045806ec2d75d8c158dc40d7de98 Diff: https://reviews.apache.org/r/32225/diff/ Testing --- ./gradlew jmh Thanks, Maxim Khutornenko
Re: Review Request 32220: Making preemptor asynchronous. Part 2 - async handling.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32220/#review76986 --- Ship it! Master (6396410) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 19, 2015, 12:29 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32220/ --- (Updated March 19, 2015, 12:29 a.m.) Review request for Aurora, Bill Farner and Zameer Manji. Bugs: AURORA-1158 https://issues.apache.org/jira/browse/AURORA-1158 Repository: aurora Description --- This diff makes preemption asynchronous wrt the scheduling loop. New flow works as follows: - TaskScheduler is unable to schedule a task and calls into PreemptorImpl.attemptPreemptionFor() to acquire a reservation. - PreemptorImpl keeps track of found preemption slots in PreemptionSlotCache, fires an async slot search request and replies back with empty result. - A search is finished and a slot (if found) is added into internal slot cache. - TaskScheduler calls into attemptPreemptionFor(), finds a preemption slot, validates a slot is still valid and preempts tasks. A reservation for a slave is created. This is still an intermediate milestone on the way to a fully independent background preemptor. Benchmark refactoring will be addressed in a separate diff. Diffs - src/main/java/org/apache/aurora/scheduler/async/OfferManager.java 7d2cb46aa86dd4c3c6d53848725eed1542307ebd src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptionSlotCache.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptionSlotFinder.java e748b42eaa8f54e0cf1a0a883da4aceff3d7a3b8 src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 1808b71546423dfe80ccb1902e8cebd545674a27 src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorMetrics.java 801a6d7f3cb0e9987e2029fd8c4c89015e8d3b65 src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorModule.java dbfebf99bc6028faf433a69db4308a239ff61290 src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptionSlotCacheTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImplTest.java cfbc1a039262d92481ded2733d50ac51293a5b91 src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptorSlotFinderTest.java e329358f70028f52a807cd987378cbc002af36a9 Diff: https://reviews.apache.org/r/32220/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 32221: Remove excessively low timeout in SIGTERM swallowing test.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/#review76992 --- @ReviewBot retry - Brian Wickman On March 19, 2015, 1:20 a.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/ --- (Updated March 19, 2015, 1:20 a.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-1054 https://issues.apache.org/jira/browse/AURORA-1054 Repository: aurora Description --- Remove excessively low timeout in SIGTERM swallowing test. Diffs - src/test/python/apache/aurora/executor/test_thermos_task_runner.py 6b24bbb2ab7ca16f97961aabeed945b61e5b5908 Diff: https://reviews.apache.org/r/32221/diff/ Testing --- Cannot reproduce locally, but 5 seconds is an impossibly small timeout, even if we aren't testing SIGTERM swallowing. If this fails, we will get tripped by 60s timeout instead. Thanks, Brian Wickman
Re: Review Request 32221: Remove excessively low timeout in SIGTERM swallowing test.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/#review76991 --- Ship it! Master (6396410) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 19, 2015, 1:20 a.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/ --- (Updated March 19, 2015, 1:20 a.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-1054 https://issues.apache.org/jira/browse/AURORA-1054 Repository: aurora Description --- Remove excessively low timeout in SIGTERM swallowing test. Diffs - src/test/python/apache/aurora/executor/test_thermos_task_runner.py 6b24bbb2ab7ca16f97961aabeed945b61e5b5908 Diff: https://reviews.apache.org/r/32221/diff/ Testing --- Cannot reproduce locally, but 5 seconds is an impossibly small timeout, even if we aren't testing SIGTERM swallowing. If this fails, we will get tripped by 60s timeout instead. Thanks, Brian Wickman
Re: Review Request 32221: Remove excessively low timeout in SIGTERM swallowing test.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/#review76999 --- Ship it! Master (6396410) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 19, 2015, 1:20 a.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/ --- (Updated March 19, 2015, 1:20 a.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-1054 https://issues.apache.org/jira/browse/AURORA-1054 Repository: aurora Description --- Remove excessively low timeout in SIGTERM swallowing test. Diffs - src/test/python/apache/aurora/executor/test_thermos_task_runner.py 6b24bbb2ab7ca16f97961aabeed945b61e5b5908 Diff: https://reviews.apache.org/r/32221/diff/ Testing --- Cannot reproduce locally, but 5 seconds is an impossibly small timeout, even if we aren't testing SIGTERM swallowing. If this fails, we will get tripped by 60s timeout instead. Thanks, Brian Wickman
Re: Review Request 32221: Remove excessively low timeout in SIGTERM swallowing test.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/ --- (Updated March 19, 2015, 1:20 a.m.) Review request for Aurora and Bill Farner. Changes --- The low timeout is actually what allows the test to even work in the first place. It should've worked in the past, but at least this way we can possibly eke out the original reason it was failing. Bugs: AURORA-1054 https://issues.apache.org/jira/browse/AURORA-1054 Repository: aurora Description --- Remove excessively low timeout in SIGTERM swallowing test. Diffs (updated) - src/test/python/apache/aurora/executor/test_thermos_task_runner.py 6b24bbb2ab7ca16f97961aabeed945b61e5b5908 Diff: https://reviews.apache.org/r/32221/diff/ Testing --- Cannot reproduce locally, but 5 seconds is an impossibly small timeout, even if we aren't testing SIGTERM swallowing. If this fails, we will get tripped by 60s timeout instead. Thanks, Brian Wickman
Re: Review Request 32221: Remove excessively low timeout in SIGTERM swallowing test.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/#review76965 --- Master (6396410) is red with this patch. ./build-support/jenkins/build.sh src.test.python.apache.aurora.client.cli.plugins . SUCCESS src.test.python.apache.aurora.client.cli.quota . SUCCESS src.test.python.apache.aurora.client.cli.sla . SUCCESS src.test.python.apache.aurora.client.cli.supdate . SUCCESS src.test.python.apache.aurora.client.cli.task . SUCCESS src.test.python.apache.aurora.client.cli.update . SUCCESS src.test.python.apache.aurora.client.cli.version . SUCCESS src.test.python.apache.aurora.client.config . SUCCESS src.test.python.apache.aurora.client.factory . SUCCESS src.test.python.apache.aurora.client.hooks.hooked_api . SUCCESS src.test.python.apache.aurora.client.hooks.non_hooked_api . SUCCESS src.test.python.apache.aurora.common.test_aurora_job_key . SUCCESS src.test.python.apache.aurora.common.test_cluster . SUCCESS src.test.python.apache.aurora.common.test_cluster_option . SUCCESS src.test.python.apache.aurora.common.test_clusters . SUCCESS src.test.python.apache.aurora.common.test_http_signaler . SUCCESS src.test.python.apache.aurora.common.test_pex_version . SUCCESS src.test.python.apache.aurora.common.test_shellify . SUCCESS src.test.python.apache.aurora.common.test_transport . SUCCESS src.test.python.apache.aurora.config.test_base . SUCCESS src.test.python.apache.aurora.config.test_constraint_parsing . SUCCESS src.test.python.apache.aurora.config.test_loader . SUCCESS src.test.python.apache.aurora.config.test_thrift . SUCCESS src.test.python.apache.aurora.executor.common.path_detector . SUCCESS src.test.python.apache.aurora.executor.common.task_info . SUCCESS src.test.python.apache.aurora.executor.executor_base . SUCCESS src.test.python.apache.aurora.executor.executor_vars . SUCCESS src.test.python.apache.aurora.executor.status_manager . SUCCESS src.test.python.apache.aurora.executor.thermos_task_runner . FAILURE src.test.python.apache.thermos.cli.commands.commands . SUCCESS src.test.python.apache.thermos.cli.common . SUCCESS src.test.python.apache.thermos.cli.main . SUCCESS src.test.python.apache.thermos.common.test_pathspec . SUCCESS src.test.python.apache.thermos.core.test_runner_integration . SUCCESS src.test.python.apache.thermos.monitoring.test_disk . SUCCESS FAILURE [31m FAILURE[0m I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 18, 2015, 10:44 p.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/ --- (Updated March 18, 2015, 10:44 p.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-1054
Re: Review Request 32221: Remove excessively low timeout in SIGTERM swallowing test.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/#review76998 --- @ReviewBot retry - Brian Wickman On March 19, 2015, 1:20 a.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/ --- (Updated March 19, 2015, 1:20 a.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-1054 https://issues.apache.org/jira/browse/AURORA-1054 Repository: aurora Description --- Remove excessively low timeout in SIGTERM swallowing test. Diffs - src/test/python/apache/aurora/executor/test_thermos_task_runner.py 6b24bbb2ab7ca16f97961aabeed945b61e5b5908 Diff: https://reviews.apache.org/r/32221/diff/ Testing --- Cannot reproduce locally, but 5 seconds is an impossibly small timeout, even if we aren't testing SIGTERM swallowing. If this fails, we will get tripped by 60s timeout instead. Thanks, Brian Wickman
Review Request 32221: Remove excessively low timeout in SIGTERM swallowing test.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/ --- Review request for Aurora and Bill Farner. Bugs: AURORA-1054 https://issues.apache.org/jira/browse/AURORA-1054 Repository: aurora Description --- Remove excessively low timeout in SIGTERM swallowing test. Diffs - src/test/python/apache/aurora/executor/test_thermos_task_runner.py 6b24bbb2ab7ca16f97961aabeed945b61e5b5908 Diff: https://reviews.apache.org/r/32221/diff/ Testing --- Cannot reproduce locally, but 5 seconds is an impossibly small timeout, even if we aren't testing SIGTERM swallowing. If this fails, we will get tripped by 60s timeout instead. Thanks, Brian Wickman
Re: Review Request 32220: Making preemptor asynchronous. Part 2 - async handling.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32220/#review76969 --- Ship it! Master (6396410) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 18, 2015, 11:04 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32220/ --- (Updated March 18, 2015, 11:04 p.m.) Review request for Aurora, Bill Farner and Zameer Manji. Bugs: AURORA-1158 https://issues.apache.org/jira/browse/AURORA-1158 Repository: aurora Description --- This diff makes preemption asynchronous wrt the scheduling loop. New flow works as follows: - TaskScheduler is unable to schedule a task and calls into PreemptorImpl.attemptPreemptionFor() to acquire a reservation. - PreemptorImpl keeps track of found preemption slots in PreemptionSlotCache, fires an async slot search request and replies back with empty result. - A search is finished and a slot (if found) is added into internal slot cache. - TaskScheduler calls into attemptPreemptionFor(), finds a preemption slot, validates a slot is still valid and preempts tasks. A reservation for a slave is created. This is still an intermediate milestone on the way to a fully independent background preemptor. Benchmark refactoring will be addressed in a separate diff. Diffs - src/main/java/org/apache/aurora/scheduler/async/OfferManager.java 7d2cb46aa86dd4c3c6d53848725eed1542307ebd src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptionSlotCache.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptionSlotFinder.java e748b42eaa8f54e0cf1a0a883da4aceff3d7a3b8 src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImpl.java 1808b71546423dfe80ccb1902e8cebd545674a27 src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorMetrics.java 801a6d7f3cb0e9987e2029fd8c4c89015e8d3b65 src/main/java/org/apache/aurora/scheduler/async/preemptor/PreemptorModule.java dbfebf99bc6028faf433a69db4308a239ff61290 src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptionSlotCacheTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptorImplTest.java cfbc1a039262d92481ded2733d50ac51293a5b91 src/test/java/org/apache/aurora/scheduler/async/preemptor/PreemptorSlotFinderTest.java e329358f70028f52a807cd987378cbc002af36a9 Diff: https://reviews.apache.org/r/32220/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 32221: Remove excessively low timeout in SIGTERM swallowing test.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/#review76997 --- Ship it! Master (6396410) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 19, 2015, 1:20 a.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32221/ --- (Updated March 19, 2015, 1:20 a.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-1054 https://issues.apache.org/jira/browse/AURORA-1054 Repository: aurora Description --- Remove excessively low timeout in SIGTERM swallowing test. Diffs - src/test/python/apache/aurora/executor/test_thermos_task_runner.py 6b24bbb2ab7ca16f97961aabeed945b61e5b5908 Diff: https://reviews.apache.org/r/32221/diff/ Testing --- Cannot reproduce locally, but 5 seconds is an impossibly small timeout, even if we aren't testing SIGTERM swallowing. If this fails, we will get tripped by 60s timeout instead. Thanks, Brian Wickman
Re: Review Request 32231: AURORA-1189: Adding check to see if java version is below 1.8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32231/#review77005 --- Master (6396410) is red with this patch. ./build-support/jenkins/build.sh :api:processResources UP-TO-DATE :api:classes :api:jar :compileJavaNote: Writing file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/com/twitter/common/args/apt/cmdline.arg.info.txt.2 :processResources :classes :jar :startScripts :distTar :distZip :assemble :compileJmhJava :processJmhResources UP-TO-DATE :jmhClasses :checkstyleJmh :jsHint :checkstyleMain[ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java:70: Line has trailing spaces. [ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java:245:7: 'if' is not followed by whitespace. [ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java:247:92: '+' should be on a new line. [ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java:248:17: '+' should be on a new line. [ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java:249:17: '+' should be on a new line. [ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java:250:83: '+' should be on a new line. [ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java:251:67: '+' should be on a new line. [ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java:252:17: '+' should be on a new line. [ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java:253:17: '+' should be on a new line. FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':checkstyleMain'. Checkstyle rule violations were found. See the report at: file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/main.xml * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. BUILD FAILED Total time: 1 mins 34.294 secs I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 19, 2015, 3:21 a.m., Florian Pfeiffer wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32231/ --- (Updated March 19, 2015, 3:21 a.m.) Review request for Aurora. Bugs: AURORA-1189 https://issues.apache.org/jira/browse/AURORA-1189 Repository: aurora Description --- AURORA-1189: Adding check to see if java version is below 1.8 I think it should be fine to check the java version that naive (comparing the char at 3rd posion of java.version) since this check will probably already be removed again until java 1.10 comes out. Diffs - src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 66fb432028c26ea9ed8da475da6c3cbee5e535fc Diff: https://reviews.apache.org/r/32231/diff/ Testing --- * ./gradlew test * running scheduler on vagrant machine to check if log message appears * couldn't get the scheduler run properly on java 1.8 to check if the message doesn't appear there Thanks, Florian Pfeiffer
Re: Review Request 32231: AURORA-1189: Adding check to see if java version is below 1.8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32231/#review77008 --- Master (6396410) is green with this patch. ./build-support/jenkins/build.sh However, it appears that it might lack test coverage. I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 19, 2015, 3:46 a.m., Florian Pfeiffer wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32231/ --- (Updated March 19, 2015, 3:46 a.m.) Review request for Aurora. Bugs: AURORA-1189 https://issues.apache.org/jira/browse/AURORA-1189 Repository: aurora Description --- AURORA-1189: Adding check to see if java version is below 1.8 I think it should be fine to check the java version that naive (comparing the char at 3rd posion of java.version) since this check will probably already be removed again until java 1.10 comes out. Diffs - src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 66fb432028c26ea9ed8da475da6c3cbee5e535fc Diff: https://reviews.apache.org/r/32231/diff/ Testing --- * ./gradlew test * running scheduler on vagrant machine to check if log message appears * couldn't get the scheduler run properly on java 1.8 to check if the message doesn't appear there Thanks, Florian Pfeiffer
Review Request 32231: AURORA-1189: Adding check to see if java version is below 1.8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32231/ --- Review request for Aurora. Bugs: AURORA-1189 https://issues.apache.org/jira/browse/AURORA-1189 Repository: aurora Description --- AURORA-1189: Adding check to see if java version is below 1.8 I think it should be fine to check the java version that naive (comparing the char at 3rd posion of java.version) since this check will probably already be removed again until java 1.10 comes out. Diffs - src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 66fb432028c26ea9ed8da475da6c3cbee5e535fc Diff: https://reviews.apache.org/r/32231/diff/ Testing --- * ./gradlew test * running scheduler on vagrant machine to check if log message appears * couldn't get the scheduler run properly on java 1.8 to check if the message doesn't appear there Thanks, Florian Pfeiffer
Re: Review Request 32231: AURORA-1189: Adding check to see if java version is below 1.8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32231/#review77007 --- @ReviewBot retry - Florian Pfeiffer On März 19, 2015, 3:46 vorm., Florian Pfeiffer wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32231/ --- (Updated März 19, 2015, 3:46 vorm.) Review request for Aurora. Bugs: AURORA-1189 https://issues.apache.org/jira/browse/AURORA-1189 Repository: aurora Description --- AURORA-1189: Adding check to see if java version is below 1.8 I think it should be fine to check the java version that naive (comparing the char at 3rd posion of java.version) since this check will probably already be removed again until java 1.10 comes out. Diffs - src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 66fb432028c26ea9ed8da475da6c3cbee5e535fc Diff: https://reviews.apache.org/r/32231/diff/ Testing --- * ./gradlew test * running scheduler on vagrant machine to check if log message appears * couldn't get the scheduler run properly on java 1.8 to check if the message doesn't appear there Thanks, Florian Pfeiffer
Re: Review Request 32231: AURORA-1189: Adding check to see if java version is below 1.8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32231/ --- (Updated März 19, 2015, 3:46 vorm.) Review request for Aurora. Bugs: AURORA-1189 https://issues.apache.org/jira/browse/AURORA-1189 Repository: aurora Description --- AURORA-1189: Adding check to see if java version is below 1.8 I think it should be fine to check the java version that naive (comparing the char at 3rd posion of java.version) since this check will probably already be removed again until java 1.10 comes out. Diffs (updated) - src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 66fb432028c26ea9ed8da475da6c3cbee5e535fc Diff: https://reviews.apache.org/r/32231/diff/ Testing --- * ./gradlew test * running scheduler on vagrant machine to check if log message appears * couldn't get the scheduler run properly on java 1.8 to check if the message doesn't appear there Thanks, Florian Pfeiffer
Re: Review Request 32208: Reduce loglevel for insufficient GC resources to fine
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32208/#review76919 --- Ship it! Ship It! - Maxim Khutornenko On March 18, 2015, 6:28 p.m., Stephan Erb wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32208/ --- (Updated March 18, 2015, 6:28 p.m.) Review request for Aurora. Repository: aurora Description --- Reduce loglevel for insufficient GC resources to fine Diffs - src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 31aa2bbaab3d97875493ad75c4d2c7c82ac7fa58 Diff: https://reviews.apache.org/r/32208/diff/ Testing --- ./gradlew -Pq build Thanks, Stephan Erb
Re: Review Request 32208: Reduce loglevel for insufficient GC resources to fine
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32208/#review76934 --- @ReviewBot retry - Stephan Erb On March 18, 2015, 7:28 p.m., Stephan Erb wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32208/ --- (Updated March 18, 2015, 7:28 p.m.) Review request for Aurora. Repository: aurora Description --- Reduce loglevel for insufficient GC resources to fine Diffs - src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 31aa2bbaab3d97875493ad75c4d2c7c82ac7fa58 Diff: https://reviews.apache.org/r/32208/diff/ Testing --- ./gradlew -Pq build Thanks, Stephan Erb
Re: Review Request 32208: Reduce loglevel for insufficient GC resources to fine
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32208/#review76937 --- Master (a419863) is green with this patch. ./build-support/jenkins/build.sh However, it appears that it might lack test coverage. I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 18, 2015, 6:28 p.m., Stephan Erb wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32208/ --- (Updated March 18, 2015, 6:28 p.m.) Review request for Aurora. Repository: aurora Description --- Reduce loglevel for insufficient GC resources to fine Diffs - src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 31aa2bbaab3d97875493ad75c4d2c7c82ac7fa58 Diff: https://reviews.apache.org/r/32208/diff/ Testing --- ./gradlew -Pq build Thanks, Stephan Erb
Re: Review Request 32208: Reduce loglevel for insufficient GC resources to fine
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32208/#review76947 --- Ship it! Ship It! - Bill Farner On March 18, 2015, 6:28 p.m., Stephan Erb wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32208/ --- (Updated March 18, 2015, 6:28 p.m.) Review request for Aurora. Repository: aurora Description --- Reduce loglevel for insufficient GC resources to fine Diffs - src/main/java/org/apache/aurora/scheduler/async/GcExecutorLauncher.java 31aa2bbaab3d97875493ad75c4d2c7c82ac7fa58 Diff: https://reviews.apache.org/r/32208/diff/ Testing --- ./gradlew -Pq build Thanks, Stephan Erb
