Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/ --- (Updated March 3, 2015, 3:21 p.m.) Review request for Aurora, Jay Buffington and Bill Farner. Bugs: AURORA-1107 https://issues.apache.org/jira/browse/AURORA-1107 Repository: aurora Description --- Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run. This is the first portion of allowing per-job mounts, however, I wanted to get this out first since more people want it. I'll implement per-job mounts in a future review. Diffs (updated) - api/src/main/thrift/org/apache/aurora/gen/api.thrift a5198dfe1c56e05d50e32dd14222a984515c0d07 config/checkstyle/checkstyle.xml 580d9d3737c0f52da2d33f5b99ef9406b55c5807 docs/deploying-aurora-scheduler.md d1123359961fd59ddb8c1a07f80f293bdd46019f src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 3239eaa139e35e8c3acdacf6375f492de2b5bfee src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 24b61c1e4f615295acf28d904588e1512972d3f4 src/main/java/org/apache/aurora/scheduler/app/VolumeParser.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java baacb71403d55c5b90fc11cb2a23f552a32e8ba5 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 5340d651b298ec8aa079e73d6d2f652fdf876293 src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 6575b7d420f17ec68d6e2a83e7b380f684577d4f src/test/java/org/apache/aurora/scheduler/app/VolumeParserTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 444d6d3fdaf86eb84612f846eaa326eb75c49898 src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java efe62ceb502ead88a2f0cd6d09a76664e465d9bc Diff: https://reviews.apache.org/r/31338/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 31423: Stop the announcer and status checkers before starting to kill the runners
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31423/#review74958 --- Is there anything else here? Is this ready to be merged? - Steve Niemitz On Feb. 26, 2015, 3:20 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31423/ --- (Updated Feb. 26, 2015, 3:20 p.m.) Review request for Aurora, Brian Wickman and Zameer Manji. Repository: aurora Description --- Stop the announcer and status checkers before starting to kill the runners. This allows the task to be removed from the ZK ensemble before it begins getting killed. The delay can be significant if the task takes some time to shutdown, and during the time it stops responding to requests. Diffs - src/main/python/apache/aurora/executor/aurora_executor.py 9c0282392dbb9cca308baf47adc1750c1f5cacc6 src/test/python/apache/aurora/executor/BUILD 2ee9b1233e9db47455ddffbc48691d379222 src/test/python/apache/aurora/executor/test_thermos_executor.py 8dbfb1db5eb7a6548820ff7cf82a9c7092f61d28 Diff: https://reviews.apache.org/r/31423/diff/ Testing --- We're now running this in our production environments. Watching ZK, I can confirm that the nodes are removed before process shutdown begins. Watching the executor log also confirms this. I couldn't observe any other side effects either. Thanks, Steve Niemitz
Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/#review75000 --- @ReviewBot retry - Bill Farner On March 3, 2015, 3:23 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/ --- (Updated March 3, 2015, 3:23 p.m.) Review request for Aurora, Jay Buffington and Bill Farner. Bugs: AURORA-1107 https://issues.apache.org/jira/browse/AURORA-1107 Repository: aurora Description --- Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run. This is the first portion of allowing per-job mounts, however, I wanted to get this out first since more people want it. I'll implement per-job mounts in a future review. Diffs - api/src/main/thrift/org/apache/aurora/gen/api.thrift a5198dfe1c56e05d50e32dd14222a984515c0d07 config/checkstyle/checkstyle.xml 580d9d3737c0f52da2d33f5b99ef9406b55c5807 docs/deploying-aurora-scheduler.md d1123359961fd59ddb8c1a07f80f293bdd46019f src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 3239eaa139e35e8c3acdacf6375f492de2b5bfee src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 24b61c1e4f615295acf28d904588e1512972d3f4 src/main/java/org/apache/aurora/scheduler/app/VolumeParser.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java baacb71403d55c5b90fc11cb2a23f552a32e8ba5 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 5340d651b298ec8aa079e73d6d2f652fdf876293 src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 6575b7d420f17ec68d6e2a83e7b380f684577d4f src/test/java/org/apache/aurora/scheduler/app/VolumeParserTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 444d6d3fdaf86eb84612f846eaa326eb75c49898 src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java efe62ceb502ead88a2f0cd6d09a76664e465d9bc Diff: https://reviews.apache.org/r/31338/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
On March 3, 2015, 6:28 p.m., Aurora ReviewBot wrote: Master (782e3e7) is red with this patch. ./build-support/jenkins/build.sh src.test.python.apache.aurora.client.cli.update . SUCCESS src.test.python.apache.aurora.client.cli.version . SUCCESS src.test.python.apache.aurora.client.config . SUCCESS src.test.python.apache.aurora.client.factory . SUCCESS src.test.python.apache.aurora.client.hooks.hooked_api . SUCCESS src.test.python.apache.aurora.client.hooks.non_hooked_api . SUCCESS src.test.python.apache.aurora.common.test_aurora_job_key . SUCCESS src.test.python.apache.aurora.common.test_cluster . SUCCESS src.test.python.apache.aurora.common.test_cluster_option . SUCCESS src.test.python.apache.aurora.common.test_clusters . SUCCESS src.test.python.apache.aurora.common.test_http_signaler . SUCCESS src.test.python.apache.aurora.common.test_pex_version . SUCCESS src.test.python.apache.aurora.common.test_shellify . SUCCESS src.test.python.apache.aurora.common.test_transport . SUCCESS src.test.python.apache.aurora.config.test_base . SUCCESS src.test.python.apache.aurora.config.test_constraint_parsing . SUCCESS src.test.python.apache.aurora.config.test_loader . SUCCESS src.test.python.apache.aurora.config.test_thrift . SUCCESS src.test.python.apache.aurora.executor.common.announcer . SUCCESS src.test.python.apache.aurora.executor.common.directory_sandbox . SUCCESS src.test.python.apache.aurora.executor.common.executor_detector . SUCCESS src.test.python.apache.aurora.executor.common.executor_timeout . SUCCESS src.test.python.apache.aurora.executor.common.health_checker . SUCCESS src.test.python.apache.aurora.executor.common.kill_manager . SUCCESS src.test.python.apache.aurora.executor.common.path_detector . SUCCESS src.test.python.apache.aurora.executor.common.status_checker . SUCCESS src.test.python.apache.aurora.executor.common.task_info . SUCCESS src.test.python.apache.aurora.executor.executor_base . SUCCESS src.test.python.apache.aurora.executor.executor_vars . SUCCESS src.test.python.apache.aurora.executor.gc_executor . FAILURE src.test.python.apache.aurora.executor.status_manager . SUCCESS src.test.python.apache.aurora.executor.thermos_task_runner . SUCCESS src.test.python.apache.thermos.common.test_pathspec . SUCCESS src.test.python.apache.thermos.core.test_runner_integration . SUCCESS src.test.python.apache.thermos.monitoring.test_disk . SUCCESS FAILURE [31m FAILURE[0m I will refresh this build result if you post a review containing @ReviewBot retry Whack-a-mole https://issues.apache.org/jira/browse/AURORA-1162 - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/#review75003 --- On March 3, 2015, 3:23 p.m., Steve Niemitz wrote:
Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/#review75005 --- @ReviewBot retry - Bill Farner On March 3, 2015, 3:23 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/ --- (Updated March 3, 2015, 3:23 p.m.) Review request for Aurora, Jay Buffington and Bill Farner. Bugs: AURORA-1107 https://issues.apache.org/jira/browse/AURORA-1107 Repository: aurora Description --- Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run. This is the first portion of allowing per-job mounts, however, I wanted to get this out first since more people want it. I'll implement per-job mounts in a future review. Diffs - api/src/main/thrift/org/apache/aurora/gen/api.thrift a5198dfe1c56e05d50e32dd14222a984515c0d07 config/checkstyle/checkstyle.xml 580d9d3737c0f52da2d33f5b99ef9406b55c5807 docs/deploying-aurora-scheduler.md d1123359961fd59ddb8c1a07f80f293bdd46019f src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 3239eaa139e35e8c3acdacf6375f492de2b5bfee src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 24b61c1e4f615295acf28d904588e1512972d3f4 src/main/java/org/apache/aurora/scheduler/app/VolumeParser.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java baacb71403d55c5b90fc11cb2a23f552a32e8ba5 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 5340d651b298ec8aa079e73d6d2f652fdf876293 src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 6575b7d420f17ec68d6e2a83e7b380f684577d4f src/test/java/org/apache/aurora/scheduler/app/VolumeParserTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 444d6d3fdaf86eb84612f846eaa326eb75c49898 src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java efe62ceb502ead88a2f0cd6d09a76664e465d9bc Diff: https://reviews.apache.org/r/31338/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 31659: Clean up end-to-end tests.
On March 3, 2015, 6:10 p.m., Maxim Khutornenko wrote: src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh, line 130 https://reviews.apache.org/r/31659/diff/1/?file=882656#file882656line130 Not directly related but adding --batch-size=2 would make it even faster. This crossed my mind as well. I've gone ahead and used an update configuration to express this instead. - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/#review74994 --- On March 3, 2015, 3:23 a.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/ --- (Updated March 3, 2015, 3:23 a.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Repository: aurora Description --- Major changes: - applying DRY by not invoking everything via `vagrant ssh` (making tests easier to write and much faster to run) - pulled 'test cases' into individual functions - simplified (maybe?) setup for tests using SSH This is being done in preparation for AURORA-1157. Diffs - src/test/sh/org/apache/aurora/e2e/test_common.sh b621975bab6f8bbf193e61360654e34337e36943 src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 1d599c3f71b7e07c1cb8bee01a4d3b5cfcf92631 src/test/sh/org/apache/aurora/e2e/test_run.sh ab91b95a60f7b459b95dc7e13a29692badca5ff7 Diff: https://reviews.apache.org/r/31659/diff/ Testing --- Ran the script. Thanks, Bill Farner
Re: Review Request 31659: Clean up end-to-end tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/#review75049 --- Ship it! Master (782e3e7) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 3, 2015, 8:55 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/ --- (Updated March 3, 2015, 8:55 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Repository: aurora Description --- Major changes: - applying DRY by not invoking everything via `vagrant ssh` (making tests easier to write and much faster to run) - pulled 'test cases' into individual functions - simplified (maybe?) setup for tests using SSH This is being done in preparation for AURORA-1157. Diffs - src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 0f6423521ad2c85032b4825a5b9793dc522a3b70 src/test/sh/org/apache/aurora/e2e/http/http_example_docker.aurora 00fa2fb6c272fe01700c4696f92713f11d62230b src/test/sh/org/apache/aurora/e2e/http/http_example_docker_updated.aurora d987d637da4061cd7946074243daabee4b9a150f src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 072bbb76707e6696815509f21d50b6b5446241b3 src/test/sh/org/apache/aurora/e2e/test_common.sh b621975bab6f8bbf193e61360654e34337e36943 src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 1d599c3f71b7e07c1cb8bee01a4d3b5cfcf92631 src/test/sh/org/apache/aurora/e2e/test_run.sh ab91b95a60f7b459b95dc7e13a29692badca5ff7 Diff: https://reviews.apache.org/r/31659/diff/ Testing --- Ran the script. Thanks, Bill Farner
Re: Review Request 31338: Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/#review75050 --- Ship it! Master (782e3e7) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 3, 2015, 3:23 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31338/ --- (Updated March 3, 2015, 3:23 p.m.) Review request for Aurora, Jay Buffington and Bill Farner. Bugs: AURORA-1107 https://issues.apache.org/jira/browse/AURORA-1107 Repository: aurora Description --- Added a command line flag, -global_container_mounts, to allow mounting paths from the slaves into the (docker) containers they run. This is the first portion of allowing per-job mounts, however, I wanted to get this out first since more people want it. I'll implement per-job mounts in a future review. Diffs - api/src/main/thrift/org/apache/aurora/gen/api.thrift a5198dfe1c56e05d50e32dd14222a984515c0d07 config/checkstyle/checkstyle.xml 580d9d3737c0f52da2d33f5b99ef9406b55c5807 docs/deploying-aurora-scheduler.md d1123359961fd59ddb8c1a07f80f293bdd46019f src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 3239eaa139e35e8c3acdacf6375f492de2b5bfee src/main/java/org/apache/aurora/scheduler/app/SchedulerMain.java 24b61c1e4f615295acf28d904588e1512972d3f4 src/main/java/org/apache/aurora/scheduler/app/VolumeParser.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/mesos/ExecutorSettings.java baacb71403d55c5b90fc11cb2a23f552a32e8ba5 src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 5340d651b298ec8aa079e73d6d2f652fdf876293 src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java 6575b7d420f17ec68d6e2a83e7b380f684577d4f src/test/java/org/apache/aurora/scheduler/app/VolumeParserTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 444d6d3fdaf86eb84612f846eaa326eb75c49898 src/test/java/org/apache/aurora/scheduler/mesos/TaskExecutors.java efe62ceb502ead88a2f0cd6d09a76664e465d9bc Diff: https://reviews.apache.org/r/31338/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 31659: Clean up end-to-end tests.
On March 3, 2015, 5:23 p.m., Maxim Khutornenko wrote: src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh, line 64 https://reviews.apache.org/r/31659/diff/1/?file=882656#file882656line64 Is this statement still accurate now that ssh is dropped from the command? It is not! Thanks, removed. - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/#review74983 --- On March 3, 2015, 3:23 a.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/ --- (Updated March 3, 2015, 3:23 a.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Repository: aurora Description --- Major changes: - applying DRY by not invoking everything via `vagrant ssh` (making tests easier to write and much faster to run) - pulled 'test cases' into individual functions - simplified (maybe?) setup for tests using SSH This is being done in preparation for AURORA-1157. Diffs - src/test/sh/org/apache/aurora/e2e/test_common.sh b621975bab6f8bbf193e61360654e34337e36943 src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 1d599c3f71b7e07c1cb8bee01a4d3b5cfcf92631 src/test/sh/org/apache/aurora/e2e/test_run.sh ab91b95a60f7b459b95dc7e13a29692badca5ff7 Diff: https://reviews.apache.org/r/31659/diff/ Testing --- Ran the script. Thanks, Bill Farner
Re: Review Request 31559: Split thermos cli into pieces. Add custom aurora thermos entry point.
On March 1, 2015, 9:03 p.m., Joe Smith wrote: src/main/python/apache/aurora/tools/thermos.py, line 1 https://reviews.apache.org/r/31559/diff/5/?file=881006#file881006line1 Add apache copyright stanzas to these files? done On March 1, 2015, 9:03 p.m., Joe Smith wrote: src/test/python/apache/thermos/cli/test_common.py, line 3 https://reviews.apache.org/r/31559/diff/5/?file=881031#file881031line3 Adding tests for `get_task_from_options` and `task_from_re` would be valuable added test for get_task_from_options since it's the most complex. task_from_re is simpler and the cost to cover is pretty high, so i'd like to elect to punt until we're sure it's sticking around. - Brian --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31559/#review74711 --- On March 3, 2015, 10:14 p.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31559/ --- (Updated March 3, 2015, 10:14 p.m.) Review request for Aurora, Joe Smith and Zameer Manji. Bugs: AURORA-1027 https://issues.apache.org/jira/browse/AURORA-1027 Repository: aurora Description --- This splits src/main/python/apache/thermos/bin/thermos.py into src/main/python/apache/thermos/cli/**. This also adds a path detector registry (AURORA-1027) for the client so that we can easily add a customized aurora entry point for thermos that is capable of finding checkpoint roots in executor sandboxes. Added said entry point in src/main/apache/aurora/tools/thermos.py -- open for other suggestions where this should live (possibly under aurora/executor/bin?) This is a drop-in replacement for the existing thermos binary. The cli/commands/*.py are mostly no-op splits with only a few lines changed in gc, status and tail to get_path_detector. Added a basic testing framework and an example test for simplerun but did not go with full coverage, instead opting to ship AURORA-1027 for now. Diffs - src/main/python/apache/aurora/tools/BUILD PRE-CREATION src/main/python/apache/aurora/tools/thermos.py PRE-CREATION src/main/python/apache/thermos/bin/BUILD 1a002da4f8294b3ecd43ab845eb15bc8ebd12b26 src/main/python/apache/thermos/bin/thermos.py 0853a9892399824385bee9e72db4c108f46fceda src/main/python/apache/thermos/cli/BUILD PRE-CREATION src/main/python/apache/thermos/cli/bin/BUILD PRE-CREATION src/main/python/apache/thermos/cli/bin/thermos.py PRE-CREATION src/main/python/apache/thermos/cli/commands/BUILD PRE-CREATION src/main/python/apache/thermos/cli/commands/gc.py PRE-CREATION src/main/python/apache/thermos/cli/commands/help.py PRE-CREATION src/main/python/apache/thermos/cli/commands/inspect.py PRE-CREATION src/main/python/apache/thermos/cli/commands/kill.py PRE-CREATION src/main/python/apache/thermos/cli/commands/read.py PRE-CREATION src/main/python/apache/thermos/cli/commands/run.py PRE-CREATION src/main/python/apache/thermos/cli/commands/simplerun.py PRE-CREATION src/main/python/apache/thermos/cli/commands/status.py PRE-CREATION src/main/python/apache/thermos/cli/commands/tail.py PRE-CREATION src/main/python/apache/thermos/cli/common.py PRE-CREATION src/main/python/apache/thermos/cli/main.py PRE-CREATION src/test/python/apache/thermos/BUILD df50964851c6be89c646015edf91d07992743497 src/test/python/apache/thermos/bin/BUILD 596aeb200dc55b077ed73fc342b9187045732890 src/test/python/apache/thermos/bin/test_thermos.py 2d9d33397ae01b31ab011d958f5457311ef7ef20 src/test/python/apache/thermos/cli/commands/BUILD PRE-CREATION src/test/python/apache/thermos/cli/commands/test_import.py PRE-CREATION src/test/python/apache/thermos/cli/commands/test_simplerun.py PRE-CREATION src/test/python/apache/thermos/cli/test_common.py PRE-CREATION Diff: https://reviews.apache.org/r/31559/diff/ Testing --- ./pants test src/test/python/apache/thermos/cli:: Thanks, Brian Wickman
Re: Review Request 31659: Clean up end-to-end tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/#review75045 --- Ship it! Ship It! - Maxim Khutornenko On March 3, 2015, 8:53 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/ --- (Updated March 3, 2015, 8:53 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Repository: aurora Description --- Major changes: - applying DRY by not invoking everything via `vagrant ssh` (making tests easier to write and much faster to run) - pulled 'test cases' into individual functions - simplified (maybe?) setup for tests using SSH This is being done in preparation for AURORA-1157. Diffs - src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 0f6423521ad2c85032b4825a5b9793dc522a3b70 src/test/sh/org/apache/aurora/e2e/http/http_example_docker.aurora 00fa2fb6c272fe01700c4696f92713f11d62230b src/test/sh/org/apache/aurora/e2e/http/http_example_docker_updated.aurora d987d637da4061cd7946074243daabee4b9a150f src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 072bbb76707e6696815509f21d50b6b5446241b3 src/test/sh/org/apache/aurora/e2e/test_common.sh b621975bab6f8bbf193e61360654e34337e36943 src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 1d599c3f71b7e07c1cb8bee01a4d3b5cfcf92631 src/test/sh/org/apache/aurora/e2e/test_run.sh ab91b95a60f7b459b95dc7e13a29692badca5ff7 Diff: https://reviews.apache.org/r/31659/diff/ Testing --- Ran the script. Thanks, Bill Farner
Re: Review Request 31659: Clean up end-to-end tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/#review75048 --- Ship it! Just tried this out locally. So much faster, thanks for doing this! - Joshua Cohen On March 3, 2015, 8:55 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/ --- (Updated March 3, 2015, 8:55 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Repository: aurora Description --- Major changes: - applying DRY by not invoking everything via `vagrant ssh` (making tests easier to write and much faster to run) - pulled 'test cases' into individual functions - simplified (maybe?) setup for tests using SSH This is being done in preparation for AURORA-1157. Diffs - src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 0f6423521ad2c85032b4825a5b9793dc522a3b70 src/test/sh/org/apache/aurora/e2e/http/http_example_docker.aurora 00fa2fb6c272fe01700c4696f92713f11d62230b src/test/sh/org/apache/aurora/e2e/http/http_example_docker_updated.aurora d987d637da4061cd7946074243daabee4b9a150f src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 072bbb76707e6696815509f21d50b6b5446241b3 src/test/sh/org/apache/aurora/e2e/test_common.sh b621975bab6f8bbf193e61360654e34337e36943 src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 1d599c3f71b7e07c1cb8bee01a4d3b5cfcf92631 src/test/sh/org/apache/aurora/e2e/test_run.sh ab91b95a60f7b459b95dc7e13a29692badca5ff7 Diff: https://reviews.apache.org/r/31659/diff/ Testing --- Ran the script. Thanks, Bill Farner
Re: Review Request 31659: Clean up end-to-end tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/ --- (Updated March 3, 2015, 8:53 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Repository: aurora Description --- Major changes: - applying DRY by not invoking everything via `vagrant ssh` (making tests easier to write and much faster to run) - pulled 'test cases' into individual functions - simplified (maybe?) setup for tests using SSH This is being done in preparation for AURORA-1157. Diffs (updated) - src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 0f6423521ad2c85032b4825a5b9793dc522a3b70 src/test/sh/org/apache/aurora/e2e/http/http_example_docker.aurora 00fa2fb6c272fe01700c4696f92713f11d62230b src/test/sh/org/apache/aurora/e2e/http/http_example_docker_updated.aurora d987d637da4061cd7946074243daabee4b9a150f src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 072bbb76707e6696815509f21d50b6b5446241b3 src/test/sh/org/apache/aurora/e2e/test_common.sh b621975bab6f8bbf193e61360654e34337e36943 src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 1d599c3f71b7e07c1cb8bee01a4d3b5cfcf92631 src/test/sh/org/apache/aurora/e2e/test_run.sh ab91b95a60f7b459b95dc7e13a29692badca5ff7 Diff: https://reviews.apache.org/r/31659/diff/ Testing --- Ran the script. Thanks, Bill Farner
Re: Review Request 31559: Split thermos cli into pieces. Add custom aurora thermos entry point.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31559/ --- (Updated March 3, 2015, 10:14 p.m.) Review request for Aurora, Joe Smith and Zameer Manji. Changes --- Address joe's feedback Bugs: AURORA-1027 https://issues.apache.org/jira/browse/AURORA-1027 Repository: aurora Description --- This splits src/main/python/apache/thermos/bin/thermos.py into src/main/python/apache/thermos/cli/**. This also adds a path detector registry (AURORA-1027) for the client so that we can easily add a customized aurora entry point for thermos that is capable of finding checkpoint roots in executor sandboxes. Added said entry point in src/main/apache/aurora/tools/thermos.py -- open for other suggestions where this should live (possibly under aurora/executor/bin?) This is a drop-in replacement for the existing thermos binary. The cli/commands/*.py are mostly no-op splits with only a few lines changed in gc, status and tail to get_path_detector. Added a basic testing framework and an example test for simplerun but did not go with full coverage, instead opting to ship AURORA-1027 for now. Diffs (updated) - src/main/python/apache/aurora/tools/BUILD PRE-CREATION src/main/python/apache/aurora/tools/thermos.py PRE-CREATION src/main/python/apache/thermos/bin/BUILD 1a002da4f8294b3ecd43ab845eb15bc8ebd12b26 src/main/python/apache/thermos/bin/thermos.py 0853a9892399824385bee9e72db4c108f46fceda src/main/python/apache/thermos/cli/BUILD PRE-CREATION src/main/python/apache/thermos/cli/bin/BUILD PRE-CREATION src/main/python/apache/thermos/cli/bin/thermos.py PRE-CREATION src/main/python/apache/thermos/cli/commands/BUILD PRE-CREATION src/main/python/apache/thermos/cli/commands/gc.py PRE-CREATION src/main/python/apache/thermos/cli/commands/help.py PRE-CREATION src/main/python/apache/thermos/cli/commands/inspect.py PRE-CREATION src/main/python/apache/thermos/cli/commands/kill.py PRE-CREATION src/main/python/apache/thermos/cli/commands/read.py PRE-CREATION src/main/python/apache/thermos/cli/commands/run.py PRE-CREATION src/main/python/apache/thermos/cli/commands/simplerun.py PRE-CREATION src/main/python/apache/thermos/cli/commands/status.py PRE-CREATION src/main/python/apache/thermos/cli/commands/tail.py PRE-CREATION src/main/python/apache/thermos/cli/common.py PRE-CREATION src/main/python/apache/thermos/cli/main.py PRE-CREATION src/test/python/apache/thermos/BUILD df50964851c6be89c646015edf91d07992743497 src/test/python/apache/thermos/bin/BUILD 596aeb200dc55b077ed73fc342b9187045732890 src/test/python/apache/thermos/bin/test_thermos.py 2d9d33397ae01b31ab011d958f5457311ef7ef20 src/test/python/apache/thermos/cli/commands/BUILD PRE-CREATION src/test/python/apache/thermos/cli/commands/test_import.py PRE-CREATION src/test/python/apache/thermos/cli/commands/test_simplerun.py PRE-CREATION src/test/python/apache/thermos/cli/test_common.py PRE-CREATION Diff: https://reviews.apache.org/r/31559/diff/ Testing --- ./pants test src/test/python/apache/thermos/cli:: Thanks, Brian Wickman
Re: Review Request 31710: Fix query and timestamp display when running beta-update status.
On March 3, 2015, 3:21 p.m., Zameer Manji wrote:
src/test/python/apache/aurora/client/cli/test_supdate.py, line 211
https://reviews.apache.org/r/31710/diff/1/?file=883801#file883801line211
For the calls like this you can do
`mock_api.pause_job_update.assert_called_once_with(self.TEST_JOBKEY)`
instead of comparing against an array of call objects.
Bill Farner wrote:
In case the diff didn't make it clear, i think we should settle on
`mock_calls` for uniformity. Additionally, i think the error messages from
asserting on `mock_calls` is better:
```
_mock_self = MagicMock name='ctime' id='140532173693328', args = (1,)
kwargs = {}, self = MagicMock name='ctime' id='140532173693328'
msg = 'Expected to be called once. Called 9 times.'
???
E AssertionError: Expected to be called once. Called 9 times.
```
vs
```
assert self.mock_ctime.mock_calls == [call(1)]
E AssertionError: assert [call(1),\n ca...(8),\n call(9)] ==
[call(1)]
E Left contains more items, first extra item: call(2)
E Full diff:
E - [call(1),
E ? ^
E + [call(1)]
E ? ^
E - call(2),
E - call(3),
E - call(4),
E - call(5),
E - call(6),
E - call(7),
E - call(8),
E - call(9)]
```
Moreover, `assert x.mock_calls == [...]` is not vulnerable to typos. In
this test, the first typoed line passes, but the second fails.
```
self.mock_ctime.assret_called_once_with(1)
assert self.mock_ctime.mock_clls == [call(n) for n in range(1, 10)]
```
Your reasoning is sound. I would like to point out the nice error messages come
from using `assert` statements under a py.test runner.
- Zameer
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31710/#review75083
---
On March 3, 2015, 3:16 p.m., Bill Farner wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31710/
---
(Updated March 3, 2015, 3:16 p.m.)
Review request for Aurora and Maxim Khutornenko.
Bugs: AURORA-1165 and AURORA-1166
https://issues.apache.org/jira/browse/AURORA-1165
https://issues.apache.org/jira/browse/AURORA-1166
Repository: aurora
Description
---
Discovered these while working on AURORA-1157
Diffs
-
src/main/python/apache/aurora/client/cli/update.py
ee8146be0cb6e3f3139a5491c883864950f245bd
src/test/python/apache/aurora/client/cli/test_supdate.py
1173650998d397a68a7fabf38201fb9feebf2977
Diff: https://reviews.apache.org/r/31710/diff/
Testing
---
Thanks,
Bill Farner
Re: Review Request 31716: Add an end-to-end test for scheduler-driven job updates.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31716/#review75121 --- Ship it! Master (270faf7) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 4, 2015, 1:21 a.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31716/ --- (Updated March 4, 2015, 1:21 a.m.) Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-1157 https://issues.apache.org/jira/browse/AURORA-1157 Repository: aurora Description --- Add an end-to-end test for scheduler-driven job updates. Diffs - src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 6a75528aeb62c119e4f8ab602d0e2081dc07fe49 Diff: https://reviews.apache.org/r/31716/diff/ Testing --- Ran end-to-end tests. Thanks, Bill Farner
Re: Review Request 31451: Port thermos observer to the path detector interface
On March 1, 2015, 8:47 p.m., Joe Smith wrote: src/main/python/apache/aurora/executor/common/resource_manager.py, line 96 https://reviews.apache.org/r/31451/diff/6/?file=880982#file880982line96 Why is this moving to kwargs? (My normal assumption is to stick with actual arguments) this is mostly so we don't have to duplicate all the options for TaskResourceMonitor. These options are really just around for mocking. On March 1, 2015, 8:47 p.m., Joe Smith wrote: src/main/python/apache/thermos/observer/detector.py, line 8 https://reviews.apache.org/r/31451/diff/6/?file=880992#file880992line8 You think something like this might help for a docstring? ``` class ObserverTaskDetector(object): The canonical way to watch (and be notified) of tasks transitioning state from active - finished - removed. ``` ObserverTaskDetector turns on-disk thermos task transitions into callback events. On March 1, 2015, 8:47 p.m., Joe Smith wrote: src/test/python/apache/aurora/executor/common/test_resource_manager_integration.py, line 79 https://reviews.apache.org/r/31451/diff/6/?file=880998#file880998line79 Maybe add a TODO not to write to the filesystem? No. It is an integration test. On March 1, 2015, 8:47 p.m., Joe Smith wrote: src/test/python/apache/thermos/observer/test_detector.py, line 1 https://reviews.apache.org/r/31451/diff/6/?file=881004#file881004line1 add apache copywrite header? fixed On March 1, 2015, 8:47 p.m., Joe Smith wrote: src/test/python/apache/thermos/observer/test_detector.py, line 70 https://reviews.apache.org/r/31451/diff/6/?file=881004#file881004line70 putting this in a `setUp` method would remove the need for remembering `on_finished.reset_mock()` right? this would be true if each contextmanager block could live in its own test_... function, but because the state of the ObserverTaskDetector needs to be carried from context to context (e.g. active_tasks=TASK1, then finished_tasks=TASK1 to ensure that on_finished is called, etc) we can't tear down all test state, and instead need to reset the mocks. On March 1, 2015, 8:47 p.m., Joe Smith wrote: src/test/python/apache/thermos/observer/test_detector.py, line 76 https://reviews.apache.org/r/31451/diff/6/?file=881004#file881004line76 I think it's worthy to make this (and others) explicitly: assert on_active.mock_calls == [mock.call(TASK1[0], TASK1[1]] on_active.assert_called_once_with(X) is the same as assert on_active.mock_calls == [mock.call(X)], right? - Brian --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31451/#review74709 --- On Feb. 27, 2015, 11:36 p.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31451/ --- (Updated Feb. 27, 2015, 11:36 p.m.) Review request for Aurora, Joe Smith and Zameer Manji. Bugs: AURORA-1026 https://issues.apache.org/jira/browse/AURORA-1026 Repository: aurora Description --- This creates a new abstraction, the ObserverTaskDetector, which is responsible for managing state transitions for tasks for the observer. Adds some tests and better debug logging. Diffs - src/main/python/apache/aurora/executor/common/BUILD 8cef41d04aff9edc7da7053e05341274a9bd0834 src/main/python/apache/aurora/executor/common/resource_manager.py 08e02e41b581f275f070228bb23c4cf2a0489f9a src/main/python/apache/aurora/executor/thermos_task_runner.py 7a28e3255842e3e13a0866d6ad1bfc4cb64781e1 src/main/python/apache/thermos/bin/thermos.py 0853a9892399824385bee9e72db4c108f46fceda src/main/python/apache/thermos/common/path.py 846f507e2e097fc04fe0098a7250b40fefcfc6e2 src/main/python/apache/thermos/monitoring/disk.py 175ed3af6515e6107e297d91d4e30cbb3034faf7 src/main/python/apache/thermos/monitoring/monitor.py 11423bc1764c8380d8de4ad095c1e2748ebb78f8 src/main/python/apache/thermos/monitoring/resource.py b4cb881c87a09bb90a740f369a7a5fc5d75dbf04 src/main/python/apache/thermos/observer/BUILD ee65f3a46e1d339620e76cadae92c6678fc3510f src/main/python/apache/thermos/observer/bin/BUILD 15a03f74f204f58856f0843b9db05e83b89d1138 src/main/python/apache/thermos/observer/bin/thermos_observer.py effa8c19f963bf2792497f4a06049214ae30dfa5 src/main/python/apache/thermos/observer/detector.py PRE-CREATION src/main/python/apache/thermos/observer/http/file_browser.py 87ef9c8a29689c78a5e39a46cc53e4675e36a381 src/main/python/apache/thermos/observer/observed_task.py f33aecbc8f3c0a461ae3dba66fbd4986f544dc04
Re: Review Request 31451: Port thermos observer to the path detector interface
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31451/#review75074 --- Ship it! Ship It! - Joe Smith On March 3, 2015, 2:58 p.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31451/ --- (Updated March 3, 2015, 2:58 p.m.) Review request for Aurora, Joe Smith and Zameer Manji. Bugs: AURORA-1026 https://issues.apache.org/jira/browse/AURORA-1026 Repository: aurora Description --- This creates a new abstraction, the ObserverTaskDetector, which is responsible for managing state transitions for tasks for the observer. Adds some tests and better debug logging. Diffs - src/main/python/apache/aurora/executor/common/BUILD 8cef41d04aff9edc7da7053e05341274a9bd0834 src/main/python/apache/aurora/executor/common/resource_manager.py 08e02e41b581f275f070228bb23c4cf2a0489f9a src/main/python/apache/aurora/executor/thermos_task_runner.py 7a28e3255842e3e13a0866d6ad1bfc4cb64781e1 src/main/python/apache/thermos/bin/thermos.py 0853a9892399824385bee9e72db4c108f46fceda src/main/python/apache/thermos/common/path.py 846f507e2e097fc04fe0098a7250b40fefcfc6e2 src/main/python/apache/thermos/monitoring/disk.py 175ed3af6515e6107e297d91d4e30cbb3034faf7 src/main/python/apache/thermos/monitoring/monitor.py 11423bc1764c8380d8de4ad095c1e2748ebb78f8 src/main/python/apache/thermos/monitoring/resource.py b4cb881c87a09bb90a740f369a7a5fc5d75dbf04 src/main/python/apache/thermos/observer/BUILD ee65f3a46e1d339620e76cadae92c6678fc3510f src/main/python/apache/thermos/observer/bin/BUILD 15a03f74f204f58856f0843b9db05e83b89d1138 src/main/python/apache/thermos/observer/bin/thermos_observer.py effa8c19f963bf2792497f4a06049214ae30dfa5 src/main/python/apache/thermos/observer/detector.py PRE-CREATION src/main/python/apache/thermos/observer/http/file_browser.py 87ef9c8a29689c78a5e39a46cc53e4675e36a381 src/main/python/apache/thermos/observer/observed_task.py f33aecbc8f3c0a461ae3dba66fbd4986f544dc04 src/main/python/apache/thermos/observer/task_observer.py cd528dcca3f5a330359cf38005f3a1a0329a4886 src/test/python/apache/aurora/executor/BUILD b8dd28c3e64248d0181ad606d4c6de5d915ac706 src/test/python/apache/aurora/executor/common/BUILD 7b73f693d161cfd205435e4acb398f553b92389f src/test/python/apache/aurora/executor/common/test_resource_manager_integration.py 8f288f6115ab52265dfada3f41d81271c55a src/test/python/apache/aurora/executor/test_thermos_executor.py 6cc928edc60d7a0f3b0e668edf68eb81b49cdb22 src/test/python/apache/thermos/bin/test_thermos.py 2d9d33397ae01b31ab011d958f5457311ef7ef20 src/test/python/apache/thermos/core/test_staged_kill.py faa23ae6afcaa443d23c65e5e027902961a0e52b src/test/python/apache/thermos/monitoring/test_resource.py 52d813946379bdc70c40ad079c74b54f60bd4b41 src/test/python/apache/thermos/observer/BUILD PRE-CREATION src/test/python/apache/thermos/observer/test_detector.py PRE-CREATION Diff: https://reviews.apache.org/r/31451/diff/ Testing --- Manually launched observer and ran some thermos tasks. + mba=aurora=; ./pants test src/test/python/apache/thermos/observer/:: Thanks, Brian Wickman
Re: Review Request 31451: Port thermos observer to the path detector interface
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31451/#review75078 --- Ship it! Master (2bf03dc) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 3, 2015, 10:58 p.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31451/ --- (Updated March 3, 2015, 10:58 p.m.) Review request for Aurora, Joe Smith and Zameer Manji. Bugs: AURORA-1026 https://issues.apache.org/jira/browse/AURORA-1026 Repository: aurora Description --- This creates a new abstraction, the ObserverTaskDetector, which is responsible for managing state transitions for tasks for the observer. Adds some tests and better debug logging. Diffs - src/main/python/apache/aurora/executor/common/BUILD 8cef41d04aff9edc7da7053e05341274a9bd0834 src/main/python/apache/aurora/executor/common/resource_manager.py 08e02e41b581f275f070228bb23c4cf2a0489f9a src/main/python/apache/aurora/executor/thermos_task_runner.py 7a28e3255842e3e13a0866d6ad1bfc4cb64781e1 src/main/python/apache/thermos/bin/thermos.py 0853a9892399824385bee9e72db4c108f46fceda src/main/python/apache/thermos/common/path.py 846f507e2e097fc04fe0098a7250b40fefcfc6e2 src/main/python/apache/thermos/monitoring/disk.py 175ed3af6515e6107e297d91d4e30cbb3034faf7 src/main/python/apache/thermos/monitoring/monitor.py 11423bc1764c8380d8de4ad095c1e2748ebb78f8 src/main/python/apache/thermos/monitoring/resource.py b4cb881c87a09bb90a740f369a7a5fc5d75dbf04 src/main/python/apache/thermos/observer/BUILD ee65f3a46e1d339620e76cadae92c6678fc3510f src/main/python/apache/thermos/observer/bin/BUILD 15a03f74f204f58856f0843b9db05e83b89d1138 src/main/python/apache/thermos/observer/bin/thermos_observer.py effa8c19f963bf2792497f4a06049214ae30dfa5 src/main/python/apache/thermos/observer/detector.py PRE-CREATION src/main/python/apache/thermos/observer/http/file_browser.py 87ef9c8a29689c78a5e39a46cc53e4675e36a381 src/main/python/apache/thermos/observer/observed_task.py f33aecbc8f3c0a461ae3dba66fbd4986f544dc04 src/main/python/apache/thermos/observer/task_observer.py cd528dcca3f5a330359cf38005f3a1a0329a4886 src/test/python/apache/aurora/executor/BUILD b8dd28c3e64248d0181ad606d4c6de5d915ac706 src/test/python/apache/aurora/executor/common/BUILD 7b73f693d161cfd205435e4acb398f553b92389f src/test/python/apache/aurora/executor/common/test_resource_manager_integration.py 8f288f6115ab52265dfada3f41d81271c55a src/test/python/apache/aurora/executor/test_thermos_executor.py 6cc928edc60d7a0f3b0e668edf68eb81b49cdb22 src/test/python/apache/thermos/bin/test_thermos.py 2d9d33397ae01b31ab011d958f5457311ef7ef20 src/test/python/apache/thermos/core/test_staged_kill.py faa23ae6afcaa443d23c65e5e027902961a0e52b src/test/python/apache/thermos/monitoring/test_resource.py 52d813946379bdc70c40ad079c74b54f60bd4b41 src/test/python/apache/thermos/observer/BUILD PRE-CREATION src/test/python/apache/thermos/observer/test_detector.py PRE-CREATION Diff: https://reviews.apache.org/r/31451/diff/ Testing --- Manually launched observer and ran some thermos tasks. + mba=aurora=; ./pants test src/test/python/apache/thermos/observer/:: Thanks, Brian Wickman
Re: Review Request 31710: Fix query and timestamp display when running beta-update status.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31710/#review75100 --- Ship it! Master (c0b3701) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 3, 2015, 11:44 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31710/ --- (Updated March 3, 2015, 11:44 p.m.) Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-1165 and AURORA-1166 https://issues.apache.org/jira/browse/AURORA-1165 https://issues.apache.org/jira/browse/AURORA-1166 Repository: aurora Description --- Discovered these while working on AURORA-1157 Diffs - src/main/python/apache/aurora/client/cli/update.py ee8146be0cb6e3f3139a5491c883864950f245bd src/test/python/apache/aurora/client/cli/test_supdate.py 1173650998d397a68a7fabf38201fb9feebf2977 Diff: https://reviews.apache.org/r/31710/diff/ Testing --- Thanks, Bill Farner
Review Request 31714: Suppress empty RPC response logging from the client.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31714/ --- Review request for Aurora and Zameer Manji. Repository: aurora Description --- The client used to log an empty line for every successful RPC. This make the client much less noisy. Diffs - src/main/python/apache/aurora/client/cli/context.py d3113ff693eb87be3ec383f238b204f4f4b411c4 Diff: https://reviews.apache.org/r/31714/diff/ Testing --- Ran end-to-end tests, output is more sane. Thanks, Bill Farner
Re: Review Request 31559: Split thermos cli into pieces. Add custom aurora thermos entry point.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31559/#review75063 --- Ship it! src/main/python/apache/thermos/cli/common.py https://reviews.apache.org/r/31559/#comment121996 shouldn't closing the checkpoint and killing the task runner be in a finally clause? - Zameer Manji On March 3, 2015, 2:14 p.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31559/ --- (Updated March 3, 2015, 2:14 p.m.) Review request for Aurora, Joe Smith and Zameer Manji. Bugs: AURORA-1027 https://issues.apache.org/jira/browse/AURORA-1027 Repository: aurora Description --- This splits src/main/python/apache/thermos/bin/thermos.py into src/main/python/apache/thermos/cli/**. This also adds a path detector registry (AURORA-1027) for the client so that we can easily add a customized aurora entry point for thermos that is capable of finding checkpoint roots in executor sandboxes. Added said entry point in src/main/apache/aurora/tools/thermos.py -- open for other suggestions where this should live (possibly under aurora/executor/bin?) This is a drop-in replacement for the existing thermos binary. The cli/commands/*.py are mostly no-op splits with only a few lines changed in gc, status and tail to get_path_detector. Added a basic testing framework and an example test for simplerun but did not go with full coverage, instead opting to ship AURORA-1027 for now. Diffs - src/main/python/apache/aurora/tools/BUILD PRE-CREATION src/main/python/apache/aurora/tools/thermos.py PRE-CREATION src/main/python/apache/thermos/bin/BUILD 1a002da4f8294b3ecd43ab845eb15bc8ebd12b26 src/main/python/apache/thermos/bin/thermos.py 0853a9892399824385bee9e72db4c108f46fceda src/main/python/apache/thermos/cli/BUILD PRE-CREATION src/main/python/apache/thermos/cli/bin/BUILD PRE-CREATION src/main/python/apache/thermos/cli/bin/thermos.py PRE-CREATION src/main/python/apache/thermos/cli/commands/BUILD PRE-CREATION src/main/python/apache/thermos/cli/commands/gc.py PRE-CREATION src/main/python/apache/thermos/cli/commands/help.py PRE-CREATION src/main/python/apache/thermos/cli/commands/inspect.py PRE-CREATION src/main/python/apache/thermos/cli/commands/kill.py PRE-CREATION src/main/python/apache/thermos/cli/commands/read.py PRE-CREATION src/main/python/apache/thermos/cli/commands/run.py PRE-CREATION src/main/python/apache/thermos/cli/commands/simplerun.py PRE-CREATION src/main/python/apache/thermos/cli/commands/status.py PRE-CREATION src/main/python/apache/thermos/cli/commands/tail.py PRE-CREATION src/main/python/apache/thermos/cli/common.py PRE-CREATION src/main/python/apache/thermos/cli/main.py PRE-CREATION src/test/python/apache/thermos/BUILD df50964851c6be89c646015edf91d07992743497 src/test/python/apache/thermos/bin/BUILD 596aeb200dc55b077ed73fc342b9187045732890 src/test/python/apache/thermos/bin/test_thermos.py 2d9d33397ae01b31ab011d958f5457311ef7ef20 src/test/python/apache/thermos/cli/commands/BUILD PRE-CREATION src/test/python/apache/thermos/cli/commands/test_import.py PRE-CREATION src/test/python/apache/thermos/cli/commands/test_simplerun.py PRE-CREATION src/test/python/apache/thermos/cli/test_common.py PRE-CREATION Diff: https://reviews.apache.org/r/31559/diff/ Testing --- ./pants test src/test/python/apache/thermos/cli:: Thanks, Brian Wickman
Re: Review Request 31653: docs: Add info on granting production quota.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31653/#review75098 --- Ship it! Ship It! - Bill Farner On March 3, 2015, 12:58 a.m., Ricardo Cervera-Navarro wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31653/ --- (Updated March 3, 2015, 12:58 a.m.) Review request for Aurora and Bill Farner. Repository: aurora Description --- docs: Add info on granting production quota. Diffs - docs/configuration-reference.md 0da4a9b4c46843f4f810f9c61fee738568619ec6 docs/configuration-tutorial.md 0825d085b32b67127709111ca5c163a89992c839 Diff: https://reviews.apache.org/r/31653/diff/ Testing --- Thanks, Ricardo Cervera-Navarro
Re: Review Request 31451: Port thermos observer to the path detector interface
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31451/ --- (Updated March 3, 2015, 10:58 p.m.) Review request for Aurora, Joe Smith and Zameer Manji. Changes --- Feedbackses Bugs: AURORA-1026 https://issues.apache.org/jira/browse/AURORA-1026 Repository: aurora Description --- This creates a new abstraction, the ObserverTaskDetector, which is responsible for managing state transitions for tasks for the observer. Adds some tests and better debug logging. Diffs (updated) - src/main/python/apache/aurora/executor/common/BUILD 8cef41d04aff9edc7da7053e05341274a9bd0834 src/main/python/apache/aurora/executor/common/resource_manager.py 08e02e41b581f275f070228bb23c4cf2a0489f9a src/main/python/apache/aurora/executor/thermos_task_runner.py 7a28e3255842e3e13a0866d6ad1bfc4cb64781e1 src/main/python/apache/thermos/bin/thermos.py 0853a9892399824385bee9e72db4c108f46fceda src/main/python/apache/thermos/common/path.py 846f507e2e097fc04fe0098a7250b40fefcfc6e2 src/main/python/apache/thermos/monitoring/disk.py 175ed3af6515e6107e297d91d4e30cbb3034faf7 src/main/python/apache/thermos/monitoring/monitor.py 11423bc1764c8380d8de4ad095c1e2748ebb78f8 src/main/python/apache/thermos/monitoring/resource.py b4cb881c87a09bb90a740f369a7a5fc5d75dbf04 src/main/python/apache/thermos/observer/BUILD ee65f3a46e1d339620e76cadae92c6678fc3510f src/main/python/apache/thermos/observer/bin/BUILD 15a03f74f204f58856f0843b9db05e83b89d1138 src/main/python/apache/thermos/observer/bin/thermos_observer.py effa8c19f963bf2792497f4a06049214ae30dfa5 src/main/python/apache/thermos/observer/detector.py PRE-CREATION src/main/python/apache/thermos/observer/http/file_browser.py 87ef9c8a29689c78a5e39a46cc53e4675e36a381 src/main/python/apache/thermos/observer/observed_task.py f33aecbc8f3c0a461ae3dba66fbd4986f544dc04 src/main/python/apache/thermos/observer/task_observer.py cd528dcca3f5a330359cf38005f3a1a0329a4886 src/test/python/apache/aurora/executor/BUILD b8dd28c3e64248d0181ad606d4c6de5d915ac706 src/test/python/apache/aurora/executor/common/BUILD 7b73f693d161cfd205435e4acb398f553b92389f src/test/python/apache/aurora/executor/common/test_resource_manager_integration.py 8f288f6115ab52265dfada3f41d81271c55a src/test/python/apache/aurora/executor/test_thermos_executor.py 6cc928edc60d7a0f3b0e668edf68eb81b49cdb22 src/test/python/apache/thermos/bin/test_thermos.py 2d9d33397ae01b31ab011d958f5457311ef7ef20 src/test/python/apache/thermos/core/test_staged_kill.py faa23ae6afcaa443d23c65e5e027902961a0e52b src/test/python/apache/thermos/monitoring/test_resource.py 52d813946379bdc70c40ad079c74b54f60bd4b41 src/test/python/apache/thermos/observer/BUILD PRE-CREATION src/test/python/apache/thermos/observer/test_detector.py PRE-CREATION Diff: https://reviews.apache.org/r/31451/diff/ Testing --- Manually launched observer and ran some thermos tasks. + mba=aurora=; ./pants test src/test/python/apache/thermos/observer/:: Thanks, Brian Wickman
Re: Review Request 31710: Fix query and timestamp display when running beta-update status.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31710/#review75087 --- Master (c0b3701) is red with this patch. ./build-support/jenkins/build.sh SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/cli/test_status.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/cli/test_plugins.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/cli/test_update.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/cli/test_task.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/cli/util.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/cli/test_create.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_task_util.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_job_monitor.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_restarter.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_health_check.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_instance_watcher.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_scheduler_mux.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_sla.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_api.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_updater_util.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/__init__.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_updater.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_quota_check.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/api/test_scheduler_client.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/hooks/test_hooked_api.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/aurora/client/hooks/test_non_hooked_api.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/core/test_staged_kill.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/core/test_helper.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/core/test_angry.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/core/test_process.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/core/test_runner_integration.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/core/test_finalization.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/core/test_failure_limit.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/core/test_ephemerals.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/core/test_failing_runner.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/common/test_pathspec.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/common/test_planner.py Everything Looks Good! SUCCESS: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/test/python/apache/thermos/common/test_task_planner.py Everything Looks Good! SUCCESS:
Re: Review Request 31659: Clean up end-to-end tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/#review75090 --- Ship it! Ship It! - Kevin Sweeney On March 3, 2015, 12:55 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31659/ --- (Updated March 3, 2015, 12:55 p.m.) Review request for Aurora, Kevin Sweeney and Maxim Khutornenko. Repository: aurora Description --- Major changes: - applying DRY by not invoking everything via `vagrant ssh` (making tests easier to write and much faster to run) - pulled 'test cases' into individual functions - simplified (maybe?) setup for tests using SSH This is being done in preparation for AURORA-1157. Diffs - src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 0f6423521ad2c85032b4825a5b9793dc522a3b70 src/test/sh/org/apache/aurora/e2e/http/http_example_docker.aurora 00fa2fb6c272fe01700c4696f92713f11d62230b src/test/sh/org/apache/aurora/e2e/http/http_example_docker_updated.aurora d987d637da4061cd7946074243daabee4b9a150f src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 072bbb76707e6696815509f21d50b6b5446241b3 src/test/sh/org/apache/aurora/e2e/test_common.sh b621975bab6f8bbf193e61360654e34337e36943 src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 1d599c3f71b7e07c1cb8bee01a4d3b5cfcf92631 src/test/sh/org/apache/aurora/e2e/test_run.sh ab91b95a60f7b459b95dc7e13a29692badca5ff7 Diff: https://reviews.apache.org/r/31659/diff/ Testing --- Ran the script. Thanks, Bill Farner
Re: Review Request 31652: Returning pending reason for all tasks in a group
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31652/#review75096 --- Ship it! Nice! src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java https://reviews.apache.org/r/31652/#comment122032 s/taskId/groupKey/ src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java https://reviews.apache.org/r/31652/#comment122034 Should this cache be renamed? - Bill Farner On March 3, 2015, 12:58 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31652/ --- (Updated March 3, 2015, 12:58 a.m.) Review request for Aurora, Bill Farner and Zameer Manji. Bugs: AURORA-911 https://issues.apache.org/jira/browse/AURORA-911 Repository: aurora Description --- Modifying `Vetoed` event to broadcast `TaskGroupKey` instead of task ID and storing veto reasons by TaskGroupKey in NearestFit. Depends on https://reviews.apache.org/r/31646/. Diffs - src/main/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilter.java edaf2f4f845544c13b2fb9bc77c34f6e6d96fb48 src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java c103472b9404df1c690b3a6019d64d42e15f2fed src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 53582c63ddee23e643bd4654cad2bef75dfba36d src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 13520eb5846022ed0b43b402096fe02565103aa9 src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java ab7817f929bbcc96a6046043ea17921a388fdb9f src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02 src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java ce5a62650cebab9a53743460f5a5119f62efec1c Diff: https://reviews.apache.org/r/31652/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 31710: Fix query and timestamp display when running beta-update status.
On March 3, 2015, 11:48 p.m., Maxim Khutornenko wrote: src/main/python/apache/aurora/client/cli/update.py, line 302 https://reviews.apache.org/r/31710/diff/1/?file=883800#file883800line302 You may want to extract an internal method to do this, e.g.: ``` def timestamp(time_msec): return time.ctime(time_msec / 1000) ``` Done. - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31710/#review75089 --- On March 3, 2015, 11:44 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31710/ --- (Updated March 3, 2015, 11:44 p.m.) Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-1165 and AURORA-1166 https://issues.apache.org/jira/browse/AURORA-1165 https://issues.apache.org/jira/browse/AURORA-1166 Repository: aurora Description --- Discovered these while working on AURORA-1157 Diffs - src/main/python/apache/aurora/client/cli/update.py ee8146be0cb6e3f3139a5491c883864950f245bd src/test/python/apache/aurora/client/cli/test_supdate.py 1173650998d397a68a7fabf38201fb9feebf2977 Diff: https://reviews.apache.org/r/31710/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 31710: Fix query and timestamp display when running beta-update status.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31710/ --- (Updated March 4, 2015, 12:19 a.m.) Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-1165 and AURORA-1166 https://issues.apache.org/jira/browse/AURORA-1165 https://issues.apache.org/jira/browse/AURORA-1166 Repository: aurora Description --- Discovered these while working on AURORA-1157 Diffs (updated) - src/main/python/apache/aurora/client/cli/update.py ee8146be0cb6e3f3139a5491c883864950f245bd src/test/python/apache/aurora/client/cli/test_supdate.py 1173650998d397a68a7fabf38201fb9feebf2977 Diff: https://reviews.apache.org/r/31710/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 31451: Port thermos observer to the path detector interface
On March 1, 2015, 12:47 p.m., Joe Smith wrote: src/test/python/apache/thermos/observer/test_detector.py, line 76 https://reviews.apache.org/r/31451/diff/6/?file=881004#file881004line76 I think it's worthy to make this (and others) explicitly: assert on_active.mock_calls == [mock.call(TASK1[0], TASK1[1]] Brian Wickman wrote: on_active.assert_called_once_with(X) is the same as assert on_active.mock_calls == [mock.call(X)], right? Hm.. not sure what I was going for here, maybe I missed the *? Anyways, you're right and +1. - Joe --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31451/#review74709 --- On March 3, 2015, 2:58 p.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31451/ --- (Updated March 3, 2015, 2:58 p.m.) Review request for Aurora, Joe Smith and Zameer Manji. Bugs: AURORA-1026 https://issues.apache.org/jira/browse/AURORA-1026 Repository: aurora Description --- This creates a new abstraction, the ObserverTaskDetector, which is responsible for managing state transitions for tasks for the observer. Adds some tests and better debug logging. Diffs - src/main/python/apache/aurora/executor/common/BUILD 8cef41d04aff9edc7da7053e05341274a9bd0834 src/main/python/apache/aurora/executor/common/resource_manager.py 08e02e41b581f275f070228bb23c4cf2a0489f9a src/main/python/apache/aurora/executor/thermos_task_runner.py 7a28e3255842e3e13a0866d6ad1bfc4cb64781e1 src/main/python/apache/thermos/bin/thermos.py 0853a9892399824385bee9e72db4c108f46fceda src/main/python/apache/thermos/common/path.py 846f507e2e097fc04fe0098a7250b40fefcfc6e2 src/main/python/apache/thermos/monitoring/disk.py 175ed3af6515e6107e297d91d4e30cbb3034faf7 src/main/python/apache/thermos/monitoring/monitor.py 11423bc1764c8380d8de4ad095c1e2748ebb78f8 src/main/python/apache/thermos/monitoring/resource.py b4cb881c87a09bb90a740f369a7a5fc5d75dbf04 src/main/python/apache/thermos/observer/BUILD ee65f3a46e1d339620e76cadae92c6678fc3510f src/main/python/apache/thermos/observer/bin/BUILD 15a03f74f204f58856f0843b9db05e83b89d1138 src/main/python/apache/thermos/observer/bin/thermos_observer.py effa8c19f963bf2792497f4a06049214ae30dfa5 src/main/python/apache/thermos/observer/detector.py PRE-CREATION src/main/python/apache/thermos/observer/http/file_browser.py 87ef9c8a29689c78a5e39a46cc53e4675e36a381 src/main/python/apache/thermos/observer/observed_task.py f33aecbc8f3c0a461ae3dba66fbd4986f544dc04 src/main/python/apache/thermos/observer/task_observer.py cd528dcca3f5a330359cf38005f3a1a0329a4886 src/test/python/apache/aurora/executor/BUILD b8dd28c3e64248d0181ad606d4c6de5d915ac706 src/test/python/apache/aurora/executor/common/BUILD 7b73f693d161cfd205435e4acb398f553b92389f src/test/python/apache/aurora/executor/common/test_resource_manager_integration.py 8f288f6115ab52265dfada3f41d81271c55a src/test/python/apache/aurora/executor/test_thermos_executor.py 6cc928edc60d7a0f3b0e668edf68eb81b49cdb22 src/test/python/apache/thermos/bin/test_thermos.py 2d9d33397ae01b31ab011d958f5457311ef7ef20 src/test/python/apache/thermos/core/test_staged_kill.py faa23ae6afcaa443d23c65e5e027902961a0e52b src/test/python/apache/thermos/monitoring/test_resource.py 52d813946379bdc70c40ad079c74b54f60bd4b41 src/test/python/apache/thermos/observer/BUILD PRE-CREATION src/test/python/apache/thermos/observer/test_detector.py PRE-CREATION Diff: https://reviews.apache.org/r/31451/diff/ Testing --- Manually launched observer and ran some thermos tasks. + mba=aurora=; ./pants test src/test/python/apache/thermos/observer/:: Thanks, Brian Wickman
Re: Review Request 31559: Split thermos cli into pieces. Add custom aurora thermos entry point.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31559/#review75065 --- Master (2bf03dc) is red with this patch. ./build-support/jenkins/build.sh src.test.python.apache.aurora.executor.common.directory_sandbox . SUCCESS src.test.python.apache.aurora.executor.common.executor_detector . SUCCESS src.test.python.apache.aurora.executor.common.executor_timeout . SUCCESS src.test.python.apache.aurora.executor.common.health_checker . SUCCESS src.test.python.apache.aurora.executor.common.kill_manager . SUCCESS src.test.python.apache.aurora.executor.common.path_detector . SUCCESS src.test.python.apache.aurora.executor.common.resource_manager_integration . FAILURE src.test.python.apache.aurora.executor.common.status_checker . SUCCESS src.test.python.apache.aurora.executor.common.task_info . SUCCESS src.test.python.apache.aurora.executor.executor_base . SUCCESS src.test.python.apache.aurora.executor.executor_vars . SUCCESS src.test.python.apache.aurora.executor.gc_executor . SUCCESS src.test.python.apache.aurora.executor.status_manager . SUCCESS src.test.python.apache.aurora.executor.thermos_executor . SUCCESS src.test.python.apache.aurora.executor.thermos_task_runner . SUCCESS src.test.python.apache.thermos.cli.commands.commands . SUCCESS src.test.python.apache.thermos.cli.common . SUCCESS src.test.python.apache.thermos.cli.main . SUCCESS src.test.python.apache.thermos.common.test_pathspec . SUCCESS src.test.python.apache.thermos.common.test_planner . SUCCESS src.test.python.apache.thermos.common.test_task_planner . SUCCESS src.test.python.apache.thermos.config.test_schema . SUCCESS src.test.python.apache.thermos.core.test_angry . SUCCESS src.test.python.apache.thermos.core.test_ephemerals . SUCCESS src.test.python.apache.thermos.core.test_failing_runner . SUCCESS src.test.python.apache.thermos.core.test_failure_limit . SUCCESS src.test.python.apache.thermos.core.test_finalization . SUCCESS src.test.python.apache.thermos.core.test_helper . SUCCESS src.test.python.apache.thermos.core.test_process . SUCCESS src.test.python.apache.thermos.core.test_runner_integration . SUCCESS src.test.python.apache.thermos.core.test_staged_kill . SUCCESS src.test.python.apache.thermos.monitoring.test_detector . SUCCESS src.test.python.apache.thermos.monitoring.test_disk . SUCCESS src.test.python.apache.thermos.monitoring.test_garbage . SUCCESS src.test.python.apache.thermos.monitoring.test_resource . SUCCESS FAILURE [31m FAILURE[0m I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 3, 2015, 10:14 p.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31559/ --- (Updated March 3, 2015, 10:14 p.m.) Review request for Aurora, Joe Smith and Zameer Manji.
Re: Review Request 31451: Port thermos observer to the path detector interface
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31451/#review75082 --- Ship it! Ship It! - Zameer Manji On March 3, 2015, 2:58 p.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31451/ --- (Updated March 3, 2015, 2:58 p.m.) Review request for Aurora, Joe Smith and Zameer Manji. Bugs: AURORA-1026 https://issues.apache.org/jira/browse/AURORA-1026 Repository: aurora Description --- This creates a new abstraction, the ObserverTaskDetector, which is responsible for managing state transitions for tasks for the observer. Adds some tests and better debug logging. Diffs - src/main/python/apache/aurora/executor/common/BUILD 8cef41d04aff9edc7da7053e05341274a9bd0834 src/main/python/apache/aurora/executor/common/resource_manager.py 08e02e41b581f275f070228bb23c4cf2a0489f9a src/main/python/apache/aurora/executor/thermos_task_runner.py 7a28e3255842e3e13a0866d6ad1bfc4cb64781e1 src/main/python/apache/thermos/bin/thermos.py 0853a9892399824385bee9e72db4c108f46fceda src/main/python/apache/thermos/common/path.py 846f507e2e097fc04fe0098a7250b40fefcfc6e2 src/main/python/apache/thermos/monitoring/disk.py 175ed3af6515e6107e297d91d4e30cbb3034faf7 src/main/python/apache/thermos/monitoring/monitor.py 11423bc1764c8380d8de4ad095c1e2748ebb78f8 src/main/python/apache/thermos/monitoring/resource.py b4cb881c87a09bb90a740f369a7a5fc5d75dbf04 src/main/python/apache/thermos/observer/BUILD ee65f3a46e1d339620e76cadae92c6678fc3510f src/main/python/apache/thermos/observer/bin/BUILD 15a03f74f204f58856f0843b9db05e83b89d1138 src/main/python/apache/thermos/observer/bin/thermos_observer.py effa8c19f963bf2792497f4a06049214ae30dfa5 src/main/python/apache/thermos/observer/detector.py PRE-CREATION src/main/python/apache/thermos/observer/http/file_browser.py 87ef9c8a29689c78a5e39a46cc53e4675e36a381 src/main/python/apache/thermos/observer/observed_task.py f33aecbc8f3c0a461ae3dba66fbd4986f544dc04 src/main/python/apache/thermos/observer/task_observer.py cd528dcca3f5a330359cf38005f3a1a0329a4886 src/test/python/apache/aurora/executor/BUILD b8dd28c3e64248d0181ad606d4c6de5d915ac706 src/test/python/apache/aurora/executor/common/BUILD 7b73f693d161cfd205435e4acb398f553b92389f src/test/python/apache/aurora/executor/common/test_resource_manager_integration.py 8f288f6115ab52265dfada3f41d81271c55a src/test/python/apache/aurora/executor/test_thermos_executor.py 6cc928edc60d7a0f3b0e668edf68eb81b49cdb22 src/test/python/apache/thermos/bin/test_thermos.py 2d9d33397ae01b31ab011d958f5457311ef7ef20 src/test/python/apache/thermos/core/test_staged_kill.py faa23ae6afcaa443d23c65e5e027902961a0e52b src/test/python/apache/thermos/monitoring/test_resource.py 52d813946379bdc70c40ad079c74b54f60bd4b41 src/test/python/apache/thermos/observer/BUILD PRE-CREATION src/test/python/apache/thermos/observer/test_detector.py PRE-CREATION Diff: https://reviews.apache.org/r/31451/diff/ Testing --- Manually launched observer and ran some thermos tasks. + mba=aurora=; ./pants test src/test/python/apache/thermos/observer/:: Thanks, Brian Wickman
Review Request 31710: Fix query and timestamp display when running beta-update status.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31710/ --- Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-1165 and AURORA-1166 https://issues.apache.org/jira/browse/AURORA-1165 https://issues.apache.org/jira/browse/AURORA-1166 Repository: aurora Description --- Discovered these while working on AURORA-1157 Diffs - src/main/python/apache/aurora/client/cli/update.py ee8146be0cb6e3f3139a5491c883864950f245bd src/test/python/apache/aurora/client/cli/test_supdate.py 1173650998d397a68a7fabf38201fb9feebf2977 Diff: https://reviews.apache.org/r/31710/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 31646: Moving GroupKey to scheduler.base.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31646/#review75093 --- src/main/java/org/apache/aurora/scheduler/base/TaskGroupKey.java https://reviews.apache.org/r/31646/#comment122028 s/Identifying/Identifer for/ It's not until reading this diff that i wonder why TaskGroupKey/GroupKey even exists. The best justification i have is separation of concerns from how a task is configured and how it is identified. If you agree, it would be nice to document that here. src/main/java/org/apache/aurora/scheduler/base/TaskGroupKey.java https://reviews.apache.org/r/31646/#comment122031 Why the proxy method to the constructor? - Bill Farner On March 3, 2015, 12:07 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31646/ --- (Updated March 3, 2015, 12:07 a.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-911 https://issues.apache.org/jira/browse/AURORA-911 Repository: aurora Description --- Simple IDE-driven refactoring in preparation for the AURORA-911 fix. Diffs - src/main/java/org/apache/aurora/scheduler/async/OfferManager.java 7d3713972f1df54da4c8cbaae0fca59a8a6f3d77 src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java e5067e18af7c477d2927d83eacec063f3dec575a src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java 8cd6c966c9aca2e869311787fb5d5caba7439d3e src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java d0fe3e133cbec2418f31160bf8ab8adaa45bb958 src/main/java/org/apache/aurora/scheduler/base/TaskGroupKey.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/async/OfferManagerImplTest.java 0cbc71d50612323aa4d8acc33e74b879c0a76aff src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 4ee13c8e5d46ba863f4d9871884c7d494d07758d src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 87bc531d2a72f21c36ddd0c1bd3b2367826cc422 Diff: https://reviews.apache.org/r/31646/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 31646: Moving GroupKey to scheduler.base.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31646/#review75095 --- Ship it! Ship It! - Bill Farner On March 3, 2015, 12:07 a.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31646/ --- (Updated March 3, 2015, 12:07 a.m.) Review request for Aurora and Bill Farner. Bugs: AURORA-911 https://issues.apache.org/jira/browse/AURORA-911 Repository: aurora Description --- Simple IDE-driven refactoring in preparation for the AURORA-911 fix. Diffs - src/main/java/org/apache/aurora/scheduler/async/OfferManager.java 7d3713972f1df54da4c8cbaae0fca59a8a6f3d77 src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java e5067e18af7c477d2927d83eacec063f3dec575a src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java 8cd6c966c9aca2e869311787fb5d5caba7439d3e src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java d0fe3e133cbec2418f31160bf8ab8adaa45bb958 src/main/java/org/apache/aurora/scheduler/base/TaskGroupKey.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/async/OfferManagerImplTest.java 0cbc71d50612323aa4d8acc33e74b879c0a76aff src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 4ee13c8e5d46ba863f4d9871884c7d494d07758d src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 87bc531d2a72f21c36ddd0c1bd3b2367826cc422 Diff: https://reviews.apache.org/r/31646/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 31714: Suppress empty RPC response logging from the client.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31714/#review75123 --- Ship it! Ship It! - Zameer Manji On March 3, 2015, 4:59 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31714/ --- (Updated March 3, 2015, 4:59 p.m.) Review request for Aurora and Zameer Manji. Repository: aurora Description --- The client used to log an empty line for every successful RPC. This make the client much less noisy. Diffs - src/main/python/apache/aurora/client/cli/context.py d3113ff693eb87be3ec383f238b204f4f4b411c4 Diff: https://reviews.apache.org/r/31714/diff/ Testing --- Ran end-to-end tests, output is more sane. Thanks, Bill Farner
Re: Review Request 31714: Suppress empty RPC response logging from the client.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31714/#review75126 --- Master (270faf7) is green with this patch. ./build-support/jenkins/build.sh However, it appears that it might lack test coverage. I will refresh this build result if you post a review containing @ReviewBot retry - Aurora ReviewBot On March 4, 2015, 12:59 a.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31714/ --- (Updated March 4, 2015, 12:59 a.m.) Review request for Aurora and Zameer Manji. Repository: aurora Description --- The client used to log an empty line for every successful RPC. This make the client much less noisy. Diffs - src/main/python/apache/aurora/client/cli/context.py d3113ff693eb87be3ec383f238b204f4f4b411c4 Diff: https://reviews.apache.org/r/31714/diff/ Testing --- Ran end-to-end tests, output is more sane. Thanks, Bill Farner
Re: Review Request 31559: Split thermos cli into pieces. Add custom aurora thermos entry point.
On March 3, 2015, 10:31 p.m., Zameer Manji wrote: src/main/python/apache/thermos/cli/common.py, line 101 https://reviews.apache.org/r/31559/diff/6/?file=883670#file883670line101 shouldn't closing the checkpoint and killing the task runner be in a finally clause? No, it's ^C that should indicate whether or not we should kill the task. If the task exits normally, .close_ckpt/.kill will fail since the checkpoint is already closed and the task is in a terminal state. - Brian --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31559/#review75063 --- On March 3, 2015, 10:14 p.m., Brian Wickman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31559/ --- (Updated March 3, 2015, 10:14 p.m.) Review request for Aurora, Joe Smith and Zameer Manji. Bugs: AURORA-1027 https://issues.apache.org/jira/browse/AURORA-1027 Repository: aurora Description --- This splits src/main/python/apache/thermos/bin/thermos.py into src/main/python/apache/thermos/cli/**. This also adds a path detector registry (AURORA-1027) for the client so that we can easily add a customized aurora entry point for thermos that is capable of finding checkpoint roots in executor sandboxes. Added said entry point in src/main/apache/aurora/tools/thermos.py -- open for other suggestions where this should live (possibly under aurora/executor/bin?) This is a drop-in replacement for the existing thermos binary. The cli/commands/*.py are mostly no-op splits with only a few lines changed in gc, status and tail to get_path_detector. Added a basic testing framework and an example test for simplerun but did not go with full coverage, instead opting to ship AURORA-1027 for now. Diffs - src/main/python/apache/aurora/tools/BUILD PRE-CREATION src/main/python/apache/aurora/tools/thermos.py PRE-CREATION src/main/python/apache/thermos/bin/BUILD 1a002da4f8294b3ecd43ab845eb15bc8ebd12b26 src/main/python/apache/thermos/bin/thermos.py 0853a9892399824385bee9e72db4c108f46fceda src/main/python/apache/thermos/cli/BUILD PRE-CREATION src/main/python/apache/thermos/cli/bin/BUILD PRE-CREATION src/main/python/apache/thermos/cli/bin/thermos.py PRE-CREATION src/main/python/apache/thermos/cli/commands/BUILD PRE-CREATION src/main/python/apache/thermos/cli/commands/gc.py PRE-CREATION src/main/python/apache/thermos/cli/commands/help.py PRE-CREATION src/main/python/apache/thermos/cli/commands/inspect.py PRE-CREATION src/main/python/apache/thermos/cli/commands/kill.py PRE-CREATION src/main/python/apache/thermos/cli/commands/read.py PRE-CREATION src/main/python/apache/thermos/cli/commands/run.py PRE-CREATION src/main/python/apache/thermos/cli/commands/simplerun.py PRE-CREATION src/main/python/apache/thermos/cli/commands/status.py PRE-CREATION src/main/python/apache/thermos/cli/commands/tail.py PRE-CREATION src/main/python/apache/thermos/cli/common.py PRE-CREATION src/main/python/apache/thermos/cli/main.py PRE-CREATION src/test/python/apache/thermos/BUILD df50964851c6be89c646015edf91d07992743497 src/test/python/apache/thermos/bin/BUILD 596aeb200dc55b077ed73fc342b9187045732890 src/test/python/apache/thermos/bin/test_thermos.py 2d9d33397ae01b31ab011d958f5457311ef7ef20 src/test/python/apache/thermos/cli/commands/BUILD PRE-CREATION src/test/python/apache/thermos/cli/commands/test_import.py PRE-CREATION src/test/python/apache/thermos/cli/commands/test_simplerun.py PRE-CREATION src/test/python/apache/thermos/cli/test_common.py PRE-CREATION Diff: https://reviews.apache.org/r/31559/diff/ Testing --- ./pants test src/test/python/apache/thermos/cli:: Thanks, Brian Wickman
Re: Review Request 31710: Fix query and timestamp display when running beta-update status.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31710/#review75083 --- src/test/python/apache/aurora/client/cli/test_supdate.py https://reviews.apache.org/r/31710/#comment122016 For the calls like this you can do `mock_api.pause_job_update.assert_called_once_with(self.TEST_JOBKEY)` instead of comparing against an array of call objects. - Zameer Manji On March 3, 2015, 3:16 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31710/ --- (Updated March 3, 2015, 3:16 p.m.) Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-1165 and AURORA-1166 https://issues.apache.org/jira/browse/AURORA-1165 https://issues.apache.org/jira/browse/AURORA-1166 Repository: aurora Description --- Discovered these while working on AURORA-1157 Diffs - src/main/python/apache/aurora/client/cli/update.py ee8146be0cb6e3f3139a5491c883864950f245bd src/test/python/apache/aurora/client/cli/test_supdate.py 1173650998d397a68a7fabf38201fb9feebf2977 Diff: https://reviews.apache.org/r/31710/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 31710: Fix query and timestamp display when running beta-update status.
On March 3, 2015, 11:21 p.m., Zameer Manji wrote:
src/test/python/apache/aurora/client/cli/test_supdate.py, line 211
https://reviews.apache.org/r/31710/diff/1/?file=883801#file883801line211
For the calls like this you can do
`mock_api.pause_job_update.assert_called_once_with(self.TEST_JOBKEY)`
instead of comparing against an array of call objects.
In case the diff didn't make it clear, i think we should settle on `mock_calls`
for uniformity. Additionally, i think the error messages from asserting on
`mock_calls` is better:
```
_mock_self = MagicMock name='ctime' id='140532173693328', args = (1,)
kwargs = {}, self = MagicMock name='ctime' id='140532173693328'
msg = 'Expected to be called once. Called 9 times.'
???
E AssertionError: Expected to be called once. Called 9 times.
```
vs
```
assert self.mock_ctime.mock_calls == [call(1)]
E AssertionError: assert [call(1),\n ca...(8),\n call(9)] == [call(1)]
E Left contains more items, first extra item: call(2)
E Full diff:
E - [call(1),
E ? ^
E + [call(1)]
E ? ^
E - call(2),
E - call(3),
E - call(4),
E - call(5),
E - call(6),
E - call(7),
E - call(8),
E - call(9)]
```
Moreover, `assert x.mock_calls == [...]` is not vulnerable to typos. In this
test, the first typoed line passes, but the second fails.
```
self.mock_ctime.assret_called_once_with(1)
assert self.mock_ctime.mock_clls == [call(n) for n in range(1, 10)]
```
- Bill
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31710/#review75083
---
On March 3, 2015, 11:16 p.m., Bill Farner wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31710/
---
(Updated March 3, 2015, 11:16 p.m.)
Review request for Aurora and Maxim Khutornenko.
Bugs: AURORA-1165 and AURORA-1166
https://issues.apache.org/jira/browse/AURORA-1165
https://issues.apache.org/jira/browse/AURORA-1166
Repository: aurora
Description
---
Discovered these while working on AURORA-1157
Diffs
-
src/main/python/apache/aurora/client/cli/update.py
ee8146be0cb6e3f3139a5491c883864950f245bd
src/test/python/apache/aurora/client/cli/test_supdate.py
1173650998d397a68a7fabf38201fb9feebf2977
Diff: https://reviews.apache.org/r/31710/diff/
Testing
---
Thanks,
Bill Farner
Re: Review Request 31710: Fix query and timestamp display when running beta-update status.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31710/#review75089 --- Ship it! src/main/python/apache/aurora/client/cli/update.py https://reviews.apache.org/r/31710/#comment122025 You may want to extract an internal method to do this, e.g.: ``` def timestamp(time_msec): return time.ctime(time_msec / 1000) ``` - Maxim Khutornenko On March 3, 2015, 11:44 p.m., Bill Farner wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31710/ --- (Updated March 3, 2015, 11:44 p.m.) Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-1165 and AURORA-1166 https://issues.apache.org/jira/browse/AURORA-1165 https://issues.apache.org/jira/browse/AURORA-1166 Repository: aurora Description --- Discovered these while working on AURORA-1157 Diffs - src/main/python/apache/aurora/client/cli/update.py ee8146be0cb6e3f3139a5491c883864950f245bd src/test/python/apache/aurora/client/cli/test_supdate.py 1173650998d397a68a7fabf38201fb9feebf2977 Diff: https://reviews.apache.org/r/31710/diff/ Testing --- Thanks, Bill Farner
Re: Review Request 31652: Returning pending reason for all tasks in a group
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31652/#review75099 --- Ship it! Ship It! - Zameer Manji On March 2, 2015, 4:58 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31652/ --- (Updated March 2, 2015, 4:58 p.m.) Review request for Aurora, Bill Farner and Zameer Manji. Bugs: AURORA-911 https://issues.apache.org/jira/browse/AURORA-911 Repository: aurora Description --- Modifying `Vetoed` event to broadcast `TaskGroupKey` instead of task ID and storing veto reasons by TaskGroupKey in NearestFit. Depends on https://reviews.apache.org/r/31646/. Diffs - src/main/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilter.java edaf2f4f845544c13b2fb9bc77c34f6e6d96fb48 src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java c103472b9404df1c690b3a6019d64d42e15f2fed src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java 53582c63ddee23e643bd4654cad2bef75dfba36d src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 13520eb5846022ed0b43b402096fe02565103aa9 src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java ab7817f929bbcc96a6046043ea17921a388fdb9f src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02 src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java ce5a62650cebab9a53743460f5a5119f62efec1c Diff: https://reviews.apache.org/r/31652/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Re: Review Request 31525: Improving NearestFit reporting accuracy.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31525/#review75103 --- Ship it! Ship It! - Bill Farner On March 2, 2015, 11:40 p.m., Maxim Khutornenko wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31525/ --- (Updated March 2, 2015, 11:40 p.m.) Review request for Aurora, Kevin Sweeney and Bill Farner. Bugs: AURORA-1159 https://issues.apache.org/jira/browse/AURORA-1159 Repository: aurora Description --- Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with break early rule in SchedulingFilter, this ensures heavier vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance. Diffs - src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963 src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147 src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424 src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02 Diff: https://reviews.apache.org/r/31525/diff/ Testing --- ./gradlew -Pq build Thanks, Maxim Khutornenko
Review Request 31716: Add an end-to-end test for scheduler-driven job updates.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31716/ --- Review request for Aurora and Maxim Khutornenko. Bugs: AURORA-1157 https://issues.apache.org/jira/browse/AURORA-1157 Repository: aurora Description --- Add an end-to-end test for scheduler-driven job updates. Diffs - src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 6a75528aeb62c119e4f8ab602d0e2081dc07fe49 Diff: https://reviews.apache.org/r/31716/diff/ Testing --- Ran end-to-end tests. Thanks, Bill Farner
