Re: Review Request 20950: Moving kill wait to the client (Part 1: client changes)

2014-05-01 Thread Brian Wickman 


> On May 1, 2014, 8:59 p.m., Brian Wickman wrote:
> > src/main/python/apache/aurora/client/api/job_monitor.py, line 81
> > 
> >
> > >=
> 
> Maxim Khutornenko wrote:
> It's safe as is but since Mark asked about this too changing it for 
> better readability.

it's not safe.  you assume that min interval and max interval are powers of 2 
of each other.  plus then you run into python LOLisms like this

mba=aurora=; python
Python 2.6.9 (unknown, Apr 15 2014, 11:11:58) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 1.1
1.1001
>>> 3 * 1.1 == 3.3
False


- Brian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20950/#review41947
---


On May 1, 2014, 10:21 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20950/
> ---
> 
> (Updated May 1, 2014, 10:21 p.m.)
> 
> 
> Review request for Aurora, Mark Chu-Carroll and Brian Wickman.
> 
> 
> Bugs: AURORA-370
> https://issues.apache.org/jira/browse/AURORA-370
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Migration plan:
> 1. Prepare client to poll for the status of tasks being killed. This is 
> mostly a noop at this point while the scheduler wait with a shorter timeout 
> is still in place.
> 2. Cut off scheduler killTasks wait thus relying on the client poll (coming 
> next).
> 
> This RB addresses client changes:
> - Repurposing JobMonitor to poll for all job tasks regardless of their 
> arrival time.
> - Introducing a timeout cap into JobMonitor to interrupt wait upon reaching 
> the max_poll_interval.
> - Setting the JobMonitor max_poll_interval to be higher than the scheduler 
> killTasks backoff to ensure smooth migration.
> - Added wait_util calls into kill/killall/updater calls.
> 
> Once this ticket is closed, the next step would be to make kill/killall truly 
> async: AURORA-371
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/api/BUILD 
> 32097d220a6f8a58555260ce081ace0c58b9fa99 
>   src/main/python/apache/aurora/client/api/job_monitor.py 
> b694ef62ba708c4b67c5cc931b0edfd93702027f 
>   src/main/python/apache/aurora/client/api/updater.py 
> 0acf45034db124ed369d0bcb8095fe5545605eed 
>   src/main/python/apache/aurora/client/cli/jobs.py 
> 782b34800def55c54b67ca5f13da66c4997a0777 
>   src/main/python/apache/aurora/client/cli/task.py 
> 62747ed3e3f5f6ecd9b9c46f6f175c634c068358 
>   src/main/python/apache/aurora/client/commands/core.py 
> 39190e06f3eac1ab4dabf48055418bc383be14ed 
>   src/test/python/apache/aurora/client/api/BUILD 
> dd9b79764db0bbc3a11c51051e43b93a9b5d370a 
>   src/test/python/apache/aurora/client/api/test_job_monitor.py 
> 32609e4e97afc7b8b164f3db438c60efd0ebbe11 
>   src/test/python/apache/aurora/client/api/test_updater.py 
> e7eb1e783baaabe9437e55ecb848f6a42de13dd7 
>   src/test/python/apache/aurora/client/cli/test_command_hooks.py 
> 7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 
>   src/test/python/apache/aurora/client/cli/test_create.py 
> 875573e2ee534ea50da71fac2174ecc8877a714d 
>   src/test/python/apache/aurora/client/cli/test_kill.py 
> cf5df648206e0cc234d603c1fd56dc7ab1311fa9 
>   src/test/python/apache/aurora/client/cli/test_plugins.py 
> 2dab749d1dcba99a1e3c792c8f26ee86441673d1 
>   src/test/python/apache/aurora/client/cli/test_update.py 
> cf077e821ebe32104b3e1345014f5c7b07c44c34 
>   src/test/python/apache/aurora/client/cli/util.py 
> 2225ab071dff72be547c43f66ffdc890171c3288 
>   src/test/python/apache/aurora/client/commands/test_create.py 
> e0ecb523db4ce3b1901adcc96a2fd5ce8184f621 
>   src/test/python/apache/aurora/client/commands/test_kill.py 
> db820a543b0ea11e5a40cc2a00209aa8fa6186c9 
>   src/test/python/apache/aurora/client/commands/test_update.py 
> 6e145db9ff8213c8f098b53b9a9668532d664249 
> 
> Diff: https://reviews.apache.org/r/20950/diff/
> 
> 
> Testing
> ---
> 
> ./pants src/test/python:all
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 20950: Moving kill wait to the client (Part 1: client changes)

2014-05-01 Thread Maxim Khutornenko 


> On May 1, 2014, 8:59 p.m., Brian Wickman wrote:
> > src/main/python/apache/aurora/client/api/job_monitor.py, line 81
> > 
> >
> > >=

It's safe as is but since Mark asked about this too changing it for better 
readability.


> On May 1, 2014, 8:59 p.m., Brian Wickman wrote:
> > src/main/python/apache/aurora/client/api/job_monitor.py, line 44
> > 
> >
> > i'm fine with tuning the min_poll_interval and max_poll_interval but 
> > possibly make them configurable via __init__, ie. __init__(..., 
> > min_poll_interval=MIN_POLL_INTERVAL, max_poll_interval=MAX_POLL_INTERVAL)

Sure, works for me.


> On May 1, 2014, 8:59 p.m., Brian Wickman wrote:
> > src/main/python/apache/aurora/client/api/job_monitor.py, line 56
> > 
> >
> > as a library this should probably be log.error

Done.


> On May 1, 2014, 8:59 p.m., Brian Wickman wrote:
> > src/main/python/apache/aurora/client/api/job_monitor.py, lines 74-75
> > 
> >
> > update the docstring to reflect new return behavior

Done.


> On May 1, 2014, 8:59 p.m., Brian Wickman wrote:
> > src/main/python/apache/aurora/client/api/job_monitor.py, lines 76-77
> > 
> >
> > i don't like the idea of wait_until having side effects -- this 
> > effectively renders it not thread safe.
> > 
> > can you clone the query and update instanceIds on the cloned query, 
> > then change iter_query to take a query parameter?

Great catch. Fixed.


> On May 1, 2014, 8:59 p.m., Brian Wickman wrote:
> > src/main/python/apache/aurora/client/api/updater.py, line 284
> > 
> >
> > pass the kwarg so that this is more self-documenting, i.e. 
> > with_timeout=True

Done.


> On May 1, 2014, 8:59 p.m., Brian Wickman wrote:
> > src/main/python/apache/aurora/client/commands/core.py, lines 101-102
> > 
> >
> > weird indent.
> > 
> > also this should use die('Tasks were not killed in time.')

Done and done.


- Maxim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20950/#review41947
---


On May 1, 2014, 8:42 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20950/
> ---
> 
> (Updated May 1, 2014, 8:42 p.m.)
> 
> 
> Review request for Aurora, Mark Chu-Carroll and Brian Wickman.
> 
> 
> Bugs: AURORA-370
> https://issues.apache.org/jira/browse/AURORA-370
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Migration plan:
> 1. Prepare client to poll for the status of tasks being killed. This is 
> mostly a noop at this point while the scheduler wait with a shorter timeout 
> is still in place.
> 2. Cut off scheduler killTasks wait thus relying on the client poll (coming 
> next).
> 
> This RB addresses client changes:
> - Repurposing JobMonitor to poll for all job tasks regardless of their 
> arrival time.
> - Introducing a timeout cap into JobMonitor to interrupt wait upon reaching 
> the max_poll_interval.
> - Setting the JobMonitor max_poll_interval to be higher than the scheduler 
> killTasks backoff to ensure smooth migration.
> - Added wait_util calls into kill/killall/updater calls.
> 
> Once this ticket is closed, the next step would be to make kill/killall truly 
> async: AURORA-371
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/api/BUILD 
> 32097d220a6f8a58555260ce081ace0c58b9fa99 
>   src/main/python/apache/aurora/client/api/job_monitor.py 
> b694ef62ba708c4b67c5cc931b0edfd93702027f 
>   src/main/python/apache/aurora/client/api/updater.py 
> 0acf45034db124ed369d0bcb8095fe5545605eed 
>   src/main/python/apache/aurora/client/cli/jobs.py 
> 782b34800def55c54b67ca5f13da66c4997a0777 
>   src/main/python/apache/aurora/client/cli/task.py 
> 62747ed3e3f5f6ecd9b9c46f6f175c634c068358 
>   src/main/python/apache/aurora/client/commands/core.py 
> 39190e06f3eac1ab4dabf48055418bc383be14ed 
>   src/test/python/apache/aurora/client/api/BUILD 
> dd9b79764db0bbc3a11c51051e43b93a9b5d370a 
>   src/test/python/apache/aurora/client/api/test_job_monitor.py 
> 32609e4e97afc7b8b164f3db438c60efd0ebbe11 
>   src/test/python/apache/aurora/client/api/test_updater.py 
> e7eb1e783baaabe9437e55ecb848f6a42de13dd7 
>   src/test/python/apache/aurora/client/cli/test_command_hooks.py 
> 7c6f70c7ef7534e9dd498

Re: Review Request 20950: Moving kill wait to the client (Part 1: client changes)

2014-05-01 Thread Maxim Khutornenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20950/
---

(Updated May 1, 2014, 10:21 p.m.)


Review request for Aurora, Mark Chu-Carroll and Brian Wickman.


Changes
---

CR comments.


Bugs: AURORA-370
https://issues.apache.org/jira/browse/AURORA-370


Repository: aurora


Description
---

Migration plan:
1. Prepare client to poll for the status of tasks being killed. This is mostly 
a noop at this point while the scheduler wait with a shorter timeout is still 
in place.
2. Cut off scheduler killTasks wait thus relying on the client poll (coming 
next).

This RB addresses client changes:
- Repurposing JobMonitor to poll for all job tasks regardless of their arrival 
time.
- Introducing a timeout cap into JobMonitor to interrupt wait upon reaching the 
max_poll_interval.
- Setting the JobMonitor max_poll_interval to be higher than the scheduler 
killTasks backoff to ensure smooth migration.
- Added wait_util calls into kill/killall/updater calls.

Once this ticket is closed, the next step would be to make kill/killall truly 
async: AURORA-371


Diffs (updated)
-

  src/main/python/apache/aurora/client/api/BUILD 
32097d220a6f8a58555260ce081ace0c58b9fa99 
  src/main/python/apache/aurora/client/api/job_monitor.py 
b694ef62ba708c4b67c5cc931b0edfd93702027f 
  src/main/python/apache/aurora/client/api/updater.py 
0acf45034db124ed369d0bcb8095fe5545605eed 
  src/main/python/apache/aurora/client/cli/jobs.py 
782b34800def55c54b67ca5f13da66c4997a0777 
  src/main/python/apache/aurora/client/cli/task.py 
62747ed3e3f5f6ecd9b9c46f6f175c634c068358 
  src/main/python/apache/aurora/client/commands/core.py 
39190e06f3eac1ab4dabf48055418bc383be14ed 
  src/test/python/apache/aurora/client/api/BUILD 
dd9b79764db0bbc3a11c51051e43b93a9b5d370a 
  src/test/python/apache/aurora/client/api/test_job_monitor.py 
32609e4e97afc7b8b164f3db438c60efd0ebbe11 
  src/test/python/apache/aurora/client/api/test_updater.py 
e7eb1e783baaabe9437e55ecb848f6a42de13dd7 
  src/test/python/apache/aurora/client/cli/test_command_hooks.py 
7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 
  src/test/python/apache/aurora/client/cli/test_create.py 
875573e2ee534ea50da71fac2174ecc8877a714d 
  src/test/python/apache/aurora/client/cli/test_kill.py 
cf5df648206e0cc234d603c1fd56dc7ab1311fa9 
  src/test/python/apache/aurora/client/cli/test_plugins.py 
2dab749d1dcba99a1e3c792c8f26ee86441673d1 
  src/test/python/apache/aurora/client/cli/test_update.py 
cf077e821ebe32104b3e1345014f5c7b07c44c34 
  src/test/python/apache/aurora/client/cli/util.py 
2225ab071dff72be547c43f66ffdc890171c3288 
  src/test/python/apache/aurora/client/commands/test_create.py 
e0ecb523db4ce3b1901adcc96a2fd5ce8184f621 
  src/test/python/apache/aurora/client/commands/test_kill.py 
db820a543b0ea11e5a40cc2a00209aa8fa6186c9 
  src/test/python/apache/aurora/client/commands/test_update.py 
6e145db9ff8213c8f098b53b9a9668532d664249 

Diff: https://reviews.apache.org/r/20950/diff/


Testing
---

./pants src/test/python:all


Thanks,

Maxim Khutornenko

Review Request 20986: Adding min_instance_count threshold to improve safe domain results.

2014-05-01 Thread Maxim Khutornenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20986/
---

Review request for Aurora and Brian Wickman.


Bugs: AURORA-305
https://issues.apache.org/jira/browse/AURORA-305


Repository: aurora


Description
---

Low instance count jobs prevent hosts from showing up in the safe domain list. 
Adding an option to control jobs included into results.


Diffs
-

  src/main/python/apache/aurora/client/api/__init__.py 
d8a23274bc9d0ba2d0ebc232652ff8c4c5da6eb1 
  src/main/python/apache/aurora/client/api/sla.py 
bb705caab4ef7749971613e23c8f6b0d3968ba23 
  src/main/python/apache/aurora/client/commands/admin.py 
912851af1da0f6f706e55e28f6e949e1750b9897 
  src/test/python/apache/aurora/client/api/test_sla.py 
359599af400d621e5652c2e56d01802a804abcc9 
  src/test/python/apache/aurora/client/commands/test_admin_sla.py 
aaf6c7f05caf8c809429a122e3d02e8ebb2fa54e 

Diff: https://reviews.apache.org/r/20986/diff/


Testing
---

./pants src/test/python:all


Thanks,

Maxim Khutornenko

Re: Review Request 20928: Command hooks: stage 2.

2014-05-01 Thread David McLaughlin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20928/#review41958
---

Ship it!


lgtm

- David McLaughlin


On May 1, 2014, 9:39 p.m., Mark Chu-Carroll wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20928/
> ---
> 
> (Updated May 1, 2014, 9:39 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin and Suman Karumuri.
> 
> 
> Bugs: aurora-270
> https://issues.apache.org/jira/browse/aurora-270
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> The second half of command hooks: 
> - Dynamically registered hook exceptions are provided, by fetching a hooks 
> skip rules file from a 
>   URL.
> - Hooks are loaded and activated by the noun/verb framework.
> - Hook selection and dispatch has been substantially updated.
> 
> Also did some long overdue cleanup of string quoting consistency.
> 
> 
> Diffs
> -
> 
>   3rdparty/python/BUILD 122f71db0bc6f37c19ae0f3cb2fcade8321404e6 
>   docs/design/command-hooks.md ee320ed3922928408d23a2dfdf3c42ef96e62ff7 
>   src/main/python/apache/aurora/client/cli/BUILD 
> 1bd565effb3dbe2aeb5d6156575e9316bd77c6a8 
>   src/main/python/apache/aurora/client/cli/__init__.py 
> 1e396b2263451b41cd223708f4cc4cdb1eeddbf0 
>   src/main/python/apache/aurora/client/cli/command_hooks.py 
> 2d200682209e1df83c03f9d515f3b118aaa85a99 
>   src/test/python/apache/aurora/client/cli/test_command_hooks.py 
> 7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 
> 
> Diff: https://reviews.apache.org/r/20928/diff/
> 
> 
> Testing
> ---
> 
> [sun-wukong incubator-aurora (command-hooks-two)]$ !./p
> ./pants src/test/python/apache/aurora/client/cli:all
> Build operating on targets: 
> OrderedSet([PythonTestSuite(src/test/python/apache/aurora/client/cli/BUILD:all)])
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 4 items
> 
> src/test/python/apache/aurora/client/cli/test_bridge.py 
> 
> === 4 passed in 0.02 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 9 items
> 
> src/test/python/apache/aurora/client/cli/test_command_hooks.py .
> 
> === 9 passed in 0.80 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 5 items
> 
> src/test/python/apache/aurora/client/cli/test_help.py .
> 
> === 5 passed in 0.70 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 36 items
> 
> src/test/python/apache/aurora/client/cli/test_cancel_update.py ..
> src/test/python/apache/aurora/client/cli/test_create.py 
> src/test/python/apache/aurora/client/cli/test_diff.py ...
> src/test/python/apache/aurora/client/cli/test_kill.py .
> src/test/python/apache/aurora/client/cli/test_open.py .
> src/test/python/apache/aurora/client/cli/test_restart.py ...
> src/test/python/apache/aurora/client/cli/test_status.py ...
> src/test/python/apache/aurora/client/cli/test_update.py ...
> 
> == 36 passed in 2.02 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 1 items
> 
> src/test/python/apache/aurora/client/cli/test_logging.py .
> 
> === 1 passed in 0.65 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 3 items
> 
> src/test/python/apache/aurora/client/cli/test_plugins.py ...
> 
> === 3 passed in 0.67 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 4 items
> 
> src/test/python/apache/aurora/client/cli/test_quota.py 
> 
> === 4 passed in 0.67 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20

Re: Review Request 20928: Command hooks: stage 2.

2014-05-01 Thread Mark Chu-Carroll 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20928/
---

(Updated May 1, 2014, 5:39 p.m.)


Review request for Aurora, David McLaughlin and Suman Karumuri.


Changes
---

Address review: 
- Fix docs. 
- Simplify the hook-skip check.


Bugs: aurora-270
https://issues.apache.org/jira/browse/aurora-270


Repository: aurora


Description
---

The second half of command hooks: 
- Dynamically registered hook exceptions are provided, by fetching a hooks skip 
rules file from a 
  URL.
- Hooks are loaded and activated by the noun/verb framework.
- Hook selection and dispatch has been substantially updated.

Also did some long overdue cleanup of string quoting consistency.


Diffs (updated)
-

  3rdparty/python/BUILD 122f71db0bc6f37c19ae0f3cb2fcade8321404e6 
  docs/design/command-hooks.md ee320ed3922928408d23a2dfdf3c42ef96e62ff7 
  src/main/python/apache/aurora/client/cli/BUILD 
1bd565effb3dbe2aeb5d6156575e9316bd77c6a8 
  src/main/python/apache/aurora/client/cli/__init__.py 
1e396b2263451b41cd223708f4cc4cdb1eeddbf0 
  src/main/python/apache/aurora/client/cli/command_hooks.py 
2d200682209e1df83c03f9d515f3b118aaa85a99 
  src/test/python/apache/aurora/client/cli/test_command_hooks.py 
7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 

Diff: https://reviews.apache.org/r/20928/diff/


Testing
---

[sun-wukong incubator-aurora (command-hooks-two)]$ !./p
./pants src/test/python/apache/aurora/client/cli:all
Build operating on targets: 
OrderedSet([PythonTestSuite(src/test/python/apache/aurora/client/cli/BUILD:all)])
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 4 items

src/test/python/apache/aurora/client/cli/test_bridge.py 

=== 4 passed in 0.02 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 9 items

src/test/python/apache/aurora/client/cli/test_command_hooks.py .

=== 9 passed in 0.80 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 5 items

src/test/python/apache/aurora/client/cli/test_help.py .

=== 5 passed in 0.70 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 36 items

src/test/python/apache/aurora/client/cli/test_cancel_update.py ..
src/test/python/apache/aurora/client/cli/test_create.py 
src/test/python/apache/aurora/client/cli/test_diff.py ...
src/test/python/apache/aurora/client/cli/test_kill.py .
src/test/python/apache/aurora/client/cli/test_open.py .
src/test/python/apache/aurora/client/cli/test_restart.py ...
src/test/python/apache/aurora/client/cli/test_status.py ...
src/test/python/apache/aurora/client/cli/test_update.py ...

== 36 passed in 2.02 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 1 items

src/test/python/apache/aurora/client/cli/test_logging.py .

=== 1 passed in 0.65 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 3 items

src/test/python/apache/aurora/client/cli/test_plugins.py ...

=== 3 passed in 0.67 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 4 items

src/test/python/apache/aurora/client/cli/test_quota.py 

=== 4 passed in 0.67 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 5 items

src/test/python/apache/aurora/client/cli/test_sla.py .

=== 5 passed in 0.72 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 2 items

src/test/python/apache/aurora/client/cli/test_task_run.py ..

=== 2 passed in 0.65 seconds ===
src.test

Re: Review Request 20928: Command hooks: stage 2.

2014-05-01 Thread Mark Chu-Carroll 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20928/#review41951
---



docs/design/command-hooks.md


Good catch, thanks!



src/main/python/apache/aurora/client/cli/command_hooks.py


The intention is that any field of a hook skip rule that is left blank 
means "match anything, as long as the rest of the rule matches", so that is 
intentional. 

But I think you're right - that's not the best approach. I'll modify it, 
and update the docs.




src/test/python/apache/aurora/client/cli/test_command_hooks.py


They do - the cases that use exception 'b".



- Mark Chu-Carroll


On May 1, 2014, 4:47 p.m., Mark Chu-Carroll wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20928/
> ---
> 
> (Updated May 1, 2014, 4:47 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin and Suman Karumuri.
> 
> 
> Bugs: aurora-270
> https://issues.apache.org/jira/browse/aurora-270
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> The second half of command hooks: 
> - Dynamically registered hook exceptions are provided, by fetching a hooks 
> skip rules file from a 
>   URL.
> - Hooks are loaded and activated by the noun/verb framework.
> - Hook selection and dispatch has been substantially updated.
> 
> Also did some long overdue cleanup of string quoting consistency.
> 
> 
> Diffs
> -
> 
>   3rdparty/python/BUILD 122f71db0bc6f37c19ae0f3cb2fcade8321404e6 
>   docs/design/command-hooks.md ee320ed3922928408d23a2dfdf3c42ef96e62ff7 
>   src/main/python/apache/aurora/client/cli/BUILD 
> 1bd565effb3dbe2aeb5d6156575e9316bd77c6a8 
>   src/main/python/apache/aurora/client/cli/__init__.py 
> 1e396b2263451b41cd223708f4cc4cdb1eeddbf0 
>   src/main/python/apache/aurora/client/cli/command_hooks.py 
> 2d200682209e1df83c03f9d515f3b118aaa85a99 
>   src/test/python/apache/aurora/client/cli/test_command_hooks.py 
> 7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 
> 
> Diff: https://reviews.apache.org/r/20928/diff/
> 
> 
> Testing
> ---
> 
> [sun-wukong incubator-aurora (command-hooks-two)]$ !./p
> ./pants src/test/python/apache/aurora/client/cli:all
> Build operating on targets: 
> OrderedSet([PythonTestSuite(src/test/python/apache/aurora/client/cli/BUILD:all)])
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 4 items
> 
> src/test/python/apache/aurora/client/cli/test_bridge.py 
> 
> === 4 passed in 0.02 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 9 items
> 
> src/test/python/apache/aurora/client/cli/test_command_hooks.py .
> 
> === 9 passed in 0.80 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 5 items
> 
> src/test/python/apache/aurora/client/cli/test_help.py .
> 
> === 5 passed in 0.70 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 36 items
> 
> src/test/python/apache/aurora/client/cli/test_cancel_update.py ..
> src/test/python/apache/aurora/client/cli/test_create.py 
> src/test/python/apache/aurora/client/cli/test_diff.py ...
> src/test/python/apache/aurora/client/cli/test_kill.py .
> src/test/python/apache/aurora/client/cli/test_open.py .
> src/test/python/apache/aurora/client/cli/test_restart.py ...
> src/test/python/apache/aurora/client/cli/test_status.py ...
> src/test/python/apache/aurora/client/cli/test_update.py ...
> 
> == 36 passed in 2.02 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 1 items
> 
> src/test/python/apache/aurora/client/cli/test_logging.py .
> 
> === 1 passed in 0.65 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugin

Re: Review Request 20928: Command hooks: stage 2.

2014-05-01 Thread David McLaughlin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20928/#review41945
---



docs/design/command-hooks.md


This isn't true anymore, right?



src/main/python/apache/aurora/client/cli/command_hooks.py


This is a confusing and potentially dangerous way to write this. Wouldn't 
it better to return False by default, and only return True if the conditions 
for skipping were met? 

e.g. if I define json with no hooks option, this will always return True. 
Is that desired behaviour here? (It's unclear from the docs). 



src/test/python/apache/aurora/client/cli/test_command_hooks.py


Further to above, could you add some test cases for some partially defined 
JSON documents? 


- David McLaughlin


On May 1, 2014, 8:47 p.m., Mark Chu-Carroll wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20928/
> ---
> 
> (Updated May 1, 2014, 8:47 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin and Suman Karumuri.
> 
> 
> Bugs: aurora-270
> https://issues.apache.org/jira/browse/aurora-270
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> The second half of command hooks: 
> - Dynamically registered hook exceptions are provided, by fetching a hooks 
> skip rules file from a 
>   URL.
> - Hooks are loaded and activated by the noun/verb framework.
> - Hook selection and dispatch has been substantially updated.
> 
> Also did some long overdue cleanup of string quoting consistency.
> 
> 
> Diffs
> -
> 
>   3rdparty/python/BUILD 122f71db0bc6f37c19ae0f3cb2fcade8321404e6 
>   docs/design/command-hooks.md ee320ed3922928408d23a2dfdf3c42ef96e62ff7 
>   src/main/python/apache/aurora/client/cli/BUILD 
> 1bd565effb3dbe2aeb5d6156575e9316bd77c6a8 
>   src/main/python/apache/aurora/client/cli/__init__.py 
> 1e396b2263451b41cd223708f4cc4cdb1eeddbf0 
>   src/main/python/apache/aurora/client/cli/command_hooks.py 
> 2d200682209e1df83c03f9d515f3b118aaa85a99 
>   src/test/python/apache/aurora/client/cli/test_command_hooks.py 
> 7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 
> 
> Diff: https://reviews.apache.org/r/20928/diff/
> 
> 
> Testing
> ---
> 
> [sun-wukong incubator-aurora (command-hooks-two)]$ !./p
> ./pants src/test/python/apache/aurora/client/cli:all
> Build operating on targets: 
> OrderedSet([PythonTestSuite(src/test/python/apache/aurora/client/cli/BUILD:all)])
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 4 items
> 
> src/test/python/apache/aurora/client/cli/test_bridge.py 
> 
> === 4 passed in 0.02 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 9 items
> 
> src/test/python/apache/aurora/client/cli/test_command_hooks.py .
> 
> === 9 passed in 0.80 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 5 items
> 
> src/test/python/apache/aurora/client/cli/test_help.py .
> 
> === 5 passed in 0.70 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 36 items
> 
> src/test/python/apache/aurora/client/cli/test_cancel_update.py ..
> src/test/python/apache/aurora/client/cli/test_create.py 
> src/test/python/apache/aurora/client/cli/test_diff.py ...
> src/test/python/apache/aurora/client/cli/test_kill.py .
> src/test/python/apache/aurora/client/cli/test_open.py .
> src/test/python/apache/aurora/client/cli/test_restart.py ...
> src/test/python/apache/aurora/client/cli/test_status.py ...
> src/test/python/apache/aurora/client/cli/test_update.py ...
> 
> == 36 passed in 2.02 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> plugins: cov
> collected 1 items
> 
> src/test/python/apache/aurora/client/cli/test_logging.py .
> 
> === 1 passed in 0.65 seconds 
> ===
> = test session star

Re: Review Request 20950: Moving kill wait to the client (Part 1: client changes)

2014-05-01 Thread Brian Wickman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20950/#review41947
---



src/main/python/apache/aurora/client/api/job_monitor.py


i'm fine with tuning the min_poll_interval and max_poll_interval but 
possibly make them configurable via __init__, ie. __init__(..., 
min_poll_interval=MIN_POLL_INTERVAL, max_poll_interval=MAX_POLL_INTERVAL)



src/main/python/apache/aurora/client/api/job_monitor.py


as a library this should probably be log.error



src/main/python/apache/aurora/client/api/job_monitor.py


update the docstring to reflect new return behavior



src/main/python/apache/aurora/client/api/job_monitor.py


i don't like the idea of wait_until having side effects -- this effectively 
renders it not thread safe.

can you clone the query and update instanceIds on the cloned query, then 
change iter_query to take a query parameter?



src/main/python/apache/aurora/client/api/job_monitor.py


>=



src/main/python/apache/aurora/client/api/updater.py


pass the kwarg so that this is more self-documenting, i.e. with_timeout=True



src/main/python/apache/aurora/client/commands/core.py


weird indent.

also this should use die('Tasks were not killed in time.')


- Brian Wickman


On May 1, 2014, 8:42 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20950/
> ---
> 
> (Updated May 1, 2014, 8:42 p.m.)
> 
> 
> Review request for Aurora, Mark Chu-Carroll and Brian Wickman.
> 
> 
> Bugs: AURORA-370
> https://issues.apache.org/jira/browse/AURORA-370
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Migration plan:
> 1. Prepare client to poll for the status of tasks being killed. This is 
> mostly a noop at this point while the scheduler wait with a shorter timeout 
> is still in place.
> 2. Cut off scheduler killTasks wait thus relying on the client poll (coming 
> next).
> 
> This RB addresses client changes:
> - Repurposing JobMonitor to poll for all job tasks regardless of their 
> arrival time.
> - Introducing a timeout cap into JobMonitor to interrupt wait upon reaching 
> the max_poll_interval.
> - Setting the JobMonitor max_poll_interval to be higher than the scheduler 
> killTasks backoff to ensure smooth migration.
> - Added wait_util calls into kill/killall/updater calls.
> 
> Once this ticket is closed, the next step would be to make kill/killall truly 
> async: AURORA-371
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/api/BUILD 
> 32097d220a6f8a58555260ce081ace0c58b9fa99 
>   src/main/python/apache/aurora/client/api/job_monitor.py 
> b694ef62ba708c4b67c5cc931b0edfd93702027f 
>   src/main/python/apache/aurora/client/api/updater.py 
> 0acf45034db124ed369d0bcb8095fe5545605eed 
>   src/main/python/apache/aurora/client/cli/jobs.py 
> 782b34800def55c54b67ca5f13da66c4997a0777 
>   src/main/python/apache/aurora/client/cli/task.py 
> 62747ed3e3f5f6ecd9b9c46f6f175c634c068358 
>   src/main/python/apache/aurora/client/commands/core.py 
> 39190e06f3eac1ab4dabf48055418bc383be14ed 
>   src/test/python/apache/aurora/client/api/BUILD 
> dd9b79764db0bbc3a11c51051e43b93a9b5d370a 
>   src/test/python/apache/aurora/client/api/test_job_monitor.py 
> 32609e4e97afc7b8b164f3db438c60efd0ebbe11 
>   src/test/python/apache/aurora/client/api/test_updater.py 
> e7eb1e783baaabe9437e55ecb848f6a42de13dd7 
>   src/test/python/apache/aurora/client/cli/test_command_hooks.py 
> 7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 
>   src/test/python/apache/aurora/client/cli/test_create.py 
> 875573e2ee534ea50da71fac2174ecc8877a714d 
>   src/test/python/apache/aurora/client/cli/test_kill.py 
> cf5df648206e0cc234d603c1fd56dc7ab1311fa9 
>   src/test/python/apache/aurora/client/cli/test_plugins.py 
> 2dab749d1dcba99a1e3c792c8f26ee86441673d1 
>   src/test/python/apache/aurora/client/cli/test_update.py 
> cf077e821ebe32104b3e1345014f5c7b07c44c34 
>   src/test/python/apache/aurora/client/cli/util.py 
> 2225ab071dff72be547c43f66ffdc890171c3288 
>   src/test/python/apache/aurora/client/commands/test_create.py 
> e0ecb523db4ce3b1901adcc96a2fd5ce8184f621 
>   src/test/python/apache/aurora/client/commands/test_kill.py 
> db820a543b0ea11e5a40cc2a00209aa8fa6186c9 
>   src/test/python/apache/aurora/client/commands/test_update.py 
> 6e145db9ff8213c8f098b53b9a9668532d664249 
> 
> Diff: https://rev

Re: Review Request 20950: Moving kill wait to the client (Part 1: client changes)

2014-05-01 Thread Maxim Khutornenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20950/
---

(Updated May 1, 2014, 8:42 p.m.)


Review request for Aurora, Mark Chu-Carroll and Brian Wickman.


Changes
---

CR comments.


Bugs: AURORA-370
https://issues.apache.org/jira/browse/AURORA-370


Repository: aurora


Description
---

Migration plan:
1. Prepare client to poll for the status of tasks being killed. This is mostly 
a noop at this point while the scheduler wait with a shorter timeout is still 
in place.
2. Cut off scheduler killTasks wait thus relying on the client poll (coming 
next).

This RB addresses client changes:
- Repurposing JobMonitor to poll for all job tasks regardless of their arrival 
time.
- Introducing a timeout cap into JobMonitor to interrupt wait upon reaching the 
max_poll_interval.
- Setting the JobMonitor max_poll_interval to be higher than the scheduler 
killTasks backoff to ensure smooth migration.
- Added wait_util calls into kill/killall/updater calls.

Once this ticket is closed, the next step would be to make kill/killall truly 
async: AURORA-371


Diffs (updated)
-

  src/main/python/apache/aurora/client/api/BUILD 
32097d220a6f8a58555260ce081ace0c58b9fa99 
  src/main/python/apache/aurora/client/api/job_monitor.py 
b694ef62ba708c4b67c5cc931b0edfd93702027f 
  src/main/python/apache/aurora/client/api/updater.py 
0acf45034db124ed369d0bcb8095fe5545605eed 
  src/main/python/apache/aurora/client/cli/jobs.py 
782b34800def55c54b67ca5f13da66c4997a0777 
  src/main/python/apache/aurora/client/cli/task.py 
62747ed3e3f5f6ecd9b9c46f6f175c634c068358 
  src/main/python/apache/aurora/client/commands/core.py 
39190e06f3eac1ab4dabf48055418bc383be14ed 
  src/test/python/apache/aurora/client/api/BUILD 
dd9b79764db0bbc3a11c51051e43b93a9b5d370a 
  src/test/python/apache/aurora/client/api/test_job_monitor.py 
32609e4e97afc7b8b164f3db438c60efd0ebbe11 
  src/test/python/apache/aurora/client/api/test_updater.py 
e7eb1e783baaabe9437e55ecb848f6a42de13dd7 
  src/test/python/apache/aurora/client/cli/test_command_hooks.py 
7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 
  src/test/python/apache/aurora/client/cli/test_create.py 
875573e2ee534ea50da71fac2174ecc8877a714d 
  src/test/python/apache/aurora/client/cli/test_kill.py 
cf5df648206e0cc234d603c1fd56dc7ab1311fa9 
  src/test/python/apache/aurora/client/cli/test_plugins.py 
2dab749d1dcba99a1e3c792c8f26ee86441673d1 
  src/test/python/apache/aurora/client/cli/test_update.py 
cf077e821ebe32104b3e1345014f5c7b07c44c34 
  src/test/python/apache/aurora/client/cli/util.py 
2225ab071dff72be547c43f66ffdc890171c3288 
  src/test/python/apache/aurora/client/commands/test_create.py 
e0ecb523db4ce3b1901adcc96a2fd5ce8184f621 
  src/test/python/apache/aurora/client/commands/test_kill.py 
db820a543b0ea11e5a40cc2a00209aa8fa6186c9 
  src/test/python/apache/aurora/client/commands/test_update.py 
6e145db9ff8213c8f098b53b9a9668532d664249 

Diff: https://reviews.apache.org/r/20950/diff/


Testing
---

./pants src/test/python:all


Thanks,

Maxim Khutornenko

Re: Review Request 20928: Command hooks: stage 2.

2014-05-01 Thread Mark Chu-Carroll 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20928/
---

(Updated May 1, 2014, 4:47 p.m.)


Review request for Aurora, David McLaughlin and Suman Karumuri.


Changes
---

Fix logging glitch: move logging statement to after plugin activation.


Bugs: aurora-270
https://issues.apache.org/jira/browse/aurora-270


Repository: aurora


Description
---

The second half of command hooks: 
- Dynamically registered hook exceptions are provided, by fetching a hooks skip 
rules file from a 
  URL.
- Hooks are loaded and activated by the noun/verb framework.
- Hook selection and dispatch has been substantially updated.

Also did some long overdue cleanup of string quoting consistency.


Diffs (updated)
-

  3rdparty/python/BUILD 122f71db0bc6f37c19ae0f3cb2fcade8321404e6 
  docs/design/command-hooks.md ee320ed3922928408d23a2dfdf3c42ef96e62ff7 
  src/main/python/apache/aurora/client/cli/BUILD 
1bd565effb3dbe2aeb5d6156575e9316bd77c6a8 
  src/main/python/apache/aurora/client/cli/__init__.py 
1e396b2263451b41cd223708f4cc4cdb1eeddbf0 
  src/main/python/apache/aurora/client/cli/command_hooks.py 
2d200682209e1df83c03f9d515f3b118aaa85a99 
  src/test/python/apache/aurora/client/cli/test_command_hooks.py 
7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 

Diff: https://reviews.apache.org/r/20928/diff/


Testing
---

[sun-wukong incubator-aurora (command-hooks-two)]$ !./p
./pants src/test/python/apache/aurora/client/cli:all
Build operating on targets: 
OrderedSet([PythonTestSuite(src/test/python/apache/aurora/client/cli/BUILD:all)])
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 4 items

src/test/python/apache/aurora/client/cli/test_bridge.py 

=== 4 passed in 0.02 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 9 items

src/test/python/apache/aurora/client/cli/test_command_hooks.py .

=== 9 passed in 0.80 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 5 items

src/test/python/apache/aurora/client/cli/test_help.py .

=== 5 passed in 0.70 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 36 items

src/test/python/apache/aurora/client/cli/test_cancel_update.py ..
src/test/python/apache/aurora/client/cli/test_create.py 
src/test/python/apache/aurora/client/cli/test_diff.py ...
src/test/python/apache/aurora/client/cli/test_kill.py .
src/test/python/apache/aurora/client/cli/test_open.py .
src/test/python/apache/aurora/client/cli/test_restart.py ...
src/test/python/apache/aurora/client/cli/test_status.py ...
src/test/python/apache/aurora/client/cli/test_update.py ...

== 36 passed in 2.02 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 1 items

src/test/python/apache/aurora/client/cli/test_logging.py .

=== 1 passed in 0.65 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 3 items

src/test/python/apache/aurora/client/cli/test_plugins.py ...

=== 3 passed in 0.67 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 4 items

src/test/python/apache/aurora/client/cli/test_quota.py 

=== 4 passed in 0.67 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 5 items

src/test/python/apache/aurora/client/cli/test_sla.py .

=== 5 passed in 0.72 seconds ===
= test session starts ==
platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
plugins: cov
collected 2 items

src/test/python/apache/aurora/client/cli/test_task_run.py ..

=== 2 passed in 0.65 seconds ===

Re: Review Request 20723: Add batch options to kill and killall.

2014-05-01 Thread Mark Chu-Carroll 


> On April 25, 2014, 3:18 p.m., David McLaughlin wrote:
> > lgtm

ping, suman?


- Mark


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20723/#review41509
---


On April 25, 2014, 3:15 p.m., Mark Chu-Carroll wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20723/
> ---
> 
> (Updated April 25, 2014, 3:15 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin and Suman Karumuri.
> 
> 
> Bugs: aurora-356
> https://issues.apache.org/jira/browse/aurora-356
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Add batch options to kill and killall. (This brings the kill commands in 
> client v1 into parity with client v2.
> At the moment, in order to avoid disrupting current users, v1 still defaults 
> to non-batched operation.)
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/commands/core.py 
> 67c061eace86b7a28d2c3ae03e23c7d77fc7957f 
>   src/main/python/apache/aurora/client/options.py 
> 0d85c360dc4c62ab0baf58315a68a32c304fcffc 
>   src/test/python/apache/aurora/client/commands/test_kill.py 
> 825f5f13da1c8bee843581a34f3357c0fa2dbb00 
> 
> Diff: https://reviews.apache.org/r/20723/diff/
> 
> 
> Testing
> ---
> 
> [sun-wukong incubator-aurora (batch_v1)]$ ./pants 
> src/test/python/apache/aurora/client/commands:all
> Build operating on targets: 
> OrderedSet([PythonTestSuite(src/test/python/apache/aurora/client/commands/BUILD:all)])
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> collected 13 items
> 
> src/test/python/apache/aurora/client/commands/test_admin_sla.py .
> 
> == 13 passed in 0.57 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> collected 28 items
> 
> src/test/python/apache/aurora/client/commands/test_cancel_update.py ..
> src/test/python/apache/aurora/client/commands/test_create.py ..
> src/test/python/apache/aurora/client/commands/test_diff.py ...
> src/test/python/apache/aurora/client/commands/test_kill.py ..
> src/test/python/apache/aurora/client/commands/test_listjobs.py ..
> src/test/python/apache/aurora/client/commands/test_restart.py ...
> src/test/python/apache/aurora/client/commands/test_status.py ...
> src/test/python/apache/aurora/client/commands/test_update.py ...
> 
> == 28 passed in 1.47 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> collected 3 items
> 
> src/test/python/apache/aurora/client/commands/test_hooks.py ...
> 
> === 3 passed in 1.21 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> collected 4 items
> 
> src/test/python/apache/aurora/client/commands/test_maintenance.py 
> 
> === 4 passed in 0.48 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> collected 1 items
> 
> src/test/python/apache/aurora/client/commands/test_run.py .
> 
> === 1 passed in 0.58 seconds 
> ===
> = test session starts 
> ==
> platform darwin -- Python 2.6.8 -- py-1.4.20 -- pytest-2.5.2
> collected 1 items
> 
> src/test/python/apache/aurora/client/commands/test_ssh.py .
> 
> === 1 passed in 0.47 seconds 
> ===
> src.test.python.apache.aurora.client.commands.admin   
>   .   SUCCESS
> src.test.python.apache.aurora.client.commands.core
>   .   SUCCESS
> src.test.python.apache.aurora.client.commands.hooks   
>   .   SUCCESS
> src.test.python.apache.aurora.client.commands.maintenance 
>   .   SUCCESS
> src.test.python.apache.aurora.client.commands.run 
>   .   SUCCESS
> src.test.python.apache.aurora.client.commands.ssh 
>   .   SUCCESS
> [sun-wukong incubator-aurora (batch_v1)]$
> 
> 
> Thanks,
> 
> Mark Chu-Carroll
> 
>

Re: Review Request 20950: Moving kill wait to the client (Part 1: client changes)

2014-05-01 Thread Mark Chu-Carroll 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20950/#review41943
---

Ship it!


Ship It!

- Mark Chu-Carroll


On May 1, 2014, 4:42 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20950/
> ---
> 
> (Updated May 1, 2014, 4:42 p.m.)
> 
> 
> Review request for Aurora, Mark Chu-Carroll and Brian Wickman.
> 
> 
> Bugs: AURORA-370
> https://issues.apache.org/jira/browse/AURORA-370
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Migration plan:
> 1. Prepare client to poll for the status of tasks being killed. This is 
> mostly a noop at this point while the scheduler wait with a shorter timeout 
> is still in place.
> 2. Cut off scheduler killTasks wait thus relying on the client poll (coming 
> next).
> 
> This RB addresses client changes:
> - Repurposing JobMonitor to poll for all job tasks regardless of their 
> arrival time.
> - Introducing a timeout cap into JobMonitor to interrupt wait upon reaching 
> the max_poll_interval.
> - Setting the JobMonitor max_poll_interval to be higher than the scheduler 
> killTasks backoff to ensure smooth migration.
> - Added wait_util calls into kill/killall/updater calls.
> 
> Once this ticket is closed, the next step would be to make kill/killall truly 
> async: AURORA-371
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/api/BUILD 
> 32097d220a6f8a58555260ce081ace0c58b9fa99 
>   src/main/python/apache/aurora/client/api/job_monitor.py 
> b694ef62ba708c4b67c5cc931b0edfd93702027f 
>   src/main/python/apache/aurora/client/api/updater.py 
> 0acf45034db124ed369d0bcb8095fe5545605eed 
>   src/main/python/apache/aurora/client/cli/jobs.py 
> 782b34800def55c54b67ca5f13da66c4997a0777 
>   src/main/python/apache/aurora/client/cli/task.py 
> 62747ed3e3f5f6ecd9b9c46f6f175c634c068358 
>   src/main/python/apache/aurora/client/commands/core.py 
> 39190e06f3eac1ab4dabf48055418bc383be14ed 
>   src/test/python/apache/aurora/client/api/BUILD 
> dd9b79764db0bbc3a11c51051e43b93a9b5d370a 
>   src/test/python/apache/aurora/client/api/test_job_monitor.py 
> 32609e4e97afc7b8b164f3db438c60efd0ebbe11 
>   src/test/python/apache/aurora/client/api/test_updater.py 
> e7eb1e783baaabe9437e55ecb848f6a42de13dd7 
>   src/test/python/apache/aurora/client/cli/test_command_hooks.py 
> 7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 
>   src/test/python/apache/aurora/client/cli/test_create.py 
> 875573e2ee534ea50da71fac2174ecc8877a714d 
>   src/test/python/apache/aurora/client/cli/test_kill.py 
> cf5df648206e0cc234d603c1fd56dc7ab1311fa9 
>   src/test/python/apache/aurora/client/cli/test_plugins.py 
> 2dab749d1dcba99a1e3c792c8f26ee86441673d1 
>   src/test/python/apache/aurora/client/cli/test_update.py 
> cf077e821ebe32104b3e1345014f5c7b07c44c34 
>   src/test/python/apache/aurora/client/cli/util.py 
> 2225ab071dff72be547c43f66ffdc890171c3288 
>   src/test/python/apache/aurora/client/commands/test_create.py 
> e0ecb523db4ce3b1901adcc96a2fd5ce8184f621 
>   src/test/python/apache/aurora/client/commands/test_kill.py 
> db820a543b0ea11e5a40cc2a00209aa8fa6186c9 
>   src/test/python/apache/aurora/client/commands/test_update.py 
> 6e145db9ff8213c8f098b53b9a9668532d664249 
> 
> Diff: https://reviews.apache.org/r/20950/diff/
> 
> 
> Testing
> ---
> 
> ./pants src/test/python:all
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 20950: Moving kill wait to the client (Part 1: client changes)

2014-05-01 Thread Maxim Khutornenko 


> On May 1, 2014, 7:44 p.m., Mark Chu-Carroll wrote:
> > src/main/python/apache/aurora/client/api/job_monitor.py, line 81
> > 
> >
> > shouldn't this be >=?

Line 84 guarantees it never exceeds the max value:

poll_interval = min(self.MAX_POLL_INTERVAL, 2 * poll_interval)


> On May 1, 2014, 7:44 p.m., Mark Chu-Carroll wrote:
> > src/test/python/apache/aurora/client/api/test_job_monitor.py, line 103
> > 
> >
> > You can just "assert monitor.wait_until...".
> >

Sure, changed.


- Maxim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20950/#review41938
---


On May 1, 2014, 5:01 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20950/
> ---
> 
> (Updated May 1, 2014, 5:01 p.m.)
> 
> 
> Review request for Aurora, Mark Chu-Carroll and Brian Wickman.
> 
> 
> Bugs: AURORA-370
> https://issues.apache.org/jira/browse/AURORA-370
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Migration plan:
> 1. Prepare client to poll for the status of tasks being killed. This is 
> mostly a noop at this point while the scheduler wait with a shorter timeout 
> is still in place.
> 2. Cut off scheduler killTasks wait thus relying on the client poll (coming 
> next).
> 
> This RB addresses client changes:
> - Repurposing JobMonitor to poll for all job tasks regardless of their 
> arrival time.
> - Introducing a timeout cap into JobMonitor to interrupt wait upon reaching 
> the max_poll_interval.
> - Setting the JobMonitor max_poll_interval to be higher than the scheduler 
> killTasks backoff to ensure smooth migration.
> - Added wait_util calls into kill/killall/updater calls.
> 
> Once this ticket is closed, the next step would be to make kill/killall truly 
> async: AURORA-371
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/api/BUILD 
> 32097d220a6f8a58555260ce081ace0c58b9fa99 
>   src/main/python/apache/aurora/client/api/job_monitor.py 
> b694ef62ba708c4b67c5cc931b0edfd93702027f 
>   src/main/python/apache/aurora/client/api/updater.py 
> 0acf45034db124ed369d0bcb8095fe5545605eed 
>   src/main/python/apache/aurora/client/cli/jobs.py 
> 782b34800def55c54b67ca5f13da66c4997a0777 
>   src/main/python/apache/aurora/client/cli/task.py 
> 62747ed3e3f5f6ecd9b9c46f6f175c634c068358 
>   src/main/python/apache/aurora/client/commands/core.py 
> 39190e06f3eac1ab4dabf48055418bc383be14ed 
>   src/test/python/apache/aurora/client/api/BUILD 
> dd9b79764db0bbc3a11c51051e43b93a9b5d370a 
>   src/test/python/apache/aurora/client/api/test_job_monitor.py 
> 32609e4e97afc7b8b164f3db438c60efd0ebbe11 
>   src/test/python/apache/aurora/client/api/test_updater.py 
> e7eb1e783baaabe9437e55ecb848f6a42de13dd7 
>   src/test/python/apache/aurora/client/cli/test_command_hooks.py 
> 7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 
>   src/test/python/apache/aurora/client/cli/test_create.py 
> 875573e2ee534ea50da71fac2174ecc8877a714d 
>   src/test/python/apache/aurora/client/cli/test_kill.py 
> cf5df648206e0cc234d603c1fd56dc7ab1311fa9 
>   src/test/python/apache/aurora/client/cli/test_plugins.py 
> 2dab749d1dcba99a1e3c792c8f26ee86441673d1 
>   src/test/python/apache/aurora/client/cli/test_update.py 
> cf077e821ebe32104b3e1345014f5c7b07c44c34 
>   src/test/python/apache/aurora/client/cli/util.py 
> 2225ab071dff72be547c43f66ffdc890171c3288 
>   src/test/python/apache/aurora/client/commands/test_create.py 
> e0ecb523db4ce3b1901adcc96a2fd5ce8184f621 
>   src/test/python/apache/aurora/client/commands/test_kill.py 
> db820a543b0ea11e5a40cc2a00209aa8fa6186c9 
>   src/test/python/apache/aurora/client/commands/test_update.py 
> 6e145db9ff8213c8f098b53b9a9668532d664249 
> 
> Diff: https://reviews.apache.org/r/20950/diff/
> 
> 
> Testing
> ---
> 
> ./pants src/test/python:all
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Re: Review Request 20950: Moving kill wait to the client (Part 1: client changes)

2014-05-01 Thread Mark Chu-Carroll 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20950/#review41938
---



src/main/python/apache/aurora/client/api/job_monitor.py


shouldn't this be >=?



src/test/python/apache/aurora/client/api/test_job_monitor.py


You can just "assert monitor.wait_until...".



- Mark Chu-Carroll


On May 1, 2014, 1:01 p.m., Maxim Khutornenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20950/
> ---
> 
> (Updated May 1, 2014, 1:01 p.m.)
> 
> 
> Review request for Aurora, Mark Chu-Carroll and Brian Wickman.
> 
> 
> Bugs: AURORA-370
> https://issues.apache.org/jira/browse/AURORA-370
> 
> 
> Repository: aurora
> 
> 
> Description
> ---
> 
> Migration plan:
> 1. Prepare client to poll for the status of tasks being killed. This is 
> mostly a noop at this point while the scheduler wait with a shorter timeout 
> is still in place.
> 2. Cut off scheduler killTasks wait thus relying on the client poll (coming 
> next).
> 
> This RB addresses client changes:
> - Repurposing JobMonitor to poll for all job tasks regardless of their 
> arrival time.
> - Introducing a timeout cap into JobMonitor to interrupt wait upon reaching 
> the max_poll_interval.
> - Setting the JobMonitor max_poll_interval to be higher than the scheduler 
> killTasks backoff to ensure smooth migration.
> - Added wait_util calls into kill/killall/updater calls.
> 
> Once this ticket is closed, the next step would be to make kill/killall truly 
> async: AURORA-371
> 
> 
> Diffs
> -
> 
>   src/main/python/apache/aurora/client/api/BUILD 
> 32097d220a6f8a58555260ce081ace0c58b9fa99 
>   src/main/python/apache/aurora/client/api/job_monitor.py 
> b694ef62ba708c4b67c5cc931b0edfd93702027f 
>   src/main/python/apache/aurora/client/api/updater.py 
> 0acf45034db124ed369d0bcb8095fe5545605eed 
>   src/main/python/apache/aurora/client/cli/jobs.py 
> 782b34800def55c54b67ca5f13da66c4997a0777 
>   src/main/python/apache/aurora/client/cli/task.py 
> 62747ed3e3f5f6ecd9b9c46f6f175c634c068358 
>   src/main/python/apache/aurora/client/commands/core.py 
> 39190e06f3eac1ab4dabf48055418bc383be14ed 
>   src/test/python/apache/aurora/client/api/BUILD 
> dd9b79764db0bbc3a11c51051e43b93a9b5d370a 
>   src/test/python/apache/aurora/client/api/test_job_monitor.py 
> 32609e4e97afc7b8b164f3db438c60efd0ebbe11 
>   src/test/python/apache/aurora/client/api/test_updater.py 
> e7eb1e783baaabe9437e55ecb848f6a42de13dd7 
>   src/test/python/apache/aurora/client/cli/test_command_hooks.py 
> 7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 
>   src/test/python/apache/aurora/client/cli/test_create.py 
> 875573e2ee534ea50da71fac2174ecc8877a714d 
>   src/test/python/apache/aurora/client/cli/test_kill.py 
> cf5df648206e0cc234d603c1fd56dc7ab1311fa9 
>   src/test/python/apache/aurora/client/cli/test_plugins.py 
> 2dab749d1dcba99a1e3c792c8f26ee86441673d1 
>   src/test/python/apache/aurora/client/cli/test_update.py 
> cf077e821ebe32104b3e1345014f5c7b07c44c34 
>   src/test/python/apache/aurora/client/cli/util.py 
> 2225ab071dff72be547c43f66ffdc890171c3288 
>   src/test/python/apache/aurora/client/commands/test_create.py 
> e0ecb523db4ce3b1901adcc96a2fd5ce8184f621 
>   src/test/python/apache/aurora/client/commands/test_kill.py 
> db820a543b0ea11e5a40cc2a00209aa8fa6186c9 
>   src/test/python/apache/aurora/client/commands/test_update.py 
> 6e145db9ff8213c8f098b53b9a9668532d664249 
> 
> Diff: https://reviews.apache.org/r/20950/diff/
> 
> 
> Testing
> ---
> 
> ./pants src/test/python:all
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Review Request 20959: Moving kill wait to the client (Part 2: server changes)

2014-05-01 Thread Maxim Khutornenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20959/
---

Review request for Aurora and Bill Farner.


Bugs: AURORA-370
https://issues.apache.org/jira/browse/AURORA-370


Repository: aurora


Description
---

Dropping killTask wait from the scheduler.


Diffs
-

  
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 
abf47d791182680f5bc3b92e4cef937633fc1a2a 
  
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java
 169c4039bc609ca1b82f9c56661a9648c8d47116 

Diff: https://reviews.apache.org/r/20959/diff/


Testing
---

gradle build


Thanks,

Maxim Khutornenko

Review Request 20950: Moving kill wait to the client (Part 1: client changes)

2014-05-01 Thread Maxim Khutornenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20950/
---

Review request for Aurora, Mark Chu-Carroll and Brian Wickman.


Bugs: AURORA-370
https://issues.apache.org/jira/browse/AURORA-370


Repository: aurora


Description
---

Migration plan:
1. Prepare client to poll for the status of tasks being killed. This is mostly 
a noop at this point while the scheduler wait with a shorter timeout is still 
in place.
2. Cut off scheduler killTasks wait thus relying on the client poll (coming 
next).

This RB addresses client changes:
- Repurposing JobMonitor to poll for all job tasks regardless of their arrival 
time.
- Introducing a timeout cap into JobMonitor to interrupt wait upon reaching the 
max_poll_interval.
- Setting the JobMonitor max_poll_interval to be higher than the scheduler 
killTasks backoff to ensure smooth migration.
- Added wait_util calls into kill/killall/updater calls.

Once this ticket is closed, the next step would be to make kill/killall truly 
async: AURORA-371


Diffs
-

  src/main/python/apache/aurora/client/api/BUILD 
32097d220a6f8a58555260ce081ace0c58b9fa99 
  src/main/python/apache/aurora/client/api/job_monitor.py 
b694ef62ba708c4b67c5cc931b0edfd93702027f 
  src/main/python/apache/aurora/client/api/updater.py 
0acf45034db124ed369d0bcb8095fe5545605eed 
  src/main/python/apache/aurora/client/cli/jobs.py 
782b34800def55c54b67ca5f13da66c4997a0777 
  src/main/python/apache/aurora/client/cli/task.py 
62747ed3e3f5f6ecd9b9c46f6f175c634c068358 
  src/main/python/apache/aurora/client/commands/core.py 
39190e06f3eac1ab4dabf48055418bc383be14ed 
  src/test/python/apache/aurora/client/api/BUILD 
dd9b79764db0bbc3a11c51051e43b93a9b5d370a 
  src/test/python/apache/aurora/client/api/test_job_monitor.py 
32609e4e97afc7b8b164f3db438c60efd0ebbe11 
  src/test/python/apache/aurora/client/api/test_updater.py 
e7eb1e783baaabe9437e55ecb848f6a42de13dd7 
  src/test/python/apache/aurora/client/cli/test_command_hooks.py 
7c6f70c7ef7534e9dd4986364331c67f6f7c36b1 
  src/test/python/apache/aurora/client/cli/test_create.py 
875573e2ee534ea50da71fac2174ecc8877a714d 
  src/test/python/apache/aurora/client/cli/test_kill.py 
cf5df648206e0cc234d603c1fd56dc7ab1311fa9 
  src/test/python/apache/aurora/client/cli/test_plugins.py 
2dab749d1dcba99a1e3c792c8f26ee86441673d1 
  src/test/python/apache/aurora/client/cli/test_update.py 
cf077e821ebe32104b3e1345014f5c7b07c44c34 
  src/test/python/apache/aurora/client/cli/util.py 
2225ab071dff72be547c43f66ffdc890171c3288 
  src/test/python/apache/aurora/client/commands/test_create.py 
e0ecb523db4ce3b1901adcc96a2fd5ce8184f621 
  src/test/python/apache/aurora/client/commands/test_kill.py 
db820a543b0ea11e5a40cc2a00209aa8fa6186c9 
  src/test/python/apache/aurora/client/commands/test_update.py 
6e145db9ff8213c8f098b53b9a9668532d664249 

Diff: https://reviews.apache.org/r/20950/diff/


Testing
---

./pants src/test/python:all


Thanks,

Maxim Khutornenko