> On Feb. 13, 2017, 8:48 p.m., Santhosh Kumar Shanmugham wrote: > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java, > > lines 1128-1129 > > <https://reviews.apache.org/r/56629/diff/5/?file=1632842#file1632842line1128> > > > > Depending on th query this can be a full-scan on the store, which can > > be inefficient. As mentioned in https://reviews.apache.org/r/56575/ this > > approach can cause GC pressure. We should use the `TaskStore.pruneHistory` > > implementation which might be more efficient. > > David McLaughlin wrote: > pruneHistory would search for candidates that are eligible for pruning > according to the pruning settings. This is a way to circumvent the grace > periods and other limits, and prune all terminal tasks that match the > provided query. Since this would be called explicitly and can be supplied > with a limit, the operator is able to manage the GC pressure themselves.
Since this command has the potential to hurt the Scheduler, should we make sure that atleast one of the filters is provided? Or force a minimum default `limit` to avoid operators from accidentally bringing down the scheduler? - Santhosh Kumar ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56629/#review165440 ----------------------------------------------------------- On Feb. 13, 2017, 8:28 p.m., David McLaughlin wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56629/ > ----------------------------------------------------------- > > (Updated Feb. 13, 2017, 8:28 p.m.) > > > Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer > Manji. > > > Repository: aurora > > > Description > ------- > > Expose task pruning endpoint in aurora_admin. Useful for scale testing in > order to 'clean up' after a test run, but also useful in production if you > have a bad actor inflating the size of your task index. > > > Diffs > ----- > > api/src/main/thrift/org/apache/aurora/gen/api.thrift > 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1 > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > a211483caf59755b1f9b286b58f183e00db6eee6 > src/main/python/apache/aurora/admin/admin.py > 070c348d2ca5db1edecf832efd9aa5481bddaa4b > src/main/python/apache/aurora/client/api/__init__.py > 1250ccd16f906eec08c159df3300d1b94b306d8e > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 0cdd9829417bb3ef0215278a9458a3dd78a49a20 > src/test/python/apache/aurora/admin/test_admin.py > 66abade378d3974dbf031a7bbed9cd6fc4f22d5f > src/test/python/apache/aurora/client/api/test_scheduler_client.py > fab97986dcecf7761e161f4074d5108a8c362e6e > > Diff: https://reviews.apache.org/r/56629/diff/ > > > Testing > ------- > > Ran various combinations of task pruning via aurora_admin in Vagrant. Works > as expected. > > Also: > > ./pants test src/test/python/apache/aurora/admin:: > > > Thanks, > > David McLaughlin > >
