aurora git commit: Fix exception thrown in SchedulingBenchmarks set up.

2016-01-08 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 024bac9dc -> 217640f8e Fix exception thrown in SchedulingBenchmarks set up. SchedulingBenchmarks were broken because of a missing binding to `TeirConfig` and an invalid parameter to `PreemptorModule`. Testing Done: ./gradlew jmh -Pbenchma

aurora git commit: Bump JMH to 1.11.2.

2016-01-08 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 217640f8e -> c595228af Bump JMH to 1.11.2. Bump JMH to the latest available release which is 1.11.2. There isn't a CHANGELOG but the commit history shows several bug fixes: http://hg.openjdk.java.net/code-tools/jmh/ Testing Done: ./gradle

aurora git commit: Enable H2 query statistics collection.

2016-01-11 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master e4c9c73f5 -> f064dc106 Enable H2 query statistics collection. With this enabled operators can visit the H2 console at /h2console and run queries like `SELECT * FROM INFORMATION_SCHEMA.QUERY_STATISTICS ORDER BY MAX_EXECUTION_TIME DESC;` to

aurora git commit: Add `--show-error` to curl when bootstrapping thrift.

2016-01-12 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master d542bd1d5 -> 952ef6db3 Add `--show-error` to curl when bootstrapping thrift. >From the curl documentation: -S, --show-error When used with -s it makes curl show an error message if it fails. It's possible for curl to fail when

aurora git commit: Add metric for counting uncaught exceptions in async executor.

2016-01-15 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 4dff5da84 -> b56367902 Add metric for counting uncaught exceptions in async executor. Add metric "async_executor_uncaught_exceptions" for tracking uncaught exceptions in async executor. Bugs closed: AURORA-1582 Reviewed at https://review

aurora git commit: Turn TaskHistoryPruner into a service and trigger shutdown on pruning failure.

2016-01-21 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master a2c7ccc17 -> c89fecbcd Turn TaskHistoryPruner into a service and trigger shutdown on pruning failure. Task pruning is key to operating a large cluster and failure to prune should trigger shutdown to prevent unbounded growth of storage. Thi

aurora git commit: Enable H2 logging to slf4j.

2016-01-27 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master e2a973e1e -> c417f9d09 Enable H2 logging to slf4j. On a test cluster with DbTaskStore enabled there are several lines in the log that look like: 2016-01-26 13:07:14 jdbc[15]: exception There is no other information with these lin

aurora git commit: Enable ping query to prevent use of invalid pooled connections.

2016-02-01 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master de0029bb6 -> 6c539fe5a Enable ping query to prevent use of invalid pooled connections. Bugs closed: AURORA-1596 Reviewed at https://reviews.apache.org/r/42979/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://g

aurora git commit: Bump virtualenv version for in repo tools.

2016-02-01 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 6c539fe5a -> 513f4b81b Bump virtualenv version for in repo tools. Reviewed at https://reviews.apache.org/r/43066/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://git-wip-us.apache.org/repos/asf/aurora/commit/51

aurora git commit: Add a flag to configure H2 LOCK_TIMEOUT.

2016-02-01 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 166ca416d -> 824e396ab Add a flag to configure H2 LOCK_TIMEOUT. Bugs closed: AURORA-1596 Reviewed at https://reviews.apache.org/r/42985/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://git-wip-us.apache.org/re

aurora git commit: Make --announcer-enable optional no-op instead of removing it completely.

2016-02-02 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 2f485e856 -> d033d5393 Make --announcer-enable optional no-op instead of removing it completely. Reviewed at https://reviews.apache.org/r/43112/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://git-wip-us.apache

aurora git commit: Expose MyBatis PoolState via stats.

2016-02-03 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 2d91e18fb -> 52c19cf05 Expose MyBatis PoolState via stats. To better understand the MyBatis connection pool this patch exposes the pool state via stats. Reviewed at https://reviews.apache.org/r/43150/ Project: http://git-wip-us.apache.o

aurora git commit: Remove unused entry in TaskMapper.

2016-02-04 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 52c19cf05 -> 42bff1961 Remove unused entry in TaskMapper. The property `taskConfigRowId` doesn't exist on `DbScheduledTask` so this line has no use. Testing Done: ./gadlew test Reviewed at https://reviews.apache.org/r/43178/ Project:

aurora git commit: Add benchmark for feching tasks.

2016-02-07 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 2d59b697a -> 784deaf40 Add benchmark for feching tasks. This adds a benchmark for the throughput of fetching tasks from both the DB and InMem store. Reviewed at https://reviews.apache.org/r/43237/ Project: http://git-wip-us.apache.org/r

aurora git commit: Reject NULL for enum values in H2.

2016-02-10 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 86b6d7919 -> d657f952a Reject NULL for enum values in H2. While I was working on something else, I introduced an error where MyBatis was populating a thrift enum value and was reading in NULL. I realized that the code would currently treat

aurora git commit: Increase throughput of DbTaskStore

2016-02-12 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 17ddc484f -> 9340d468f Increase throughput of DbTaskStore Profiling master indicated that the bottleneck was MyBatis populating ResultSets and populating the resulting objects. This patch removes subselects, which reduces the number of Res

aurora git commit: Do not create a set inside DbTaskStore#fetchTasks.

2016-02-19 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 9d642dcfa -> 8501d8219 Do not create a set inside DbTaskStore#fetchTasks. The TaskStore interface allows for an Iterable to be returned. Taking advantage of this and returning the list MyBatis creates results in a performance improvement.

aurora git commit: Bump mybatis dependency to 3.3.1. The release notes[1] indicate a few changes including one performance improvement[2].

2016-02-23 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 14f867cde -> 0df94f6e7 Bump mybatis dependency to 3.3.1. The release notes[1] indicate a few changes including one performance improvement[2]. [1]: http://blog.mybatis.org/2016/02/mybatis-331-and-mybatis-spring-124.html [2]: https://github

aurora git commit: Do not modify list populated by MyBatis.

2016-02-26 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 7e32c4381 -> 8f98e6e8a Do not modify list populated by MyBatis. A ConcurrentModificationException can occur when multiple threads are sorting the same task event list. This is possible because we have enabled MyBatis caching with the readO

aurora git commit: Complete TODO in Tasks.java.

2016-03-03 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 35de88645 -> 98a2bc194 Complete TODO in Tasks.java. This completes the TODO in Tasks.java by removing the unused code. Reviewed at https://reviews.apache.org/r/44219/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: h

aurora git commit: Install thrift package from the offical debs.

2016-03-04 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master f6506d903 -> a4e7b3a43 Install thrift package from the offical debs. `twopercent` pointed out on IRC that due to the people.a.o change our vagrant provisioning was broken. This ports the change from aurora-packaging to our vagrant script.

aurora git commit: Show principal in auth failure messaage.

2016-03-04 Thread zmanji
extends EasyMockTest { .andReturn(false); expect(subject.isPermitted(interceptor.makeTargetPermission("killTasks", JOB_KEY))) .andReturn(false); +expect(subject.getPrincipal()).andReturn("zmanji"); replayAndInitialize();

aurora git commit: Revert "Add support for storing and fetching images as properties of task configs."

2016-03-22 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master d5d7ec0eb -> b5c9e1bc4 Revert "Add support for storing and fetching images as properties of task configs." This reverts commit d5d7ec0eb5703d6bda8c43cd0586684a550a575a. As outlined in `AURORA-1648` the schema changes here are not necessar

aurora git commit: Implement custom MyBatis cache to record cache statistics.

2016-04-06 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 1a391d75f -> 9a93955a1 Implement custom MyBatis cache to record cache statistics. This swaps out our use of the MyBatis default cache with a custom one backed by Guava. The benefit of this is that we can now plumb cache statistics out whic

aurora git commit: Replace org.apache.aurora.common.base.Closure with java.util.function.Consumer

2016-04-15 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 02ffef5de -> 03ec02389 Replace org.apache.aurora.common.base.Closure with java.util.function.Consumer Commons came with a `Closure` type which is identical to the Java 8 type `Consumer`. This replaces the former with the latter in the inte

aurora git commit: Make MyBatis connection pool size configurable.

2016-04-18 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master e543415d2 -> 96c990875 Make MyBatis connection pool size configurable. This adds two arguments which enables operators to configure the internal MyBatis connection pool size. Increasing the size will allow for greater concurrency in larger

aurora git commit: Add @Timed annotations to cron stores.

2016-04-18 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master a6197316c -> 96d5b648e Add @Timed annotations to cron stores. Both cron stores don't have @Timed annoations. This adds them so operators can look at the throughput of storage. Bugs closed: AURORA-1665 Reviewed at https://reviews.apache.o

aurora git commit: Deleting --setuid-health-checks and switching to --nosetuid-health-checks flag to to control whether the executor runs health checks as the job role's user.

2016-04-19 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 96d5b648e -> d1dfd16a3 Deleting --setuid-health-checks and switching to --nosetuid-health-checks flag to to control whether the executor runs health checks as the job role's user. Bugs closed: AURORA-1666 Reviewed at https://reviews.apach

aurora git commit: Improve `executorLost` error message by including the slave id.

2016-07-27 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 08792d433 -> dde2c92f7 Improve `executorLost` error message by including the slave id. Reviewed at https://reviews.apache.org/r/50478/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://git-wip-us.apache.org/repos

aurora git commit: Support TBinaryProtocol over HTTP

2016-08-03 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master a071af345 -> 0105a151b Support TBinaryProtocol over HTTP This replaces the `TServlet` servlet from thrift with our own servlet which dispatches thrift responses based on the content type of the request. This enables a client to use either

aurora git commit: Populate the source field of ExecutorInfo.

2016-08-05 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 78d35269f -> 122780872 Populate the source field of ExecutorInfo. b912e17 stopped populating the source field of the executor. For backwards compatibility we should continue to populate this field and the `source` label. Bugs closed: AURO

aurora git commit: Fix typo in `RELEASE-NOTES.md`.

2016-08-05 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 122780872 -> 581262ccf Fix typo in `RELEASE-NOTES.md`. Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://git-wip-us.apache.org/repos/asf/aurora/commit/581262cc Tree: http://git-wip-us.apache.org/repos/asf/aurora/t

aurora git commit: Remove unnecessary guice container parameters.

2016-08-10 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 581262ccf -> c465fb1e2 Remove unnecessary guice container parameters. I noticed these configuration parameters have no effect. Both the API and JAX-RS endpoints like /vars return gipped content. Testing Done: $ curl -I -X GET http://192.1

aurora git commit: Bump jetty dependency to the latest release.

2016-08-10 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master c465fb1e2 -> 4d7730bc4 Bump jetty dependency to the latest release. A useful fix from the jetty-9.3.10.v20160621 release: > 623 Add --gzip suffix to 304 responses with ETAGs Without this fix adding ETAG support to the scheduler with gzip

aurora git commit: Add rollback functionality to the scheduler

2016-08-12 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 90846640b -> a4fdf284d Add rollback functionality to the scheduler For active job updates in ROLLING_FORWARD, ROLL_BACK_PAUSED, ROLL_BACK_AWAITING_PULSE, ROLL_FORWARD_PAUSED or ROLL_FORWARD_AWAITING_PULSE state it is possible now to initia

aurora git commit: Reduce static method exposure for Stats.

2016-08-22 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 370813f11 -> 9b34a4036 Reduce static method exposure for Stats. `org.apache.aurora.common.stats.Stats` has several static methods that are not used in our codebase. This patch deletes the unused methods and reduces the visability of other

aurora git commit: Fix thermos killing heuristic to permit setuid(2).

2016-08-23 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master c115ac6bc -> 50f47ccc9 Fix thermos killing heuristic to permit setuid(2). Previously this process killing heuristic would not allow killing of a process if the uid it was launched with differs from the real uid of the currently running pro

aurora git commit: Remove static Stats method `exportSize`.

2016-09-06 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 5d3f94549 -> 059b08621 Remove static Stats method `exportSize`. Reviewed at https://reviews.apache.org/r/51469/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://git-wip-us.apache.org/repos/asf/aurora/commit/059b

aurora git commit: Add MEDIAN_TIME_TO_STARTING as a new metric.

2016-09-06 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 059b08621 -> 0c90c862a Add MEDIAN_TIME_TO_STARTING as a new metric. A new MTTS (Median Time To Starting) metric is added to the sla module in addition to MTTA and MTTR. This review request is related to my previous review request: https:/

aurora git commit: Remove HttpServletRequestParams.

2016-09-06 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 0c90c862a -> 19866b516 Remove HttpServletRequestParams. `HttpServletRequestParams` is dead code can be removed safely. Reviewed at https://reviews.apache.org/r/51667/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: h

aurora git commit: Modify the watch_secs assertion on scheduler

2016-09-07 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 19866b516 -> 8fca74501 Modify the watch_secs assertion on scheduler This feature intends to improve reliability and performance of the Aurora scheduler job updater by relying on health check status rather than watch_secs timeout when decid

aurora git commit: Implement `toString` on lazy modules.

2016-09-12 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master c4903d873 -> f0ff1c4fb Implement `toString` on lazy modules. This will change the help output from: `-shiro_realm_modules (default [org.apache.aurora.scheduler.app.MoreModules$1@158a8276])` to `-shiro_realm_modules (default [class org.ap

aurora git commit: Extend getJobUpdateDetails to accept JobUpdateQuery

2016-09-13 Thread zmanji
org/apache/aurora/gen/api.thrift @@ -896,7 +896,9 @@ struct GetJobUpdateSummariesResult { /** Result of the getJobUpdateDetails call. */ struct GetJobUpdateDetailsResult { + // TODO(zmanji): Remove this once we complete AURORA-1765 1: JobUpdateDetails details + 2: list detailsL

aurora git commit: Aurora admin commands for reconciliation

2016-09-13 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 795a2728c -> 633948ab0 Aurora admin commands for reconciliation * A new command for task reconciliation `reconcile_tasks` was added to the aurora_admin CLI. It takes type of reconciliation and the batch size(for explicit reconciliation

aurora git commit: Fixing connection leak in webhook by making sure stream is closed.

2016-09-26 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 4ead1893b -> 60e5e4e67 Fixing connection leak in webhook by making sure stream is closed. Last refactoring of Webhook did not correctly close out connections so some webhook requests would not complete. Testing Done: Verified in vagrant +

aurora git commit: Add Criteo to Aurora user list

2016-09-28 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 69cba786e -> 1c1c0a597 Add Criteo to Aurora user list Testing Done: N/A Reviewed at https://reviews.apache.org/r/52343/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://git-wip-us.apache.org/repos/asf/aurora/co

aurora git commit: Replace wfarner with zmanji as default reviewer.

2016-10-04 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 249245db0 -> ebdaa6b86 Replace wfarner with zmanji as default reviewer. Reviewed at https://reviews.apache.org/r/52521/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://git-wip-us.apache.org/repos/asf/aur

aurora git commit: Build Debian Jessie and Ubuntu Xenial specific mesos eggs

2016-10-05 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 251ef5a56 -> 640f07bab Build Debian Jessie and Ubuntu Xenial specific mesos eggs Support for making mesos native python eggs for Debian Jessie and Ubuntu Xenial. Testing Done: ./make-mesos-native-egg jessie64 1.0.0 $(pwd)/test ./make-meso

aurora git commit: Add support for receiving min_consecutive_successes in health checker

2016-10-05 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 640f07bab -> e91130e49 Add support for receiving min_consecutive_successes in health checker - Add support for receiving a new HealthCheckConfig attribute "min_consecutive_successes" in health checker. - Add an entry in release note that

aurora git commit: Upgrade pystachio to 0.8.3

2016-10-12 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master e9abb2239 -> 03eff2838 Upgrade pystachio to 0.8.3 Testing Done: src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh Reviewed at https://reviews.apache.org/r/52790/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: htt

aurora git commit: Handle the case where content type header is null.

2016-10-18 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 8e228b681 -> 8c1200894 Handle the case where content type header is null. Per the [documentation](http://docs.oracle.com/javaee/7/api/javax/servlet/ServletRequestWrapper.html#getContentType--) `getContentType` can return `null`. This now h

aurora git commit: Adding an error message when the mesos_containerizer_path is not set correctly.

2016-10-18 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master ad77de1e6 -> 8e228b681 Adding an error message when the mesos_containerizer_path is not set correctly. Testing Done: I verified the new error makes its way to the UI when mesos_containerizer_path is set to a file that does not exist and al

aurora git commit: Enable per task volume mounts via scheduler API

2016-10-31 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 2a8c667ec -> 13d48619e Enable per task volume mounts via scheduler API This allows users to specify volume mounts for tasks using the unified containerizer if the operator permits them. This is analogous to enabling docker parameters per t

aurora git commit: Log TaskInfo and Assigned Task on task startup.

2016-11-04 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 35435cba9 -> fb936b545 Log TaskInfo and Assigned Task on task startup. The executor logs `ExecutorInfo`, `FrameworkInfo`, `SlaveInfo` on startup. This adds logging of `TaskInfo` and the Assigned Task object when it is received. Testing Do

aurora git commit: Send SIGTERM to daemonized processes on shutdown.

2016-11-04 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master fb936b545 -> 5410c229f Send SIGTERM to daemonized processes on shutdown. Problem Processes can deamonize and escape the supervision of a coordinator. Using the Docker Containerizer or the Mesos Containerizer with pid isolation means that

aurora git commit: Fix regression in 5410c22.

2016-11-04 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 5410c229f -> 84bde070f Fix regression in 5410c22. The hard dependency on `prctl` broke thermos unit tests both on Apache Jenkins and OS X. This adopts serb's suggestion https://reviews.apache.org/r/53418/ and wraps the `prcl(2)` call in a

aurora git commit: Upgrade guava to 20.0

2016-11-16 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 889f4b2e3 -> 85e249390 Upgrade guava to 20.0 Release Notes: https://github.com/google/guava/wiki/Release20 It's the usual mix of new features and deprecations. The additions of `Quantiles` and `Stats` could give us some quick improvements

[2/2] aurora git commit: Change job updates to rely on `health-checks` rather than on `watch_secs`.

2016-11-17 Thread zmanji
Change job updates to rely on `health-checks` rather than on `watch_secs`. Make RUNNING a first class state to indicate that the task is running and is healthy. It is achieved by introducing a new configuration parameter `min_consecutive_successes`, which will dictate when to move a task into RUNN

[1/2] aurora git commit: Change job updates to rely on `health-checks` rather than on `watch_secs`.

2016-11-17 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 05f082a1c -> 2992c8b4d http://git-wip-us.apache.org/repos/asf/aurora/blob/2992c8b4/src/test/python/apache/aurora/executor/test_status_manager.py -- diff --git a/src/test/py

aurora git commit: Add benchmarks for `StateManagerImpl`.

2016-11-23 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 4797dfe33 -> b099e2f0f Add benchmarks for `StateManagerImpl`. `StateManagerImpl` is in the middle of every task state transition in the scheduler. Performance improvements here could yield scheduling throughput improvements across the boar

aurora git commit: Improve scheduling throughput via logging changes.

2016-12-02 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 3ea0331d4 -> 4bc524614 Improve scheduling throughput via logging changes. This patch makes two logging performance changes. First, it reduces the cost of logging by replacing the costly class and line patterns with the cheaper logger patt

aurora git commit: Expose stats on SlotSizeCounter runs.

2017-01-13 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 06a221fda -> 264527896 Expose stats on SlotSizeCounter runs. Bugs closed: AURORA-1874 Reviewed at https://reviews.apache.org/r/55477/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://git-wip-us.apache.org/repos

aurora git commit: Exposed stats on number of offers rescinded and number of slaves lost.

2017-01-13 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 264527896 -> b75dfb838 Exposed stats on number of offers rescinded and number of slaves lost. Testing Done: curl -w '\n' 192.168.33.7:8081/vars | grep offers_rescinded % Total% Received % Xferd Average Speed TimeTime Time C

aurora git commit: Make leader elections resilient to ZK disconnections.

2017-01-23 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 75129b694 -> 1e2a9e160 Make leader elections resilient to ZK disconnections. As documented in AURORA-1840 the Curator `LeaderLatch` recipe abdicates leadership if the ZK connection is lost or if there is a timeout. This is not compatible w

aurora git commit: Capture health check output.

2017-01-25 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 3cdcd17a5 -> a8afa59fb Capture health check output. Users really could really benefit from seeing the output of the shell health check failure, so plumbing through the output. Testing Done: added unit tests e2e tests screenshot attached.

aurora git commit: Move deprecated resource validations so they happen after the thrift backfill.

2017-01-30 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 7be7ad6f1 -> 07065b502 Move deprecated resource validations so they happen after the thrift backfill. As the validations for NumCpus, RamMb and DiskMb happened before the thrift backfill, those values needed to be set, even though they are

aurora git commit: Add message parameter to killTasks

2017-02-06 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 7de6d34be -> f88b7f3bf Add message parameter to killTasks RPC's such as pauseJobUpdate include a parameter for "a user-specified message to include with the induced job update state change." This diff provides a similar optional parameter

[2/2] aurora git commit: Move Aurora to v1 Protobufs.

2017-02-06 Thread zmanji
Move Aurora to v1 Protobufs. This is the first step in moving Aurora to the V1 API from Mesos. This patch moves most of the code to v1 Protobufs. This means all peices of code that do not interact with Mesos now handle only v1 Protobufs. Classes that interact with Mesos directly are: * `org.apac

[1/2] aurora git commit: Move Aurora to v1 Protobufs.

2017-02-06 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master f88b7f3bf -> cd4df56c9 http://git-wip-us.apache.org/repos/asf/aurora/blob/cd4df56c/src/test/java/org/apache/aurora/scheduler/mesos/MesosSchedulerImplTest.java -- diff --git

aurora git commit: Add additional tests for the conversion of TaskStatus.

2017-02-08 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 5b7042c4e -> ad3377ab2 Add additional tests for the conversion of TaskStatus. This adds additional testing for the `ProtosConversions` class, ensuring there is the correct conversion between `SlaveID` and `AgentID`. Reviewed at https://re

aurora git commit: Displaying update id after 'Killed for job update' message for the update that resulted in the task getting killed.

2017-02-13 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master ad3377ab2 -> 40d91feb7 Displaying update id after 'Killed for job update' message for the update that resulted in the task getting killed. Testing Done: Tests: -- aurora job create devcluster/www-data/devel/hello_world my_jobs/new_hel

aurora git commit: Add DSL and E2E changes for per task volume mounts.

2017-02-14 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 0e9c0864e -> 9ea897978 Add DSL and E2E changes for per task volume mounts. Enables the client DSL to set per task volume mounts. This also adds a E2E test that tests per task volume mounting. Testing Done: sh ./src/test/sh/org/apache/auro

aurora git commit: Add best effort pulse timestamp recovery.

2017-02-16 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 9ea897978 -> 4ab4b2b2c Add best effort pulse timestamp recovery. Currently the scheduler causes all coordinated ("pulsed") updates into ROLL_FORWARD_AWAITING_PULSE, or ROLL_BACK_AWAITING_PULSE on scheduler startup/recovery. This is because

aurora git commit: Fix scheduler_framework_disconnects stat.

2017-02-27 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master fadfd554d -> 2652fe02a Fix scheduler_framework_disconnects stat. Refactoring in r/31550 has disabled incrementing scheduler_framework_disconnects stats. This change brings it back. Testing Done: Added a check to `MesosSchedulerImplTest.te

[2/2] aurora git commit: Enable Mesos HTTP API.

2017-03-02 Thread zmanji
") private static final Arg ALLOW_GPU_RESOURCE = Arg.create(false); + public enum DriverKind { +// TODO(zmanji): Remove this option once V0_DRIVER has been proven out in production. +// This is the original driver that libmesos shipped with. Uses unversioned protobufs

[1/2] aurora git commit: Enable Mesos HTTP API.

2017-03-02 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 2652fe02a -> 705dbc7cd http://git-wip-us.apache.org/repos/asf/aurora/blob/705dbc7c/src/test/java/org/apache/aurora/scheduler/mesos/MesosSchedulerImplTest.java -- diff --git

aurora git commit: Remove SerializableClock interface.

2017-03-06 Thread zmanji
+ b/commons/src/main/java/org/apache/aurora/common/util/Clock.java @@ -13,20 +13,17 @@ */ package org.apache.aurora.common.util; -import java.io.Serializable; - /** * An abstraction of the system clock. - * * @author John Sirois */ public interface Clock { - + // TODO(zmanji): Consi

aurora git commit: Reduce log output in `VersionedSchedulerDriverService`.

2017-03-08 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 932e90871 -> a07b9edf5 Reduce log output in `VersionedSchedulerDriverService`. The `acceptOffers` log message outputs the entire `Operation` object which for the `LAUNCH` type includes the entire `TaskInfo` protobuf. This makes the log out

aurora git commit: Populate `host` and `webURL`fields of FrameworkInfo.

2017-03-17 Thread zmanji
import org.apache.aurora.scheduler.stats.StatsModule; import org.apache.aurora.scheduler.storage.Storage; @@ -82,6 +83,7 @@ public class SchedulerMain { @CmdLine(name = "serverset_path", help = "ZooKeeper ServerSet path to register at.") private static final Arg SERVERSET_PATH = Arg.create(); +

aurora git commit: Support Mesos Maintenance

2017-03-23 Thread zmanji
til; +import java.time.Instant; + /** * An abstraction of the system clock. * @author John Sirois @@ -21,7 +23,6 @@ public interface Clock { // TODO(zmanji): Consider replacing this with java.time.Clock /** * A clock that returns the the actual time reported by the system. - * This clock

aurora git commit: Reset `framework_registered` metric on disconnection.

2017-03-29 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master c32f14c75 -> 3a9aabda9 Reset `framework_registered` metric on disconnection. Previously the `framework_registered` metric only transitioned from 0 to 1 on the first registration. On disconnection and registration loss, the metric was not u

aurora git commit: Ensure enum tables are complete after a snapshot restore.

2017-03-30 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 42a497438 -> 076d9177b Ensure enum tables are complete after a snapshot restore. In our in memory database, we model enums as two column tables. The two columns would be `id` which corresponds to the integer value in the thrift enum and `n

aurora git commit: Reliably subscribe to Mesos in the HTTP Driver.

2017-04-06 Thread zmanji
kHandler.class).to(MesosCallbackHandlerImpl.class); bind(MesosCallbackHandlerImpl.class).in(Singleton.class); // TODO(zmanji): Create singleThreadedExecutor (non-scheduled) variant. - bind(Executor.class).annotatedWith(MesosCallbackHandlerImpl.SchedulerExecutor.class) +bind(Executor.class).annotatedWith(Sched

aurora git commit: Fix schema to allow multiple task volumes per task.

2017-04-07 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 656cf9ac5 -> cc2aa46f7 Fix schema to allow multiple task volumes per task. The original commit adding this feature added an artifical constraint to the schema that prevented more than one task volume per task. This is because there was a `

aurora git commit: Process rescinds in the same thread pool as offers.

2017-06-06 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 2cbaeecce -> 73234d09d Process rescinds in the same thread pool as offers. In a a production environment I was able to observe the following: ``` I0606 00:31:32.510 [Thread-77638, MesosCallbackHandler$MesosCallbackHandlerImpl:229] Offer r

svn commit: r19973 - /release/aurora/KEYS

2017-06-09 Thread zmanji
Author: zmanji Date: Fri Jun 9 22:13:24 2017 New Revision: 19973 Log: Add GPG key for santhk@ Modified: release/aurora/KEYS Modified: release/aurora/KEYS == --- release/aurora/KEYS (original) +++ release/aurora

aurora git commit: Remove restriction on task id length.

2017-06-12 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 06518c86b -> 40d9d4dbe Remove restriction on task id length. To work around an old Mesos bug (MESOS-691) we would reject jobs that resulted in Mesos task ids longer than 255 characters. This is because Mesos used to use the task id to gene

aurora git commit: Remove url related methods out of AuroraCommandContext

2015-04-07 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 5506bfa27 -> e8e1caaa7 Remove url related methods out of AuroraCommandContext This is a refactor of AuroraCommandContext which removes all url related methods out of it. The objective of this refactor is to remove functionality from Aurora

aurora git commit: Bump pants version to 0.0.32.

2015-04-08 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master e8e1caaa7 -> 35f3e8dae Bump pants version to 0.0.32. >From the [release notes](http://pantsbuild.github.io/changelog.html) we get PEX improvements and a bunch of python related bug fixes. Testing Done: build-support/jenkins/build.sh src/t

aurora git commit: Revert "Bump pants version to 0.0.32."

2015-04-08 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 35f3e8dae -> 246058df3 Revert "Bump pants version to 0.0.32." qhis reverts commit 35f3e8daea815facccd1f5e577fe5a682bd02516. This reverts pants to 0.0.28. After some extensive testing I have found some flakyness in how pants 0.0.32 handles

aurora git commit: Place api_util.py inside a `pants_library` target.

2015-05-18 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 16c990f59 -> 920263b0e Place api_util.py inside a `pants_library` target. This places `api_util.py` inside a `pants_library` target and has the two consumer targets depend on that file explicitly. Testing Done: ./pants test.pytest --no-fa

aurora git commit: Use TaskStatus Reason to set memory limit message.

2015-06-03 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 54d561eee -> ed4415c12 Use TaskStatus Reason to set memory limit message. Use TaskStatus Reason to set memory limit message instead of checking the contents of the message field. Future versions of Mesos can change the diagnostic informati

aurora git commit: Allow equals in bound variables passed via cmdline

2015-06-22 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 3351b09b8 -> 2df2db951 Allow equals in bound variables passed via cmdline The parsing for the --bind parameter splits the parameter value on equals and enforces only two elements. This prevents parameters whose value contains an equals to

aurora git commit: Remove "enable_legacy_constraints" flag.

2015-06-26 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 9ab3ede57 -> 2ef6a05e2 Remove "enable_legacy_constraints" flag. Remove the "enable_legacy_constraints" flag and associated behaviour. Testing Done: ./gradlew build -Pq Bugs closed: AURORA-1074 Reviewed at https://reviews.apache.org/r/35

aurora git commit: Prevent quota from being set below current production reservation.

2015-07-01 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 1024a8579 -> 97520ba4d Prevent quota from being set below current production reservation. Having quota below current production reservation is illogical and the scheduler should reject requests that would result in this. Bugs closed: AURO

aurora git commit: Removes version on virtual pacakge in debian control

2015-07-15 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master d9dac92d3 -> e63c518e7 Removes version on virtual pacakge in debian control java-runtime-headless is a virtual package and cannot have a version specified. The consequence of this is that the built packages require openjdk-8 be installed w

aurora git commit: Added fix for client to connect to correct zookeeper-hosts address/port pairs.

2015-07-22 Thread zmanji
Repository: aurora Updated Branches: refs/heads/master 0070a5fd1 -> 8bdfb8500 Added fix for client to connect to correct zookeeper-hosts address/port pairs. When more than one zookeeper host was specified in the clusters.json file, only the last host was accessed through the specified zk_port.

[20/37] aurora git commit: Import of Twitter Commons.

2015-08-25 Thread zmanji
http://git-wip-us.apache.org/repos/asf/aurora/blob/86a547b9/commons/src/main/java/com/twitter/common/util/Stat.java -- diff --git a/commons/src/main/java/com/twitter/common/util/Stat.java b/commons/src/main/java/com/twitter/common

[15/37] aurora git commit: Import of Twitter Commons.

2015-08-25 Thread zmanji
http://git-wip-us.apache.org/repos/asf/aurora/blob/86a547b9/commons/src/main/resources/com/twitter/common/application/http/graphview/dygraph-extra.js -- diff --git a/commons/src/main/resources/com/twitter/common/application/http/g

[22/37] aurora git commit: Import of Twitter Commons.

2015-08-25 Thread zmanji
http://git-wip-us.apache.org/repos/asf/aurora/blob/86a547b9/commons/src/main/java/com/twitter/common/thrift/ThriftFactory.java -- diff --git a/commons/src/main/java/com/twitter/common/thrift/ThriftFactory.java b/commons/src/main/j

  1   2   3   >