----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57487/ -----------------------------------------------------------
(Updated July 5, 2017, 9:50 p.m.) Review request for Aurora, Mehrdad Nurolahzade, Stephan Erb, and Zameer Manji. Changes ------- Full patch running in our prod. Note: if you use dynamic resource estimator (for oversubscription), use Mesos > 1.2.1. Also, this patch contains small orthogonal fixes: allowing rootfs and changing task id length. I believe no harm in adding them since I don't want to unresolve merge conflicts. Repository: aurora Description ------- Esteemed reviewers, here is the latest iteration on the implementation of dynamic reservations. Some changes are merging of the patches into a single one, updated design document with a more high level overview and user stories told from an operator’s point of view. Unit TESTS are going to be done as soon as we agree on the approach, as I have tested this patch on local vagrant and a multi-node dev cluster. Jenkins build is expected to fail as tested are incomplete. For reference, here are previous two patches which feedback I addressed in this new single patch. Previous 2 patches: https://reviews.apache.org/r/56690/ https://reviews.apache.org/r/56691/ RFC document: https://docs.google.com/document/d/15n29HSQPXuFrnxZAgfVINTRP1Iv47_jfcstJNuMwr5A Design Doc [UPDATED]: https://docs.google.com/document/d/1L2EKEcKKBPmuxRviSUebyuqiNwaO-2hsITBjt3SgWvE Diffs (updated) ----- examples/vagrant/mesos_config/etc_mesos-slave/resources aa0e97e1c4a6c1a76cc712549159db9336d051eb examples/vagrant/upstart/aurora-scheduler.conf 63fcc87be653835cb3c3f25dae4164f1d7c8d4da src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java f2296a9d7a88be7e43124370edecfe64415df00f src/jmh/java/org/apache/aurora/benchmark/StatusUpdateBenchmark.java c81387f24d554bcb2ee73e49028ba068ad11e4d6 src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java 6f2ca35c5d83dde29c24865b4826d4932e96da80 src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1 src/main/java/org/apache/aurora/scheduler/TaskVars.java 676dfd9f9d7ee0633c05424f788fd0ab116976bb src/main/java/org/apache/aurora/scheduler/TierInfo.java c45b949ae7946fc92d7e62f94696ddc4f0790cfa src/main/java/org/apache/aurora/scheduler/base/InstanceKeys.java b12ac83168401c15fb1d30179ea8e4816f09cd3d src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java f0b148cd158d61cd89cc51dca9f3fa4c6feb1b49 src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java 754fde0fdc976b673d78ae15d8ccd8c85b792373 src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorModule.java 4dac9757a65e144142d36ee921b85a02a5311fe5 src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 src/main/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilter.java f6c759f03c4152ae93317692fc9db202fe251122 src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 36608a9f027c95723c31f9915852112beb367223 src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java df51d4cf4893899613683603ab4aa9aefa88faa6 src/main/java/org/apache/aurora/scheduler/mesos/FrameworkInfoFactory.java 8ad9d216ad14534ec53ab506e92b669632db891a src/main/java/org/apache/aurora/scheduler/mesos/MesosCallbackHandler.java 5a5281aeaea1e2a4e0eab67069605838ee809c6c src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 0d639f66db456858278b0485c91c40975c3b45ac src/main/java/org/apache/aurora/scheduler/mesos/TestExecutorSettings.java e1cd81e6fbd98f23046e6e775be268be4310c62a src/main/java/org/apache/aurora/scheduler/mesos/VersionedSchedulerDriverService.java 5e86504c70083065278864e6ab1cc85c83a45a28 src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 78255e6dfa31c4920afc0221ee60ec4f8c2a12c4 src/main/java/org/apache/aurora/scheduler/offers/OfferSettings.java adf7f33e4a72d87c3624f84dfe4998e20dc75fdc src/main/java/org/apache/aurora/scheduler/offers/OffersModule.java 317a2d26d8bfa27988c60a7706b9fb3aa9b4e2a2 src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 5ed578cc4c11b49f607db5f7e516d9e6022a926c src/main/java/org/apache/aurora/scheduler/resources/AcceptedOffer.java 291d5c95916915afc48a7143759e523fccd52feb src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java 9aa263a9cfae03a9a0c5bc7fe3a1405397d3009c src/main/java/org/apache/aurora/scheduler/resources/ResourceMapper.java 375f93c5277a78666fc4823382c82ac4d179f39d src/main/java/org/apache/aurora/scheduler/scheduling/ReservationTimeoutCalculator.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/scheduling/SchedulingModule.java 03a0e8485d1a392f107fda5b4af05b7f8f6067c6 src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 203f62bacc47470545d095e4d25f7e0f25990ed9 src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java a177b301203143539b052524d14043ec8a85a46d src/main/java/org/apache/aurora/scheduler/stats/AsyncStatsModule.java 40451e91aed45866c2030d901160cc4e084834df src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 059fbb86a575f5b3d78a63c9a7b5a9eebb6cb3ae src/main/resources/org/apache/aurora/scheduler/tiers.json 34ddb1dc769a73115c209c9b2ee158cd364392d8 src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 src/test/java/org/apache/aurora/scheduler/app/SchedulerIT.java a7c9c83eebbbea7ae8a6c807f98d3ce8bd050137 src/test/java/org/apache/aurora/scheduler/base/InstanceKeysTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java 50d7499f4332a3feb0e2301cb707f2cea6bb2e98 src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java bf9c2b46b186125061c29ac777134a871c727066 src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java d2fe7a3d78c9c12ca3d0b40843eb7351217fe1fb src/test/java/org/apache/aurora/scheduler/http/OffersTest.java 30699596a1c95199df7504f62c5c18cab1be1c6c src/test/java/org/apache/aurora/scheduler/mesos/FrameworkInfoFactoryImplTest.java b22f047ba8ead5d840e13e379ec4471288b21118 src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 93cc34cf8393f969087cd0fd6f577228c00170e9 src/test/java/org/apache/aurora/scheduler/offers/HostOffers.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java d7addc0effb60c196cf339081ad81de541d05385 src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java c39b00d65c8c2f2f74c836c6117a651fb7f9cd05 src/test/java/org/apache/aurora/scheduler/resources/AcceptedOfferTest.java dded9c34749cf599d197ed312ffb6bf63b6033f1 src/test/java/org/apache/aurora/scheduler/resources/PortMapperTest.java cef795bff131df46a50b632eaf2261df0b8dc3e7 src/test/java/org/apache/aurora/scheduler/resources/ResourceManagerTest.java b8b8edb1a21ba89b8b60f8f8451c8c776fc23ae8 src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 676d305d257585e53f0a05b359ba7eb11f1b23be src/test/java/org/apache/aurora/scheduler/scheduling/ReservationTimeoutCalculatorTest.java PRE-CREATION src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java fa1a81785802b82542030e1aae786fe9570d9827 src/test/java/org/apache/aurora/scheduler/sla/SlaTestUtil.java 78f440f7546de9ed6842cb51db02b3bddc9a74ff src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java cf2d25ec2e407df7159e0021ddb44adf937e1777 src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 016859ca3bf83f64d2576b4c7109729770f9e25c src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 0361b36be3bc1f7d925b616d67dde5c64fd7a909 src/test/sh/org/apache/aurora/e2e/http/http_example_bad_healthcheck.aurora c826a54809ea8bcd25fa04a8fa15ecd414a53e30 src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora b85907a955a1ba83012a26912938b6466f0476ac src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 1a81dc5dcde0400510c5576ac523f4d3f14424ca Diff: https://reviews.apache.org/r/57487/diff/8/ Changes: https://reviews.apache.org/r/57487/diff/7-8/ Testing ------- Tested on local vagrant for following scenarios: Reserving a task Making sure returned offer comes back Making sure offer is unreserved Thanks, Dmitriy Shirchenko
