> On Jan. 21, 2014, 11:39 p.m., Benjamin Hindman wrote: > > src/slave/container/containerizer.hpp, line 153 > > <https://reviews.apache.org/r/16147/diff/6/?file=425462#file425462line153> > > > > Doing a fetch should not block. We should probably have some > > abstractions around doing the operations required for a fetch > > asynchronously. Worst case, we should do the fetching out of process and > > have an abstraction for doing that asynchronously. Doing fetch out of > > process might make sense anyway because then one could integrate with their > > own caching or storage layers more easily.
Moved fetch/extract out to an external fetcher which can be executed in a Subprocess. > On Jan. 21, 2014, 11:39 p.m., Benjamin Hindman wrote: > > src/slave/container/mesos_containerizer.cpp, lines 528-529 > > <https://reviews.apache.org/r/16147/diff/6/?file=425466#file425466line528> > > > > The flow around container termination seems a bit flawed. If you're > > asking the launche to destroy a container AND setting the promise below > > that means that the slave is possibly finding out about a terminated > > container before it's actually completely destroyed. I think the launcher > > should have something like 'wait' as well that the containerizer uses to > > gate when it returns to the slave. In fact, you might be able to make > > Containerizer::wait be a simple wrapper around Launcher::wait. Reworked flow so all destroy paths merge at MesosContainerizer::destroy() and promise is set after all processes have been killed, the isolators have cleaned up, and the executor pid has been reaped. > On Jan. 21, 2014, 11:39 p.m., Benjamin Hindman wrote: > > src/slave/slave.cpp, line 2041 > > <https://reviews.apache.org/r/16147/diff/6/?file=425478#file425478line2041> > > > > What are the ramifications of the overcommit with respect to other > > containers? Should we be moving this closer to the acceptance of the task > > (i.e., in the master?). > > Ian Downes wrote: > Previously, the (initial) overcommit was done in the cgroups isolator > where default values for memory and cpu were used if unspecified. I think > better than adding an allocation is (a) if an executor is specified then its > resources must be too, and (b) if no executor is specified then the task > resources are used. Thoughts? We now include the first task's resources when launching a new executor so this code has been pruned. - Ian ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16147/#review32409 ----------------------------------------------------------- On Jan. 31, 2014, 11:51 p.m., Ian Downes wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/16147/ > ----------------------------------------------------------- > > (Updated Jan. 31, 2014, 11:51 p.m.) > > > Review request for mesos, Benjamin Hindman, Ben Mahler, Chi Zhang, Niklas > Nielsen, samya, and Jason Dusek. > > > Repository: mesos-git > > > Description > ------- > > The proposed Containerizer interface is to replace the existing Isolator. > > One ContainerizerProcess has been written: > MesosContainerizerProcess - implements containerizeration internally using a > Launcher and one or more Isolators (following review) > > The intent is to also support a generic ExternalContainerizerProcess that can > delegate containerizeration by making external calls. Other Containerizers > could interface with specific external containerization techniques such as > Docker or LXC. > > > Diffs > ----- > > 3rdparty/libprocess/3rdparty/stout/Makefile.am > 5d5a76021900bf27113fe5b730cfd2b6a537d132 > 3rdparty/libprocess/3rdparty/stout/include/stout/format.hpp > 3eadaefad1945a2cf55ed67e5ee02eaffce76fea > 3rdparty/libprocess/3rdparty/stout/include/stout/list.hpp > 60f0c8c67317872883857a1cd8f39d6d6a45d0c1 > 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp > 08428b8f38f08674743ec0d07ec53bee8649fadd > 3rdparty/libprocess/Makefile.am 40f01a7b3803696ccca440c8326e1d6d7c377459 > 3rdparty/libprocess/include/process/statistics.hpp > a4f1db3a8a219c39193a1d237477f0350e47e681 > 3rdparty/libprocess/include/process/timeseries.hpp > b368b9bb9cd63fb331b2c137dcde759d0ee59ee6 > 3rdparty/libprocess/src/process.cpp > bc7a1c5890df7c763b0b140b47966e457e519208 > 3rdparty/libprocess/src/statistics.cpp > 75aac4074d33cb5054da6c8b0bd4a890c2eaf80e > 3rdparty/libprocess/src/tests/statistics_tests.cpp > 3521bd565dae8fcbba464f2539b3b14a37a037f0 > 3rdparty/libprocess/src/tests/timeseries_tests.cpp > fd906b869e8bbf5c07c67c5319dec82681e811b7 > CHANGELOG 905c48996c37b33253deb00c156682fd1b14eedf > configure.ac aa6ee45bc477dd81b6f0cb389ef9f16f8216d6df > docs/home.md df8e05a1aed4040325ae4258f05f7cbda6dcdb8e > docs/logging-and-debugging.md 1f037d6cdbc9e76976a796961c300d8842a16601 > docs/powered-by-mesos.md 6df9490c3df28f2097927d3e707a62099fe2b460 > docs/upgrades.md dac0ef8ac62e97e6dc88b0e1a42e042a8d775f0c > include/mesos/mesos.proto 1503e73f0d299f8efb209e0b10966c9a7f8db5f8 > src/Makefile.am d58b46e99e0a041cf2a26abe44bbd1504a9539c0 > src/common/type_utils.hpp fe6bf71d689f7bfd8b6ae1b8fab9b2e76e28e7a8 > src/launcher/fetcher.cpp PRE-CREATION > src/launcher/launcher.hpp 104fe812ddd2b75b14ec0b40d192f58cdba3603a > src/launcher/launcher.cpp d5ab66704429a95eeb8eda5188e33d8e691221af > src/launcher/main.cpp de64609905ee63096c0173fe7e64a1eafea5d6bf > src/local/local.cpp 83a7f913afb1441e9137c7fcec8fd81815714905 > src/log/recover.cpp ee38acd280fab53b8584f68ea9dfbb496820d022 > src/master/constants.hpp cdaaad060d4ee777f8b0838b63c0fd031da861ea > src/master/constants.cpp 8a48bbbe1d452919562b21f9b73680c42271a207 > src/master/contender.hpp 0048ee026afc7034a09dd71f0a65c0fb55e8c77f > src/master/contender.cpp e3b0737195aba42c805a05c96b054cec1471b502 > src/master/detector.hpp 277c9d907120a9ba30948d8ab4baaa2586293a62 > src/master/detector.cpp 2b169c551affe7acd2feac7806a27b46eb99bb88 > src/master/master.hpp 99b81811d596a777ee83dce7b157c1b11c064fab > src/master/master.cpp c7d9186b9c22fb40b05aa6d5bc6f2d38fb7f73ea > src/slave/cgroups_isolator.hpp e86062e9abaaa263c32c55e9dbfefd700f605886 > src/slave/cgroups_isolator.cpp 80155a3f0cd684a6aad9a37f61c5204d86032735 > src/slave/containerizer/containerizer.hpp PRE-CREATION > src/slave/containerizer/containerizer.cpp PRE-CREATION > src/slave/containerizer/mesos_containerizer.hpp PRE-CREATION > src/slave/containerizer/mesos_containerizer.cpp PRE-CREATION > src/slave/flags.hpp 827b2d0d6dc8fa279f3187a09e5dc6d4799d17cd > src/slave/http.cpp c8357e214d2adf2cd712072f58d07b07badb79dc > src/slave/isolator.hpp 9634535d8c746597b4bb6e278587a1b9ca8f1608 > src/slave/isolator.cpp c9643cf9c5657bc142482a71fb161233bffb3b9f > src/slave/main.cpp e0cae7b205c2599e05c4db990cc9c8e9e3673c37 > src/slave/monitor.hpp b677410e20eeafa7eac9acb4e687af7c8d53809f > src/slave/monitor.cpp bb3723e967b0723bc2b925019b6e9c4170dcc071 > src/slave/paths.hpp 70ee0f31ebb294011e5ec6b05ce3ad28b2c4bacc > src/slave/process_isolator.hpp 4ae093fe65775a2b9bec42071961dd58aa0c3d8b > src/slave/process_isolator.cpp 0bc698f04f7c8eaad166dc9d646e13310129dd01 > src/slave/slave.hpp b8acc1866353f4863aee5e2d3db4ac1f9d2a48a5 > src/slave/slave.cpp aeecca0b8631c4b947d505a99d705f909d5986f7 > src/slave/state.hpp 78b20ffae1f2f629d851784302648e3bf5349321 > src/slave/state.cpp 6c382cd30602ede60b7d526acc272adc16285a93 > src/slave/status_update_manager.hpp > 06ea4659cdd24cb1b82f389f404566ba14a663fb > src/slave/status_update_manager.cpp > 03f5eafefd6ed748bfd4511d654c23c7b460db66 > src/tests/group_tests.cpp 5d4240c5fb32db97a8a0920d90af5400a0900dfd > src/tests/log_tests.cpp e493af4f2f2435efe168d07acd267b61afd37fe4 > src/tests/master_contender_detector_tests.cpp > 5223200ee59b9ce0e1d228320e8ad9e84a567288 > src/tests/master_tests.cpp f1486cecec2fdc1d56624de150bc8cb18df76b65 > src/tests/monitor_tests.cpp 7988c90b4b383bd822d1294a7f0318fbd9160dc6 > src/tests/slave_recovery_tests.cpp 999e59857fd5a02de24bd15c9253d7a42df816ac > src/tests/zookeeper_tests.cpp 615338adf2a66bdf2d8c48e63783f054612bb958 > src/webui/master/static/js/controllers.js > afb24fb9c2184772f7314162f5637dbabaa2ab94 > src/webui/master/static/js/services.js > 8403aad9b2586ac3c1bea3f75ca38f6f907087a2 > src/webui/master/static/slave.html 09bfcf29a700168938f4ec4fa42ca8383fa55511 > src/webui/master/static/slave_executor.html > 7c66405090f46f89bdd29806a58c05dc76c0ad23 > src/webui/master/static/slave_framework.html > 90eff2f3b4de08f98283b27e1635ba23a855e488 > src/zookeeper/contender.hpp 65292450d8c800d3e6a7e8e4db7c8d48bc79a79d > src/zookeeper/contender.cpp 6710da4e64fc0a43c1eabfc0f39fb0133c13df14 > src/zookeeper/detector.hpp 73235c0f2a339e27fa86f3bcb5e9a7a08e96f7f8 > src/zookeeper/detector.cpp e186e5110341dd2fb94ddf50fed1b2819f70afb4 > src/zookeeper/group.hpp e51ebb2cf5f09a633462c101f913ee8272be9a6c > src/zookeeper/group.cpp ecb6c002e8194b8d67e262826d988f747414f9f3 > src/zookeeper/zookeeper.hpp f50aca6e7035c8084c3e76fd56b9d1ef7f9d9902 > support/tag.sh 3977f4fce0200069f2265bfd8a03a611c2fa1849 > > Diff: https://reviews.apache.org/r/16147/diff/ > > > Testing > ------- > > > Thanks, > > Ian Downes > >
