Re: Review Request 31985: Mesos container ID available to the executor through an environment variable.
On March 14, 2015, 1:49 a.m., Timothy Chen wrote: The change looks good, but I'm not sure how exposing the container id is the right thing to do overall yet. Container id as I know of is meant to be a internal id that is used only in mesos, and I believe the whole motivation was for users to be able to guess the docker container name from the container id. However, container id - docker container name mapping might change since we manage that, and I'm progress changing it to include the slave id. I personally think we should think about exposing container specific information. Do you know if there is any other use case to know the container id? Alexander Rojas wrote: 1. I am not aware of any other use case. However I asked benh about this and he mentioned it was ok to have this as extra info on the update, as long as it was no docker specific data. 2. After asking some questions to the original reporter, the idea was more to be able to group tasks assigned to the same container, but not as a way to extract the specific container. 3. I am not sure, then, what would be considered mesos private info and info which can be shared. For example, why can the framework id and the executor id be shared but no the container id? Vinod Kone wrote: group tasks assigned to the same container .. What does this mean? IIUC, our docker containerizer only supports single task containers. regarding why framework id and executor id are exposed: framework id is needed by frameworks to reregister with master. executor id (and task id) is generated by the framework and not mesos. Alexander Rojas wrote: So I guess then, I will discard this patch and set the issue to won't fix? Vinod Kone wrote: I think we should try to understand the root of the issue that the reporter is having before jumping onto a specific implementation. Alexander Rojas wrote: Well, I felt like I had it clear but apparently not. Can you please ask the questions in the Jira entry vinod? I was also wondering, As mentioned above, we want to keep the `containerId` private within mesos, but patch [32426](https://reviews.apache.org/r/32426) effectively makes it public. Alexander Rojas wrote: I got it now… it is still quite private. Forget my question on the second paragraph. Alexander Rojas wrote: One more thing Vinod, you wrote one container per task, however if you check the code (Slave.cpp at `Executor* Framework::launchExecutor(ExecutorInfo, TaskInfo)`) what we have is one container per executor. Timothy Chen wrote: Docker containerizer supports multiple tasks only if the container is a custom executor, so you get grouping naturally through that. From what I understand most people wants the container ID so they can get docker specific information like name, IP, etc. I think by sending this all back through the docker executor we shouldn't need this anymore. Hey guys, I asked the reporter of the issue the reason why he needs it. He wrote a full explanation on the [JIRA Issue](https://issues.apache.org/jira/browse/MESOS-2191?focusedCommentId=14387724page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14387724), so if you can read that and then decide if it is worth commiting this patch or not. - Alexander --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31985/#review76470 --- On March 20, 2015, 10:27 a.m., Alexander Rojas wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31985/ --- (Updated March 20, 2015, 10:27 a.m.) Review request for mesos, Bernd Mathiske, Isabel Jimenez, Joerg Schad, Till Toenshoff, and Vinod Kone. Bugs: MESOS-2191 https://issues.apache.org/jira/browse/MESOS-2191 Repository: mesos Description --- When the executor is created, the container ID where it runs is made accesible through an environment variable. Diffs - src/exec/exec.cpp d678f0682d803b0b080c3a6c296067ac9ab5dbf8 src/slave/containerizer/containerizer.hpp 129e60f20835f5d151701e934330b81825887af1 src/slave/containerizer/containerizer.cpp 4d66e767de1f877cb66b37826ba7c9d00639a7c0 src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88 src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 src/slave/containerizer/external_containerizer.cpp 42c67f548caf7bddbe131e0dfa7d74227d8c2593 src/slave/containerizer/mesos/containerizer.cpp fbd1c0a0e5f4f227adb022f0baaa6d2c7e3ad748 src/tests/containerizer.cpp
Re: Review Request 31985: Mesos container ID available to the executor through an environment variable.
On March 14, 2015, 12:49 a.m., Timothy Chen wrote: The change looks good, but I'm not sure how exposing the container id is the right thing to do overall yet. Container id as I know of is meant to be a internal id that is used only in mesos, and I believe the whole motivation was for users to be able to guess the docker container name from the container id. However, container id - docker container name mapping might change since we manage that, and I'm progress changing it to include the slave id. I personally think we should think about exposing container specific information. Do you know if there is any other use case to know the container id? Alexander Rojas wrote: 1. I am not aware of any other use case. However I asked benh about this and he mentioned it was ok to have this as extra info on the update, as long as it was no docker specific data. 2. After asking some questions to the original reporter, the idea was more to be able to group tasks assigned to the same container, but not as a way to extract the specific container. 3. I am not sure, then, what would be considered mesos private info and info which can be shared. For example, why can the framework id and the executor id be shared but no the container id? Vinod Kone wrote: group tasks assigned to the same container .. What does this mean? IIUC, our docker containerizer only supports single task containers. regarding why framework id and executor id are exposed: framework id is needed by frameworks to reregister with master. executor id (and task id) is generated by the framework and not mesos. Alexander Rojas wrote: So I guess then, I will discard this patch and set the issue to won't fix? Vinod Kone wrote: I think we should try to understand the root of the issue that the reporter is having before jumping onto a specific implementation. Alexander Rojas wrote: Well, I felt like I had it clear but apparently not. Can you please ask the questions in the Jira entry vinod? I was also wondering, As mentioned above, we want to keep the `containerId` private within mesos, but patch [32426](https://reviews.apache.org/r/32426) effectively makes it public. Alexander Rojas wrote: I got it now… it is still quite private. Forget my question on the second paragraph. Alexander Rojas wrote: One more thing Vinod, you wrote one container per task, however if you check the code (Slave.cpp at `Executor* Framework::launchExecutor(ExecutorInfo, TaskInfo)`) what we have is one container per executor. Timothy Chen wrote: Docker containerizer supports multiple tasks only if the container is a custom executor, so you get grouping naturally through that. From what I understand most people wants the container ID so they can get docker specific information like name, IP, etc. I think by sending this all back through the docker executor we shouldn't need this anymore. Alexander Rojas wrote: Hey guys, I asked the reporter of the issue the reason why he needs it. He wrote a full explanation on the [JIRA Issue](https://issues.apache.org/jira/browse/MESOS-2191?focusedCommentId=14387724page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14387724), so if you can read that and then decide if it is worth commiting this patch or not. I think we shouldn't, as I guessed correctly he just wants the container name. Since the container is internally managed by Mesos, it's subject to change any time and you can't always guess that it's mesos- + containerId. I'm proposed a solution that I plan to put in that JIRA. - Timothy --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31985/#review76470 --- On March 20, 2015, 9:27 a.m., Alexander Rojas wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31985/ --- (Updated March 20, 2015, 9:27 a.m.) Review request for mesos, Bernd Mathiske, Isabel Jimenez, Joerg Schad, Till Toenshoff, and Vinod Kone. Bugs: MESOS-2191 https://issues.apache.org/jira/browse/MESOS-2191 Repository: mesos Description --- When the executor is created, the container ID where it runs is made accesible through an environment variable. Diffs - src/exec/exec.cpp d678f0682d803b0b080c3a6c296067ac9ab5dbf8 src/slave/containerizer/containerizer.hpp 129e60f20835f5d151701e934330b81825887af1 src/slave/containerizer/containerizer.cpp 4d66e767de1f877cb66b37826ba7c9d00639a7c0 src/slave/containerizer/docker.hpp b7bf54ac65d6c61622e485ac253513eaac2e4f88
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/#review80018 --- Ship it! LGTM assuming the changes only involve code move only :-). src/authentication/cram_md5/authenticatee.hpp https://reviews.apache.org/r/32850/#comment129768 I think this `#include` can be moved to the cpp file as well. - Kapil Arya On April 9, 2015, 10:58 p.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 9, 2015, 10:58 p.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs - src/Makefile.am fa609da src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Review Request 33174: Fix for docker not configuring CFS quotas correctly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33174/ --- Review request for mesos. Bugs: MESOS-2617 https://issues.apache.org/jira/browse/MESOS-2617 Repository: mesos Description --- Fix for docker containerizer not configuring CFS quotas correctly. It would be nice to refactor all this isolation code in a way that can be shared between all containerizers, as this is basically just copied from the CgroupsCpushareIsolator, but that's a much bigger undertaking. Diffs - src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 Diff: https://reviews.apache.org/r/33174/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 32961: Allow framework re-registeration to update master http fields.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32961/ --- (Updated April 14, 2015, 4:30 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- add framework id to warning log. Summary (updated) - Allow framework re-registeration to update master http fields. Bugs: MESOS-1218, MESOS-2614 and MESOS-703 https://issues.apache.org/jira/browse/MESOS-1218 https://issues.apache.org/jira/browse/MESOS-2614 https://issues.apache.org/jira/browse/MESOS-703 Repository: mesos Description --- Fields: 'name', 'hostname', 'failover_timeout', 'webui_url' Diffs (updated) - src/master/master.hpp 6141917644b84edfed9836fa0a005d55a36880e3 src/master/master.cpp 44b0a0147f5354824d86332a67b30018634c9a36 Diff: https://reviews.apache.org/r/32961/diff/ Testing --- make check. re-registered no_executor_framework with different 'name', 'hostname', 'failover_timeout', and 'webui_url' Thanks, Joris Van Remoortere
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 14, 2015, 2:20 p.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Changes --- Addressed Kapil's comment. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs (updated) - src/Makefile.am d15a373 src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Review Request 33174: Fix for docker not configuring CFS quotas correctly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33174/#review80053 --- Patch looks great! Reviews applied: [33174] All tests passed. - Mesos ReviewBot On April 14, 2015, 4:44 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33174/ --- (Updated April 14, 2015, 4:44 p.m.) Review request for mesos, Ian Downes, Jie Yu, and Timothy Chen. Bugs: MESOS-2617 https://issues.apache.org/jira/browse/MESOS-2617 Repository: mesos Description --- Fix for docker containerizer not configuring CFS quotas correctly. It would be nice to refactor all this isolation code in a way that can be shared between all containerizers, as this is basically just copied from the CgroupsCpushareIsolator, but that's a much bigger undertaking. Diffs - src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 Diff: https://reviews.apache.org/r/33174/diff/ Testing --- Thanks, Steve Niemitz
Re: Oversubscription design doc
We have a dedicated IRC channel for this: mesos-oversubscription. Feel free to drop in and chat. On Tue, Apr 14, 2015 at 11:09 AM, Niklas Nielsen nik...@mesosphere.io wrote: Hi everyone, In context of some of the recent discussions on 'Resource overcommittal', we are very interested and invested in enabling oversubscription in Mesos in a safe and efficient way. Some of the community members have been working on an architecture document and prototypes recently and feel we have something we can start working out of: https://docs.google.com/document/d/1pUnElxHy1uWfHY_FOvvRC73QaOGgdXE0OXN-gbxdXA0/edit# We will continue meeting up and discussing the approach, as it is a significant change to how workloads are run on Mesos. Feel free to jump in, add suggestions, raise concerns and participate in the development of this feature. Cheers, Niklas
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/#review80072 --- src/authentication/cram_md5/authenticatee.cpp https://reviews.apache.org/r/32850/#comment129844 Actually according to Google's styleguide this should be the first include, shouldn't it? http://google-styleguide.googlecode.com/svn/trunk/cppguide.html#Names_and_Order_of_Includes I know we don't do that at other places but actually it would prevent a number of Alexs comments (and be consistent with the styleguide)... - Joerg Schad On April 14, 2015, 2:20 p.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 14, 2015, 2:20 p.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs - src/Makefile.am d15a373 src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
On April 14, 2015, 7:22 p.m., Kapil Arya wrote: src/authentication/cram_md5/authenticatee.cpp, lines 46-47 https://reviews.apache.org/r/32850/diff/3/?file=922370#file922370line46 According to our style guide, this is okay but we keep seeing issues being raised about this. Should we just update the style guide instead? The styleguide is IMHO unclear on this case as it proposes similar examples but only being ok. Definitely a case for an update suggestion. - Till --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/#review80082 --- On April 14, 2015, 2:20 p.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 14, 2015, 2:20 p.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs - src/Makefile.am d15a373 src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Review Request 32954: Added a 'slave_shutdowns_completed' metric.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32954/#review80139 --- Ship it! Ship It! - Vinod Kone On April 8, 2015, 2:29 a.m., Ben Mahler wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32954/ --- (Updated April 8, 2015, 2:29 a.m.) Review request for mesos and Vinod Kone. Repository: mesos Description --- This allows us to determine the number of pending slave shutdowns, as the scheduled shutdowns must either be canceled or completed. Diffs - src/master/master.cpp 618db68ee4163b06e479cf3413eda4b63c9c5a4b src/master/metrics.hpp 52a83289cfe7e6b6fd8d5bff0774ebe5ce51d0ed src/master/metrics.cpp 14486bf7130250f561c9fb7a43c95f3fc1e76a4b Diff: https://reviews.apache.org/r/32954/diff/ Testing --- make check Thanks, Ben Mahler
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 15, 2015, 12:44 a.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Changes --- Addressed most of Alex's comments. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs (updated) - src/Makefile.am d15a373 src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Suggestion: Mesos 0.22.1 point release
Also, this one: https://issues.apache.org/jira/browse/MESOS-2601 This sounds like a non trivial fix. - Jie On Tue, Apr 14, 2015 at 6:35 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: Per Nik's comment here: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. I will move the other tickets back to 0.23.0 if you don't have any objections - let me know if you have any tickets which were regressions in 0.22.0. I expected there to be fewer tickets in the spreadsheet, are the extra tickets (e.g. https://issues.apache.org/jira/browse/MESOS-2614) going to be included after all? On Tue, Apr 14, 2015 at 6:20 PM, Joris Van Remoortere jo...@mesosphere.io wrote: I think the plan is to cut a new RC by sometime tomorrow. The spreadsheet is up-to-date, just need to cherry-pick and modify the change-log. Joris On Tue, Apr 14, 2015 at 5:37 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: Hey Nik, any progress on this? Is the spreadsheet up-to-date? On Wed, Apr 8, 2015 at 1:00 AM, Adam Bordelon a...@mesosphere.io wrote: Hi Adam, Yes, once we have finalized the scope of the point release, Niklas will send out an announcement of Mesos 0.22.1-rc1 (release candidate) which we would love you to test any way you can. The email will contain instructions for building the release candidate and voting in the thread. See the vote thread from 0.22.0-rc4 (became final): http://www.mail-archive.com/dev%40mesos.apache.org/msg30668.html The current thread is to collect suggestions for bug fixes to include in this point release. Cheers, -Adam- On Tue, Apr 7, 2015 at 9:22 AM, Adam Avilla a...@avil.la wrote: On Fri, Apr 3, 2015 at 3:47 PM, Niklas Nielsen nik...@mesosphere.io wrote: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. Can I help test these in any way? -- /adam
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
On April 14, 2015, 2:16 p.m., Alexander Rukletsov wrote: src/authentication/cram_md5/authenticatee.cpp, lines 46-47 https://reviews.apache.org/r/32850/diff/3/?file=922370#file922370line46 Let's move to newline to avoid jaggeddness. Btw, what does clang-format suggest? Kapil Arya wrote: According to our style guide, this is okay but we keep seeing issues being raised about this. Should we just update the style guide instead? Alexander Rukletsov wrote: I would like to have just one way of doing this. I think the one with newline scales better. This was exactly clang-format's suggestion. Raised https://issues.apache.org/jira/browse/MESOS-2618 - Till --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/#review80020 --- On April 15, 2015, 12:44 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 15, 2015, 12:44 a.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs - src/Makefile.am d15a373 src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
On April 14, 2015, 2:16 p.m., Alexander Rukletsov wrote: Thanks a bunch for your review Alex, much appreciated. - Till --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/#review80020 --- On April 15, 2015, 12:44 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 15, 2015, 12:44 a.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs - src/Makefile.am d15a373 src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Suggestion: Mesos 0.22.1 point release
Per Nik's comment here: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. I will move the other tickets back to 0.23.0 if you don't have any objections - let me know if you have any tickets which were regressions in 0.22.0. I expected there to be fewer tickets in the spreadsheet, are the extra tickets (e.g. https://issues.apache.org/jira/browse/MESOS-2614) going to be included after all? On Tue, Apr 14, 2015 at 6:20 PM, Joris Van Remoortere jo...@mesosphere.io wrote: I think the plan is to cut a new RC by sometime tomorrow. The spreadsheet is up-to-date, just need to cherry-pick and modify the change-log. Joris On Tue, Apr 14, 2015 at 5:37 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: Hey Nik, any progress on this? Is the spreadsheet up-to-date? On Wed, Apr 8, 2015 at 1:00 AM, Adam Bordelon a...@mesosphere.io wrote: Hi Adam, Yes, once we have finalized the scope of the point release, Niklas will send out an announcement of Mesos 0.22.1-rc1 (release candidate) which we would love you to test any way you can. The email will contain instructions for building the release candidate and voting in the thread. See the vote thread from 0.22.0-rc4 (became final): http://www.mail-archive.com/dev%40mesos.apache.org/msg30668.html The current thread is to collect suggestions for bug fixes to include in this point release. Cheers, -Adam- On Tue, Apr 7, 2015 at 9:22 AM, Adam Avilla a...@avil.la wrote: On Fri, Apr 3, 2015 at 3:47 PM, Niklas Nielsen nik...@mesosphere.io wrote: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. Can I help test these in any way? -- /adam
Re: Suggestion: Mesos 0.22.1 point release
Hey Nik, any progress on this? Is the spreadsheet up-to-date? On Wed, Apr 8, 2015 at 1:00 AM, Adam Bordelon a...@mesosphere.io wrote: Hi Adam, Yes, once we have finalized the scope of the point release, Niklas will send out an announcement of Mesos 0.22.1-rc1 (release candidate) which we would love you to test any way you can. The email will contain instructions for building the release candidate and voting in the thread. See the vote thread from 0.22.0-rc4 (became final): http://www.mail-archive.com/dev%40mesos.apache.org/msg30668.html The current thread is to collect suggestions for bug fixes to include in this point release. Cheers, -Adam- On Tue, Apr 7, 2015 at 9:22 AM, Adam Avilla a...@avil.la wrote: On Fri, Apr 3, 2015 at 3:47 PM, Niklas Nielsen nik...@mesosphere.io wrote: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. Can I help test these in any way? -- /adam
Re: Suggestion: Mesos 0.22.1 point release
I think the plan is to cut a new RC by sometime tomorrow. The spreadsheet is up-to-date, just need to cherry-pick and modify the change-log. Joris On Tue, Apr 14, 2015 at 5:37 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: Hey Nik, any progress on this? Is the spreadsheet up-to-date? On Wed, Apr 8, 2015 at 1:00 AM, Adam Bordelon a...@mesosphere.io wrote: Hi Adam, Yes, once we have finalized the scope of the point release, Niklas will send out an announcement of Mesos 0.22.1-rc1 (release candidate) which we would love you to test any way you can. The email will contain instructions for building the release candidate and voting in the thread. See the vote thread from 0.22.0-rc4 (became final): http://www.mail-archive.com/dev%40mesos.apache.org/msg30668.html The current thread is to collect suggestions for bug fixes to include in this point release. Cheers, -Adam- On Tue, Apr 7, 2015 at 9:22 AM, Adam Avilla a...@avil.la wrote: On Fri, Apr 3, 2015 at 3:47 PM, Niklas Nielsen nik...@mesosphere.io wrote: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. Can I help test these in any way? -- /adam
Re: Review Request 30263: Added test for CRAM-MD5 support of SASL within configuration phase.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30263/ --- (Updated April 15, 2015, 2:50 a.m.) Review request for mesos, Alexander Rojas and Cody Maloney. Changes --- Fixed OSX distcheck issue. Bugs: MESOS-2165 https://issues.apache.org/jira/browse/MESOS-2165 Repository: mesos Description --- see summary Diffs (updated) - configure.ac 868c041 Diff: https://reviews.apache.org/r/30263/diff/ Testing (updated) --- make distcheck (with and without CRAM-MD5 installed, Linux OSX) Thanks, Till Toenshoff
Re: Review Request 30263: Added test for CRAM-MD5 support of SASL within configuration phase.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30263/#review80151 --- Patch looks great! Reviews applied: [30263] All tests passed. - Mesos ReviewBot On April 15, 2015, 2:50 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30263/ --- (Updated April 15, 2015, 2:50 a.m.) Review request for mesos, Alexander Rojas and Cody Maloney. Bugs: MESOS-2165 https://issues.apache.org/jira/browse/MESOS-2165 Repository: mesos Description --- see summary Diffs - configure.ac 868c041 Diff: https://reviews.apache.org/r/30263/diff/ Testing --- make distcheck (with and without CRAM-MD5 installed, Linux OSX) Thanks, Till Toenshoff
Re: Suggestion: Mesos 0.22.1 point release
Yes, fixing it in 0.23.0 SGTM. On Tue, Apr 14, 2015 at 10:02 PM, Jie Yu yujie@gmail.com wrote: I am just asking if you guys want to fix that for 0.22.1 or not. It sounds to me a non trivial fix. Given the bug is there for quite a while, maybe we can fix it in 0.23.0? - Jie On Tue, Apr 14, 2015 at 9:55 PM, Benjamin Hindman b...@eecs.berkeley.edu wrote: We are going to include MESOS-2614 (it's a really trivial fix). Jie, where did you get MESOS-2601 from? That's definitely not in the spreadsheet. On Tue, Apr 14, 2015 at 7:40 PM, Jie Yu yujie@gmail.com wrote: Also, this one: https://issues.apache.org/jira/browse/MESOS-2601 This sounds like a non trivial fix. - Jie On Tue, Apr 14, 2015 at 6:35 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: Per Nik's comment here: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. I will move the other tickets back to 0.23.0 if you don't have any objections - let me know if you have any tickets which were regressions in 0.22.0. I expected there to be fewer tickets in the spreadsheet, are the extra tickets (e.g. https://issues.apache.org/jira/browse/MESOS-2614) going to be included after all? On Tue, Apr 14, 2015 at 6:20 PM, Joris Van Remoortere jo...@mesosphere.io wrote: I think the plan is to cut a new RC by sometime tomorrow. The spreadsheet is up-to-date, just need to cherry-pick and modify the change-log. Joris On Tue, Apr 14, 2015 at 5:37 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: Hey Nik, any progress on this? Is the spreadsheet up-to-date? On Wed, Apr 8, 2015 at 1:00 AM, Adam Bordelon a...@mesosphere.io wrote: Hi Adam, Yes, once we have finalized the scope of the point release, Niklas will send out an announcement of Mesos 0.22.1-rc1 (release candidate) which we would love you to test any way you can. The email will contain instructions for building the release candidate and voting in the thread. See the vote thread from 0.22.0-rc4 (became final): http://www.mail-archive.com/dev%40mesos.apache.org/msg30668.html The current thread is to collect suggestions for bug fixes to include in this point release. Cheers, -Adam- On Tue, Apr 7, 2015 at 9:22 AM, Adam Avilla a...@avil.la wrote: On Fri, Apr 3, 2015 at 3:47 PM, Niklas Nielsen nik...@mesosphere.io wrote: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. Can I help test these in any way? -- /adam
Re: Suggestion: Mesos 0.22.1 point release
I am just asking if you guys want to fix that for 0.22.1 or not. It sounds to me a non trivial fix. Given the bug is there for quite a while, maybe we can fix it in 0.23.0? - Jie On Tue, Apr 14, 2015 at 9:55 PM, Benjamin Hindman b...@eecs.berkeley.edu wrote: We are going to include MESOS-2614 (it's a really trivial fix). Jie, where did you get MESOS-2601 from? That's definitely not in the spreadsheet. On Tue, Apr 14, 2015 at 7:40 PM, Jie Yu yujie@gmail.com wrote: Also, this one: https://issues.apache.org/jira/browse/MESOS-2601 This sounds like a non trivial fix. - Jie On Tue, Apr 14, 2015 at 6:35 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: Per Nik's comment here: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. I will move the other tickets back to 0.23.0 if you don't have any objections - let me know if you have any tickets which were regressions in 0.22.0. I expected there to be fewer tickets in the spreadsheet, are the extra tickets (e.g. https://issues.apache.org/jira/browse/MESOS-2614) going to be included after all? On Tue, Apr 14, 2015 at 6:20 PM, Joris Van Remoortere jo...@mesosphere.io wrote: I think the plan is to cut a new RC by sometime tomorrow. The spreadsheet is up-to-date, just need to cherry-pick and modify the change-log. Joris On Tue, Apr 14, 2015 at 5:37 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: Hey Nik, any progress on this? Is the spreadsheet up-to-date? On Wed, Apr 8, 2015 at 1:00 AM, Adam Bordelon a...@mesosphere.io wrote: Hi Adam, Yes, once we have finalized the scope of the point release, Niklas will send out an announcement of Mesos 0.22.1-rc1 (release candidate) which we would love you to test any way you can. The email will contain instructions for building the release candidate and voting in the thread. See the vote thread from 0.22.0-rc4 (became final): http://www.mail-archive.com/dev%40mesos.apache.org/msg30668.html The current thread is to collect suggestions for bug fixes to include in this point release. Cheers, -Adam- On Tue, Apr 7, 2015 at 9:22 AM, Adam Avilla a...@avil.la wrote: On Fri, Apr 3, 2015 at 3:47 PM, Niklas Nielsen nik...@mesosphere.io wrote: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. Can I help test these in any way? -- /adam
Review Request 33208: Delete detector in MesosSchedulerDriver::stop
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33208/ --- Review request for mesos, Benjamin Hindman, Ben Mahler, Niklas Nielsen, and Vinod Kone. Bugs: MESOS-1634 https://issues.apache.org/jira/browse/MESOS-1634 Repository: mesos Description --- When the Mesos Scheduler Driver stops, it does not delete the detector process before the object is garbage collected, which leaves ZooKeeper connections hanging around unnecessarily. This deletes the process on stop as well, not only on destruction. Diffs - src/sched/sched.cpp 66fd2b3146568900356cc48e0f17c6720665ae80 Diff: https://reviews.apache.org/r/33208/diff/ Testing --- make check, manual testing Thanks, Robert Lacroix
Re: Suggestion: Mesos 0.22.1 point release
We are going to include MESOS-2614 (it's a really trivial fix). Jie, where did you get MESOS-2601 from? That's definitely not in the spreadsheet. On Tue, Apr 14, 2015 at 7:40 PM, Jie Yu yujie@gmail.com wrote: Also, this one: https://issues.apache.org/jira/browse/MESOS-2601 This sounds like a non trivial fix. - Jie On Tue, Apr 14, 2015 at 6:35 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: Per Nik's comment here: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. I will move the other tickets back to 0.23.0 if you don't have any objections - let me know if you have any tickets which were regressions in 0.22.0. I expected there to be fewer tickets in the spreadsheet, are the extra tickets (e.g. https://issues.apache.org/jira/browse/MESOS-2614) going to be included after all? On Tue, Apr 14, 2015 at 6:20 PM, Joris Van Remoortere jo...@mesosphere.io wrote: I think the plan is to cut a new RC by sometime tomorrow. The spreadsheet is up-to-date, just need to cherry-pick and modify the change-log. Joris On Tue, Apr 14, 2015 at 5:37 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: Hey Nik, any progress on this? Is the spreadsheet up-to-date? On Wed, Apr 8, 2015 at 1:00 AM, Adam Bordelon a...@mesosphere.io wrote: Hi Adam, Yes, once we have finalized the scope of the point release, Niklas will send out an announcement of Mesos 0.22.1-rc1 (release candidate) which we would love you to test any way you can. The email will contain instructions for building the release candidate and voting in the thread. See the vote thread from 0.22.0-rc4 (became final): http://www.mail-archive.com/dev%40mesos.apache.org/msg30668.html The current thread is to collect suggestions for bug fixes to include in this point release. Cheers, -Adam- On Tue, Apr 7, 2015 at 9:22 AM, Adam Avilla a...@avil.la wrote: On Fri, Apr 3, 2015 at 3:47 PM, Niklas Nielsen nik...@mesosphere.io wrote: Based on input from Vinod and Adam; I will reduce the scope on the point release to focus on MESOS-1795 and MESOS-2583. Can I help test these in any way? -- /adam
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/#review80020 --- src/authentication/cram_md5/authenticatee.hpp https://reviews.apache.org/r/32850/#comment129770 `#include stout/try.hpp`? src/authentication/cram_md5/authenticatee.hpp https://reviews.apache.org/r/32850/#comment129771 `#include process/future.hpp`? src/authentication/cram_md5/authenticatee.cpp https://reviews.apache.org/r/32850/#comment129772 Should stay in the header. src/authentication/cram_md5/authenticatee.cpp https://reviews.apache.org/r/32850/#comment129773 Let's move to newline to avoid jaggeddness. Btw, what does clang-format suggest? src/authentication/cram_md5/authenticatee.cpp https://reviews.apache.org/r/32850/#comment129774 `#include logging/logging.hpp`? src/authentication/cram_md5/authenticatee.cpp https://reviews.apache.org/r/32850/#comment129775 `#include process/dispatch.hpp? - Alexander Rukletsov On April 10, 2015, 2:58 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 10, 2015, 2:58 a.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs - src/Makefile.am fa609da src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Review Request 33174: Fix for docker not configuring CFS quotas correctly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33174/#review80126 --- Patch looks great! Reviews applied: [33174] All tests passed. - Mesos ReviewBot On April 14, 2015, 8:32 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33174/ --- (Updated April 14, 2015, 8:32 p.m.) Review request for mesos, Ian Downes, Jie Yu, and Timothy Chen. Bugs: MESOS-2617 https://issues.apache.org/jira/browse/MESOS-2617 Repository: mesos Description --- Fix for docker containerizer not configuring CFS quotas correctly. It would be nice to refactor all this isolation code in a way that can be shared between all containerizers, as this is basically just copied from the CgroupsCpushareIsolator, but that's a much bigger undertaking. Diffs - src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 Diff: https://reviews.apache.org/r/33174/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/#review80082 --- src/authentication/cram_md5/authenticatee.cpp https://reviews.apache.org/r/32850/#comment129850 According to our style guide, this is okay but we keep seeing issues being raised about this. Should we just update the style guide instead? - Kapil Arya On April 14, 2015, 10:20 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 14, 2015, 10:20 a.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs - src/Makefile.am d15a373 src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
On April 14, 2015, 10:16 a.m., Alexander Rukletsov wrote: src/authentication/cram_md5/authenticatee.cpp, lines 46-47 https://reviews.apache.org/r/32850/diff/3/?file=922370#file922370line46 Let's move to newline to avoid jaggeddness. Btw, what does clang-format suggest? According to our style guide, this is okay but we keep seeing issues being raised about this. Should we just update the style guide instead? - Kapil --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/#review80020 --- On April 14, 2015, 10:20 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 14, 2015, 10:20 a.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs - src/Makefile.am d15a373 src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Review Request 31666: Piped hashmapSlaveID, Resources from master through to allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31666/ --- (Updated April 14, 2015, 7:43 p.m.) Review request for mesos, Alexander Rukletsov and Ben Mahler. Bugs: MESOS-2373 https://issues.apache.org/jira/browse/MESOS-2373 Repository: mesos Description --- Piped the master through by `s/sum(framework-usedResources.values())/framework-usedResources/` Lots of `s/Resources()/hashmapSlaveID, Resources()/` in `src/tests/hierarchical_allocator_tests.cpp`. Diffs (updated) - src/master/allocator/allocator.hpp 91f80501aa9bc733fd53f9cb1ac0f15949a74964 src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f src/master/allocator/mesos/hierarchical.hpp 9f9a631ee52a3ab194e678f9be139e8dabc7f3db src/master/master.cpp 44b0a0147f5354824d86332a67b30018634c9a36 src/tests/hierarchical_allocator_tests.cpp 8861bf398e4bb17b0f74eab4f4af26202447ccef src/tests/mesos.hpp 42e42ac425a448fcc5e93db1cef1112cbf5e67c4 Diff: https://reviews.apache.org/r/31666/diff/ Testing --- make check Thanks, Michael Park
Re: Review Request 31666: Piped hashmapSlaveID, Resources from master through to allocator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31666/ --- (Updated April 14, 2015, 7:44 p.m.) Review request for mesos, Alexander Rukletsov and Ben Mahler. Bugs: MESOS-2373 https://issues.apache.org/jira/browse/MESOS-2373 Repository: mesos Description (updated) --- Piped the master through by `s/framework-usedResources)/framework-usedResourcesBySlaveId/` Lots of `s/Resources()/hashmapSlaveID, Resources()/` in `src/tests/hierarchical_allocator_tests.cpp`. Diffs - src/master/allocator/allocator.hpp 91f80501aa9bc733fd53f9cb1ac0f15949a74964 src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f src/master/allocator/mesos/hierarchical.hpp 9f9a631ee52a3ab194e678f9be139e8dabc7f3db src/master/master.cpp 44b0a0147f5354824d86332a67b30018634c9a36 src/tests/hierarchical_allocator_tests.cpp 8861bf398e4bb17b0f74eab4f4af26202447ccef src/tests/mesos.hpp 42e42ac425a448fcc5e93db1cef1112cbf5e67c4 Diff: https://reviews.apache.org/r/31666/diff/ Testing --- make check Thanks, Michael Park
Re: Review Request 27113: Libprocess benchmark cleanup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/ --- (Updated April 14, 2015, 7:46 p.m.) Review request for mesos, Ben Mahler, Cody Maloney, Joerg Schad, and Michael Park. Changes --- minor fixes for Joerg. Bugs: MESOS-1980 https://issues.apache.org/jira/browse/MESOS-1980 Repository: mesos Description --- Clean up Libprocess for readability / extensibility. Diffs (updated) - 3rdparty/libprocess/src/tests/benchmarks.cpp a927e4ecd8c8955cd9f85e716173a73a9a21c6cd Diff: https://reviews.apache.org/r/27113/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 33174: Fix for docker not configuring CFS quotas correctly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33174/#review80087 --- I just realized this doesn't work against 0.22.x, because the containerizer now checks if the resources have changed before doing an update. I'm working on a fix for that now. - Steve Niemitz On April 14, 2015, 4:44 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33174/ --- (Updated April 14, 2015, 4:44 p.m.) Review request for mesos, Ian Downes, Jie Yu, and Timothy Chen. Bugs: MESOS-2617 https://issues.apache.org/jira/browse/MESOS-2617 Repository: mesos Description --- Fix for docker containerizer not configuring CFS quotas correctly. It would be nice to refactor all this isolation code in a way that can be shared between all containerizers, as this is basically just copied from the CgroupsCpushareIsolator, but that's a much bigger undertaking. Diffs - src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 Diff: https://reviews.apache.org/r/33174/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 31665: Added 'hashmapSlaveID, Resources (used|offered)ResourcesBySlaveId' to master::Framework.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31665/ --- (Updated April 14, 2015, 7:40 p.m.) Review request for mesos, Alexander Rukletsov and Ben Mahler. Summary (updated) - Added 'hashmapSlaveID, Resources (used|offered)ResourcesBySlaveId' to master::Framework. Bugs: MESOS-2373 https://issues.apache.org/jira/browse/MESOS-2373 Repository: mesos Description --- `master::Framework` holds 2 member variables of type `Resources`: `usedResources` and `offeredResources`. Both of these are aggregates of resources from multiple slaves. We add the `hashmapSlaveID, Resources` versions since we can lose non-scalar resources by summing them up across multiple slaves. For further details refer to [MESOS-2373](https://issues.apache.org/jira/browse/MESOS-2373). In [r31666](https://reviews.apache.org/r/31666/), we report `usedResourcesBySlaveId` rather than `usedResources` when adding the framework to the allocator. We don't actually use `offeredResourcesBySlaveId` currently, but it should be kept for symmetry. There will be a ticket to expose `usedResourcesBySlaveId` as well as `offeredResourcesBySlaveId` via the HTTP endpoint. Diffs - src/master/master.hpp 6141917644b84edfed9836fa0a005d55a36880e3 Diff: https://reviews.apache.org/r/31665/diff/ Testing --- make check Thanks, Michael Park
Re: Review Request 31665: Added 'hashmapSlaveID, Resources usedResourcesBySlaveId' to master::Framework.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31665/ --- (Updated April 14, 2015, 7:40 p.m.) Review request for mesos, Alexander Rukletsov and Ben Mahler. Summary (updated) - Added 'hashmapSlaveID, Resources usedResourcesBySlaveId' to master::Framework. Bugs: MESOS-2373 https://issues.apache.org/jira/browse/MESOS-2373 Repository: mesos Description (updated) --- `master::Framework` holds 2 member variables of type `Resources`: `usedResources` and `offeredResources`. Both of these are aggregates of resources from multiple slaves. We add the `hashmapSlaveID, Resources` versions since we can lose non-scalar resources by summing them up across multiple slaves. For further details refer to [MESOS-2373](https://issues.apache.org/jira/browse/MESOS-2373). In [r31666](https://reviews.apache.org/r/31666/), we report `usedResourcesBySlaveId` rather than `usedResources` when adding the framework to the allocator. We don't actually use `offeredResourcesBySlaveId` currently, but it should be kept for symmetry. There will be a ticket to expose `usedResourcesBySlaveId` as well as `offeredResourcesBySlaveId` via the HTTP endpoint. Diffs (updated) - src/master/master.hpp 6141917644b84edfed9836fa0a005d55a36880e3 Diff: https://reviews.apache.org/r/31665/diff/ Testing --- make check Thanks, Michael Park
Re: Review Request 31666: Send 'framework-usedResourcesBySlaveId' to allocator instead for 'addFramework'.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31666/ --- (Updated April 14, 2015, 7:48 p.m.) Review request for mesos, Alexander Rukletsov and Ben Mahler. Summary (updated) - Send 'framework-usedResourcesBySlaveId' to allocator instead for 'addFramework'. Bugs: MESOS-2373 https://issues.apache.org/jira/browse/MESOS-2373 Repository: mesos Description (updated) --- Piped the master through by `s/framework-usedResources/framework-usedResourcesBySlaveId/` Lots of `s/Resources()/hashmapSlaveID, Resources()/` in `src/tests/hierarchical_allocator_tests.cpp`. Diffs - src/master/allocator/allocator.hpp 91f80501aa9bc733fd53f9cb1ac0f15949a74964 src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f src/master/allocator/mesos/hierarchical.hpp 9f9a631ee52a3ab194e678f9be139e8dabc7f3db src/master/master.cpp 44b0a0147f5354824d86332a67b30018634c9a36 src/tests/hierarchical_allocator_tests.cpp 8861bf398e4bb17b0f74eab4f4af26202447ccef src/tests/mesos.hpp 42e42ac425a448fcc5e93db1cef1112cbf5e67c4 Diff: https://reviews.apache.org/r/31666/diff/ Testing --- make check Thanks, Michael Park
Re: Review Request 33174: Fix for docker not configuring CFS quotas correctly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33174/ --- (Updated April 14, 2015, 8:32 p.m.) Review request for mesos, Ian Downes, Jie Yu, and Timothy Chen. Bugs: MESOS-2617 https://issues.apache.org/jira/browse/MESOS-2617 Repository: mesos Description --- Fix for docker containerizer not configuring CFS quotas correctly. It would be nice to refactor all this isolation code in a way that can be shared between all containerizers, as this is basically just copied from the CgroupsCpushareIsolator, but that's a much bigger undertaking. Diffs (updated) - src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 Diff: https://reviews.apache.org/r/33174/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 33174: Fix for docker not configuring CFS quotas correctly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33174/#review80099 --- src/slave/containerizer/docker.cpp https://reviews.apache.org/r/33174/#comment129860 There must be a better way to do this, but I'm not very familiar w/ boost. - Steve Niemitz On April 14, 2015, 8:32 p.m., Steve Niemitz wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33174/ --- (Updated April 14, 2015, 8:32 p.m.) Review request for mesos, Ian Downes, Jie Yu, and Timothy Chen. Bugs: MESOS-2617 https://issues.apache.org/jira/browse/MESOS-2617 Repository: mesos Description --- Fix for docker containerizer not configuring CFS quotas correctly. It would be nice to refactor all this isolation code in a way that can be shared between all containerizers, as this is basically just copied from the CgroupsCpushareIsolator, but that's a much bigger undertaking. Diffs - src/slave/containerizer/docker.cpp 5f4b4ce49a9523e4743e5c79da4050e6f9e29ed7 Diff: https://reviews.apache.org/r/33174/diff/ Testing --- Thanks, Steve Niemitz
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
On April 14, 2015, 6:17 p.m., Joerg Schad wrote: src/authentication/cram_md5/authenticatee.cpp, line 33 https://reviews.apache.org/r/32850/diff/4/?file=927033#file927033line33 Actually according to Google's styleguide this should be the first include, shouldn't it? http://google-styleguide.googlecode.com/svn/trunk/cppguide.html#Names_and_Order_of_Includes I know we don't do that at other places but actually it would prevent a number of Alexs comments (and be consistent with the styleguide)... I think we don't follow google styleguide whe including headers (update the styleguide?), we have our own grouping: by project, by directory, alphabetically. - Alexander --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/#review80072 --- On April 14, 2015, 2:20 p.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 14, 2015, 2:20 p.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs - src/Makefile.am d15a373 src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
On April 14, 2015, 6:17 p.m., Joerg Schad wrote: src/authentication/cram_md5/authenticatee.cpp, line 33 https://reviews.apache.org/r/32850/diff/4/?file=927033#file927033line33 Actually according to Google's styleguide this should be the first include, shouldn't it? http://google-styleguide.googlecode.com/svn/trunk/cppguide.html#Names_and_Order_of_Includes I know we don't do that at other places but actually it would prevent a number of Alexs comments (and be consistent with the styleguide)... Alexander Rukletsov wrote: I think we don't follow google styleguide whe including headers (update the styleguide?), we have our own grouping: by project, by directory, alphabetically. Already on Agenda for next Review/C++ meeting. - Joerg --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/#review80072 --- On April 14, 2015, 2:20 p.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 14, 2015, 2:20 p.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs - src/Makefile.am d15a373 src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Review Request 32850: Moved cram-md5 authenticatee process definition into implementation file.
On April 14, 2015, 2:16 p.m., Alexander Rukletsov wrote: src/authentication/cram_md5/authenticatee.cpp, lines 46-47 https://reviews.apache.org/r/32850/diff/3/?file=922370#file922370line46 Let's move to newline to avoid jaggeddness. Btw, what does clang-format suggest? Kapil Arya wrote: According to our style guide, this is okay but we keep seeing issues being raised about this. Should we just update the style guide instead? I would like to have just one way of doing this. I think the one with newline scales better. - Alexander --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/#review80020 --- On April 14, 2015, 2:20 p.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32850/ --- (Updated April 14, 2015, 2:20 p.m.) Review request for mesos, Adam B, Joris Van Remoortere, and switched to 'mcypark'. Bugs: MESOS-2584 https://issues.apache.org/jira/browse/MESOS-2584 Repository: mesos-incubating Description --- Removing the process from the header is much cleaner and also fixes the linked clang 3.4.2 JIRA. Apart from that moving, no code is changed. Diffs - src/Makefile.am d15a373 src/authentication/cram_md5/authenticatee.hpp 55fac68 src/authentication/cram_md5/authenticatee.cpp PRE-CREATION Diff: https://reviews.apache.org/r/32850/diff/ Testing --- make check Thanks, Till Toenshoff
Re: Inlined code
+1 I will try to do so when I have free cycles. On 10 Apr 2015, at 20:22, Benjamin Mahler benjamin.mah...@gmail.com wrote: Let's also try to measure the impact on build time, going forward. Cody did a great job recently here: https://reviews.apache.org/r/32558/ On Fri, Apr 10, 2015 at 8:01 AM, Alexander Rojas alexan...@mesosphere.io wrote: I agree with the fact that stout is headers only, but this is code in mess itself. On 10 Apr 2015, at 13:15, Alex Rukletsov a...@mesosphere.com wrote: wherever possible, but we are not going to m
Re: Review Request 32536: Updated variable naming style.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32536/#review80015 --- Ship it! Thanks for bringing this up Alex - I find it very clear and helpful. - Till Toenshoff On March 26, 2015, 4:10 p.m., Alexander Rukletsov wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32536/ --- (Updated March 26, 2015, 4:10 p.m.) Review request for mesos, Benjamin Hindman, Ben Mahler, Michael Park, Niklas Nielsen, and Timothy Chen. Repository: mesos-incubating Description --- Documents the patterns we use to name variables and function arguments in our codebase. Leading underscores to avoid ambiguity. === We use this pattern extensively in libprocess, stout and mesos, a few examples below. * `stout/try.hpp:105` ``` Try(State _state, T* _t = NULL, const std::string _message = ) : state(_state), t(_t), message(_message) {} ``` * `process/http.hpp:480` ``` URL(const std::string _scheme, const std::string _domain, const uint16_t _port = 80, const std::string _path = /, const hashmapstd::string, std::string _query = (hashmapstd::string, std::string()), const Optionstd::string _fragment = None()) : scheme(_scheme), domain(_domain), port(_port), path(_path), query(_query), fragment(_fragment) {} ``` * `slave/containerizer/linux_launcher.cpp:56` ``` LinuxLauncher::LinuxLauncher( const Flags _flags, int _namespaces, const string _hierarchy) : flags(_flags), namespaces(_namespaces), hierarchy(_hierarchy) {} ``` Trailing undescores as prime symbols. = We use this pattern in the code, though not extensively. I would like to see more pass-by-value instead of creating copies from a variable passed by const reference. * `master.cpp:2942` ``` // Create and add the slave id. SlaveInfo slaveInfo_ = slaveInfo; slaveInfo_.mutable_id()-CopyFrom(newSlaveId()); ``` * `slave.cpp:4180` ``` ExecutorInfo executorInfo_ = executor-info; Resources resources = executorInfo_.resources(); resources += taskInfo.resources(); executorInfo_.mutable_resources()-CopyFrom(resources); ``` * `status_update_manager.cpp:474` ``` // Bounded exponential backoff. Duration duration_ = std::min(duration * 2, STATUS_UPDATE_RETRY_INTERVAL_MAX); ``` * `containerizer/mesos/containerizer.cpp:109` ``` // Modify the flags to include any changes to isolation. Flags flags_ = flags; flags_.isolation = isolation; ``` Passing arguments by value. === * `slave.cpp:2480` ``` void Slave::statusUpdate(StatusUpdate update, const UPID pid) { ... // Set the source before forwarding the status update. update.mutable_status()-set_source( pid == UPID() ? TaskStatus::SOURCE_SLAVE : TaskStatus::SOURCE_EXECUTOR); ... } ``` * `process/metrics/timer.hpp:103` ``` static void _time(Time start, Timer that) { const Time stop = Clock::now(); double value; process::internal::acquire(that.data-lock); { that.data-lastValue = T(stop - start).value(); value = that.data-lastValue.get(); } process::internal::release(that.data-lock); that.push(value); } ``` Diffs - docs/mesos-c++-style-guide.md 439fe12 Diff: https://reviews.apache.org/r/32536/diff/ Testing --- None (not a functional change). Thanks, Alexander Rukletsov
Re: Review Request 32001: Required a period in trailing comments in the style guide.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32001/#review80011 --- docs/mesos-c++-style-guide.md https://reviews.apache.org/r/32001/#comment129761 I actually think that MPark's draft fits very well. My suggestion would be: End each sentence within a comment with punctuation - this applies to incomplete sentences as well. Please note that we generally prefer periods. - Till Toenshoff On March 26, 2015, 1:08 p.m., Alexander Rukletsov wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32001/ --- (Updated March 26, 2015, 1:08 p.m.) Review request for mesos and Ben Mahler. Repository: mesos Description --- See summary. Diffs - docs/mesos-c++-style-guide.md 439fe12ad137c1bed3a21d5a097c60a48f10a4f9 Diff: https://reviews.apache.org/r/32001/diff/ Testing --- None (not a functional change). Thanks, Alexander Rukletsov
Re: Review Request 32536: Updated variable naming style.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32536/ --- (Updated April 14, 2015, 1:08 p.m.) Review request for mesos, Benjamin Hindman, Ben Mahler, Michael Park, Niklas Nielsen, and Timothy Chen. Changes --- added JIRA Bugs: MESOS-2616 https://issues.apache.org/jira/browse/MESOS-2616 Repository: mesos-incubating Description --- Documents the patterns we use to name variables and function arguments in our codebase. Leading underscores to avoid ambiguity. === We use this pattern extensively in libprocess, stout and mesos, a few examples below. * `stout/try.hpp:105` ``` Try(State _state, T* _t = NULL, const std::string _message = ) : state(_state), t(_t), message(_message) {} ``` * `process/http.hpp:480` ``` URL(const std::string _scheme, const std::string _domain, const uint16_t _port = 80, const std::string _path = /, const hashmapstd::string, std::string _query = (hashmapstd::string, std::string()), const Optionstd::string _fragment = None()) : scheme(_scheme), domain(_domain), port(_port), path(_path), query(_query), fragment(_fragment) {} ``` * `slave/containerizer/linux_launcher.cpp:56` ``` LinuxLauncher::LinuxLauncher( const Flags _flags, int _namespaces, const string _hierarchy) : flags(_flags), namespaces(_namespaces), hierarchy(_hierarchy) {} ``` Trailing undescores as prime symbols. = We use this pattern in the code, though not extensively. I would like to see more pass-by-value instead of creating copies from a variable passed by const reference. * `master.cpp:2942` ``` // Create and add the slave id. SlaveInfo slaveInfo_ = slaveInfo; slaveInfo_.mutable_id()-CopyFrom(newSlaveId()); ``` * `slave.cpp:4180` ``` ExecutorInfo executorInfo_ = executor-info; Resources resources = executorInfo_.resources(); resources += taskInfo.resources(); executorInfo_.mutable_resources()-CopyFrom(resources); ``` * `status_update_manager.cpp:474` ``` // Bounded exponential backoff. Duration duration_ = std::min(duration * 2, STATUS_UPDATE_RETRY_INTERVAL_MAX); ``` * `containerizer/mesos/containerizer.cpp:109` ``` // Modify the flags to include any changes to isolation. Flags flags_ = flags; flags_.isolation = isolation; ``` Passing arguments by value. === * `slave.cpp:2480` ``` void Slave::statusUpdate(StatusUpdate update, const UPID pid) { ... // Set the source before forwarding the status update. update.mutable_status()-set_source( pid == UPID() ? TaskStatus::SOURCE_SLAVE : TaskStatus::SOURCE_EXECUTOR); ... } ``` * `process/metrics/timer.hpp:103` ``` static void _time(Time start, Timer that) { const Time stop = Clock::now(); double value; process::internal::acquire(that.data-lock); { that.data-lastValue = T(stop - start).value(); value = that.data-lastValue.get(); } process::internal::release(that.data-lock); that.push(value); } ``` Diffs - docs/mesos-c++-style-guide.md 439fe12 Diff: https://reviews.apache.org/r/32536/diff/ Testing --- None (not a functional change). Thanks, Alexander Rukletsov
Re: Review Request 32583: Marked RunTaskMessage::framework_id as optional.
On April 11, 2015, 4:11 a.m., Adam B wrote: Did you ever (manually?) Test for upgrade path? Sorry for not replying earlier. I tested manually with 0.22.0 and HEAD. I tried combinations of old/new pairs of master, slave and test-framework. I also tried Niklas's upgrade script (https://reviews.apache.org/r/31645/) but had to make a small modification to it (possibly a bug in the script itself). Everything seemed to work fine. - Kapil --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32583/#review79798 --- On April 7, 2015, 12:59 p.m., Kapil Arya wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32583/ --- (Updated April 7, 2015, 12:59 p.m.) Review request for mesos, Adam B and Niklas Nielsen. Bugs: MESOS-2558 https://issues.apache.org/jira/browse/MESOS-2558 Repository: mesos Description --- The new preferred location for FrameworkID is FrameworkInfo.id. This patchset achieves this goal by incrementally deprecating other locations for FrameworkID. Here is a plan to deal with the upgrade path: For this release (N), we still keep setting RunTaskMessage::framework_id - this would handle older Slaves with newer Master. - added code to handle it being unset in the Slave (handles older Master with newer Slaves). In the following release (N+1), stop reading/setting RunTaskMessage::framework_id - the previous version would handle the unset case. In release N+2, remove the field altogether: - the previous release is not setting/reading it. Diffs - src/messages/messages.proto 97c45c01dfcea38b1ae555c036d61e10c152c2c8 src/slave/slave.cpp c7e65a6c095963feaa9d5fdbb519c68f8f761d16 Diff: https://reviews.apache.org/r/32583/diff/ Testing --- make check. TODO: Test for upgrade path. Thanks, Kapil Arya
Re: Review Request 27113: Libprocess benchmark cleanup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/#review80103 --- Patch looks great! Reviews applied: [25448, 27113] All tests passed. - Mesos ReviewBot On April 14, 2015, 7:46 p.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/ --- (Updated April 14, 2015, 7:46 p.m.) Review request for mesos, Ben Mahler, Cody Maloney, Joerg Schad, and Michael Park. Bugs: MESOS-1980 https://issues.apache.org/jira/browse/MESOS-1980 Repository: mesos Description --- Clean up Libprocess for readability / extensibility. Diffs - 3rdparty/libprocess/src/tests/benchmarks.cpp a927e4ecd8c8955cd9f85e716173a73a9a21c6cd Diff: https://reviews.apache.org/r/27113/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 31645: Added upgrade path test script
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31645/#review80102 --- In order to make this script work, I had to do the following: export LD_LIBRARY_PATH=$HOME/mesos/master/build/src/.libs:$HOME/mesos/mesos-0.22.0/build/src/.libs Otherwise, the test-executor has problem finding the correct libraries. support/test-upgrade.py https://reviews.apache.org/r/31645/#comment129862 s/prev_slave/prev_master/ The previous slave has already been killed on line 161. We do need to kill the prev_master though, otherwise it complains about address being in use when launching the new master. - Kapil Arya On March 2, 2015, 6:44 p.m., Niklas Nielsen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31645/ --- (Updated March 2, 2015, 6:44 p.m.) Review request for mesos, Ben Mahler and Vinod Kone. Bugs: MESOS-2372 https://issues.apache.org/jira/browse/MESOS-2372 Repository: mesos Description --- Upgrade path test script runs previous and new framework versions (i.e. different versions of test framework+libmesos) against different versions of mesos slave and master. We can hopefully generate the upgrade paths we want to test systematically (applying some combinatorics) and cover those in this script. Diffs - support/test-upgrade.py PRE-CREATION Diff: https://reviews.apache.org/r/31645/diff/ Testing --- The output from running the script against 0.21.0 and HEAD (0.23.0): $ python ./support/run-upgrade.py --prev=../mesos/build-0.21.0/ --next=build Running upgrade test from mesos 0.21.0 to mesos 0.23.0 +--+++---+ | Test case| Framework| Master | Slave | +--+++---+ |#1| mesos 0.21.0 | mesos 0.21.0 | mesos 0.21.0 | |#2 (live) | mesos 0.21.0 | mesos 0.21.0 | mesos 0.23.0 | |#3| mesos 0.23.0 | mesos 0.21.0 | mesos 0.23.0 | |#4| mesos 0.23.0 | mesos 0.23.0 | mesos 0.23.0 | +--+++---+ NOTE: live denotes that master process keeps running from previous case. Test case 1 (Run of previous setup) # Starting mesos 0.21.0 master # Run ['../mesos/build-0.21.0/bin/mesos-master.sh', '--ip=127.0.0.1', '--work_dir=/var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmp2UMcJv', '--credentials=/var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T /tmpEMpQQd'], output: /var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmpEXE5wH # Starting mesos 0.21.0 slave # Run ['../mesos/build-0.21.0/bin/mesos-slave.sh', '--master=localhost:5050', '--credential=/var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmpEMpQQd'], output: /var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4 gn/T/tmpnGiJfB # Starting mesos 0.21.0 framework # Waiting for mesos 0.21.0 framework to complete (10 sec max)... Run ['../mesos/build-0.21.0/src/test-framework', '--master=localhost:5050'], output: /var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmp5cfI3A Test case 2 (Upgrade of slave) # Stopping mesos 0.21.0 slave # # Starting mesos 0.23.0 slave # Run ['build/bin/mesos-slave.sh', '--master=localhost:5050', '--credential=/var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmpEMpQQd'], output: /var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmpj9sM2K # Starting mesos 0.21.0 framework # Waiting for mesos 0.21.0 framework to complete (10 sec max)... Run ['../mesos/build-0.21.0/src/test-framework', '--master=localhost:5050'], output: /var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmpltfC_p Test case 3 (Upgrade framework) # Starting mesos 0.23.0 framework # Waiting for mesos 0.23.0 framework to complete (10 sec max)... Run ['build/src/test-framework', '--master=localhost:5050'], output: /var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmpJVWJjW Test case 4 (Run of next setup) # Stopping mesos 0.23.0 slave # Stopping mesos 0.21.0 slave Run ['build/bin/mesos-master.sh', '--ip=127.0.0.1', '--work_dir=/var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmpALCpmO', '--credentials=/var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmpEMpQQd'], o$tput: /var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmp6oLkTt # Starting mesos 0.23.0 slave # Run ['build/bin/mesos-slave.sh', '--master=localhost:5050', '--credential=/var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmpEMpQQd'], output: /var/folders/y3/w04yjljd5gbcvbvxvd5bhmp4gn/T/tmpYOCLCM # Starting mesos 0.23.0 framework
Re: Review Request 27113: Libprocess benchmark cleanup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/#review80101 --- Wow, the client and server parts of this are looking a lot clearer, thanks Joris! Let's focus on the test body next, it was difficult for me to grasp it fully, I left a comment below :) 3rdparty/libprocess/src/tests/benchmarks.cpp https://reviews.apache.org/r/27113/#comment129861 How about 'concurrency' or 'concurrentRequests' instead of 'queueDepth'? Which queue is this variable referring to? 3rdparty/libprocess/src/tests/benchmarks.cpp https://reviews.apache.org/r/27113/#comment129863 This seems to suggest support for multiple runs of the client. However, if a /run request arrives while the previous run is in progress, it looks like this will behave strangely. Did you want to deny requests while a run is in progress, or chain the runs after one another? 3rdparty/libprocess/src/tests/benchmarks.cpp https://reviews.apache.org/r/27113/#comment129864 Why do totalRequests and the concurrency factor need to be reset here? Looks like these are required. 3rdparty/libprocess/src/tests/benchmarks.cpp https://reviews.apache.org/r/27113/#comment129871 How about: ``` hashmapstring, Optionstring parameters { {server, request.query.get(server)}, {messageSize, request.query.get(messageSize)}, {totalRequests, request.query.get(totalRequests)}, {concurrentRequests, request.query.get(concurrentRequests)}, }; // Ensure all parameters were provided. foreachpair (const string parameter, const Optionstring value, parameters) { if (value.isNone()) { return http::BadRequest(Missing ' + parameter + ' parameter); } } server = UPID(parameters[server].get()); TryBytes bytes = Bytes::parse(parameters[messageSize].get()); if (bytes.isError()) { return http::BadRequest(Invalid 'messageSize': + bytes.error()); } messageSize = bytes.get(); Trysize_t numify = numifysize_t(parameters[totalRequests].get()); if (numify.isError()) { return http::BadRequest(Invalid 'totalRequests': + numify.error()); } totalRequests = numify.get(); numify = numifysize_t(parameters[concurrentRequests].get()); if (numify.isError()) { return http::BadRequest(Invalid 'concurrentRequests': + numify.error()); } concurrentRequests = numify.get(); ``` 3rdparty/libprocess/src/tests/benchmarks.cpp https://reviews.apache.org/r/27113/#comment129865 How about using a `Bytes` for this? ``` TryBytes messageSize = Bytes::parse(value.get()); if (messageSize.isError()) { } message = string(messageSize.get().bytes(), '1'); 3rdparty/libprocess/src/tests/benchmarks.cpp https://reviews.apache.org/r/27113/#comment129866 Can you include the error message from the Try here and elsewhere? 3rdparty/libprocess/src/tests/benchmarks.cpp https://reviews.apache.org/r/27113/#comment129873 Why do you need to reject this? If the number of concurrent requests is larger than the total number of requests, that seems well defined (concurrentRequests == totalRequests). Any reason not to just do the right thing for what was requested? 3rdparty/libprocess/src/tests/benchmarks.cpp https://reviews.apache.org/r/27113/#comment129875 'concurrency' or 'concurrentRequests' might make this clearer 3rdparty/libprocess/src/tests/benchmarks.cpp https://reviews.apache.org/r/27113/#comment129877 Comment seems unnecessary? 3rdparty/libprocess/src/tests/benchmarks.cpp https://reviews.apache.org/r/27113/#comment129881 Could you do a self review of this test body, from the perspective of someone approaching this code for the first time? I had a hard time understanding this, at a basic level what is the overall structure of the forking here? It's also a bit tricky to figure out the lower level details, like what the `requestPipes` and `resultPipes` are for, and why they're needed. What the vectors of pipes are used for, why they're needed. Etc. In the process of being able to articulate what this does, we might figure out how we can make it more easily understandable :) - Ben Mahler On April 14, 2015, 7:46 p.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/ --- (Updated April 14, 2015, 7:46 p.m.) Review request for mesos, Ben Mahler, Cody Maloney, Joerg Schad, and Michael Park. Bugs: MESOS-1980 https://issues.apache.org/jira/browse/MESOS-1980 Repository: mesos Description --- Clean up Libprocess for readability / extensibility.
Oversubscription design doc
Hi everyone, In context of some of the recent discussions on 'Resource overcommittal', we are very interested and invested in enabling oversubscription in Mesos in a safe and efficient way. Some of the community members have been working on an architecture document and prototypes recently and feel we have something we can start working out of: https://docs.google.com/document/d/1pUnElxHy1uWfHY_FOvvRC73QaOGgdXE0OXN-gbxdXA0/edit# We will continue meeting up and discussing the approach, as it is a significant change to how workloads are run on Mesos. Feel free to jump in, add suggestions, raise concerns and participate in the development of this feature. Cheers, Niklas