> On Sept. 15, 2016, 5:49 p.m., Jiang Yan Xu wrote: > > The addition of `parseText` and `parseValidResources` feels unnecessary to > > me. I think we can reuse `convertJSON()` to achieve the same. > > > > Overall I think the following steps would be simpler and more elegant. > > > > ## Patch 1: Change convertJSON() to return a Try<vector<Resource>> > > > > This way the empty disks in the result are not dropped. > > > > ``` > > vector<Resource> convertJSON( > > const JSON::Array& resourcesJSON, > > const string& defaultRole) > > ``` > > > > I also don't see why `convertJSON()` needs to fail on empty resources since > > they are allowed in the regular string formatted resources specification. > > Maybe we can have a **patch 0** that changes that? > > > > The current lone call site can just construct `Resources` from > > `vector<Resource>`. > > > > Then make convertJSON a public static member method. > > > > ## Patch 2: Let Resources class answer if a resource is a MOUNT disk, PATH > > disk or ROOT disk > > > > e.g., > > > > ``` > > static bool Resources::isMountDisk(const Resource& resource); > > ``` > > > > These are pretty general purpose and purely for convenience. Add simple > > tests to verify these methods. > > > > ## Patch 3: Autodetection for MOUNT and ROOT disks > > > > In `Containerizer::resources()`, process each disk from the result of > > `convertJSON()`. > > > > For MOUNT disks that are empty (`isEmpty()`), detect their sizes and modify > > the Resource objects. > > What about the ROOT disk? If `disk:0` is in a plain string we don't do > > anything. With the JSON input the operator can choose to not specify a root > > disk to have it autodetected, so I guess we don't have to separately deal > > with ROOT disks with an explict "zero" size. If all disks have been > > processed and no ROOT disk is encountered, we auto-detect it. > > > > How does it sound?
Refactored this RR into 4 separate RRs: (1) https://reviews.apache.org/r/51999/ (Patch 1): Refactor parsing of resources to include all valid resources. - Modified `convertJSON()` to not fail on empty resources (now is in sync with text representation). - Converted `convertJSON()` to return `Try<vector<Resource>>`. - `Resources::parse()` is refactored to use a new function `Resources::parseText()` that returns `Try<vector<Resource>>` so that existing calls to `Resources::parse()` are not affected. (2) https://reviews.apache.org/r/52001/: Allow text based disk resources to indicate source type and root. (3) https://reviews.apache.org/r/52002/ (Patch 2): Added helper methods to determine types of disk resources. (4) https://reviews.apache.org/r/51879/ (Patch 3): Determine disk size when not specified in static resources. - As discussed offline, we do not add root disk if missing when `--resources` does not include the root disk is to allow the possibility of removing the root disk should the operator desires. > On Sept. 15, 2016, 5:49 p.m., Jiang Yan Xu wrote: > > src/slave/containerizer/containerizer.cpp, line 177 > > <https://reviews.apache.org/r/51879/diff/1/?file=1498166#file1498166line177> > > > > Call them `disks` to avoid confusion with the volumes as a Mesos > > concept. Updated based on the refactor. > On Sept. 15, 2016, 5:49 p.m., Jiang Yan Xu wrote: > > src/slave/containerizer/containerizer.cpp, lines 260-265 > > <https://reviews.apache.org/r/51879/diff/1/?file=1498166#file1498166line260> > > > > This is only for the ROOT disk, i.e., we only leave some headroom for > > the system root partition. > > > > Therefore we probably don't need this method. Agreed. However, we have 2 cases for rot disks: a. When no disks are present in `--resources` b. When `--resources` indicates a disk without source. I refactored `Containerizer::diskSize()` to take in if it is a root disk and add the headroom for root disks only. - Anindya ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51879/#review149033 ----------------------------------------------------------- On Sept. 14, 2016, 2:29 a.m., Anindya Sinha wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51879/ > ----------------------------------------------------------- > > (Updated Sept. 14, 2016, 2:29 a.m.) > > > Review request for mesos and Jiang Yan Xu. > > > Bugs: MESOS-6062 > https://issues.apache.org/jira/browse/MESOS-6062 > > > Repository: mesos > > > Description > ------- > > When static resources indicate disks with a positive size, we use that > for the disk resources on the agent. However, --resources can include > disks with size of 0, which indicates that mesos agent determine the > size of those disks from the host and uses that information instead. > Note that this is not allowed for PATH disks since PATH disks can be > carved into smaller chunks and we cannot assume that the whole > physical disk is available to the PATH disk. > > With this change, JSON or textual representation for resources that > specify scalar value of 0 would not result in an error, but those > resources will not be accounted for until a valid size is determined > for such resources. A scalar value of -1 in JSON or textual formats > still results in an invalid resource. > > > Diffs > ----- > > include/mesos/resources.hpp 7ba422d57d47777e058c682df9aa780557782423 > include/mesos/v1/resources.hpp add48c7baf8fabf5cc443d60a6c96d2902fc67de > src/common/resources.cpp f841e91b81975a887e64a76a709708ed7946025f > src/slave/containerizer/containerizer.hpp > f13669d0dfc4ce3287cfe630cabd0470dc765b51 > src/slave/containerizer/containerizer.cpp > d46882baa904fd439bffb23c324828b777228f1c > src/tests/resources_tests.cpp f627862870279e5a778aa83e1ddcaa88c27ba1cb > src/v1/resources.cpp 4410a8f95ef0ebe6ec7aee38cd47f6f83f863867 > > Diff: https://reviews.apache.org/r/51879/diff/ > > > Testing > ------- > > Tests passed. > > > Thanks, > > Anindya Sinha > >
