I would suggest we also consider the possibility of adding per framework
control on `min_allocatable_resources`.
If we want to consider supporting per-framework setting, we should probably
model this as a protobuf, rather than a free form JSON. The same protobuf
can be reused for both master flag, framework API, or even supporting
Resource Request in the future. Something like the following:
message ResourceQuantityPredicate {
enum Type {
SCALAR_GE,
}
optional Type type;
optional Value.Scalar scalar;
}
message ResourceRequirement {
required string resource_name;
oneof predicates {
ResourceQuantityPredicate quantity;
}
}
message ResourceRequirementList {
// All requirements MUST be met.
repeated ResourceRequirement requirements;
}
// Resource request API.
message Request {
repeated ResoruceRequrementList accepted;
}
// `allocatable()`
message MinimalAllocatableResources {
repeated ResoruceRequrementList accepted;
}
On Mon, Jun 11, 2018 at 3:47 PM, Meng Zhu <m...@mesosphere.com> wrote:
> Hi:
>
> The allocatable
> <https://github.com/apache/mesos/blob/1.5.x/src/master/allocator/mesos/hierarchical.cpp#L2471-L2479>
> check in the allocator (shown below) was originally introduced to
>
> help alleviate the situation where a framework receives some resources,
> but no
>
> cpu/memory, thus cannot launch a task.
>
>
> constexpr double MIN_CPUS = 0.01;constexpr Bytes MIN_MEM = Megabytes(32);
> bool HierarchicalAllocatorProcess::allocatable(
> const Resources& resources)
> {
> Option<double> cpus = resources.cpus();
> Option<Bytes> mem = resources.mem();
>
> return (cpus.isSome() && cpus.get() >= MIN_CPUS) ||
> (mem.isSome() && mem.get() >= MIN_MEM);
> }
>
>
> Issues
>
> However, there has been a couple of issues surfacing lately surrounding
> the check.
>
> -
> - - MESOS-8935 Quota limit "chopping" can lead to cpu-only and
> memory-only offers.
>
> We introduced fined-grained quota-allocation (MESOS-7099) in Mesos 1.5.
> When we
>
> allocate resources to a role, we'll "chop" the available resources of the
> agent up to the
>
> quota limit for the role. However, this has the unintended consequence of
> creating
>
> cpu-only and memory-only offers, even though there might be other agents
> with both
>
> cpu and memory resources available in the cluster.
>
>
> - MESOS-8626 The 'allocatable' check in the allocator is problematic with
> multi-role frameworks.
>
> Consider roleA reserved cpu/memory on an agent and roleB reserved disk on
> the same agent.
>
> A framework under both roleA and roleB will not be able to get the
> reserved disk due to the
>
> allocatable check. With the introduction of resource providers, the
> similar situation will
>
> become more common.
>
> Proposed change
>
> Instead of hardcoding a one-size-fits-all value in Mesos, we are proposing
> to add a new master flag
>
> min_allocatable_resources. It specifies one or more scalar resources
> quantities that define the
>
> minimum allocatable resources for the allocator. The allocator will only
> offer resources that are more
>
> than at least one of the specified resources. The default behavior *is
> backward compatible* i.e.
>
> by default, the flag is set to “cpus:0.01|mem:32”.
>
> Usage
>
> The flag takes in either a simple text of resource(s) delimited by a bar
> (|) or a JSON array of JSON
>
> formatted resources. Note, the input should be “pure” scalar quantities
> i.e. the specified resource(s)
>
> should only have name, type (set to scalar) and scalar fields set.
>
>
> Examples:
>
> - - To eliminate cpu or memory only offer due to the quota chopping,
> - we could set the flag to “cpus:0.01;mem:32”
> -
> - - To enable offering disk only offer, we could set the flag to
> “disk:32”
> -
> - - For both, we could set the flag to “cpus:0.01;mem:32|disk:32”.
> - Then the allocator will only offer resources that at least contain
> “cpus:0.01;mem:32”
> - OR resources that at least contain “disk:32”.
>
>
> Let me know what you think! Thanks!
>
>
> -Meng
>
>
