Hi Martin, I'm not aware that the community is actively working on enabling the balanced host attribute constraint. If you wanna give it a try, then I'm happy to review your contribution.
Cheers, Till On Mon, Sep 17, 2018 at 5:28 AM Renjie Liu <liurenjie2...@gmail.com> wrote: > Hi, Martin: > I think a better solution would be to set the number of cores of each > container equals to that of a physical server if this mesos cluster is > dedicated to your flink cluster. > > On Mon, Sep 17, 2018 at 5:28 AM Martin Eden <martineden...@gmail.com> > wrote: > >> Hi Till, >> >> I was able to use mesos.constraints.hard.hostattribute to run all task >> managers on a particular host in my cluster. >> >> However, after looking a bit at the code, I'm not sure we can use >> mesos.constraints.hard.hostattribute for load balancing Flink task managers >> evenly across hosts in a Mesos cluster. >> >> This is because under the hood it uses the fenzo host attribute value >> constraint while we would need the fenzo balanced host attribute constraint. >> >> The LaunchableMesosWorker sets the constraints via >> the com.netflix.fenzo.TaskRequest and all of these hard constraints must be >> satisfied by a host for the task scheduler to assign this task to that >> host. Since the current implementation always return the static constraint >> value configured i.e. what is after ":", see >> org.apache.flink.mesos.runtime.clusterframework.MesosTaskManagerParameters#addHostAttrValueConstraint, >> I don't see how we can use it to load balance unless the constraint value >> would be dynamic based on the some property of the mesos task request. >> >> Am I correct in my assumptions? >> >> Any other way of load balancing? >> Maybe by not even using the DCOS Flink package (mesos flink framework) at >> all? >> Any plans to add support for the fenzo balanced host attribute constraint? >> >> Thanks, >> M >> >> >> >> >> On Fri, Sep 14, 2018 at 5:46 PM Till Rohrmann <trohrm...@apache.org> >> wrote: >> >>> Hi Martin, >>> >>> Flink supports the mesos.constraints.hard.hostattribute to specify task >>> constraints based on agent attributes [1]. I think you could use them to >>> control the task placement. >>> >>> [1] >>> https://ci.apache.org/projects/flink/flink-docs-stable/ops/config.html#mesos-constraints-hard-hostattribute >>> >>> Cheers, >>> Till >>> >>> On Fri, Sep 14, 2018 at 3:08 PM Martin Eden <martineden...@gmail.com> >>> wrote: >>> >>>> Thanks Vino! >>>> >>>> On Fri, Sep 14, 2018 at 3:37 AM vino yang <yanghua1...@gmail.com> >>>> wrote: >>>> >>>>> Hi Martin, >>>>> >>>>> Till has done most of the work of Flink on Mesos. Ping Till for you. >>>>> >>>>> Thanks, vino. >>>>> >>>>> Martin Eden <martineden...@gmail.com> 于2018年9月12日周三 下午11:21写道： >>>>> >>>>>> Hi all, >>>>>> >>>>>> We're using Flink 1.3.2 with DCOS / Mesos. >>>>>> >>>>>> We have a 3 node cluster and are running the Flink DCOS package >>>>>> (Flink Mesos framework) configured with 3 Task Managers. >>>>>> >>>>>> Our goal is to run each of them on separate hosts for better load >>>>>> balancing but it seems the task managers end up running on the same host. >>>>>> >>>>>> Looked around the docs and DCOS Flink package but could not find any >>>>>> placement policy or anything of the sorts. >>>>>> >>>>>> Is there anything like that? >>>>>> >>>>>> We are also planning to upgrade to the latest Flink version. Is >>>>>> something like that supported in this newer version? >>>>>> >>>>>> Thanks, >>>>>> M >>>>>> >>>>> -- > Liu, Renjie > Software Engineer, MVAD >
