Hi Till, I was able to use mesos.constraints.hard.hostattribute to run all task managers on a particular host in my cluster.
However, after looking a bit at the code, I'm not sure we can use mesos.constraints.hard.hostattribute for load balancing Flink task managers evenly across hosts in a Mesos cluster. This is because under the hood it uses the fenzo host attribute value constraint while we would need the fenzo balanced host attribute constraint. The LaunchableMesosWorker sets the constraints via the com.netflix.fenzo.TaskRequest and all of these hard constraints must be satisfied by a host for the task scheduler to assign this task to that host. Since the current implementation always return the static constraint value configured i.e. what is after ":", see org.apache.flink.mesos.runtime.clusterframework.MesosTaskManagerParameters#addHostAttrValueConstraint, I don't see how we can use it to load balance unless the constraint value would be dynamic based on the some property of the mesos task request. Am I correct in my assumptions? Any other way of load balancing? Maybe by not even using the DCOS Flink package (mesos flink framework) at all? Any plans to add support for the fenzo balanced host attribute constraint? Thanks, M On Fri, Sep 14, 2018 at 5:46 PM Till Rohrmann <trohrm...@apache.org> wrote: > Hi Martin, > > Flink supports the mesos.constraints.hard.hostattribute to specify task > constraints based on agent attributes [1]. I think you could use them to > control the task placement. > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/ops/config.html#mesos-constraints-hard-hostattribute > > Cheers, > Till > > On Fri, Sep 14, 2018 at 3:08 PM Martin Eden <martineden...@gmail.com> > wrote: > >> Thanks Vino! >> >> On Fri, Sep 14, 2018 at 3:37 AM vino yang <yanghua1...@gmail.com> wrote: >> >>> Hi Martin, >>> >>> Till has done most of the work of Flink on Mesos. Ping Till for you. >>> >>> Thanks, vino. >>> >>> Martin Eden <martineden...@gmail.com> 于2018年9月12日周三 下午11:21写道： >>> >>>> Hi all, >>>> >>>> We're using Flink 1.3.2 with DCOS / Mesos. >>>> >>>> We have a 3 node cluster and are running the Flink DCOS package (Flink >>>> Mesos framework) configured with 3 Task Managers. >>>> >>>> Our goal is to run each of them on separate hosts for better load >>>> balancing but it seems the task managers end up running on the same host. >>>> >>>> Looked around the docs and DCOS Flink package but could not find any >>>> placement policy or anything of the sorts. >>>> >>>> Is there anything like that? >>>> >>>> We are also planning to upgrade to the latest Flink version. Is >>>> something like that supported in this newer version? >>>> >>>> Thanks, >>>> M >>>> >>>
