Hi Kamal, In general, tasks are scheduled in topological order, which means the source tasks will be scheduled first and they will be deployed on different TM if there are enough TMs. However, as Flink does not guarantee this, in some special situations such as failover, multiple source tasks may be scheduled to a single TM.
Best, Shamon FY On Fri, Jul 21, 2023 at 10:43 AM Kamal Mittal <kamal.mit...@ericsson.com> wrote: > Hello Shammon, > > > > Thanks for quick reply. > > > > Is there any way to limit only one task executes on each task manager for > an operator like source? > > > > If parallelism is set equal to no. of task managers for an operator then > it can work and keep cluster.evenly-spread-out-slots=true? > > > > Actually there is TCP server socket custom source created with parallelism > 1 as TCP server socket can only bind once for a port and this limits this > source task to execute only at one task manager which we want to avoid and > want to scale it across task managers. > > > > Rgds, > > Kamal > > > > *From:* Shammon FY <zjur...@gmail.com> > *Sent:* 21 July 2023 07:41 AM > *To:* Kamal Mittal <kamal.mit...@ericsson.com> > *Cc:* user@flink.apache.org > *Subject:* Re: About cluster.evenly-spread-out-slots > > > > Hi Kamal, > > > > Even if `cluster.evenly-spread-out-slots` is set to true, Flink will not > guarantee that same operator multiple tasks are never executed/scheduled on > the same task manager, it just means Flink will use > `LeastUtilizationSlotMatchingStrategy` to find the matching slot for the > task. > > > > As we know there will be multiple task managers with multiple slots in > Flink cluster, there will be two strategy to find the matching slot for > task according to its resource requirement: > > > > 1> Use AnyMatchingSlotMatchingStrategy as default, it will traverse all > free slots until it finds one slot which meets the resource requirement for > the task. > > 2> Use LeastUtilizationSlotMatchingStrategy when > `cluster.evenly-spread-out-slots` is true, it will calculate utilization > according the registered slot number, free slot number for each task > manager and find the matching slot from the task manager with min > utilization. > > > > Back to your example, if all slots in TM1 and TM2 match the task > requirements, there will be one free slot in TM1 and TM2 after your job is > scheduled if `cluster.evenly-spread-out-slots` is set to true. > > > > Best, > > Shammon FY > > > > > > On Thu, Jul 20, 2023 at 3:47 PM Kamal Mittal via user < > user@flink.apache.org> wrote: > > Hello, > > > > If property “cluster.evenly-spread-out-slots” is set to TRUE then Flink > guarantees that same operator multiple tasks are never executed/scheduled > on same task manager? Definitely this will depend upon parallelism value > used for an operator and no. of task slots available. > > > > Like in below diagram both operators has parallelism value as 2 and no. of > task slots available on each task manager are 3 each. So allocation of task > slots will be as below and one task slot on each task manager will remain > free? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Rgds, > > Kamal > >
