Hi Lu Niu, TM distribution on YARN nodes is managed by YARN RM and is out of the scope of Flink. On the other hand, cluster.evenly-spread-out-slots forces even distribution of tasks among Flink TMs, and has nothing to do with your concerns. Also, the config currently only supports Standalone mode Flink clusters, and does not take effect on a Flink cluster on YARN.
Best, Zhanghao Chen ________________________________ 发件人: Lu Niu <qqib...@gmail.com> 发送时间: 2023年8月29日 4:30 收件人: Kenan Kılıçtepe <kkilict...@gmail.com> 抄送: user <user@flink.apache.org> 主题: Re: Uneven TM Distribution of Flink on YARN Thanks for the reply. We've already set cluster.evenly-spread-out-slots = true Best Lu On Mon, Aug 28, 2023 at 1:23 PM Kenan Kılıçtepe <kkilict...@gmail.com<mailto:kkilict...@gmail.com>> wrote: Have you checked config param cluster.evenly-spread-out-slots ? On Mon, Aug 28, 2023 at 10:31 PM Lu Niu <qqib...@gmail.com<mailto:qqib...@gmail.com>> wrote: Hi, Flink users We have recently observed that the allocation of Flink TaskManagers in our YARN cluster is not evenly distributed. We would like to hear your thoughts on this matter. 1. Our setup includes Flink version 1.15.1 and Hadoop 2.10.0. 2. The uneven distribution is that out of a 370-node YARN cluster, there are 16 nodes with either 0 or 1 vCore available, while 110 nodes have more than 10 vCores available. Is such behavior expected? If not, is there a fix provided in Flink? Thanks! Best Lu
