Hi experts
        I am running flink job on YARN in job cluster mode, the job is divided 
into 2 tasks, the following are some configs of the job:
        parallelism.default => 16
        taskmanager.numberOfTaskSlots => 8
        -yn => 2

        when the program starts, I found that the count of task managers is not 
set immediately, but first expand then converge, I record the number during the 
process:
                Task Managers         Task Slots         Available Task Slots
        1.      14                               104                    88
        2.      15                             120                    104
        3.      16                             128                    112
        4.       6                               48                      32
        5.       3                               24                      8
        6.       2                               16                      0

        The final state is correct. There are 2 tasks, 32 subtask in total, due 
to slot sharing, only 16 slots are enough, the number of task slots per TM are 
8, so 2 TMs are needed.
        I have the following question:
        Because I specify yn=2, why does not directly allocate 2 TMs, but goes 
the expand then converge process?  Why does it apply 16 task managers at most? 
If it is not a must, how to avoid it?

        Thanks a lot!

Best
Henry
        

Reply via email to