Re: Optimizing parallelism in reactive mode with adaptive scaling

2022-06-29 Thread Weihua Hu
Hi, Vishal The reactive mode will adjust the parallelism of tasks by slots of cluster. it will not allocate new workers automatically.[1] 1. max parallelism only works to scale up the parallelism of tasks. it will not affect the scheduling of tasks. 2. flink will enable slot sharing by default, u

Re: Optimizing parallelism in reactive mode with adaptive scaling

2022-06-29 Thread Vishal Surana
I've attached a screenshot of the job which highlights the "missing slots". [image: Screenshot 2022-06-29 at 9.38.54 PM.png] Coming to slot sharing, it seems that slot sharing isn't being honored. It doesn't matter if I put 2 or 3 of the 3 heavy weight operators - flink is simply ignoring it and a