I believe there can be advantages and disadvantages in both directions. For example, fewer containers with multiple slots reduces the effort the Flink Master has to do whenever global coordination is required, i.e., during checkpointing. And the network stack in the task managers is optimized to take advantage of locality, whenever possible.
On the other hand, if you have a lot of pressure on the heap (e.g., because you are using a heap-based state backend), then having more, smaller task managers can reduce latency by reducing the impact of garbage collection pauses. I'm sure I've overlooked some factors, but the bottom line appears to be that there's no one-size-fits-all answer. David On Wed, Sep 25, 2019 at 5:43 PM Navneeth Krishnan <reachnavnee...@gmail.com> wrote: > > Thanks Terry, the reason why I asked this is because somewhere I saw running > one slot per container is beneficial. I couldn’t find the where I saw that. > Also I think running it with multiple slots will reduce IPC since some of the > data will be processed writhing the same JVM. > > Thanks > > On Wed, Sep 25, 2019 at 1:16 AM Terry Wang <zjuwa...@gmail.com> wrote: >> >> Hi, Navneeth, >> >> I think both is ok. >> IMO, run one container with number of slots same as virtual cores may be >> better for slots can share the Flink Framework and thus reduce memory cost. >> >> Best, >> Terry Wang >> >> >> >> > 在 2019年9月25日，下午3:26，Navneeth Krishnan <reachnavnee...@gmail.com> 写道： >> > >> > Hi All, >> > >> > I’m currently running flink on amazon ecs and I have assigned task slots >> > based on vcpus per instance. Is it beneficial to run a separate container >> > with one slot each or one container with number of slots same as virtual >> > cores? >> > >> > Thanks >>
