Hi, Are you asking the question if that’s the behaviour or you have actually observed this issue? I’m not entirely sure, but I would guess that the Sink tasks would be distributed randomly across the cluster, but maybe I’m mixing this issue with resource allocations for Task Managers. Maybe Till will know something more about this?
One thing that might have solve/workaround the issue is to run those jobs in the job mode (one cluster per job), not in cluster mode, since containers for Task Managers are created/requested randomly. Piotrek > On 2 Mar 2019, at 23:53, Le Xu <sharonx...@gmail.com> wrote: > > Hello! > > I'm trying to find out if there a way to force task slot sharing within a > job. The example on the website looks like the following (as in the > screenshot) > > <image.png> > In this example, the single sink is slot-sharing with source/map (1) and > window operator (1). If I deploy multiple identical jobs shown above, all > sink operators would be placed on the first machine (which creates an > unbalanced scenario). Is there a way to avoid this situation (i.e., to have > sink operators of different jobs spread evenly across the task slots for the > entire cluster). Specifically, I was wondering if either of the following > options are possible: > 1. To force Sink[1] to be slot sharing with mapper from a different partition > on other slots such as (source[2] and window[2]). > 2. If option 1 is not possible, is there a "hacky" way for Flink to deploy > jobs starting from a different machine: e.g. For job 2, it can allocate > source/map[1], window[1], sink[1] to machine 2 instead of again on machine 1. > In this way the slot-sharing groups are still the same, but we end up having > sinks from the two jobs on different machines. > > > Thanks! > > > >
