Hi, The answer is in fact no. Flink hash-partitions keys into Key Groups [1] which are uniformly assigned to tasks, i.e., a task can process more than one key group. AFAIK, there are no plans to change this behavior. Stefan (in CC) might be able to give more details on this.
Something that might be possible in the future is to be more clever about the key group - task assignment, e.g., taking state size or number of records into account. Best, Fabian [1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/state.html#keyed-state Am Mi., 27. Feb. 2019 um 17:23 Uhr schrieb Aggarwal, Ajay < ajay.aggar...@netapp.com>: > I couldn’t find reference to it anywhere in the docs, so I thought I will > ask here. > > > > When I use KeyBy operator, say KeyBy (“customerId”) and some keys (i.e. > customers) are way too noisy than others, is there a way to ensure that too > many noisy customers do not land on the same taskslot? In general does > flink attempts to keep the load balanced across different taskslots > assigned to a KeyBy operator ? > > > > I wouldn’t be surprised if the answer is “currently no”. Would like to > know if something related is planned for future. Also would love to hear > from others who ran into similar situation and how they addressed it. > > > > Thanks. > > >
