Hi Marco, It seems to me that the imbalance problem and the state is independent for this issue: the data distribution is only decided by the KeySelector used. The only limitation for state is that the keyed state is bind to the KeySelector used across the tasks. If the imbalance is the root problem, have you checked how many keys in total does the job have ?
Best, Yun ------------------Original Mail ------------------ Sender:Marco Villalobos <mvillalo...@kineteque.com> Send Date:Sat Jun 5 23:26:09 2021 Recipients:JING ZHANG <beyond1...@gmail.com> CC:user <user@flink.apache.org> Subject:Re: Is it possible to use OperatorState, when NOT implementing a source or sink function? Oh....that won't work for me either. I needed to use MapState. Perhaps I should describe my problem. I am using a KeyedState process function, but the workload that it is processing is not distributing well across the cluster. I have four task managers, but the way my data is keyed in this operator, it only goes to one task manager node. I need state, but I don't really need it keyed. On Sat, Jun 5, 2021 at 4:56 AM Marco Villalobos <mvillalo...@kineteque.com> wrote: Does that work in the DataStream API in Batch Execution Mode? On Sat, Jun 5, 2021 at 12:04 AM JING ZHANG <beyond1...@gmail.com> wrote: Hi,please use `CheckpointedFunction`, you could initialize your operator state in `initializeState` method by using context.getOperatorStateStore().*** Best regards, JING ZHANG Marco Villalobos <mvillalo...@kineteque.com> 于2021年6月5日周六 下午1:55写道： Is it possible to use OperatorState, when NOT implementing a source or sink function? If yes, then how?
