Hi Martin, _reinterpretAsKeyedStream_ should definitely work with the reactive mode, if it doesn't it's a bug that needs to be fixed
> For test use cases (3) and (4) the state of the keyed process function (c) > seems only available for around 50% of the events processed after > scale-in/fail. > Can you please provide details on how did you verify this? Best, D. On Fri, Jan 7, 2022 at 8:10 AM Martin <mar...@sonicdev.de> wrote: > Hi, > > typo: "I run that job via Native Kubernetes deployment and use elastic > scaling in reactive mode." > -> I run it of course via standalone kubernetes deployment, to make > elastic scaling possible. > > BR > Martin > > > > mar...@sonicdev.de schrieb am 06.01.2022 21:38 (GMT +01:00): > > Hi, > > I have a job where I do a keyBy'd process function (a), then on the > downstream a normal process function (b) and and then I use > reinterpretAsKeyedStream to have yet another keyBy'd process function (c). > > The last keyed process function use keyed state for a increasing sequence > number. > > I run that job via Native Kubernetes deployment and use elastic scaling in > reactive mode. > > I use Flink 1.14.2. > > I test that job on four use cases: (1) static parallelism, (2) scale out, > (3) scale-in, (4) task manager file*. > > * via kill -SIGTERM inside the container for the flink JVM > > For test use cases (1) and (2) everything is fine. > > For test use cases (3) and (4) the state of the keyed process function (c) > seems only available for around 50% of the events processed after > scale-in/fail. > > Is the reinterpretAsKeyedStream feature in general usable with Elastic > Scaling in Reactive Mode in Flink 1.14.2? > > If yes, already any ideas what the root cause could be? > > > > BR > Martin > > > >
