Hi All, I am designing a streaming pipeline using Flink 1.8.1, which consumes messages from Kinesis and apply some business logic on per key basis using KeyedProcessFunction and Checkpointing(HeapStateBackend). It is consuming messages around 7GB per minutes from multiple Kinesis streams. I am using only one Kinesis Source which is configured with multiple streams.
The pipeline processes data and writes output to s3 as expected but I am experiencing a very weird issue when one of the stream is completely empty then it doesn't flush any file to s3 however it is consuming data from rest of the streams. When i remove only this empty stream and again submit the job then everything works fine and it writes output to s3. Regards, Ravi
