Re: Flink batch mode does not sort by event timestamp

2022-04-22 Thread David Anderson
The DataStream API's BATCH execution mode first sorts by key, and within each key, it sorts by timestamp. By operating this way, it only needs to keep state for one key at a time, so this keeps the runtime simple and efficient. Regards, David P.S. I see you also asked this question on stack

Flink batch mode does not sort by event timestamp

2022-04-21 Thread Han You
I have a custom flink Source, and I have a SerializableTimestampAssigner that assigns event timestamps to records emitted by the source. The source may emit records out of order because of the nature of the underlying data storage, however with BATCH mode, I expect Flink to sort these records