Hi all, I am opening this thread to discuss FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data. The design doc can be found at https://cwiki.apache.org/confluence/display/FLINK/FLIP-327%3A+Support+stream-batch+unified+operator+to+improve+job+throughput+when+processing+backlog+data .
This FLIP enables a Flink job to initially operate in batch mode, achieving high throughput while processing records that do not require low processing latency. Subsequently, the job can seamlessly transition to stream mode for processing real-time records with low latency. Importantly, the same state can be utilized before and after this mode switch, making it particularly valuable when users wish to bootstrap the job's state using historical data. We would greatly appreciate any comments or feedback you may have on this proposal. Cheers, Dong
