Hello everyone, First,a brief pipeline introduction: env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime) consume multi kafka topic -> union them -> assignTimestampsAndWatermarks -> keyby -> window() and so on … It's a very normal way use flink to process data like this in production environment. But, If I want to test the pipeline above I need to use the api of FlinkKafkaConsumer.setStartFromTimestamp(timestamp) to comsume 'past’ data. So my question is how to control the ’step‘ banlence as one topic produces 3 records per second while another topic produces 30000 per second.
I don’t know if I describe clearly . so any suspicion please let me know Tks