Hi Team, I have a use case where in my Kafka Source, I need to wait for 2 hours before handling a event. Currently, following is the plan, kindly let me know if this would work without issues and any gotchas I need to be aware of.
a) In Kafka consumer deserializer schema, look at the published timestamp of the event, if it is sitting in the kafka queue for < 2 hours, sleep for remaining amount b) Increase checkpoint timeout to say 3 hours, so that checkpoints succeed. c) This also implies offset commits will happen in about 2 hours for some events when checkpointing succeeds, are there any internal configs that would interfere with this. d) Using the old FlinkKafkaConsumer (1.11) , any issues w.r.t using this one. Thanks for your advice. Thanks, Johny
