Hi , Thanks in advance for your help. I am trying to write a flink function which reads from kafka using kafka-flinkconsumer and sends messages to an indexer. I am not clear on how async and checkpointing will work in this case. My flow is like this: a) Messages are ingested into kafka. b)The messages are read by kafka-flink consumer from kafka c)My code is implementing RichAsyncFunction and sends messages to external indexer, I am using threads to send multiple requests to the indexer in parallel.
I am also using unorderedwait strategy for async. I want to know what will happen in following scenarios: Assuming kafka has 100 messages and say I have 10 threads(numbered T1...T10). Also assume that threads T1 is sending messages 1..10, T2 is sending messages 11..20 asynchronously to the indexer and so on and say I get reply 200 from indexer for last 8 threads T3-T10 but servers T1,T2 dont get any reply or get non-200 reply. in this case how will the checkpointing pointer move. Thanks, Anurag
