I understand that the reception of Kafka messages may be out of order[1][2]. But I have a multiple producers/ consumers scenario where I would need to know when the last record is processed at the producer side, so once consumers process e.g. NoMoreMessage, the entire system can react such as stopping the consumers, proceeding to the next step, and so on. But due to the nature of distributed system, there is no guaranteed that NoMoreMessage issued at the producer side would be the last message processed by consumers. So I need pieces of advice if any strategies I can exploit in this scenario with Kafka?
I appreciate any suggestions. [1]. https://www.quora.com/Is-it-possible-to-consume-kafka-message-offset-based-on-timestamp [2]. http://stackoverflow.com/questions/35525356/kafka-multiple-producers-writing-to-same-topic-and-order-of-message-is-importa?rq=1
