Hi All, This is what I am observing: we have a consumer which polls data from topic, does the processing, again polls data which keeps happening continuously. At one time, there was some bad data on the topic which could not be consumed by consumer, probably because it couldn't deserialize the event due to incompatible avro schema or something similar, and consumer got error deserializing event. Since the exception wasn't handled, it crashed the consumer thread which then stopped consuming data.
The question here is how these kind of scenarios can be handled: 1. Even if I catch the exception and log it, the consumer will i think process the next event. So the bad event will be lost 2. When consumer goes for another poll, it would commit offsets of previous poll which includes bad event, So the event will be lost How can this scenario be handled in best possible way?
