Hi Jun, Thanks for taking a look at my issue and also for updating the future release plan Wiki page.
My use case is to use Kafka as if it were a JMS provider (messaging use case). I'm currently using Kafka 0.8.1.1 with Java and specifically the Spring Integration Kafka Inbound Channel Adapter. Internally that adapter uses the HighLevelConsumer which shields the caller from the internals of offsets. Let's take the case where a consumer-group reads a number of messages and then is abruptly terminated before properly processing those messages. In that scenario upon restart ideally we'd begin reading at the offset we were at prior to abruptly terminating. If we have "auto.commit.enable=true" upon restart those messages will be considered already read and will be skipped. Setting "auto.commit.enable=false" would help in this case but now we'd have to manually call on the offset manager, requiring the use of the SimpleConsumer. In my use-case, it's acceptable for some manual intervention to say "reprocess messaging X thru Y", but to do so would require us to know the exact offset we had started at prior to the chunk that was read in when the JVM abnormally terminated. Perhaps I could look at the underlying ExportZkOffsets and ImportZkOffsets Java classes mentioned in this link, but at the very least I'd need to log the timestamp just prior to my read to be used in that query per: https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Idon'twantmyconsumer'soffsetstobecommittedautomatically.CanImanuallymanagemyconsumer'soffsets? It sounds like the ConsumerAPI rewrite mentioned in these links might be helpful in my situation (potentially targeted for Apr 2015): https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPI https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.9+Consumer+Rewrite+Design In the meantime if you have any suggestions for things I might be able to use to work-around my concern I'd be appreciative. Again I'm on 0.8.1.1 but would be willing to look at 0.8.2 if it offered anything to help with my use-case. Thanks Tony
