Hi Matvey,
I have a question about the config value offsets.retention.minutes in kafka > 0.9.0.0. > Is this the timeout for when offsets get compacted in the topic or actually > deleted (as it appears from a cursory reading of > > https://github.com/apache/kafka/blob/0.9.0/core/src/main/scala/kafka/coordinator/GroupMetadataManager.scala#L532 > ) > It is actually deleted at that point. i.e., it is removed from the in-memory cache and we append a tombstone for it in the log (which means it will get deleted whenever that segment rolls over and becomes eligible for compaction). So a subsequent offset fetch will return no offset. > If it actually deletes the offsets, what happens if a topic simply doesn't > get a new message for a day (the default value for > offsets.retention.minutes). Are the offsets then forgotten? This seems > Yes > really strange and counter-intuitive. I thought offsets were supposed to be > saved forever (isn't that why a compacted topic is used)? > One reason to purge offsets older than the configured retention is to clean-up after consumers that are no longer active (especially short-lived consumers such as console consumers). Users can certainly opt to override this behavior by setting it to a very high value. (Although it sounds like you are suggesting that "forever" should be the default in the first place?) Joel