> On Nov. 21, 2014, 2:41 p.m., Joel Koshy wrote: > > core/src/main/scala/kafka/server/KafkaApis.scala, line 147 > > <https://reviews.apache.org/r/27391/diff/4/?file=766686#file766686line147> > > > > I think what you meant earlier was with < v2 you could have different > > timestamps for each partition so a global retention won't work with < v2
In the OffsetManager, the timestamp will only be overriden by the global one if it is set to Default which is -1, so for v0/1 although the retention is used it will not be overriding the timestamp. > On Nov. 21, 2014, 2:41 p.m., Joel Koshy wrote: > > core/src/main/scala/kafka/common/OffsetMetadataAndError.scala, line 17 > > <https://reviews.apache.org/r/27391/diff/4/?file=766684#file766684line17> > > > > I thought we would be going with separate format for on-disk storage? > > > > E.g., one thing that is extremely useful (until we have timestamp as a > > first-class field of messages) is to have the receive time of the > > offsetcommit in the stored offset entries. This is very useful for > > debugging. Yes they are separated: for on-disk storage the timestamp will always been stored, and for wire protocol only v0/1 will contain that value, but for v2 this value will be computed via retention. So the on-disk format is specified as OffsetAndMetadata, and when we deprecating v0/1 and adding the timestmap to message header we will replace this with OffsetMetadata. - Guozhang ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27391/#review62553 ----------------------------------------------------------- On Nov. 18, 2014, 1:42 a.m., Guozhang Wang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27391/ > ----------------------------------------------------------- > > (Updated Nov. 18, 2014, 1:42 a.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1634 > https://issues.apache.org/jira/browse/KAFKA-1634 > > > Repository: kafka > > > Description > ------- > > The timestamp field of OffsetAndMetadata is preserved since we need to be > backward compatible with older versions > > > Diffs > ----- > > clients/src/main/java/org/apache/kafka/common/protocol/Protocol.java > 7517b879866fc5dad5f8d8ad30636da8bbe7784a > > clients/src/main/java/org/apache/kafka/common/requests/OffsetCommitRequest.java > 3ee5cbad55ce836fd04bb954dcf6ef2f9bc3288f > > clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java > df37fc6d8f0db0b8192a948426af603be3444da4 > core/src/main/scala/kafka/api/OffsetCommitRequest.scala > 050615c72efe7dbaa4634f53943bd73273d20ffb > core/src/main/scala/kafka/api/OffsetFetchRequest.scala > c7604b9cdeb8f46507f04ed7d35fcb3d5f150713 > core/src/main/scala/kafka/common/OffsetMetadataAndError.scala > 4cabffeacea09a49913505db19a96a55d58c0909 > core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala > f476973eeff653473a60c3ecf36e870e386536bc > core/src/main/scala/kafka/server/KafkaApis.scala > 968b0c4f809ea9f9c3f322aef9db105f9d2e0e23 > core/src/main/scala/kafka/server/KafkaServer.scala > 4de812374e8fb1fed834d2be3f9655f55b511a74 > core/src/main/scala/kafka/server/OffsetManager.scala > 2957bc435102bc4004d8f100dbcdd56287c8ffae > core/src/test/scala/unit/kafka/api/RequestResponseSerializationTest.scala > cd16ced5465d098be7a60498326b2a98c248f343 > core/src/test/scala/unit/kafka/server/OffsetCommitTest.scala > 8c5364fa97da1be09973c176d1baeb339455d319 > > Diff: https://reviews.apache.org/r/27391/diff/ > > > Testing > ------- > > > Thanks, > > Guozhang Wang > >
