[
https://issues.apache.org/jira/browse/KAFKA-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096224#comment-15096224
]
ASF GitHub Bot commented on KAFKA-3025:
---------------------------------------
GitHub user becketqin opened a pull request:
https://github.com/apache/kafka/pull/764
KAFKA-3025 Added timetamp to Message and use relative offset.
See KIP-31 and KIP-32 for details.
A few notes on the patch:
1. This patch only implements the server side message format change. It
also implements the message format conversion.
2. All unit tests passed when set . Some of the tests require specific
message format versions. I will fix them once the change on o.a.k.clients are
done.
3. The unit tests were run with new and old message format.
4. When message format conversion occurs during consumption, the consumer
will not be able to detect the message size too large situation. I did not try
to fix this because the situation seems rare and only happen during migration
phase.
The remaining work for KIP-31 and KIP-32 is
1. Add timestamp and relative offset to o.a.k.clients.
2. Add more unit test and integration test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/becketqin/kafka KAFKA-3025
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/764.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #764
----
commit 5604b4a57fff61188c562cc634daca4dee72aa4f
Author: Jiangjie Qin <becket....@gmail.com>
Date: 2016-01-07T14:30:39Z
Added timetamp to Message and use relative offset.
See KIP-31 and KIP-32 for details.
----
> KIP-31&KIP-32 (part 1): Add timestamp field to message, configs, and
> Producer/ConsumerRecord
> --------------------------------------------------------------------------------------------
>
> Key: KAFKA-3025
> URL: https://issues.apache.org/jira/browse/KAFKA-3025
> Project: Kafka
> Issue Type: Sub-task
> Reporter: Anna Povzner
> Assignee: Jiangjie Qin
>
> This JIRA is for changes for KIP-32 excluding broker checking and acting on
> timestamp field in a message.
> This JIRA includes:
> 1. Bump up MessageAndOffset version to version 1 to:
> a. Change absolute offset to relative offset.
> b. Add time field to the message
> Timestamp => int64
> Timestamp is the number of milliseconds since Unix Epoch
> 2. Add time field to both ProducerRecord and Consumer Record
> If a user specifies the timestamp in a ProducerRecord, the ProducerRecord is
> sent with this timestamp.
> If a user does not specify the timestamp in a ProducerRecord, the producer
> stamps the ProducerRecord with current time.
> ConsumerRecord will have the timestamp of the message that were stored on
> broker.
> 3. Add two new configurations to the broker. Configuration is per topic.
> * message.timestamp.type: type of a timestamp. Possible values: CreateTime,
> LogAppendTime. Default: CreateTime
> * max.message.time.difference.ms: threshold for the acceptable time
> difference between Timestamp in the message and local time on the broker.
> Default: Long.MaxValue
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
