Elias Levy created KAFKA-4219:
---------------------------------
Summary: Permit setting of event time in stream processor
Key: KAFKA-4219
URL: https://issues.apache.org/jira/browse/KAFKA-4219
Project: Kafka
Issue Type: Improvement
Components: streams
Affects Versions: 0.10.0.1
Reporter: Elias Levy
Assignee: Guozhang Wang
Event time is assigned in stream sources via {{TimestampExtractor}}. Once the
event time has been assigned, it remains the same, regardless of any downstream
processing in the topology. This is insufficient for many processing jobs,
particularly when the output of the job is written back into a Kafka topic,
where the record's time is encoded outside of the record's value.
For instance:
* When performing windowed aggregations it may be desirable for the timestamp
of the emitted record to be lower or higher limits of the time window, rather
than the timestamp of the last processed element, which may be anywhere within
the time window.
* When joining two streams, it is non-deterministic which of the two record's
timestamps will be the timestamp of the emitted record. It would be either one
depending on what order the records are processed. Even where this
deterministic, it may be desirable for the emitted timestamp to be altogether
different from the timestamp of the joined records. For instance, setting the
timestamp to the current processing time may be desirable.
* In general, lower level processors may wish to set the timestamp of emitted
records to an arbitrary value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
