[ https://issues.apache.org/jira/browse/STORM-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056720#comment-15056720 ]
ASF GitHub Bot commented on STORM-1187: --------------------------------------- Github user Parth-Brahmbhatt commented on a diff in the pull request: https://github.com/apache/storm/pull/900#discussion_r47557478 --- Diff: docs/documentation/Windowing.md --- @@ -126,6 +126,96 @@ Time duration based tumbling window that tumbles after the specified time durati ``` +## Tuple timestamp and out of order tuples +By default the timestamp tracked in the window is the time when the tuple is processed by the bolt. The window calculations +are performed based on the processing timestamp. Storm has support for tracking windows based on the source generated timestamp. + +```java +/** +* Specify a field in the tuple that represents the timestamp as a long value. If this +* field is not present in the incoming tuple, an {@link IllegalArgumentException} will be thrown. +* +* @param fieldName the name of the field that contains the timestamp +*/ +public BaseWindowedBolt withTimestampField(String fieldName) +``` + +The value for the above `fieldName` will be looked up from the incoming tuple and considered for windowing calculations. +If the field is not present in the tuple an exception will be thrown. Along with the timestamp field name, a time lag parameter +can also be specified which indicates the max time limit for tuples with out of order timestamps. + +E.g. If the lag is 5 secs and a tuple `t1` arrived with timestamp `06:00:05` no tuples may arrive with tuple timestamp earlier than `06:00:00`. If a tuple +arrives with timestamp 05:59:59 after `t1` and the window has moved past `t1`, it will be treated as a late tuple and not processed. --- End diff -- Lets also document how users can find out number of discarded tuples? In many cases it may also be useful to provide a handler for tuples being discarded but I am fine with not including that in this patch. > Support for late and out of order events in time based windows > -------------------------------------------------------------- > > Key: STORM-1187 > URL: https://issues.apache.org/jira/browse/STORM-1187 > Project: Apache Storm > Issue Type: Sub-task > Reporter: Arun Mahadevan > Assignee: Arun Mahadevan > > Right now the time based windows uses the timestamp when the tuple is > received by the bolt. > However there are use cases where the tuples can be processed based on the > time when they are actually generated vs the time when they are received. So > we need to add support for processing events with a time lag and also have > some way to specify and read tuple timestamps. -- This message was sent by Atlassian JIRA (v6.3.4#6332)