Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/1764#discussion_r56802608
--- Diff: docs/apis/streaming/event_timestamps_watermarks.md ---
@@ -308,6 +269,72 @@ class TimeLagWatermarkGenerator extends
AssignerWithPeriodicWatermarks[MyEvent]
</div>
+#### **With Ascending timestamps**
+
+The simplest special case for periodic watermark generation is the case
where timestamps within one source occur in ascending order.
+In that case, the current timestamp can always act as a watermark, because
no lower timestamps will occur any more.
+
+Note that it is only necessary that timestamps are ascending *per parallel
data source instance*. For example, if
+in a specific setup one Kafka partition is read by one parallel data
source instance, then it is only necessary that
+timestamps are ascending within each Kafka partition. Flink's Watermark
merging mechanism will generate correct
+watermarks whenever parallel streams are shuffled, unioned, connected, or
merged.
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+{% highlight java %}
+DataStream<MyEvent> stream = ...
+
+DataStream<MyEvent> withTimestampsAndWatermarks =
+ stream.assignTimestampsAndWatermarks(new
AscendingTimestampExtractor<MyEvent>() {
+
+ @Override
+ public long extractAscendingTimestamp(MyEvent element) {
+ return element.getCreationTime();
+ }
+});
+{% endhighlight %}
+</div>
+<div data-lang="scala" markdown="1">
+{% highlight scala %}
+val stream: DataStream[MyEvent] = ...
+
+val withTimestampsAndWatermarks = stream.assignAscendingTimestamps(
_.getCreationTime )
+{% endhighlight %}
+</div>
+</div>
+
+#### **With Watermark of Fixed Allowed Lateness**
+
+Another example of periodic watermark generation is the one where the
watermark lags behind the maximum (event-time) timestamp
+seen in the stream, by a fixed amount of time. This case covers scenarios
where the maximum lateness that can be encountered in a
+stream is known in advance, e.g. when creating custom sources containing
elements with timestamps spread within a fixed period of
+time for testing. For these cases, Flink provides the
`FixedAllowedLatenessWatermarkEmitter` which takes as an argument
+the `maxAllowedLateness`, i.e. the maximum amount of time an element is
allowed to be late, before being dropped from the stream.
--- End diff --
Late elements are not dropped from the stream! (In #1819 this is also
mentioned multiple times).
Its just that late elements won't make it into their time-window anymore
(or anything else an operator implements with watermarks).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
