Jochen Niebuhr created SPARK-22480:
--------------------------------------
Summary: Dynamic Watermarking
Key: SPARK-22480
URL: https://issues.apache.org/jira/browse/SPARK-22480
Project: Spark
Issue Type: Wish
Components: Structured Streaming
Affects Versions: 2.2.0
Reporter: Jochen Niebuhr
When you're using the watermark feature, you're forced to provide an absolute
duration to identify late events. For the case we're using structured streaming
for, this is not completely working. In our case, late events will be possible
on the next business day. So I'd like to use 24 hours watermark for
Sunday-Thursday, 72 hours for Friday, 48 for Saturday. If this sounds like a
good idea, I can probably supply a pull request.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
