[
https://issues.apache.org/jira/browse/FLINK-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15763551#comment-15763551
]
ASF GitHub Bot commented on FLINK-4616:
---------------------------------------
GitHub user MayerRoman opened a pull request:
https://github.com/apache/flink/pull/3031
[FLINK-4616] Added functionality through which watermarks for each pa…
…rtition are saved and loaded via checkpointing mechanism
[FLINK-4616] Kafka consumer doesn't store last emmited watermarks per
partition in state.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/MayerRoman/flink FLINK_4616
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3031.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3031
----
commit bb9a54903dd445aa4d0750b1a0d6d1d592ab891f
Author: Roman Maier <[email protected]>
Date: 2016-12-20T07:28:12Z
[FLINK-4616] Added functionality through which watermarks for each
partition are saved and loaded via checkpointing mechanism
----
> Kafka consumer doesn't store last emmited watermarks per partition in state
> ---------------------------------------------------------------------------
>
> Key: FLINK-4616
> URL: https://issues.apache.org/jira/browse/FLINK-4616
> Project: Flink
> Issue Type: Bug
> Components: Kafka Connector
> Affects Versions: 1.1.1
> Reporter: Yuri Makhno
> Assignee: Roman Maier
> Fix For: 1.2.0
>
>
> Kafka consumers stores in state only kafka offsets and doesn't store last
> emmited watermarks, this may go to wrong state when checkpoint is restored:
> Let's say our watermark is (timestamp - 10) and in case we have the following
> messages queue results will be different after checkpoint restore and during
> normal processing:
> A(ts = 30)
> B(ts = 35)
> ------ checkpoint goes here
> C(ts=15) -- this one should be filtered by next time window
> D(ts=60)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
