[
https://issues.apache.org/jira/browse/FLINK-5018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16334601#comment-16334601
]
Eron Wright commented on FLINK-5018:
-------------------------------------
To my understanding, watermarks are either generated by a source or generated
by the 'assignTimestampsAndWatermarks' operator. The latter is a generic
solution for dealing with sources that don't support watermarks. In my
opinion, if the source is watermark-aware, it should handle all aspects of
idleness (including the implementation of an idle timeout). Similarly, if the
generic watermark operator is being used, it should provide idleness logic
My fear is that the app developer will try to create unsupported combinations
of watermark-aware sources with generic watermark logic. Maybe this idle
timeout feature should be exposed as a facility for use by the source
implementation, rather than the app directly.
> Make source idle timeout user configurable
> ------------------------------------------
>
> Key: FLINK-5018
> URL: https://issues.apache.org/jira/browse/FLINK-5018
> Project: Flink
> Issue Type: Sub-task
> Components: DataStream API
> Reporter: Tzu-Li (Gordon) Tai
> Priority: Major
> Fix For: 1.5.0
>
>
> There are 2 cases where sources are considered idle and should emit an idle
> {{StreamStatus}} downstream, taking Kafka consumer as example:
> - The source instance was not assigned any partitions
> - The source instance was assigned partitions, but they currently don't have
> any data.
> For the second case, we can only consider it idle after a timeout threshold.
> It would be good to make this timeout user configurable besides a default
> value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
