Ufuk Celebi created FLINK-7666:
----------------------------------
Summary: ContinuousFileReaderOperator swallows chained watermarks
Key: FLINK-7666
URL: https://issues.apache.org/jira/browse/FLINK-7666
Project: Flink
Issue Type: Improvement
Components: Streaming Connectors
Affects Versions: 1.3.2
Reporter: Ufuk Celebi
I use event time and read from a (finite) file. I assign watermarks right after
the {{ContinuousFileReaderOperator}} with parallelism 1.
{code}
env
.readFile(new TextInputFormat(...), ...)
.setParallelism(1)
.assignTimestampsAndWatermarks(...)
.setParallelism(1)
.map()...
{code}
The watermarks I assign never progress through the pipeline.
I can work around this by inserting a {{shuffle()}} after the file reader or
starting a new chain at the assigner:
{code}
env
.readFile(new TextInputFormat(...), ...)
.setParallelism(1)
.shuffle()
.assignTimestampsAndWatermarks(...)
.setParallelism(1)
.map()...
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
