[
https://issues.apache.org/jira/browse/FLINK-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222924#comment-14222924
]
ASF GitHub Bot commented on FLINK-1081:
---------------------------------------
Github user gyfora commented on the pull request:
https://github.com/apache/incubator-flink/pull/226#issuecomment-64183824
Thats a good point, why dont we make this optional (should be possible):
3 types:
-only new files
-reprocess with appends
-process appends from offset
So then the user can pick one that suits the use-case
> Add HDFS file-stream source for streaming
> -----------------------------------------
>
> Key: FLINK-1081
> URL: https://issues.apache.org/jira/browse/FLINK-1081
> Project: Flink
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 0.7.0-incubating
> Reporter: Gyula Fora
> Assignee: Chiwan Park
> Labels: starter
>
> Add data stream source that will monitor a slected directory on HDFS (or
> other filesystems as well) and will process all new files created.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
