Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1084#issuecomment-137390414
I'm currently trying out the module. Some comments:
- Why do we name the module `flink-connector-hdfs`. I think a name such as
`flink-connector-filesystems` or `flink-connector-hfs` would be more
appropriate because its is implemented against Hadoops `FileSystems` classes,
and not HDFS classes.
So users should be able to use the connector with other FS such as Tachyon,
NFS, S3 etc.
- Is there a way of re-using existing InputFormats with the rolling file
sink? I guess users will start asking about CSV, Avro, Parquet ...
- I think there is already some code to monitor a file system directory to
ingest it into a data stream. Maybe it would make sense to move that code out
of the core into this module?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
