Github user gyfora commented on the pull request:
https://github.com/apache/incubator-flink/pull/226#issuecomment-65709404
You are right Robert. This behavior is unexpected at best, and we will have
to do something about it. It actually applies to other sources as well. A
central monitor would be ideal until then we could figure out some workaround.
The first thing that came to my mind is to somehow partition the incoming files
in the sources for example hash the file names. We should of course try to
respect locality for performance.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
