GitHub user dossett opened a pull request:
https://github.com/apache/storm/pull/664
STORM-969: HDFS Bolt can end up in an unrecoverable state
A few notes about this PR:
- I updated the storm-hdfs pom.xml to align with other external modules.
Most significant change was probably going from hdfs version 2.2 to
${hadoop.version} (i.e. currently 2.6)
- Many errors are recovered by forcing a file rotation which opens a new,
valid File. So the rotation now occurs either according to rotation policy or
when a serious error happens. Work could probably be done to reopen the same
file name to reduce the number of rotations.
- Added unittests with MiniDFSCluster
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dossett/storm STORM-969
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/664.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #664
----
commit 795aaf93af78bf664727b91c179e0d96f673f674
Author: Aaron Dossett <[email protected]>
Date: 2015-08-02T22:22:51Z
STORM-969: HDFS Bolt can end up in an unrecoverable state
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
