GitHub user adenes opened a pull request:
https://github.com/apache/flume/pull/127
FLUME-3080. Call DistributedFileSystem.recoverLease() if close() fails to
avoid lease leak
If the HDFS Sink tries to close a file but it fails (e.g. due to timeout)
the last block might not end up in COMPLETE state. In this case block recovery
should happen but as the lease is still held by Flume the NameNode will start
the recovery process only after the hard limit of 1 hour expires.
This change adds an explicit recoverLease() call in case of close failure.
For more details see https://issues.apache.org/jira/browse/FLUME-3080
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/adenes/flume FLUME-3080
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flume/pull/127.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #127
----
commit 8cc9082c69ad0aea2e8cfa20e906261a6a417245
Author: Denes Arvay <[email protected]>
Date: 2017-04-03T15:27:19Z
FLUME-3080. call DistributedFileSystem.recoverLease() if close() fails to
avoid lease leak
If the HDFS Sink tries to close a file but it fails (e.g. due to timeout)
the last block might
not end up in COMPLETE state. In this case block recovery should happen but
as the lease is
still held by Flume the NameNode will start the recovery process only after
the hard limit of
1 hour expires.
This change adds an explicit recoverLease() call in case of close failure.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
