GitHub user mcsanady opened a pull request:
https://github.com/apache/flume/pull/180
FLUME-3190: flume shutdown hook issue when both hbase and hdfs sink aâ¦
When both hdfs and hbase sink are in use, during shutdown (KILL SIGTERM),
the hdfs sink won't be able to rename/close the .tmp hdfs file because the
underlying filesystem could be closed earlier when shutting down the other
component.
This change registers a new ShutdownHook in the hadoop's
ShutdownHookManager, which will prevent other hooks to run until Flume stops
itself.
Tested on a cluster which could reproduce the error before the change, but
eliminated after.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mcsanady/flume FLUME-3190
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flume/pull/180.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #180
----
commit f3cff0bcc6112a4067ac606049da7f4bb58c20da
Author: Miklos Csanady <[email protected]>
Date: 2017-11-03T10:21:40Z
FLUME-3190: flume shutdown hook issue when both hbase and hdfs sink are in
use
----
---
