GitHub user tillrohrmann opened a pull request:
https://github.com/apache/flink/pull/2826
[FLINK-5085] Execute CheckpointCoordinator's state discard calls
asynchronously
This PR is a back port of #2825 for the release 1.1 branch. It is based on
#2816. Thus only a70097d is relevant.
The `CheckpointCoordinator` is now given an `Executor` which is used to
execute the state discard
calls asynchronously. This will prevent blocking operations to be executed
from within the
calling thread. The provided `Executor` is the same executor as the one
used for the cleanup in the `ZooKeeperStateHandleStore`.
The executors are now gracefully shutdown after the `JobManager` has
terminated. If the executors don't shut down in the given time (akka ask
timeout), then the executors are shut down hard.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tillrohrmann/flink
backportMakeCheckpointCoordinatorNotBlocking
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2826.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2826
----
commit 357690b359a2890ec1842a20d345675b79d61cd1
Author: Till Rohrmann <[email protected]>
Date: 2016-11-15T21:45:04Z
[FLINK-5073] Use Executor to run ZooKeeper callbacks in
ZooKeeperStateHandleStore
Use dedicated Executor to run ZooKeeper callbacks in
ZooKeeperStateHandleStore instead
of running it in the ZooKeeper client's thread. The callback can be
blocking because it
discards state which might entail deleting files from disk.
Add TestExecutors
commit 640bfef9a176d57fa70d8ac21b8675897fae11ec
Author: Till Rohrmann <[email protected]>
Date: 2016-11-16T17:33:54Z
[FLINK-5082] Pull ExecutorService lifecycle management out of the JobManager
The provided ExecutorService will no longer be closed by the JobManager.
Instead the
lifecycle is managed outside of it where it was created. This will give a
nicer behaviour,
because it better seperates responsibilities.
commit 9de05526e49158a5bde1342afe602f358cae993f
Author: Till Rohrmann <[email protected]>
Date: 2016-11-16T17:51:05Z
Introduce dedicated Executor for blocking io operations
commit a70097d4ac619f9203604f6991d293a7b0f55b54
Author: Till Rohrmann <[email protected]>
Date: 2016-11-17T14:39:11Z
[FLINK-5085] Execute CheckpointCoordinator's state discard calls
asynchronously
The CheckpointCoordinator is now given an Executor which is used to execute
the state discard
calls asynchronously. This will prevent blocking operations to be executed
from within the
calling thread.
Shut down ExecutorServices gracefully
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
