C0urante commented on code in PR #12802:
URL: https://github.com/apache/kafka/pull/12802#discussion_r1039787775
##########
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java:
##########
@@ -1645,6 +1646,8 @@ private void startAndStop(Collection<Callable<Void>>
callables) {
startAndStopExecutor.invokeAll(callables);
} catch (InterruptedException e) {
// ignore
+ } catch (RejectedExecutionException re) {
+ log.error("startAndStopExecutor already shutdown or full. Not
invoking explicit connector/task shutdown");
Review Comment:
Thanks for the clarification @vamossagar12. I hadn't accounted for the
"unclean" shutdown logic that takes place in the
[DistributedHerder](https://github.com/apache/kafka/blob/2d6357f6fcb286f3db24e2b3149a6f4bbc1f832f/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L770),
[AbstractHerder](https://github.com/apache/kafka/blob/2d6357f6fcb286f3db24e2b3149a6f4bbc1f832f/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractHerder.java#L152),
and
[Worker](https://github.com/apache/kafka/blob/2d6357f6fcb286f3db24e2b3149a6f4bbc1f832f/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L224-L232)
classes.
Considering the improvements we've made to the herder's tick thread since
this shutdown logic was last altered (specifically with regards to not allowing
it to get blocked indefinitely), I wonder if it's necessary to have such a low
timeout for `herderExecutor::awaitTermination`. If we give the herder tick
thread long enough to terminate gracefully, then we never have to worry about
the `startAndStopExecutor` being shut down at all.
I'm worried that adding this kind of `try`/`catch` guard to
`DistributedHerder::startAndStop` without any additional checks makes it
possible to hide other bugs that may come up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
