kfaraz commented on code in PR #18851:
URL: https://github.com/apache/druid/pull/18851#discussion_r2644738409
##########
indexing-service/src/main/java/org/apache/druid/indexing/overlord/hrtr/HttpRemoteTaskRunner.java:
##########
@@ -564,20 +498,27 @@ public void nodeViewInitializedTimedOut()
log.info("Waiting for worker discovery...");
}
}
- log.info("[%s] Workers are discovered.", workers.size());
+ log.info("Discovered [%d] workers.", workers.size());
- // Wait till all worker state is sync'd so that we know which worker is
running/completed what tasks or else
+ // Wait till all worker state is synced so that we know which worker is
running/completed what tasks or else
// We would start assigning tasks which are pretty soon going to be
reported by discovered workers.
- for (WorkerHolder worker : workers.values()) {
- log.info("Waiting for worker[%s] to sync state...",
worker.getWorker().getHost());
- worker.waitForInitialization();
- }
- log.info("Workers have sync'd state successfully.");
+ workers.forEach((workerHost, workerEntry) -> {
+ try {
+ workerEntry.waitForInitialization();
+ }
+ catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ }
+ });
+ log.info("Workers have synced state successfully.");
Review Comment:
Oh, I see.
Can that ever cause a deadlock?
Say a worker is yet to be initialized and a thread is waiting on
`waitForInitialization()` while holding the lock for the corresponding key in
the map. Will any other thread be able to mark this worker as "initialized"
without acquiring that lock?
> I found this to be a better model that looping through a keySet, and then
indexing with those (potentially stale) keys.
Hmm, I was hoping we use `.keySet()` followed by `.compute()` itself and
move the `.compute()` part into a separate method (similar to how `TaskQueue`
does it). We could use that method wherever we add/update the entry for a
worker so that we always perform actions using a `compute()` ensuring thread
safety.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
