Github user agresch commented on a diff in the pull request:
https://github.com/apache/storm/pull/2918#discussion_r239461731
--- Diff:
storm-server/src/main/java/org/apache/storm/scheduler/blacklist/strategies/RasBlacklistStrategy.java
---
@@ -79,25 +81,46 @@
if (shortage.areAnyOverZero() || shortageSlots > 0) {
LOG.info("Need {} and {} slots more. Releasing some
blacklisted nodes to cover it.", shortage, shortageSlots);
- //release earliest blacklist
- for (String supervisor : blacklistedNodeIds) {
- SupervisorDetails sd =
availableSupervisors.get(supervisor);
- if (sd != null) {
- NormalizedResourcesWithMemory sdAvailable =
cluster.getAvailableResources(sd);
- int sdAvailableSlots =
cluster.getAvailablePorts(sd).size();
- readyToRemove.add(supervisor);
- shortage.remove(sdAvailable,
cluster.getResourceMetrics());
- shortageSlots -= sdAvailableSlots;
- LOG.debug("Releasing {} with {} and {} slots
leaving {} and {} slots to go", supervisor,
- sdAvailable, sdAvailableSlots, shortage,
shortageSlots);
- if (!shortage.areAnyOverZero() && shortageSlots <=
0) {
- // we have enough resources now...
- break;
+
+ //release earliest blacklist - but release all supervisors
on a given blacklisted host.
+ Map<String, Set<String>> hostToSupervisorIds =
createHostToSupervisorMap(blacklistedNodeIds, cluster);
+ for (Set<String> supervisorIds :
hostToSupervisorIds.values()) {
+ for (String supervisorId : supervisorIds) {
+ SupervisorDetails sd =
availableSupervisors.get(supervisorId);
+ if (sd != null) {
+ NormalizedResourcesWithMemory sdAvailable =
cluster.getAvailableResources(sd);
+ int sdAvailableSlots =
cluster.getAvailablePorts(sd).size();
+ readyToRemove.add(supervisorId);
+ shortage.remove(sdAvailable,
cluster.getResourceMetrics());
+ shortageSlots -= sdAvailableSlots;
+ LOG.info("Releasing {} with {} and {} slots
leaving {} and {} slots to go", supervisorId,
+ sdAvailable, sdAvailableSlots,
shortage, shortageSlots);
--- End diff --
It sounds like you don't agree even with the existing code that releases
blacklisted supervisors when needed for scheduling?
Should I be looking at converting all the existing code to blacklist
supervisors instead of hosts to get things functional when there are more than
one supervisor on a host?
Thanks.
---
