keith-turner commented on PR #4403:
URL: https://github.com/apache/accumulo/pull/4403#issuecomment-2011009945
Seems like there is an existing catch-22 in the code that we may want to
fix. The existing problem is that if a compactor has never checked in for a
group then there will be no entry in TIME_COMPACTOR_LAST_CHECKED and no warning
will ever be logged. So then groups that have jobs queued and are never being
serviced will never get a warning logged.
Wondering if we can fix the current code and do the needed cleanup of
`TIME_COMPACTOR_LAST_CHECKED` in the existing `idleCompactionWarning()` method.
Was playing around with the following trying to work through this, it calls
functions that do no currently exist in the code. Was just trying to work out
the concept.
```java
private void idleCompactionWarning() {
// The set of groups that currently have >0 jobs queued
Set<CompactorGroupId> queuedGroups = jobQueues.getCurrentActiveGroups();
// The set of groups that have >0 live compactors w/ a ZK lock. These
are groups that have running compactors.
Set<CompactorGroupId> liveGroups = getLiveGroups();
long now = System.currentTimeMillis();
for(var groupName : queuedGroups) {
// If the group has running compactors then do not want to log a
warning. Looking for groups that have jobs queued, no compactors running, and
the situation has been like this for a while.
if(!liveGroups.contains(groupName)) {
long lastCheckTime =
TIME_COMPACTOR_LAST_CHECKED.getOrDefault(groupName, 0);
if ((now - lastCheckTime) > getMissingCompactorWarningTime()){
LOG.warn("No compactors have checked in with coordinator for group
{} in {}ms", groupName,
getMissingCompactorWarningTime());
}
}
}
// Only care about tracking check in times for groups with compactions
currently queued
TIME_COMPACTOR_LAST_CHECKED.keySet().retainAll(queuedGroups);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
