Viktor Csomor created HIVE-25914:
------------------------------------
Summary: Cleaner updates Initiator cycle metric
Key: HIVE-25914
URL: https://issues.apache.org/jira/browse/HIVE-25914
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 4.0.0
Reporter: Viktor Csomor
Assignee: Viktor Csomor
The count of the below metrics counted incorrectly upon an exception.
- {{compaction_initator_failure_counter}}
- {{compaction_cleaner_failure_counter}}
Reasoning:
In the {{Initator}}/{{Cleaner}} class creates a list of {{CompletableFuture}}
which {{Runnable}} core exception is being wrapped to {{RuntimeExceptions}}.
The below code-snippet waits all cleaners to complete (Initiators does it
similarly).
{code:java}
try {
....
for (CompactionInfo compactionInfo : readyToClean) {
cleanerList.add(CompletableFuture.runAsync(CompactorUtil.ThrowingRunnable.unchecked(()
->
clean(compactionInfo, cleanerWaterMark, metricsEnabled)),
cleanerExecutor));
}
CompletableFuture.allOf(cleanerList.toArray(new
CompletableFuture[0])).join();
}
} catch (Throwable t) {
// the lock timeout on AUX lock, should be ignored.
if (metricsEnabled && handle != null) {
failuresCounter.inc();
}
{code}
If the {{CompleteableFututre#join}} throws an Exception then the failure
counter is incremented.
Docs:
{code}
/**
* Returns the result value when complete, or throws an
* (unchecked) exception if completed exceptionally. To better
* conform with the use of common functional forms, if a
* computation involved in the completion of this
* CompletableFuture threw an exception, this method throws an
* (unchecked) {@link CompletionException} with the underlying
* exception as its cause.
*
* @return the result value
* @throws CancellationException if the computation was cancelled
* @throws CompletionException if this future completed
* exceptionally or a completion computation threw an exception
*/
public T join() {
Object r;
return reportJoin((r = result) == null ? waitingGet(false) : r);
}
{code}
(!) Let's suppose we have 10 cleaners and the 2nd throws an exception. The
{{catch}} block will be initiated and the {{failuresCounter}} will be
incremented. If there is any consecutive error amongst the remaining cleaners
the counter won't be incremented.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
