vsinghal85 commented on code in PR #4135:
URL: https://github.com/apache/gobblin/pull/4135#discussion_r2296999970
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/Task.java:
##########
@@ -329,17 +331,37 @@ private void computeAndUpdateTaskDataQuality() {
try {
if (result != null) {
forkDataQualityStatus = DataQualityStatus.valueOf(result);
+ } else {
+ forkDataQualityStatus = DataQualityStatus.NOT_EVALUATED;
}
} catch (IllegalArgumentException e) {
Log.warn("Unknown data quality status encountered: " + result);
forkDataQualityStatus = DataQualityStatus.UNKNOWN;
}
- if (DataQualityStatus.FAILED == forkDataQualityStatus) {
- overallTaskDataQuality = DataQualityStatus.FAILED;
+ /*
+ * If any fork fails, overall status should be FAILED
+ * FAILED status cannot be overridden by subsequent successes
+ * Handle NOT_EVALUATED and UNKNOWN appropriately
+ * If forkDataQualityStatus is PASSED and overall is not
FAILED/UNKNOWN, keep PASSED
+ */
Review Comment:
this is implicit, if forkDataQuality is passed, overall will remain PASSED
until existing overall status was FAILED/UNKNOWN. If once overall status is set
to FAILED/UNKNOWN, it cannot be later set to PASSED.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
