[
https://issues.apache.org/jira/browse/GOBBLIN-2204?focusedWorklogId=970746&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-970746
]
ASF GitHub Bot logged work on GOBBLIN-2204:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 27/May/25 07:25
Start Date: 27/May/25 07:25
Worklog Time Spent: 10m
Work Description: vsinghal85 commented on code in PR #4113:
URL: https://github.com/apache/gobblin/pull/4113#discussion_r2108403326
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/SafeDatasetCommit.java:
##########
@@ -90,6 +90,7 @@ public Void call()
metricContext = Instrumented.getMetricContext(datasetState,
SafeDatasetCommit.class);
finalizeDatasetStateBeforeCommit(this.datasetState);
+
this.datasetState.computeAndStoreQualityStatus(this.jobContext.getJobState());
Class<? extends DataPublisher> dataPublisherClass;
Review Comment:
Work unit is at individual task level, and if individual task data quality
fails, it does fail that task as well. Here in this method specifically we are
computing overall data quality of the dataset, based on data quality of all
individual tasks.
Issue Time Tracking
-------------------
Worklog Id: (was: 970746)
Remaining Estimate: 0h
Time Spent: 10m
> FileSize Data Quality implementation for FileBasedCopy
> ------------------------------------------------------
>
> Key: GOBBLIN-2204
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2204
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Vaibhav Singhal
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
