[
https://issues.apache.org/jira/browse/GOBBLIN-2204?focusedWorklogId=977542&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-977542
]
ASF GitHub Bot logged work on GOBBLIN-2204:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 04/Aug/25 16:14
Start Date: 04/Aug/25 16:14
Worklog Time Spent: 10m
Work Description: khandelwal-prateek commented on code in PR #4113:
URL: https://github.com/apache/gobblin/pull/4113#discussion_r2251957581
##########
gobblin-core/src/main/java/org/apache/gobblin/qualitychecker/task/TaskLevelPolicyChecker.java:
##########
@@ -28,20 +29,13 @@
* executes each one, and then stores the output
* in a PolicyCheckResults object
*/
+@Getter
public class TaskLevelPolicyChecker {
- /**
- * An enumeration for possible statuses for Data quality checks,
- * its values will be PASSED, FAILED, in case if data quality check
- * evaluation is not performed for Job, it will be NOT_EVALUATED
- */
- public enum DataQualityStatus {
- PASSED,
- FAILED,
- NOT_EVALUATED
- }
private final List<TaskLevelPolicy> list;
private static final Logger LOG =
LoggerFactory.getLogger(TaskLevelPolicyChecker.class);
+ public static final String TASK_LEVEL_POLICY_RESULT_KEY =
"gobblin.task.level.policy.result";
Review Comment:
As discussed, the code supports marking policies as OPTIONAL. In
`jdbc-to-hdfs.template`, the policies are already defined with
`qualitychecker.task.policy.types=OPTIONAL`. Because policy results are
currently maintained in a map that stores only a single value for each failure
type, this would suppress data quality failures in such cases
Issue Time Tracking
-------------------
Worklog Id: (was: 977542)
Time Spent: 2h 20m (was: 2h 10m)
> FileSize Data Quality implementation for FileBasedCopy
> ------------------------------------------------------
>
> Key: GOBBLIN-2204
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2204
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Vaibhav Singhal
> Priority: Major
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
