vsinghal85 commented on code in PR #4127:
URL: https://github.com/apache/gobblin/pull/4127#discussion_r2262036446
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/SafeDatasetCommit.java:
##########
@@ -94,13 +94,17 @@ public Void call()
metricContext = Instrumented.getMetricContext(datasetState,
SafeDatasetCommit.class);
finalizeDatasetStateBeforeCommit(this.datasetState);
- // evaluate data quality at the dataset commit level, only when commit
source is CommitActivityImpl
- if
(SafeDatasetCommit.COMMIT_SRC_COMMIT_ACTIVITY_IMPL.equals(this.datasetCommitSrc))
{
+ // evaluate data quality at the dataset commit level, only when commit
source is CommitActivityImpl and policies are applied
+ JobState jobState = this.jobContext.getJobState();
+ String policiesApplied =
jobState.getProperties().getProperty(ConfigurationKeys.TASK_LEVEL_POLICY_LIST,
StringUtils.EMPTY);
Review Comment:
jobState directly does not have this field, it's inside properties in job
state, something like
```
{
"jobname":
...
"properties": {
"quality checker.task.policies": ....
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
