Blazer-007 commented on code in PR #4127:
URL: https://github.com/apache/gobblin/pull/4127#discussion_r2262042630
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/SafeDatasetCommit.java:
##########
@@ -94,13 +94,17 @@ public Void call()
metricContext = Instrumented.getMetricContext(datasetState,
SafeDatasetCommit.class);
finalizeDatasetStateBeforeCommit(this.datasetState);
- // evaluate data quality at the dataset commit level, only when commit
source is CommitActivityImpl
- if
(SafeDatasetCommit.COMMIT_SRC_COMMIT_ACTIVITY_IMPL.equals(this.datasetCommitSrc))
{
+ // evaluate data quality at the dataset commit level, only when commit
source is CommitActivityImpl and policies are applied
+ JobState jobState = this.jobContext.getJobState();
+ String policiesApplied =
jobState.getProperties().getProperty(ConfigurationKeys.TASK_LEVEL_POLICY_LIST,
StringUtils.EMPTY);
Review Comment:
https://github.com/apache/gobblin/blob/master/gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/util/JobStateUtils.java#L126
check this i think that function will take care of getting properties
automatically
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
