[
https://issues.apache.org/jira/browse/GOBBLIN-1552?focusedWorklogId=676674&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-676674
]
ASF GitHub Bot logged work on GOBBLIN-1552:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 04/Nov/21 19:38
Start Date: 04/Nov/21 19:38
Worklog Time Spent: 10m
Work Description: arjun4084346 commented on a change in pull request
#3403:
URL: https://github.com/apache/gobblin/pull/3403#discussion_r743147800
##########
File path:
gobblin-runtime/src/main/java/org/apache/gobblin/service/monitoring/JobStatusRetriever.java
##########
@@ -218,4 +228,42 @@ public static boolean
isFlowStatus(org.apache.gobblin.service.monitoring.JobStat
return jobStatus.getJobName() != null && jobStatus.getJobGroup() != null
&& jobStatus.getJobName().equals(JobStatusRetriever.NA_KEY) &&
jobStatus.getJobGroup().equals(JobStatusRetriever.NA_KEY);
}
+
+ public static ExecutionStatus getFlowStatusFromJobStatuses(boolean
dagManagerEnabled, Iterator<JobStatus> jobStatusIterator) {
+ ExecutionStatus flowExecutionStatus = ExecutionStatus.$UNKNOWN;
+
+ if (dagManagerEnabled) {
+ while (jobStatusIterator.hasNext()) {
+ JobStatus jobStatus = jobStatusIterator.next();
+ // Check if this is the flow status instead of a single job status
+ if (JobStatusRetriever.isFlowStatus(jobStatus)) {
+ flowExecutionStatus =
ExecutionStatus.valueOf(jobStatus.getEventName());
+ }
+ }
+ } else {
+ Set<ExecutionStatus> jobStatuses = new HashSet<>();
+ while (jobStatusIterator.hasNext()) {
+ JobStatus jobStatus = jobStatusIterator.next();
+ // because in absence of DagManager we do not get all flow level
events, we will ignore the flow level events
+ // we actually get and purely calculate flow status based on flow
statuses.
+ if (!JobStatusRetriever.isFlowStatus(jobStatus)) {
+ jobStatuses.add(ExecutionStatus.valueOf(jobStatus.getEventName()));
+ }
+ }
+
+ if (jobStatuses.contains(ExecutionStatus.FAILED)) {
+ flowExecutionStatus = ExecutionStatus.FAILED;
+ } else if (jobStatuses.contains(ExecutionStatus.CANCELLED)) {
+ flowExecutionStatus = ExecutionStatus.CANCELLED;
+ } else if (jobStatuses.contains(ExecutionStatus.ORCHESTRATED)) {
+ flowExecutionStatus = ExecutionStatus.ORCHESTRATED;
+ } else if (jobStatuses.contains(ExecutionStatus.RUNNING)) {
+ flowExecutionStatus = ExecutionStatus.RUNNING;
+ } else if (jobStatuses.contains(ExecutionStatus.COMPLETE)) {
+ flowExecutionStatus = ExecutionStatus.COMPLETE;
Review comment:
Yes, `KafkaJobStatusMonitor` should update the job status
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 676674)
Time Spent: 1h 40m (was: 1.5h)
> fix flow status reporting when dag manager is not enabled
> ---------------------------------------------------------
>
> Key: GOBBLIN-1552
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1552
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: Arjun Singh Bora
> Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> flow status is determined by looking at the flow level events. but flow level
> events are not emitted outside of dag manager. so currently flow status is
> not being determined correctly when dag manager is disabled
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
