[
https://issues.apache.org/jira/browse/GOBBLIN-1930?focusedWorklogId=885740&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-885740
]
ASF GitHub Bot logged work on GOBBLIN-1930:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 17/Oct/23 19:54
Start Date: 17/Oct/23 19:54
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3800:
URL: https://github.com/apache/gobblin/pull/3800#discussion_r1362682486
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/FlowTriggerHandler.java:
##########
@@ -117,13 +117,15 @@ public void handleTriggerEvent(Properties jobProps,
DagActionStore.DagAction flo
this.leaseObtainedCount.inc();
if (persistFlowAction(leaseObtainedStatus)) {
log.info("Successfully persisted lease: [{}, eventTimestamp: {}] ",
leaseObtainedStatus.getFlowAction(),
- leaseObtainedStatus.getEventTimestamp());
+ leaseObtainedStatus.getEventTimeMillis());
return;
}
// If persisting the flow action failed, then we set another trigger
for this event to occur immediately to
// re-attempt handling the event
+ DagActionStore.DagAction updatedFlowAction =
DagActionStore.DagAction.updateFlowExecutionId(flowAction,
+ leaseObtainedStatus.getEventTimeMillis());
scheduleReminderForEvent(jobProps,
- new MultiActiveLeaseArbiter.LeasedToAnotherStatus(flowAction,
leaseObtainedStatus.getEventTimestamp(), 0L),
+ new
MultiActiveLeaseArbiter.LeasedToAnotherStatus(updatedFlowAction, 0L),
Review Comment:
Do we still need this if we are using the event timetime millisecond, which
should be synchronized across all hosts?
Issue Time Tracking
-------------------
Worklog Id: (was: 885740)
Time Spent: 2.5h (was: 2h 20m)
> Improve Logs & Metrics around Multi-active Launch Handling
> ----------------------------------------------------------
>
> Key: GOBBLIN-1930
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1930
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Improve logging and metrics around multi-active launch flow event handling to
> identify any missing events between the {{MysqlMultiActiveLeaseArbiter}}
> committing the launch event to the {{dagActionStore}} and the
> {{DagActionMonitor}} receiving events for processing. We want to be able to
> distinguish between the following cases ofÂ
> * events that are never received by the {{DagActionMonitor}}
> * events incorrectly filtered out by the {{DagActionMonitor}}
> * any failed submissions of dags to the {{DagManager}} either upon leader
> change or from the {{DagActionChangeMonitor}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
