[jira] [Work logged] (GOBBLIN-1930) Improve Logs & Metrics around Multi-active Launch Handling

[ASF GitHub Bot (Jira)]

     [ 
https://issues.apache.org/jira/browse/GOBBLIN-1930?focusedWorklogId=885671&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-885671
 ]

ASF GitHub Bot logged work on GOBBLIN-1930:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 17/Oct/23 13:52
            Start Date: 17/Oct/23 13:52
    Worklog Time Spent: 10m 
      Work Description: phet commented on code in PR #3800:
URL: https://github.com/apache/gobblin/pull/3800#discussion_r1362157216


##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/FlowTriggerHandler.java:
##########
@@ -150,7 +150,8 @@ private boolean 
persistFlowAction(MultiActiveLeaseArbiter.LeaseObtainedStatus le
     if (this.dagActionStore.isPresent() && 
this.multiActiveLeaseArbiter.isPresent()) {
       try {
         DagActionStore.DagAction flowAction = leaseStatus.getFlowAction();
-        this.dagActionStore.get().addDagAction(flowAction.getFlowGroup(), 
flowAction.getFlowName(), flowAction.getFlowExecutionId(), 
flowAction.getFlowActionType());
+        // Replace flow execution id with trigger timestamp to easily track 
the flow
+        this.dagActionStore.get().addDagAction(flowAction.getFlowGroup(), 
flowAction.getFlowName(), String.valueOf(leaseStatus.getEventTimestamp()), 
flowAction.getFlowActionType());

Review Comment:
   seems confusing that 
`LeaseObtainedStatus.getFlowAction().getFlowExecutionId()` returns a different 
value than what we're storing in the `DagAction.getFlowExecutionId()`.  e.g. 
why not unify them by *encapsulating* the switcheroo to 
`LeaseObtainedStatus.getEventTimestamp()` inside the 
`LeaseObtainedStatus.getFlowAction()` call?
   
   then, in that location I suggest a code comment explaining why it's helpful 
and safe to replace the original value w/ this one.





Issue Time Tracking
-------------------

    Worklog Id:     (was: 885671)
    Time Spent: 2h  (was: 1h 50m)

> Improve Logs & Metrics around Multi-active Launch Handling
> ----------------------------------------------------------
>
>                 Key: GOBBLIN-1930
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1930
>             Project: Apache Gobblin
>          Issue Type: Improvement
>          Components: gobblin-service
>            Reporter: Urmi Mustafi
>            Assignee: Abhishek Tiwari
>            Priority: Major
>          Time Spent: 2h
>  Remaining Estimate: 0h
>
> Improve logging and metrics around multi-active launch flow event handling to 
> identify any missing events between the {{MysqlMultiActiveLeaseArbiter}} 
> committing the launch event to the {{dagActionStore}} and the 
> {{DagActionMonitor}} receiving events for processing. We want to be able to 
> distinguish between the following cases of 
>  * events that are never received by the {{DagActionMonitor}}
>  * events incorrectly filtered out by the {{DagActionMonitor}}
>  * any failed submissions of dags to the {{DagManager}} either upon leader 
> change or from the {{DagActionChangeMonitor}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)