[jira] [Work logged] (GOBBLIN-1921) Properly handle reminder events

[ASF GitHub Bot (Jira)]

     [ 
https://issues.apache.org/jira/browse/GOBBLIN-1921?focusedWorklogId=883191&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-883191
 ]

ASF GitHub Bot logged work on GOBBLIN-1921:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 03/Oct/23 18:10
            Start Date: 03/Oct/23 18:10
    Worklog Time Spent: 10m 
      Work Description: umustafi commented on code in PR #3790:
URL: https://github.com/apache/gobblin/pull/3790#discussion_r1344526938


##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/api/MysqlMultiActiveLeaseArbiter.java:
##########
@@ -287,19 +290,19 @@ else if (leaseValidityStatus == 2) {
         // Use our event to acquire lease, check for previous db 
eventTimestamp and leaseAcquisitionTimestamp
         int numRowsUpdated = 
attemptLeaseIfExistingRow(thisTableAcquireLeaseIfMatchingAllStatement, 
flowAction,
             true,true, dbEventTimestamp, dbLeaseAcquisitionTimestamp);
-        return evaluateStatusAfterLeaseAttempt(numRowsUpdated, flowAction, 
Optional.of(dbCurrentTimestamp));
+        return evaluateStatusAfterLeaseAttempt(numRowsUpdated, flowAction, 
Optional.of(dbCurrentTimestamp), isReminderEvent);
       } // No longer leasing this event
         if (isWithinEpsilon) {
-          log.debug("tryAcquireLease for [{}, eventTimestamp: {}] - CASE 5: 
Same event, no longer leasing event in db: "
-              + "terminate", flowAction, dbCurrentTimestamp.getTime());
+          log.debug("tryAcquireLease for [{}, is: {}, eventTimestamp: {}] - 
CASE 5: Same event, no longer leasing event"
+              + " in db", flowAction, isReminderEvent ? "reminder" : 
"original", dbCurrentTimestamp.getTime());

Review Comment:
   Yes I intentionally removed the word bc i noticed the other log statements 
only describe the case not the action that follows





Issue Time Tracking
-------------------

    Worklog Id:     (was: 883191)
    Time Spent: 3h  (was: 2h 50m)

> Properly handle reminder events
> -------------------------------
>
>                 Key: GOBBLIN-1921
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1921
>             Project: Apache Gobblin
>          Issue Type: Bug
>          Components: gobblin-service
>            Reporter: Urmi Mustafi
>            Assignee: Abhishek Tiwari
>            Priority: Major
>          Time Spent: 3h
>  Remaining Estimate: 0h
>
> Reminder flow trigger events were being improperly handled and interpreted as 
> new events because they are triggered {{linger}} time after the original 
> trigger where {{epsilon < linger}} and we use {{epsilon}} to determine event 
> distinctness. With reminder events being considered distinct events, we were 
> launching excess concurrent flows that were then being cancelled. Now we 
> handle reminder events differently from normal event triggers to ensure 
> they're properly evaluated. Because of db laundering, reminder events are 
> easy to handle - if they're older than the currently worked upon event in the 
> database they can be skipped and if they're equal to the current event in the 
> database they are handled like normal. Reminder events should never be newer 
> than the current event in the lease arbiter table because db laundering 
> always results in increasing event times. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)