phet commented on code in PR #3899:
URL: https://github.com/apache/gobblin/pull/3899#discussion_r1540627484
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/DagManagementTaskStreamImpl.java:
##########
@@ -108,6 +137,36 @@ public DagTask next() {
return null;
}
+ /**
+ * Returns a {@link
org.apache.gobblin.runtime.api.MultiActiveLeaseArbiter.LeaseAttemptStatus}
associated with the
+ * `dagAction`. If in multi-active execution mode, it retrieves the status
from calling
+ * {@link MultiActiveLeaseArbiter#tryAcquireLease(DagActionStore.DagAction,
long, boolean, boolean)}, otherwise
+ * it returns a {@link
org.apache.gobblin.runtime.api.MultiActiveLeaseArbiter.LeaseObtainedStatus}
that will not
+ * expire for a very long time to the current instance.
+ * @param dagAction
+ * @return
+ * @throws IOException
+ * @throws SchedulerException
+ */
+ private MultiActiveLeaseArbiter.LeaseAttemptStatus
retrieveLeaseStatus(DagActionStore.DagAction dagAction)
+ throws IOException, SchedulerException {
+ MultiActiveLeaseArbiter.LeaseAttemptStatus leaseAttemptStatus;
+ if (!this.isMultiActiveExecutionEnabled) {
+ leaseAttemptStatus = new
MultiActiveLeaseArbiter.LeaseObtainedStatus(dagAction,
System.currentTimeMillis(), Long.MAX_VALUE, null);
+ } else {
Review Comment:
earlier I suggested to have a "always-return-lease-obtained" MALA, but that
won't quite work, because we still need reminders. when the processing attempt
fails that deserves retry (by the single-active host).
that line of reasoning might suggest an in-memory MALA that maintains each
status based on `LeaseObtainedStatus.conclude()` having been called yet or not.
when not yet it returns `LeaseObtainedStatus`, but after it has, then it
returns `NoLongerLeasing`.
but taking one futher step back... why give up durability when running in
single-active execution mode? single-active should still support fail-over!
thus, any reason not to use a bona fide `MysqlMALA` and merely know there would
be no contention in the DB, such that `LeasedToAnotherStatus` is never returned?
regardless of our choice, don't we still need to clean up `DagAction`s in
the store? would using a genuine MALA help there too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
