[
https://issues.apache.org/jira/browse/GOBBLIN-1689?focusedWorklogId=805331&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-805331
]
ASF GitHub Bot logged work on GOBBLIN-1689:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 31/Aug/22 17:56
Start Date: 31/Aug/22 17:56
Worklog Time Spent: 10m
Work Description: ZihanLi58 commented on code in PR #3544:
URL: https://github.com/apache/gobblin/pull/3544#discussion_r959868355
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/spec_catalog/FlowCatalog.java:
##########
@@ -383,10 +387,17 @@ public Map<String, AddSpecResponse> put(Spec spec,
boolean triggerListener) thro
return responseMap;
}
}
- AddSpecResponse<String> schedulerResponse =
responseMap.getOrDefault(ServiceConfigKeys.GOBBLIN_SERVICE_JOB_SCHEDULER_LISTENER_CLASS,
new AddSpecResponse<>(null));
+ AddSpecResponse<String> compileResponse;
+ if (isWarmStandbyEnabled) {
+ compileResponse =
responseMap.getOrDefault(ServiceConfigKeys.GOBBLIN_ORCHESTRATOR_LISTENER_CLASS,
new AddSpecResponse<>(null));
+ //todo: do we check quota here? or in compiler? Quota manager need dag
to check quota which is not accessable from this class
Review Comment:
When I say check quota, it's only for adhoc and run immediate flow. Since
after this step, we put the change in mysql table and return accept to user
which means we will run the flow anyway on leader host. So before we put it
into mysql table, for achoc and run immediate flow, we need to check the quota
first, either here or in compiler
Issue Time Tracking
-------------------
Worklog Id: (was: 805331)
Time Spent: 3h (was: 2h 50m)
> Decouple compiler from scheduler in warm standby mode
> ------------------------------------------------------
>
> Key: GOBBLIN-1689
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1689
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Zihan Li
> Priority: Major
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Now, when add/update flow spec, we rely on scheduler to compile the flow and
> return status, but in warm standby mode, where every host can accept and
> process request but not running the flow, we should be able to compile flow
> without scheduler.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
