[
https://issues.apache.org/jira/browse/GOBBLIN-1689?focusedWorklogId=805023&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-805023
]
ASF GitHub Bot logged work on GOBBLIN-1689:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 30/Aug/22 21:54
Start Date: 30/Aug/22 21:54
Worklog Time Spent: 10m
Work Description: arjun4084346 commented on code in PR #3544:
URL: https://github.com/apache/gobblin/pull/3544#discussion_r958961737
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/spec_catalog/FlowCatalog.java:
##########
@@ -383,10 +387,17 @@ public Map<String, AddSpecResponse> put(Spec spec,
boolean triggerListener) thro
return responseMap;
}
}
- AddSpecResponse<String> schedulerResponse =
responseMap.getOrDefault(ServiceConfigKeys.GOBBLIN_SERVICE_JOB_SCHEDULER_LISTENER_CLASS,
new AddSpecResponse<>(null));
+ AddSpecResponse<String> compileResponse;
+ if (isWarmStandbyEnabled) {
+ compileResponse =
responseMap.getOrDefault(ServiceConfigKeys.GOBBLIN_ORCHESTRATOR_LISTENER_CLASS,
new AddSpecResponse<>(null));
+ //todo: do we check quota here? or in compiler? Quota manager need dag
to check quota which is not accessable from this class
Review Comment:
This compilation check is only to figure out if the flow should be persisted
in the flow catalog or not, right? In that case, quota check is not required,
correct?
Issue Time Tracking
-------------------
Worklog Id: (was: 805023)
Time Spent: 1h (was: 50m)
> Decouple compiler from scheduler in warm standby mode
> ------------------------------------------------------
>
> Key: GOBBLIN-1689
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1689
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Zihan Li
> Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Now, when add/update flow spec, we rely on scheduler to compile the flow and
> return status, but in warm standby mode, where every host can accept and
> process request but not running the flow, we should be able to compile flow
> without scheduler.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
