[
https://issues.apache.org/jira/browse/GOBBLIN-1689?focusedWorklogId=805043&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-805043
]
ASF GitHub Bot logged work on GOBBLIN-1689:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 30/Aug/22 23:45
Start Date: 30/Aug/22 23:45
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3544:
URL: https://github.com/apache/gobblin/pull/3544#discussion_r959018143
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/spec_catalog/FlowCatalog.java:
##########
@@ -383,10 +387,17 @@ public Map<String, AddSpecResponse> put(Spec spec,
boolean triggerListener) thro
return responseMap;
}
}
- AddSpecResponse<String> schedulerResponse =
responseMap.getOrDefault(ServiceConfigKeys.GOBBLIN_SERVICE_JOB_SCHEDULER_LISTENER_CLASS,
new AddSpecResponse<>(null));
+ AddSpecResponse<String> compileResponse;
+ if (isWarmStandbyEnabled) {
+ compileResponse =
responseMap.getOrDefault(ServiceConfigKeys.GOBBLIN_ORCHESTRATOR_LISTENER_CLASS,
new AddSpecResponse<>(null));
+ //todo: do we check quota here? or in compiler? Quota manager need dag
to check quota which is not accessable from this class
Review Comment:
Quota check should only be enabled for adhoc flows or flows who are set to
run immediately, otherwise it should be safe to store. We can't block users
from adding flows to be scheduled in the future even if their current quota is
overrun.
Issue Time Tracking
-------------------
Worklog Id: (was: 805043)
Time Spent: 1h 50m (was: 1h 40m)
> Decouple compiler from scheduler in warm standby mode
> ------------------------------------------------------
>
> Key: GOBBLIN-1689
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1689
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Zihan Li
> Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Now, when add/update flow spec, we rely on scheduler to compile the flow and
> return status, but in warm standby mode, where every host can accept and
> process request but not running the flow, we should be able to compile flow
> without scheduler.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
