[
https://issues.apache.org/jira/browse/GOBBLIN-1689?focusedWorklogId=803732&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-803732
]
ASF GitHub Bot logged work on GOBBLIN-1689:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 25/Aug/22 18:39
Start Date: 25/Aug/22 18:39
Worklog Time Spent: 10m
Work Description: umustafi commented on code in PR #3544:
URL: https://github.com/apache/gobblin/pull/3544#discussion_r955224404
##########
gobblin-api/src/main/java/org/apache/gobblin/service/ServiceConfigKeys.java:
##########
@@ -26,6 +26,7 @@ public class ServiceConfigKeys {
public static final String GOBBLIN_SERVICE_PREFIX = "gobblin.service.";
public static final String GOBBLIN_SERVICE_JOB_SCHEDULER_LISTENER_CLASS =
"org.apache.gobblin.service.modules.scheduler.GobblinServiceJobScheduler";
+ public static final String GOBBLIN_ORCHESTRATOR_LISTENER_CLASS =
"org.apache.gobblin.service.modules.orchestration.Orchestratotar";
Review Comment:
typo in the class name `Orchestratotar` instead of `Orchestrator`
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/core/GobblinServiceGuiceModule.java:
##########
@@ -135,6 +135,9 @@ public void configure(Binder binder) {
binder.bindConstant()
.annotatedWith(Names.named(InjectionNames.FLOW_CATALOG_LOCAL_COMMIT))
.to(serviceConfig.isFlowCatalogLocalCommit());
+ binder.bindConstant()
Review Comment:
what does binding this config here do?
##########
gobblin-service/src/test/java/org/apache/gobblin/service/modules/orchestration/OrchestratorTest.java:
##########
@@ -152,6 +151,32 @@ private TopologySpec initTopologySpec() {
}
private FlowSpec initFlowSpec() {
+ Properties properties = new Properties();
+ String flowName = "test_flowName";
+ String flowGroup = "test_flowGroup";
+ properties.put(ConfigurationKeys.FLOW_NAME_KEY, flowName);
+ properties.put(ConfigurationKeys.FLOW_GROUP_KEY, flowGroup);
+ properties.put("job.name", flowName);
+ properties.put("job.group", flowGroup);
+ properties.put("specStore.fs.dir", FLOW_SPEC_STORE_DIR);
+ properties.put("specExecInstance.capabilities", "source:destination");
+ properties.put("job.schedule", "0 0 0 ? * * 2050");
+ ;
+ properties.put("gobblin.flow.sourceIdentifier", "source");
+ properties.put("gobblin.flow.destinationIdentifier", "destination");
+ Config config = ConfigUtils.propertiesToConfig(properties);
+
+ FlowSpec.Builder flowSpecBuilder = null;
+ flowSpecBuilder =
FlowSpec.builder(computeTopologySpecURI(SPEC_STORE_PARENT_DIR,
+ FLOW_SPEC_GROUP_DIR))
+ .withConfig(config)
+ .withDescription(SPEC_DESCRIPTION)
+ .withVersion(SPEC_VERSION)
+ .withTemplate(URI.create("templateURI"));
+ return flowSpecBuilder.build();
+ }
+
+ private FlowSpec initBadFlowSpec() {
Properties properties = new Properties();
Review Comment:
can u add an explanation for why this spec does not compile for clarity?
##########
gobblin-service/src/test/java/org/apache/gobblin/service/modules/orchestration/OrchestratorTest.java:
##########
@@ -152,6 +151,32 @@ private TopologySpec initTopologySpec() {
}
private FlowSpec initFlowSpec() {
Review Comment:
where is this function being used? are we asserting somewhere that this flow
does compile?
Issue Time Tracking
-------------------
Worklog Id: (was: 803732)
Time Spent: 0.5h (was: 20m)
> Decouple compiler from scheduler in warm standby mode
> ------------------------------------------------------
>
> Key: GOBBLIN-1689
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1689
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Zihan Li
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Now, when add/update flow spec, we rely on scheduler to compile the flow and
> return status, but in warm standby mode, where every host can accept and
> process request but not running the flow, we should be able to compile flow
> without scheduler.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
