[
https://issues.apache.org/jira/browse/GOBBLIN-808?focusedWorklogId=275558&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-275558
]
ASF GitHub Bot logged work on GOBBLIN-808:
------------------------------------------
Author: ASF GitHub Bot
Created on: 12/Jul/19 00:04
Start Date: 12/Jul/19 00:04
Worklog Time Spent: 10m
Work Description: autumnust commented on pull request #2674:
[GOBBLIN-808] implement azkaban flow cancel when dag manager is enabled
URL: https://github.com/apache/incubator-gobblin/pull/2674#discussion_r302785048
##########
File path:
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/DagManager.java
##########
@@ -187,12 +230,37 @@ protected void startUp() {
synchronized void offer(Dag<JobExecutionPlan> dag) throws IOException {
//Persist the dag
this.dagStateStore.writeCheckpoint(dag);
- //Add it to the queue of dags
- if (!this.queue.offer(dag)) {
+ long flowExecutionId = DagManagerUtils.getFlowExecId(dag);
+ int queueId = (int) (flowExecutionId % this.numThreads);
+ // Add the dag to the specific queue determined by flowExecutionId
+ // Flow cancellation request has to be forwarded to the same
DagManagerThread where the
+ // flow create request was forwarded. This is because Azkaban Exec Id is
stored in the DagNode of the
+ // specific DagManagerThread queue
+ if (!this.queue[queueId].offer(dag)) {
throw new IOException("Could not add dag" +
DagManagerUtils.generateDagId(dag) + "to queue");
}
}
+ /**
+ * Method to submit a {@link URI} for cancellation requsts to the {@link
DagManager}.
+ * The {@link DagManager} adds the dag to the {@link BlockingQueue} to be
picked up by one of the {@link DagManagerThread}s.
+ */
+ synchronized public void offer(URI uri) throws IOException {
Review comment:
Can we rename this method to differentiate with `offer` method above?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 275558)
Time Spent: 7h (was: 6h 50m)
> implement azkaban job cancellation in Gobblin Service
> -----------------------------------------------------
>
> Key: GOBBLIN-808
> URL: https://issues.apache.org/jira/browse/GOBBLIN-808
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Arjun Singh Bora
> Priority: Major
> Time Spent: 7h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
