[
https://issues.apache.org/jira/browse/GOBBLIN-2107?focusedWorklogId=924925&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-924925
]
ASF GitHub Bot logged work on GOBBLIN-2107:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 09/Jul/24 00:29
Start Date: 09/Jul/24 00:29
Worklog Time Spent: 10m
Work Description: umustafi opened a new pull request, #3996:
URL: https://github.com/apache/gobblin/pull/3996
* if job compilation fails then ensure quota is released and flow
compilation event sent in all cases
* fix bug where adhoc flows are not deleted in multi-active scheduler
case by re-adding deletion to DagManager.addDag()
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [X] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-2107
### Description
- [X] Here are some details about my PR, including screenshots (if
applicable):
Delete adhoc `flowSpecs` from `flowCatalog` to avoid build up of adhoc
flowSpecs in catalog even when job compilation fails by adding the deletion in
a finally block in non-MA scheduler case.
1. Previous PR: https://github.com/apache/gobblin/pull/3944/files causes a
bug where adhoc flows are not deleted when multi-active scheduler is enabled.
This PR re-adds deletion of `flowSpec` to the `dagManager` to be called by the
active host for the MA scheduler case (there will only be one active host if
`dagManager` is enabled otherwise code will use `dagProcessingEngine`).
2. It also ensures quota is released and failed flow compilation event sent
by the `FlowCompilationValidationHelper` in all compilation failure cases. This
was being incorrectly handled before.
### Tests
- [X] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [X] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
3. Subject is limited to 50 characters
4. Subject does not end with a period
5. Subject uses the imperative mood ("add", not "adding")
6. Body wraps at 72 characters
7. Body explains "what" and "why", not "how"
Issue Time Tracking
-------------------
Worklog Id: (was: 924925)
Remaining Estimate: 0h
Time Spent: 10m
> Delete adhoc flowSpecs from flowCatalog
> ----------------------------------------
>
> Key: GOBBLIN-2107
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2107
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Delete adhoc flowSpecs from flowCatalog to avoid build up of adhoc flowSpecs
> in catalog even when job compilation fails by adding the deletion in a
> finally block in non-MA scheduler case.
> # Previous PR: [https://github.com/apache/gobblin/pull/3944/files] causes a
> bug where adhoc flows are not deleted when multi-active scheduler is enabled.
> This PR re-adds deletion of flowSpec to the dagManager to be called by the
> active host for the MA scheduler case (there will only be one active host if
> dagManager is enabled otherwise code will use dagProcessingEngine).
> # It also ensures quota is released and failed flow compilation event sent
> by the FlowCompilationValidationHelper in all compilation failure cases. This
> was being incorrectly handled before.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
