[
https://issues.apache.org/jira/browse/GOBBLIN-2062?focusedWorklogId=918216&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-918216
]
ASF GitHub Bot logged work on GOBBLIN-2062:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 08/May/24 00:09
Start Date: 08/May/24 00:09
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #3944:
URL: https://github.com/apache/gobblin/pull/3944#discussion_r1593229683
##########
gobblin-service/src/test/java/org/apache/gobblin/service/modules/orchestration/OrchestratorTest.java:
##########
@@ -81,12 +84,15 @@ public class OrchestratorTest {
private FlowCatalog flowCatalog;
private FlowSpec flowSpec;
- private Orchestrator orchestrator;
+
+ private FlowStatusGenerator mockFlowStatusGenerator;
+ private DagManager mockDagManager;
+ private Orchestrator dagMgrNotFlowLaunchHandlerBasedOrchestrator;
private static final String TEST_USER = "testUser";
private static final String TEST_PASSWORD = "testPassword";
private static final String TEST_TABLE = "quotas";
- @BeforeClass
+ @BeforeMethod
Review Comment:
interesting pt. of comparison: after the [PR
check](https://github.com/apache/gobblin/actions/runs/8974993667/job/24692732545?pr=3944)
test failures gave:
```
java.io.IOException: Failure creation table TestDagStateStore
```
due to:
```
java.sql.SQLNonTransientConnectionException: Too many connections
```
I realized the `ITestMetastoreDatabase` needed to be closed.
When I first put `.close()` within `@AfterMethod`, the 7 `OrchestratorTest`
cases took *1m49s*.
I then defined an `@AfterClass` for `.close()` and those same 7 tests (all
passing with either definition) took only *33s*. so definitely sticking w/
`@BeforeClass` + `@AfterClass` for that one!
##########
gobblin-service/src/test/java/org/apache/gobblin/service/modules/orchestration/OrchestratorTest.java:
##########
@@ -81,12 +84,15 @@ public class OrchestratorTest {
private FlowCatalog flowCatalog;
private FlowSpec flowSpec;
- private Orchestrator orchestrator;
+
+ private FlowStatusGenerator mockFlowStatusGenerator;
+ private DagManager mockDagManager;
+ private Orchestrator dagMgrNotFlowLaunchHandlerBasedOrchestrator;
private static final String TEST_USER = "testUser";
private static final String TEST_PASSWORD = "testPassword";
private static final String TEST_TABLE = "quotas";
- @BeforeClass
+ @BeforeMethod
Review Comment:
interesting pt. of comparison: after the [PR
check](https://github.com/apache/gobblin/actions/runs/8974993667/job/24692732545?pr=3944)
test failures gave:
```
java.io.IOException: Failure creation table TestDagStateStore
```
due to:
```
java.sql.SQLNonTransientConnectionException: Too many connections
```
I realized the `ITestMetastoreDatabase` needed to be closed.
When I first put `.close()` within `@AfterMethod`, the 7 `OrchestratorTest`
cases took **1m49s**.
I then defined an `@AfterClass` for `.close()` and those same 7 tests (all
passing with either definition) took only **33s**. so definitely sticking w/
`@BeforeClass` + `@AfterClass` for that one!
Issue Time Tracking
-------------------
Worklog Id: (was: 918216)
Time Spent: 4.5h (was: 4h 20m)
> adhoc flow failure due to concurrent execs must be removed from flow catalog
> ----------------------------------------------------------------------------
>
> Key: GOBBLIN-2062
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2062
> Project: Apache Gobblin
> Issue Type: New Feature
> Components: gobblin-service
> Reporter: Kip Kohn
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> the Orchestrator + DagManager MUST remove adhoc flows that violate concurrent
> execs from the flow catalog. otherwise gaas will continue to return '409
> Conflict' to each subsequent attempt to create an adhoc flow with the same
> flowGroup+flowName. this is despite the fact that the flow (which still
> remains in the FlowCatalog, when it shouldn't be) already has the status
> FAILED, which is a "final status".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
