[
https://issues.apache.org/jira/browse/GOBBLIN-1340?focusedWorklogId=530948&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-530948
]
ASF GitHub Bot logged work on GOBBLIN-1340:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 04/Jan/21 22:25
Start Date: 04/Jan/21 22:25
Worklog Time Spent: 10m
Work Description: aplex commented on a change in pull request #3176:
URL: https://github.com/apache/incubator-gobblin/pull/3176#discussion_r551595499
##########
File path:
gobblin-modules/gobblin-azkaban/src/main/java/org/apache/gobblin/service/modules/orchestration/AzkabanClient.java
##########
@@ -306,6 +317,22 @@ public AzkabanClientStatus deleteProject(String
projectName) throws AzkabanClien
return runWithRetry(callable, AzkabanClientStatus.class);
}
+ /**
+ * Checks if the project with specified name exists in Azkaban
+ */
+ public Boolean projectExists(String projectName) throws
AzkabanClientException {
+ try {
+ fetchProjectFlows(projectName);
+ return true;
+ } catch (AzkabanClientException e) {
+ // Azkaban does not return a strongly typed error code, so we are
checking the message
+ if (e.getCause().getMessage().contains("doesn't exist")) {
+ return false;
Review comment:
This is an outcome of a poorly-designed REST API on Azkaban side. It
does not have a direct way to check for project existence, and it does not
return machine-readable error codes -
https://azkaban.readthedocs.io/en/latest/ajaxApi.html#fetch-jobs-of-a-flow .
This can be unreliable, if they change the messages. I'll add a feature request
for them to have better error reporting in the API.
Since the error is an expected behavior here (when project does not exist),
we shouldn't log it every time it happens - it will flood the logs when nothing
bad has happened.
##########
File path:
gobblin-modules/gobblin-azkaban/src/main/java/org/apache/gobblin/service/modules/orchestration/AzkabanProjectFlowsStatus.java
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gobblin.service.modules.orchestration;
+
+import lombok.AllArgsConstructor;
+import lombok.Getter;
+
+import java.util.List;
+
+public class AzkabanProjectFlowsStatus extends
AzkabanClientStatus<AzkabanProjectFlowsStatus.Project> {
Review comment:
Added a more extended comment in code. That's how all other API calls
are designed for this Azkaban client.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 530948)
Time Spent: 50m (was: 40m)
> Extend Azkaban client
> ---------------------
>
> Key: GOBBLIN-1340
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1340
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-core
> Reporter: Alex Prokofiev
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We're going to add extra features to Azkaban client to support end-to-end
> testing framework.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
