This is an automated email from the ASF dual-hosted git repository.
pingsutw pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 12f65ef SUBMARINE-1060. Create TensorFlow model serving
12f65ef is described below
commit 12f65efa60db3f26ff7a86a81ea8f911d56fa14d
Author: KUAN-HSUN-LI <[email protected]>
AuthorDate: Mon Nov 15 14:52:06 2021 +0800
SUBMARINE-1060. Create TensorFlow model serving
### What is this PR for?
1. Use `seldon core` to create the TensorFlow model serving.
2. Create model serve rest API.
3. **Inference API is not created in this PR**
### What type of PR is it?
[Feature]
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-1060
### How should this be tested?
The test will be provided after the `minIO` client is created.
### Screenshots (if appropriate)
https://user-images.githubusercontent.com/38066413/141690103-6a3b9c36-4feb-4295-8bcd-3914c9736769.mp4
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? Yes
* Does this need new documentation? Yes
Author: KUAN-HSUN-LI <[email protected]>
Signed-off-by: Kevin <[email protected]>
Closes #800 from KUAN-HSUN-LI/SUBMARINE-1060 and squashes the following
commits:
d2658285 [KUAN-HSUN-LI] SUBMARINE-1060. Create TensorFlow Serving
ade6149b [KUAN-HSUN-LI] adjustment
ae95b1a1 [KUAN-HSUN-LI] add document
d7fba1cb [KUAN-HSUN-LI] SUBMARINE-1060. Add error handle and License
97504bd5 [KUAN-HSUN-LI] SUBMARINE-1060. Implement Delete Serve
f80285a8 [KUAN-HSUN-LI] SUBMARINE-1060. Implement Create Serve
19f99fd5 [KUAN-HSUN-LI] SUBMARINE-1060. Create TensorFlow Serving
5f3f9d36 [KUAN-HSUN-LI] SUBMARINE-1060. Create TensorFlow Serving
40c85c0c [KUAN-HSUN-LI] setup
---
helm-charts/submarine/templates/rbac.yaml | 13 +++
pom.xml | 1 +
.../artifacts/submarine/submarine-rbac.yaml | 13 +++
.../submitter-k8s => submarine-serve}/pom.xml | 56 ++-------
.../submarine/serve/seldon/SeldonDeployment.java | 126 +++++++++++++++++++++
.../apache/submarine/serve/seldon/SeldonGraph.java | 59 ++++++++++
.../submarine/serve/seldon/SeldonPredictor.java | 64 +++++++++++
.../serve/tensorflow/SeldonTFServing.java | 45 ++++++++
.../submarine/serve/utils/SeldonConstants.java | 42 +++++++
.../src/main/resources/log4j.properties | 25 ++++
.../serve/tensorflow/SeldonTFServingTest.java | 22 ++++
.../org/apache/submarine/server/api/Submitter.java | 13 +--
.../submarine/server/api/model/ServeResponse.java | 31 +++++
.../submarine/server/api/model/ServeSpec.java | 58 ++++++++++
.../server/experiment/ExperimentManager.java | 28 -----
.../submarine/server/model/ModelManager.java | 110 ++++++++++++++++++
.../submarine/server/rest/ExperimentRestApi.java | 54 ---------
.../submarine/server/rest/RestConstants.java | 4 +
.../apache/submarine/server/rest/ServeRestApi.java | 98 ++++++++++++++++
.../server-submitter/submitter-k8s/pom.xml | 6 +
.../server/submitter/k8s/K8sSubmitter.java | 96 +++++++---------
.../server/submitter/k8s/K8SJobSubmitterTest.java | 25 +++-
website/docs/api/serve.md | 78 +++++++++++++
website/sidebars.js | 1 +
24 files changed, 872 insertions(+), 196 deletions(-)
diff --git a/helm-charts/submarine/templates/rbac.yaml
b/helm-charts/submarine/templates/rbac.yaml
index 16e4c7e..0481c61 100644
--- a/helm-charts/submarine/templates/rbac.yaml
+++ b/helm-charts/submarine/templates/rbac.yaml
@@ -53,6 +53,19 @@ rules:
- patch
- update
- apiGroups:
+ - machinelearning.seldon.io
+ resources:
+ - seldondeployments
+ verbs:
+ - get
+ - list
+ - watch
+ - create
+ - delete
+ - deletecollection
+ - patch
+ - update
+- apiGroups:
- ""
resources:
- pods
diff --git a/pom.xml b/pom.xml
index e97bf46..333f269 100644
--- a/pom.xml
+++ b/pom.xml
@@ -145,6 +145,7 @@
<modules>
<module>submarine-commons</module>
<module>submarine-client</module>
+ <module>submarine-serve</module>
<module>submarine-server</module>
<module>submarine-all</module>
<module>submarine-workbench</module>
diff --git a/submarine-cloud-v2/artifacts/submarine/submarine-rbac.yaml
b/submarine-cloud-v2/artifacts/submarine/submarine-rbac.yaml
index 2ea85a7..f568dac 100644
--- a/submarine-cloud-v2/artifacts/submarine/submarine-rbac.yaml
+++ b/submarine-cloud-v2/artifacts/submarine/submarine-rbac.yaml
@@ -55,6 +55,19 @@ rules:
- patch
- update
- apiGroups:
+ - machinelearning.seldon.io
+ resources:
+ - seldondeployments
+ verbs:
+ - get
+ - list
+ - watch
+ - create
+ - delete
+ - deletecollection
+ - patch
+ - update
+- apiGroups:
- ""
resources:
- pods
diff --git a/submarine-server/server-submitter/submitter-k8s/pom.xml
b/submarine-serve/pom.xml
similarity index 68%
copy from submarine-server/server-submitter/submitter-k8s/pom.xml
copy to submarine-serve/pom.xml
index 9b0870a..57ae5da 100644
--- a/submarine-server/server-submitter/submitter-k8s/pom.xml
+++ b/submarine-serve/pom.xml
@@ -17,72 +17,28 @@
specific language governing permissions and limitations
under the License.
-->
-
<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
<parent>
- <artifactId>submarine-server-submitter</artifactId>
+ <artifactId>submarine</artifactId>
<groupId>org.apache.submarine</groupId>
<version>0.7.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
- <artifactId>submarine-submitter-k8s</artifactId>
+ <artifactId>submarine-serve</artifactId>
<version>0.7.0-SNAPSHOT</version>
- <name>Submarine: Kubernetes Submitter</name>
+ <name>Submarine: Serve</name>
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
- <version>${slf4j.version}</version>
</dependency>
-
- <dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
- <version>${slf4j.version}</version>
- </dependency>
-
<dependency>
<groupId>io.kubernetes</groupId>
<artifactId>client-java</artifactId>
- <version>${k8s.client-java.version}</version>
- </dependency>
-
- <dependency>
- <groupId>org.apache.submarine</groupId>
- <artifactId>submarine-server-api</artifactId>
- <version>${project.version}</version>
- </dependency>
-
- <dependency>
- <groupId>org.apache.submarine</groupId>
- <artifactId>submarine-server-core</artifactId>
- <version>${project.version}</version>
- <exclusions>
- <exclusion>
- <groupId>io.grpc</groupId>
- <artifactId>grpc-protobuf</artifactId>
- </exclusion>
- <exclusion>
- <groupId>com.google.code.gson</groupId>
- <artifactId>gson</artifactId>
- </exclusion>
- <exclusion>
- <groupId>org.tukaani</groupId>
- <artifactId>xz</artifactId>
- </exclusion>
- <exclusion>
- <groupId>joda-time</groupId>
- <artifactId>joda-time</artifactId>
- </exclusion>
- <exclusion>
- <groupId>mysql</groupId>
- <artifactId>mysql-connector-java</artifactId>
- </exclusion>
- </exclusions>
</dependency>
<!-- Unit Tests -->
@@ -90,6 +46,12 @@
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.apache.submarine</groupId>
+ <artifactId>submarine-commons-utils</artifactId>
+ <version>0.7.0-SNAPSHOT</version>
+ <scope>test</scope>
+ </dependency>
</dependencies>
<build>
diff --git
a/submarine-serve/src/main/java/org/apache/submarine/serve/seldon/SeldonDeployment.java
b/submarine-serve/src/main/java/org/apache/submarine/serve/seldon/SeldonDeployment.java
new file mode 100644
index 0000000..483d24c
--- /dev/null
+++
b/submarine-serve/src/main/java/org/apache/submarine/serve/seldon/SeldonDeployment.java
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.submarine.serve.seldon;
+
+import com.google.gson.annotations.SerializedName;
+import io.kubernetes.client.models.V1ObjectMeta;
+import org.apache.submarine.serve.utils.SeldonConstants;
+
+import java.util.ArrayList;
+import java.util.List;
+
+public class SeldonDeployment {
+ @SerializedName("apiVersion")
+ private String apiVersion = SeldonConstants.API_VERSION;
+
+ @SerializedName("kind")
+ private String kind = SeldonConstants.KIND;
+
+ @SerializedName("metadata")
+ private V1ObjectMeta metadata;
+
+ @SerializedName("spec")
+ private SeldonDeploymentSpec spec;
+
+ // transient to avoid being serialized
+ private transient String group = SeldonConstants.GROUP;
+
+ private transient String version = SeldonConstants.VERSION;
+
+ private transient String plural = SeldonConstants.PLURAL;
+
+ public String getApiVersion() {
+ return apiVersion;
+ }
+
+ public void setApiVersion(String apiVersion) {
+ this.apiVersion = apiVersion;
+ }
+
+ public String getKind() {
+ return kind;
+ }
+
+ public void setKind(String kind) {
+ this.kind = kind;
+ }
+
+ public V1ObjectMeta getMetadata() {
+ return metadata;
+ }
+
+ public void setMetadata(V1ObjectMeta metadata) {
+ this.metadata = metadata;
+ }
+
+ public String getGroup() {
+ return group;
+ }
+
+ public void setGroup(String group) {
+ this.group = group;
+ }
+
+ public String getVersion() {
+ return version;
+ }
+
+ public void setVersion(String version) {
+ this.version = version;
+ }
+
+ public String getPlural() {
+ return plural;
+ }
+
+ public void setPlural(String plural) {
+ this.plural = plural;
+ }
+
+ public void setSpec(SeldonDeploymentSpec seldonDeploymentSpec){
+ this.spec = seldonDeploymentSpec;
+ }
+
+ public void addPredictor(SeldonPredictor seldonPredictor) {
+ this.spec.addPredictor(seldonPredictor);
+ }
+
+ public static class SeldonDeploymentSpec {
+ public SeldonDeploymentSpec(String protocol) {
+ setProtocol(protocol);
+ }
+
+ @SerializedName("protocol")
+ private String protocol;
+ @SerializedName("predictors")
+ private List<SeldonPredictor> predictors = new ArrayList<>();
+
+ public String getProtocol() {
+ return protocol;
+ }
+
+ public void setProtocol(String protocol) {
+ this.protocol = protocol;
+ }
+
+ public void addPredictor(SeldonPredictor seldonPredictor) {
+ predictors.add(seldonPredictor);
+ }
+ }
+}
diff --git
a/submarine-serve/src/main/java/org/apache/submarine/serve/seldon/SeldonGraph.java
b/submarine-serve/src/main/java/org/apache/submarine/serve/seldon/SeldonGraph.java
new file mode 100644
index 0000000..48b7e25
--- /dev/null
+++
b/submarine-serve/src/main/java/org/apache/submarine/serve/seldon/SeldonGraph.java
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.submarine.serve.seldon;
+
+import com.google.gson.annotations.SerializedName;
+import org.apache.submarine.serve.utils.SeldonConstants;
+
+public class SeldonGraph {
+ @SerializedName("name")
+ private String name;
+ @SerializedName("implementation")
+ private String implementation;
+ @SerializedName("modelUri")
+ private String modelUri;
+ @SerializedName("storageInitializerImage")
+ private String storageInitializerImage =
SeldonConstants.STORAGE_INITIALIZER_IMAGE;
+ @SerializedName("envSecretRefName")
+ private String envSecretRefName = SeldonConstants.ENV_SECRET_REF_NAME;
+
+ public String getName() {
+ return name;
+ }
+
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ public String getImplementation() {
+ return implementation;
+ }
+
+ public void setImplementation(String implementation) {
+ this.implementation = implementation;
+ }
+
+ public String getModelUri() {
+ return modelUri;
+ }
+
+ public void setModelUri(String modelUri) {
+ this.modelUri = modelUri;
+ }
+}
diff --git
a/submarine-serve/src/main/java/org/apache/submarine/serve/seldon/SeldonPredictor.java
b/submarine-serve/src/main/java/org/apache/submarine/serve/seldon/SeldonPredictor.java
new file mode 100644
index 0000000..48e5227
--- /dev/null
+++
b/submarine-serve/src/main/java/org/apache/submarine/serve/seldon/SeldonPredictor.java
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.submarine.serve.seldon;
+
+import com.google.gson.annotations.SerializedName;
+
+public class SeldonPredictor {
+ @SerializedName("name")
+ private String name = "default";
+
+ @SerializedName("replicas")
+ private Integer replicas = 1;
+
+ @SerializedName("graph")
+ private SeldonGraph seldonGraph = new SeldonGraph();
+
+ public SeldonPredictor(String name, Integer replicas, SeldonGraph graph) {
+ setName(name);
+ setReplicas(replicas);
+ setSeldonGraph(graph);
+ }
+
+ public SeldonPredictor(){};
+
+ public String getName() {
+ return name;
+ }
+
+ public void setName(String name) {
+ this.name = name;
+ }
+
+ public Integer getReplicas() {
+ return replicas;
+ }
+
+ public void setReplicas(Integer replicas) {
+ this.replicas = replicas;
+ }
+
+ public SeldonGraph getSeldonGraph() {
+ return seldonGraph;
+ }
+
+ public void setSeldonGraph(SeldonGraph seldonGraph) {
+ this.seldonGraph = seldonGraph;
+ }
+}
diff --git
a/submarine-serve/src/main/java/org/apache/submarine/serve/tensorflow/SeldonTFServing.java
b/submarine-serve/src/main/java/org/apache/submarine/serve/tensorflow/SeldonTFServing.java
new file mode 100644
index 0000000..d63dbaa
--- /dev/null
+++
b/submarine-serve/src/main/java/org/apache/submarine/serve/tensorflow/SeldonTFServing.java
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.submarine.serve.tensorflow;
+
+import io.kubernetes.client.models.V1ObjectMeta;
+import org.apache.submarine.serve.seldon.SeldonDeployment;
+import org.apache.submarine.serve.seldon.SeldonGraph;
+import org.apache.submarine.serve.seldon.SeldonPredictor;
+import org.apache.submarine.serve.utils.SeldonConstants;
+
+public class SeldonTFServing extends SeldonDeployment {
+ public SeldonTFServing(String name, String modelURI){
+ V1ObjectMeta v1ObjectMeta = new V1ObjectMeta();
+ v1ObjectMeta.setName(name);
+ v1ObjectMeta.setNamespace(SeldonConstants.DEFAULT_NAMESPACE);
+ setMetadata(v1ObjectMeta);
+
+ setSpec(new SeldonDeploymentSpec(SeldonConstants.SELDON_PROTOCOL));
+
+ SeldonGraph seldonGraph = new SeldonGraph();
+ seldonGraph.setName(name);
+ seldonGraph.setImplementation(SeldonConstants.TFSERVING_IMPLEMENTATION);
+ seldonGraph.setModelUri(modelURI);
+ SeldonPredictor seldonPredictor = new SeldonPredictor();
+ seldonPredictor.setSeldonGraph(seldonGraph);
+
+ addPredictor(seldonPredictor);
+ }
+}
diff --git
a/submarine-serve/src/main/java/org/apache/submarine/serve/utils/SeldonConstants.java
b/submarine-serve/src/main/java/org/apache/submarine/serve/utils/SeldonConstants.java
new file mode 100644
index 0000000..bcaf0a6
--- /dev/null
+++
b/submarine-serve/src/main/java/org/apache/submarine/serve/utils/SeldonConstants.java
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.submarine.serve.utils;
+
+public class SeldonConstants {
+ public static final String API_VERSION = "machinelearning.seldon.io/v1";
+
+ public static final String KIND = "SeldonDeployment";
+
+ public static final String GROUP = "machinelearning.seldon.io";
+
+ public static final String VERSION = "v1";
+
+ public static final String PLURAL = "seldondeployments";
+
+ public static final String STORAGE_INITIALIZER_IMAGE =
"seldonio/rclone-storage-initializer:1.10.0";
+
+ public static final String ENV_SECRET_REF_NAME = "submarine-serve-secret";
+
+ public static final String DEFAULT_NAMESPACE = "default";
+
+ public static final String SELDON_PROTOCOL = "seldon";
+
+ // TensorFlow
+ public static final String TFSERVING_IMPLEMENTATION = "TENSORFLOW_SERVER";
+}
diff --git a/submarine-serve/src/main/resources/log4j.properties
b/submarine-serve/src/main/resources/log4j.properties
new file mode 100644
index 0000000..fdd0239
--- /dev/null
+++ b/submarine-serve/src/main/resources/log4j.properties
@@ -0,0 +1,25 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+log4j.rootLogger = info, stdout
+
+log4j.appender.stdout = org.apache.log4j.ConsoleAppender
+log4j.appender.stdout.Target = System.out
+log4j.appender.stdout.layout = org.apache.log4j.PatternLayout
+log4j.appender.stdout.layout.ConversionPattern = [%-5p] %d{yyyy-MM-dd
HH:mm:ss,SSS} method:%l%n%m%n
diff --git
a/submarine-serve/src/test/java/org/apache/submarine/serve/tensorflow/SeldonTFServingTest.java
b/submarine-serve/src/test/java/org/apache/submarine/serve/tensorflow/SeldonTFServingTest.java
new file mode 100644
index 0000000..8dc40db
--- /dev/null
+++
b/submarine-serve/src/test/java/org/apache/submarine/serve/tensorflow/SeldonTFServingTest.java
@@ -0,0 +1,22 @@
+// /*
+// * Licensed to the Apache Software Foundation (ASF) under one
+// * or more contributor license agreements. See the NOTICE file
+// * distributed with this work for additional information
+// * regarding copyright ownership. The ASF licenses this file
+// * to you under the Apache License, Version 2.0 (the
+// * "License"); you may not use this file except in compliance
+// * with the License. You may obtain a copy of the License at
+// *
+// * http://www.apache.org/licenses/LICENSE-2.0
+// *
+// * Unless required by applicable law or agreed to in writing,
+// * software distributed under the License is distributed on an
+// * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// * KIND, either express or implied. See the License for the
+// * specific language governing permissions and limitations
+// * under the License.
+// */
+
+package org.apache.submarine.serve.tensorflow;
+
+// TODO(KUAN-HSUN LI)
diff --git
a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/Submitter.java
b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/Submitter.java
index 6185ba4..96e0f2b 100644
---
a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/Submitter.java
+++
b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/Submitter.java
@@ -25,8 +25,8 @@ import org.apache.submarine.server.api.experiment.Experiment;
import org.apache.submarine.server.api.experiment.ExperimentLog;
import org.apache.submarine.server.api.experiment.TensorboardInfo;
import org.apache.submarine.server.api.experiment.MlflowInfo;
-import org.apache.submarine.server.api.experiment.ServeRequest;
-import org.apache.submarine.server.api.experiment.ServeResponse;
+import org.apache.submarine.server.api.model.ServeResponse;
+import org.apache.submarine.server.api.model.ServeSpec;
import org.apache.submarine.server.api.notebook.Notebook;
import org.apache.submarine.server.api.spec.ExperimentSpec;
import org.apache.submarine.server.api.spec.NotebookSpec;
@@ -126,20 +126,19 @@ public interface Submitter {
/**
* Create Serve with spec
- * @param ServeRequest
+ * @param spec
* @return object
* @throws SubmarineRuntimeException running error
*/
- ServeResponse createServe(ServeRequest spec) throws
SubmarineRuntimeException;
+ ServeResponse createServe(ServeSpec spec) throws SubmarineRuntimeException;
/**
* Delete Serve with spec
- * @param ServeRequest
- * @return object
+ * @param spec
* @throws SubmarineRuntimeException running error
*/
- ServeResponse deleteServe(ServeRequest spec) throws
SubmarineRuntimeException;
+ void deleteServe(ServeSpec spec) throws SubmarineRuntimeException;
/**
* Get tensorboard meta data
diff --git
a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/model/ServeResponse.java
b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/model/ServeResponse.java
new file mode 100644
index 0000000..d948abe
--- /dev/null
+++
b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/model/ServeResponse.java
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.submarine.server.api.model;
+
+public class ServeResponse {
+ private String url;
+
+ public String getUrl() {
+ return url;
+ }
+
+ public void setUrl(String url) {
+ this.url = url;
+ }
+}
diff --git
a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/model/ServeSpec.java
b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/model/ServeSpec.java
new file mode 100644
index 0000000..0a0f8fe
--- /dev/null
+++
b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/model/ServeSpec.java
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.submarine.server.api.model;
+
+public class ServeSpec {
+ private String modelName;
+ private Integer modelVersion;
+ private String modelType;
+ private String modelURI;
+
+ public String getModelName() {
+ return modelName;
+ }
+
+ public void setModelName(String modelName) {
+ this.modelName = modelName;
+ }
+
+ public Integer getModelVersion() {
+ return modelVersion;
+ }
+
+ public void setModelVersion(Integer modelVersion) {
+ this.modelVersion = modelVersion;
+ }
+
+ public String getModelType() {
+ return modelType;
+ }
+
+ public void setModelType(String modelType) {
+ this.modelType = modelType;
+ }
+
+ public String getModelURI() {
+ return modelURI;
+ }
+
+ public void setModelURI(String modelURI) {
+ this.modelURI = modelURI;
+ }
+}
diff --git
a/submarine-server/server-core/src/main/java/org/apache/submarine/server/experiment/ExperimentManager.java
b/submarine-server/server-core/src/main/java/org/apache/submarine/server/experiment/ExperimentManager.java
index f0776d1..e0764d1 100644
---
a/submarine-server/server-core/src/main/java/org/apache/submarine/server/experiment/ExperimentManager.java
+++
b/submarine-server/server-core/src/main/java/org/apache/submarine/server/experiment/ExperimentManager.java
@@ -41,8 +41,6 @@ import org.apache.submarine.server.api.Submitter;
import org.apache.submarine.server.api.experiment.ExperimentLog;
import org.apache.submarine.server.api.experiment.TensorboardInfo;
import org.apache.submarine.server.api.experiment.MlflowInfo;
-import org.apache.submarine.server.api.experiment.ServeRequest;
-import org.apache.submarine.server.api.experiment.ServeResponse;
import org.apache.submarine.server.api.spec.ExperimentSpec;
import org.apache.submarine.server.experiment.database.entity.ExperimentEntity;
import
org.apache.submarine.server.experiment.database.service.ExperimentService;
@@ -346,32 +344,6 @@ public class ExperimentManager {
return submitter.getMlflowInfo();
}
- /**
- * Create serve.
- *
- * @param spec spec
- * @return object
- * @throws SubmarineRuntimeException the service error
- */
- public ServeResponse createServe(ServeRequest spec) throws
SubmarineRuntimeException {
- // TODO(byronhsu): use mlflow api to make sure the model exists.
Otherwise, raise exception.
- ServeResponse serve = submitter.createServe(spec);
- return serve;
- }
-
- /**
- * Delete serve.
- *
- * @param spec spec
- * @return object
- * @throws SubmarineRuntimeException the service error
- */
- public ServeResponse deleteServe(ServeRequest spec) throws
SubmarineRuntimeException {
- ServeResponse serve = submitter.deleteServe(spec);
- return serve;
- }
-
-
private void checkSpec(ExperimentSpec spec) throws SubmarineRuntimeException
{
if (spec == null) {
throw new SubmarineRuntimeException(Status.OK.getStatusCode(), "Invalid
experiment spec.");
diff --git
a/submarine-server/server-core/src/main/java/org/apache/submarine/server/model/ModelManager.java
b/submarine-server/server-core/src/main/java/org/apache/submarine/server/model/ModelManager.java
new file mode 100644
index 0000000..49ceb6e
--- /dev/null
+++
b/submarine-server/server-core/src/main/java/org/apache/submarine/server/model/ModelManager.java
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.submarine.server.model;
+
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import javax.ws.rs.core.Response;
+
+import org.apache.submarine.commons.utils.exception.SubmarineRuntimeException;
+import org.apache.submarine.server.SubmitterManager;
+import org.apache.submarine.server.api.Submitter;
+import org.apache.submarine.server.api.model.ServeResponse;
+import org.apache.submarine.server.api.model.ServeSpec;
+import org.apache.submarine.server.model.database.service.ModelVersionService;
+
+
+
+public class ModelManager {
+ private static final Logger LOG =
LoggerFactory.getLogger(ModelManager.class);
+
+ private static volatile ModelManager manager;
+
+ private final Submitter submitter;
+
+ private final ModelVersionService modelVersionService;
+
+ private ModelManager(Submitter submitter, ModelVersionService
modelVersionService) {
+ this.submitter = submitter;
+ this.modelVersionService = modelVersionService;
+ }
+
+ /**
+ * Get the singleton instance.
+ *
+ * @return object
+ */
+ public static ModelManager getInstance() {
+ if (manager == null) {
+ synchronized (ModelManager.class) {
+ if (manager == null) {
+ manager = new ModelManager(SubmitterManager.loadSubmitter(), new
ModelVersionService());
+ }
+ }
+ }
+ return manager;
+ }
+
+ /**
+ * Create a model serve.
+ */
+ public ServeResponse createServe(ServeSpec spec) throws
SubmarineRuntimeException {
+ checkServeSpec(spec);
+ String modelURI = modelVersionService.select(spec.getModelName(),
spec.getModelVersion()).getSource();
+ spec.setModelURI(modelURI);
+
+ ServeResponse serveResponse = submitter.createServe(spec);
+ return serveResponse;
+ }
+
+ /**
+ * Delete a model serve.
+ */
+ public void deleteServe(ServeSpec spec) throws SubmarineRuntimeException {
+ checkServeSpec(spec);
+ String modelURI = modelVersionService.select(spec.getModelName(),
spec.getModelVersion()).getSource();
+ spec.setModelURI(modelURI);
+
+ submitter.deleteServe(spec);
+ }
+
+ private void checkServeSpec(ServeSpec spec) throws SubmarineRuntimeException
{
+ if (spec == null) {
+ throw new SubmarineRuntimeException(Response.Status.OK.getStatusCode(),
+ "Invalid. Serve Spec object is null.");
+ } else {
+ if (spec.getModelName() == null) {
+ throw new SubmarineRuntimeException(Response.Status.OK.getStatusCode(),
+ "Invalid. Model name in Serve Soec is null.");
+ }
+ Integer modelVersion = spec.getModelVersion();
+ if (modelVersion == null || modelVersion <= 0) {
+ throw new SubmarineRuntimeException(Response.Status.OK.getStatusCode(),
+ "Invalid. Model version must be positive, but get " +
modelVersion);
+ }
+ String modelType = spec.getModelType();
+ if (!modelType.equals("tensorflow") && !modelType.equals("pytorch")) {
+ throw new SubmarineRuntimeException(Response.Status.OK.getStatusCode(),
+ "Invalid. Serve Type can only be tensorflow or pytorch, but
get " + modelType);
+ }
+ }
+ }
+}
diff --git
a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/ExperimentRestApi.java
b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/ExperimentRestApi.java
index 4a1efa7..76331a2 100644
---
a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/ExperimentRestApi.java
+++
b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/ExperimentRestApi.java
@@ -43,8 +43,6 @@ import
org.apache.submarine.commons.utils.exception.SubmarineRuntimeException;
import org.apache.submarine.server.api.experiment.Experiment;
import org.apache.submarine.server.api.experiment.TensorboardInfo;
import org.apache.submarine.server.api.experiment.MlflowInfo;
-import org.apache.submarine.server.api.experiment.ServeRequest;
-import org.apache.submarine.server.api.experiment.ServeResponse;
import org.apache.submarine.server.experiment.ExperimentManager;
import
org.apache.submarine.server.experimenttemplate.ExperimentTemplateManager;
import org.apache.submarine.server.api.experiment.ExperimentLog;
@@ -300,58 +298,6 @@ public class ExperimentRestApi {
}
}
- /**
- * Returns the contents of {@link Serve} that submitted by user.
- *
- * @param spec spec
- * @return the contents of serve
- */
-
- @POST
- @Path("/serve")
- @Consumes({RestConstants.MEDIA_TYPE_YAML, MediaType.APPLICATION_JSON})
- @Operation(summary = "Create an serve",
- tags = {"serve"},
- responses = {
- @ApiResponse(description = "successful operation", content =
@Content(
- schema = @Schema(implementation = JsonResponse.class)))})
- public Response createServe(ServeRequest spec) {
- try {
- ServeResponse serve = experimentManager.createServe(spec);
- return new
JsonResponse.Builder<ServeResponse>(Response.Status.OK).success(true)
- .result(serve).build();
- } catch (SubmarineRuntimeException e) {
- return parseExperimentServiceException(e);
- }
- }
-
- /**
- * Delete {@link Serve} that submitted by user.
- *
- * @param spec spec
- * @return the contents of serve
- */
-
- @DELETE
- @Path("/serve")
- @Operation(summary = "Delete a serve",
- tags = {"serve"},
- responses = {
- @ApiResponse(description = "successful operation", content =
@Content(
- schema = @Schema(implementation = JsonResponse.class)))})
- public Response deleteServe(@QueryParam("modelName") String modelName,
- @QueryParam("modelVersion") String modelVersion,
@QueryParam("namespace") String namespace) {
- try {
- ServeRequest spec = new ServeRequest()
-
.modelName(modelName).modelVersion(modelVersion).namespace(namespace);
- ServeResponse serve = experimentManager.deleteServe(spec);
- return new
JsonResponse.Builder<ServeResponse>(Response.Status.OK).success(true)
- .result(serve).build();
- } catch (SubmarineRuntimeException e) {
- return parseExperimentServiceException(e);
- }
- }
-
private Response parseExperimentServiceException(SubmarineRuntimeException
e) {
return new JsonResponse.Builder<String>(e.getCode())
.message(e.getMessage().equals("Conflict") ? "Duplicated experiment
name" : e.getMessage()).build();
diff --git
a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/RestConstants.java
b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/RestConstants.java
index e6555fa..3137087 100644
---
a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/RestConstants.java
+++
b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/RestConstants.java
@@ -89,4 +89,8 @@ public class RestConstants {
public static final String MODEL_VERSION_NAME = "name";
public static final String MODEL_VERSION_VERSION = "version";
+ /**
+ * Serve.
+ */
+ public static final String SERVE = "serve";
}
diff --git
a/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/ServeRestApi.java
b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/ServeRestApi.java
new file mode 100644
index 0000000..65f9d72
--- /dev/null
+++
b/submarine-server/server-core/src/main/java/org/apache/submarine/server/rest/ServeRestApi.java
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.submarine.server.rest;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.DELETE;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import io.swagger.v3.oas.annotations.Operation;
+import io.swagger.v3.oas.annotations.media.Content;
+import io.swagger.v3.oas.annotations.media.Schema;
+import io.swagger.v3.oas.annotations.responses.ApiResponse;
+
+import org.apache.submarine.commons.utils.exception.SubmarineRuntimeException;
+import org.apache.submarine.server.api.model.ServeResponse;
+import org.apache.submarine.server.api.model.ServeSpec;
+import org.apache.submarine.server.model.ModelManager;
+import org.apache.submarine.server.response.JsonResponse;
+
+
+
+@Path(RestConstants.V1 + "/" + RestConstants.SERVE)
+@Produces({MediaType.APPLICATION_JSON + "; " + RestConstants.CHARSET_UTF8})
+public class ServeRestApi {
+
+ private final ModelManager modelManager = ModelManager.getInstance();
+
+ /**
+ * Return the Pong message for test the connectivity.
+ * @return Pong message
+ */
+ @GET
+ @Path(RestConstants.PING)
+ @Consumes(MediaType.APPLICATION_JSON)
+ @Operation(summary = "Ping submarine server",
+ tags = {"serve"},
+ description = "Return the Pong message for test the connectivity",
+ responses = {@ApiResponse(responseCode = "200", description =
"successful operation",
+ content = @Content(schema = @Schema(implementation =
String.class)))})
+ public Response ping() {
+ return new JsonResponse.Builder<String>(Response.Status.OK)
+ .success(true).result("Pong").build();
+ }
+
+ @POST
+ @Consumes({ RestConstants.MEDIA_TYPE_YAML, MediaType.APPLICATION_JSON })
+ @Operation(summary = "Create a serve instance", tags = { "serve" },
responses = {
+ @ApiResponse(description = "successful operation",
+ content = @Content(schema = @Schema(implementation =
JsonResponse.class)))})
+ public Response createServe(ServeSpec spec) {
+ try {
+ ServeResponse serveResponse = modelManager.createServe(spec);
+ return new
JsonResponse.Builder<ServeResponse>(Response.Status.OK).success(true)
+ .message("Create a serve
instance").result(serveResponse).build();
+ } catch (SubmarineRuntimeException e) {
+ return parseModelVersionServiceException(e);
+ }
+ }
+
+ @DELETE
+ @Operation(summary = "Delete the serve instance.", tags = { "serve" },
responses = {
+ @ApiResponse(description = "successful operation",
+ content = @Content(schema = @Schema(implementation =
JsonResponse.class))),
+ @ApiResponse(responseCode = "404", description = "Serve not
found.")})
+ public Response deleteServe(ServeSpec spec) {
+ try {
+ modelManager.deleteServe(spec);
+ return new JsonResponse.Builder<String>(Response.Status.OK).success(true)
+ .message("Delete the model serve instance").build();
+ } catch (SubmarineRuntimeException e) {
+ return parseModelVersionServiceException(e);
+ }
+ }
+
+ private Response parseModelVersionServiceException(SubmarineRuntimeException
e) {
+ return new
JsonResponse.Builder<String>(e.getCode()).message(e.getMessage()).build();
+ }
+}
diff --git a/submarine-server/server-submitter/submitter-k8s/pom.xml
b/submarine-server/server-submitter/submitter-k8s/pom.xml
index 9b0870a..a0e7e4d 100644
--- a/submarine-server/server-submitter/submitter-k8s/pom.xml
+++ b/submarine-server/server-submitter/submitter-k8s/pom.xml
@@ -90,6 +90,12 @@
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.apache.submarine</groupId>
+ <artifactId>submarine-serve</artifactId>
+ <version>0.7.0-SNAPSHOT</version>
+ <scope>compile</scope>
+ </dependency>
</dependencies>
<build>
diff --git
a/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java
b/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java
index cac7398..507dde9 100644
---
a/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java
+++
b/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java
@@ -49,7 +49,6 @@ import io.kubernetes.client.models.V1ObjectMeta;
import io.kubernetes.client.models.V1PersistentVolumeClaim;
import io.kubernetes.client.models.V1Pod;
import io.kubernetes.client.models.V1PodList;
-import io.kubernetes.client.models.V1Service;
import io.kubernetes.client.models.V1Status;
import io.kubernetes.client.util.Watch;
import io.kubernetes.client.util.ClientBuilder;
@@ -57,14 +56,16 @@ import io.kubernetes.client.util.KubeConfig;
import org.apache.submarine.commons.utils.SubmarineConfiguration;
import org.apache.submarine.commons.utils.exception.SubmarineRuntimeException;
+import org.apache.submarine.serve.seldon.SeldonDeployment;
+import org.apache.submarine.serve.tensorflow.SeldonTFServing;
import org.apache.submarine.server.api.Submitter;
import org.apache.submarine.server.api.exception.InvalidSpecException;
import org.apache.submarine.server.api.experiment.Experiment;
import org.apache.submarine.server.api.experiment.ExperimentLog;
import org.apache.submarine.server.api.experiment.TensorboardInfo;
import org.apache.submarine.server.api.experiment.MlflowInfo;
-import org.apache.submarine.server.api.experiment.ServeRequest;
-import org.apache.submarine.server.api.experiment.ServeResponse;
+import org.apache.submarine.server.api.model.ServeResponse;
+import org.apache.submarine.server.api.model.ServeSpec;
import org.apache.submarine.server.api.notebook.Notebook;
import org.apache.submarine.server.api.spec.ExperimentMeta;
import org.apache.submarine.server.api.spec.ExperimentSpec;
@@ -74,12 +75,10 @@ import
org.apache.submarine.server.submitter.k8s.model.NotebookCR;
import
org.apache.submarine.server.submitter.k8s.model.ingressroute.IngressRoute;
import
org.apache.submarine.server.submitter.k8s.model.ingressroute.IngressRouteSpec;
import org.apache.submarine.server.submitter.k8s.model.ingressroute.SpecRoute;
-import org.apache.submarine.server.submitter.k8s.model.middlewares.Middlewares;
import org.apache.submarine.server.submitter.k8s.model.pytorchjob.PyTorchJob;
import org.apache.submarine.server.submitter.k8s.model.tfjob.TFJob;
import org.apache.submarine.server.submitter.k8s.parser.ExperimentSpecParser;
import org.apache.submarine.server.submitter.k8s.parser.NotebookSpecParser;
-import org.apache.submarine.server.submitter.k8s.parser.ServeSpecParser;
import org.apache.submarine.server.submitter.k8s.parser.VolumeSpecParser;
import org.apache.submarine.server.submitter.k8s.util.MLJobConverter;
import org.apache.submarine.server.submitter.k8s.util.NotebookUtils;
@@ -117,6 +116,7 @@ public class K8sSubmitter implements Submitter {
public void initialize(SubmarineConfiguration conf) {
try {
String path = System.getenv(KUBECONFIG_ENV);
+ // path = System.getProperty("user.home") + "/.kube/config";
//TODO(tmp)
KubeConfig config = KubeConfig.loadKubeConfig(new FileReader(path));
client = ClientBuilder.kubeconfig(config).build();
} catch (Exception e) {
@@ -518,67 +518,39 @@ public class K8sSubmitter implements Submitter {
}
@Override
- public ServeResponse createServe(ServeRequest spec)
+ public ServeResponse createServe(ServeSpec spec)
throws SubmarineRuntimeException {
- String modelName = spec.getModelName();
- String modelVersion = spec.getModelVersion();
- String namespace = spec.getNamespace();
-
- ServeSpecParser parser = new ServeSpecParser(modelName, modelVersion,
namespace);
- V1Deployment deployment = parser.getDeployment();
- V1Service svc = parser.getService();
- IngressRoute ingressRoute = parser.getIngressRoute();
- Middlewares middleware = parser.getMiddlewares();
- ServeResponse serveInfo = new ServeResponse().url(parser.getRoutePath());
+ SeldonDeployment seldonDeployment = parseServeSpec(spec);
try {
- appsV1Api.createNamespacedDeployment(namespace, deployment, "true",
null, null);
- coreApi.createNamespacedService(namespace, svc, "true", null, null);
-
- api.createNamespacedCustomObject(
- middleware.getGroup(), middleware.getVersion(),
- middleware.getMetadata().getNamespace(),
- middleware.getPlural(), middleware, "true");
-
- api.createNamespacedCustomObject(
- ingressRoute.getGroup(), ingressRoute.getVersion(),
- ingressRoute.getMetadata().getNamespace(),
- ingressRoute.getPlural(), ingressRoute, "true");
- return serveInfo;
+ api.createNamespacedCustomObject(seldonDeployment.getGroup(),
+ seldonDeployment.getVersion(),
+ "default",
+ seldonDeployment.getPlural(),
+ seldonDeployment,
+ "true");
} catch (ApiException e) {
+ LOG.error(e.getMessage(), e);
throw new SubmarineRuntimeException(e.getCode(), e.getMessage());
}
+ return new ServeResponse();
}
@Override
- public ServeResponse deleteServe(ServeRequest spec)
+ public void deleteServe(ServeSpec spec)
throws SubmarineRuntimeException {
- String modelName = spec.getModelName();
- String modelVersion = spec.getModelVersion();
- String namespace = spec.getNamespace();
-
- ServeSpecParser parser = new ServeSpecParser(modelName, modelVersion,
namespace);
- IngressRoute ingressRoute = parser.getIngressRoute();
- Middlewares middleware = parser.getMiddlewares();
- ServeResponse serveInfo = new ServeResponse().url(parser.getRoutePath());
+ SeldonDeployment seldonDeployment = parseServeSpec(spec);
try {
- appsV1Api.deleteNamespacedDeployment(parser.getGeneralName(), namespace,
"true",
- null, null, null, null, null);
- coreApi.deleteNamespacedService(parser.getSvcName(), namespace, "true",
- null, null, null, null, null);
- api.deleteNamespacedCustomObject(
- middleware.getGroup(), middleware.getVersion(),
- middleware.getMetadata().getNamespace(), middleware.getPlural(),
parser.getMiddlewareName(),
- new
V1DeleteOptionsBuilder().withApiVersion(middleware.getApiVersion()).build(),
- null, null, null);
- api.deleteNamespacedCustomObject(
- ingressRoute.getGroup(), ingressRoute.getVersion(),
- ingressRoute.getMetadata().getNamespace(), ingressRoute.getPlural(),
parser.getRouteName(),
- new
V1DeleteOptionsBuilder().withApiVersion(ingressRoute.getApiVersion()).build(),
- null, null, null);
- return serveInfo;
+ api.deleteNamespacedCustomObject(seldonDeployment.getGroup(),
+ seldonDeployment.getVersion(),
+ "default",
+ seldonDeployment.getPlural(),
+ seldonDeployment.getMetadata().getName(),
+ new
V1DeleteOptionsBuilder().withApiVersion(seldonDeployment.getApiVersion()).build(),
+ null, null, null);
} catch (ApiException e) {
+ LOG.error(e.getMessage(), e);
throw new SubmarineRuntimeException(e.getCode(), e.getMessage());
}
}
@@ -711,7 +683,6 @@ public class K8sSubmitter implements Submitter {
}
private String getJobLabelSelector(ExperimentSpec experimentSpec) {
- // TODO(JohnTing): SELECTOR_KEY should be obtained from individual models
in MLJOB
if (experimentSpec.getMeta().getFramework()
.equalsIgnoreCase(ExperimentMeta.SupportedMLFramework.TENSORFLOW.getName())) {
return TF_JOB_SELECTOR_KEY + experimentSpec.getMeta().getExperimentId();
@@ -776,6 +747,23 @@ public class K8sSubmitter implements Submitter {
return spec;
}
+ private SeldonDeployment parseServeSpec(ServeSpec spec) throws
SubmarineRuntimeException {
+ String modelName = spec.getModelName();
+ String modelType = spec.getModelType();
+ String modelURI = spec.getModelURI();
+
+ SeldonDeployment seldonDeployment;
+ if (modelType.equals("tensorflow")){
+ seldonDeployment = new SeldonTFServing(modelName, modelURI);
+ } else if (modelType.equals("pytorch")){
+ // TODO(KUAN-HSUN LI): create pytorch serve
+ throw new SubmarineRuntimeException("Given serve type: " + modelType + "
is not supported.");
+ } else {
+ throw new SubmarineRuntimeException("Given serve type: " + modelType + "
is not supported.");
+ }
+ return seldonDeployment;
+ }
+
private void rollbackCreationPVC(String pvcName, String namespace) {
try {
deletePersistentVolumeClaim(pvcName, namespace);
diff --git
a/submarine-server/server-submitter/submitter-k8s/src/test/java/org/apache/submarine/server/submitter/k8s/K8SJobSubmitterTest.java
b/submarine-server/server-submitter/submitter-k8s/src/test/java/org/apache/submarine/server/submitter/k8s/K8SJobSubmitterTest.java
index 5be013c..64678fc 100644
---
a/submarine-server/server-submitter/submitter-k8s/src/test/java/org/apache/submarine/server/submitter/k8s/K8SJobSubmitterTest.java
+++
b/submarine-server/server-submitter/submitter-k8s/src/test/java/org/apache/submarine/server/submitter/k8s/K8SJobSubmitterTest.java
@@ -27,7 +27,7 @@ import
org.apache.submarine.commons.utils.exception.SubmarineRuntimeException;
import org.apache.submarine.server.api.experiment.Experiment;
import org.apache.submarine.server.api.experiment.TensorboardInfo;
import org.apache.submarine.server.api.experiment.MlflowInfo;
-import org.apache.submarine.server.api.experiment.ServeRequest;
+import org.apache.submarine.server.api.model.ServeSpec;
import org.apache.submarine.server.api.spec.ExperimentSpec;
import org.junit.Assert;
import org.junit.Before;
@@ -61,12 +61,25 @@ public class K8SJobSubmitterTest extends SpecBuilder {
submitter.initialize(null);
}
- @Ignore // TODO(byronhsu): make sure saving a model before create serve
+ @Ignore // TODO(KUAN-HSUN LI): make sure saving a model before create serve
@Test
- public void tmpTest() {
- ServeRequest request = new ServeRequest()
- .modelName("simple-nn-model").modelVersion("1").namespace("default");
- submitter.createServe(request);
+ public void testCreateTensorFlowServe() {
+ ServeSpec spec = new ServeSpec();
+ spec.setModelName("simple");
+ spec.setModelVersion(1);
+ spec.setModelType("tensorflow");
+ spec.setModelURI("s3://submarine/simple");
+ submitter.createServe(spec);
+ }
+
+ @Ignore // TODO(KUAN-HSUN LI): make sure saving a model before create serve
+ @Test
+ public void testDeleteTensorFlowServe() {
+ ServeSpec spec = new ServeSpec();
+ spec.setModelName("simple");
+ spec.setModelVersion(1);
+ spec.setModelType("tensorflow");
+ submitter.deleteServe(spec);
}
@Test
diff --git a/website/docs/api/serve.md b/website/docs/api/serve.md
new file mode 100644
index 0000000..b4b179f
--- /dev/null
+++ b/website/docs/api/serve.md
@@ -0,0 +1,78 @@
+---
+title: Serve REST API
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+> Note: The Serv API is in the alpha stage which is subjected to incompatible
changes in future releases.
+
+## Create a TensorFlow model serve
+`POST /api/v1/serve`
+
+**Example Request**
+> Make sure there is a model named `simple` with version `1` in the database.
+
+```sh
+curl -X POST -H "Content-Type: application/json" -d '
+{
+ "modelName": "simple",
+ "modelVersion":1,
+ "modelType":"tensorflow"
+}
+' http://127.0.0.1:32080/api/v1/serve
+```
+
+**Example Response:**
+```json
+{
+ "status":"OK",
+ "code":200,
+ "success":true,
+ "message":"Create a serve instance",
+ "result":{"url":null},
+ "attributes":{}
+}
+```
+
+## Delete the TensorFlow model serve
+`DELETE /api/v1/serve`
+
+**Example Request**
+```sh
+curl -X DELETE -H "Content-Type: application/json" -d '
+{
+ "modelName": "simple",
+ "modelVersion":1,
+ "modelType":"tensorflow"
+}
+' http://127.0.0.1:32080/api/v1/serve
+```
+
+**Example Response:**
+```json
+{
+ "status":"OK",
+ "code":200,
+ "success":true,
+ "message":"Delete the model serve instance",
+ "result":null,
+ "attributes":{}
+}
+```
diff --git a/website/sidebars.js b/website/sidebars.js
index 6679baa..794742d 100644
--- a/website/sidebars.js
+++ b/website/sidebars.js
@@ -102,5 +102,6 @@ module.exports = {
"api/experiment",
"api/experiment-template",
"api/notebook",
+ "api/serve",
],
};
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
