This is an automated email from the ASF dual-hosted git repository. pingsutw pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push: new 74f6569 SUBMARINE-1038. Replace submarine helm chart with submarine operator helm chart 74f6569 is described below commit 74f6569d652ece46334109d22f285012b782c35e Author: MortalHappiness <b07901...@ntu.edu.tw> AuthorDate: Mon Nov 22 21:13:36 2021 +0800 SUBMARINE-1038. Replace submarine helm chart with submarine operator helm chart ### What is this PR for? Replace submarine helm chart with submarine operator helm chart for consistency when developing helm charts. ### What type of PR is it? [Improvement] ### Todos * [x] - Ensure that the submarine operator creates the same resources as the original submarine helm chart does * [x] - Replace submarine helm chart with submarine operator helm chart * [x] - Make sure that k8s and python-sdk integration tests are still passed * [x] - Update the corresponding documentations ### What is the Jira issue? https://issues.apache.org/jira/projects/SUBMARINE/issues/SUBMARINE-1038 ### How should this be tested? https://github.com/MortalHappiness/submarine/actions/runs/1323214761 https://github.com/MortalHappiness/submarine/actions/runs/1323214760 ### Screenshots (if appropriate) ### Questions: * Do the license files need updating? No * Are there breaking changes for older versions? No * Does this need new documentation? No Author: MortalHappiness <b07901...@ntu.edu.tw> Signed-off-by: Kevin <pings...@apache.org> Closes #771 from MortalHappiness/SUBMARINE-1038 and squashes the following commits: df5dbc81 [MortalHappiness] SUBMARINE-1038. Update operator docker image 7c3611ff [MortalHappiness] SUBMARINE-1038. Rebase to master branch 5560b413 [MortalHappiness] SUBMARINE-1038. Update docs f32a4633 [MortalHappiness] SUBMARINE-1038. Building operator image in CI environment b6f964bf [MortalHappiness] SUBMARINE-1038. Update start-submarine.sh c7422200 [MortalHappiness] SUBMARINE-1038. Replace submarine helm chart with submarine operator helm chart 6cb2b1aa [MortalHappiness] SUBMARINE-1038. Starting submarine by using operator in CI --- .github/scripts/build-image-locally.sh | 9 +- .github/scripts/start-submarine.sh | 40 +++++- dev-support/docker-images/operator/Dockerfile | 4 +- dev-support/docker-images/operator/build.sh | 1 - helm-charts/submarine/.helmignore | 1 - helm-charts/submarine/Chart.yaml | 17 ++- .../submarine}/crds/crd.yaml | 0 .../submarine}/templates/_helpers.tpl | 0 helm-charts/submarine/templates/rbac.yaml | 155 +++++++++++---------- helm-charts/submarine/templates/storageclass.yaml | 7 +- .../submarine/templates/submarine-database.yaml | 97 ------------- .../submarine/templates/submarine-ingress.yaml | 16 --- .../submarine/templates/submarine-minio.yaml | 114 --------------- .../submarine/templates/submarine-mlflow.yaml | 131 ----------------- .../submarine}/templates/submarine-operator.yaml | 0 .../submarine/templates/submarine-server.yaml | 92 ------------ .../submarine/templates/submarine-tensorboard.yaml | 112 --------------- helm-charts/submarine/values.yaml | 49 ++----- submarine-cloud-v2/Makefile | 4 +- submarine-cloud-v2/README.md | 14 +- submarine-cloud-v2/dev.Dockerfile | 39 ------ submarine-cloud-v2/docs/developer-guide.md | 2 +- .../helm-charts/submarine-operator/.helmignore | 40 ------ .../helm-charts/submarine-operator/Chart.yaml | 32 ----- .../submarine-operator/templates/rbac.yaml | 108 -------------- .../submarine-operator/templates/storageclass.yaml | 23 --- .../helm-charts/submarine-operator/values.yaml | 32 ----- website/docs/devDocs/Development.md | 78 +---------- website/docs/gettingStarted/quickstart.md | 59 ++++---- 29 files changed, 199 insertions(+), 1077 deletions(-) diff --git a/.github/scripts/build-image-locally.sh b/.github/scripts/build-image-locally.sh index 369cb60..c53006e 100755 --- a/.github/scripts/build-image-locally.sh +++ b/.github/scripts/build-image-locally.sh @@ -17,8 +17,13 @@ # SUBMARINE_VERSION="0.7.0-SNAPSHOT" -FOLDER_LIST=("database" "mlflow" "submarine") -IMAGE_LIST=("apache/submarine:database-${SUBMARINE_VERSION}" "apache/submarine:mlflow-${SUBMARINE_VERSION}" "apache/submarine:server-${SUBMARINE_VERSION}") +FOLDER_LIST=("database" "mlflow" "submarine" "operator") +IMAGE_LIST=( + "apache/submarine:database-${SUBMARINE_VERSION}" + "apache/submarine:mlflow-${SUBMARINE_VERSION}" + "apache/submarine:server-${SUBMARINE_VERSION}" + "apache/submarine:operator-${SUBMARINE_VERSION}" +) for i in "${!IMAGE_LIST[@]}" do diff --git a/.github/scripts/start-submarine.sh b/.github/scripts/start-submarine.sh index 8d6d361..b58ef4d 100644 --- a/.github/scripts/start-submarine.sh +++ b/.github/scripts/start-submarine.sh @@ -16,12 +16,40 @@ # limitations under the License. # +wait_interval=5 +wait_timeout=900 + +wait_times=$((wait_timeout / wait_interval)) + # Fix submarine-database start failed in kind. https://github.com/kubernetes/minikube/issues/7906 sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/ sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld -helm install --wait submarine ./helm-charts/submarine -kubectl get pods -kubectl port-forward svc/submarine-database 3306:3306 & -kubectl port-forward svc/submarine-server 8080:8080 & -kubectl port-forward svc/submarine-minio-service 9000:9000 & -kubectl port-forward svc/submarine-mlflow-service 5001:5000 & \ No newline at end of file + +helm install --wait --set storageClass.provisioner=rancher.io/local-path --set storageClass.volumeBindingMode=WaitForFirstConsumer submarine ./helm-charts/submarine +kubectl apply -f ./submarine-cloud-v2/artifacts/examples/example-submarine.yaml + +# Polling waiting for the submarine to be in the RUNNING state +for ((i=0;i<$wait_times;++i)); do + state=$(kubectl get submarine -o=jsonpath='{.items[0].status.submarineState.state}') + if [[ "$state" == "RUNNING" ]]; then + echo "Submarine is running!" + kubectl describe submarine + kubectl get all + kubectl port-forward svc/submarine-database 3306:3306 & + kubectl port-forward svc/submarine-server 8080:8080 & + kubectl port-forward svc/submarine-minio-service 9000:9000 & + kubectl port-forward svc/submarine-mlflow-service 5001:5000 & + exit 0 + elif [[ "$state" == "FAILED" ]]; then + echo "Submarine failed!" 1>&2 + kubectl describe submarine + kubectl get all + exit 1 + else + sleep $wait_interval + fi +done +echo "Timeout limit reached!" 1>&2 +kubectl describe submarine +kubectl get all +exit 1 diff --git a/dev-support/docker-images/operator/Dockerfile b/dev-support/docker-images/operator/Dockerfile index bff01f6..e262784 100644 --- a/dev-support/docker-images/operator/Dockerfile +++ b/dev-support/docker-images/operator/Dockerfile @@ -23,8 +23,8 @@ WORKDIR /usr/src RUN GOOS=linux go build -o submarine-operator FROM gcr.io/distroless/base-debian10 +ADD tmp/submarine-cloud-v2/artifacts /usr/src/artifacts WORKDIR /usr/src COPY --from=build-image /usr/src/submarine-operator /usr/src/submarine-operator -ADD tmp/charts/ /usr/src/charts -CMD ["/usr/src/submarine-operator", "-incluster=true"] +CMD ["/usr/src/submarine-operator", "-incluster=true"] diff --git a/dev-support/docker-images/operator/build.sh b/dev-support/docker-images/operator/build.sh index 9c9c80b..d150dbc 100755 --- a/dev-support/docker-images/operator/build.sh +++ b/dev-support/docker-images/operator/build.sh @@ -29,7 +29,6 @@ export SUBMARINE_HOME=${CURRENT_PATH}/../../.. mkdir -p "${CURRENT_PATH}/tmp" cp -r ${SUBMARINE_HOME}/submarine-cloud-v2/ "${CURRENT_PATH}/tmp/submarine-cloud-v2/" -cp -r ${SUBMARINE_HOME}/helm-charts/submarine/charts/ "${CURRENT_PATH}/tmp/charts" cd ${CURRENT_PATH} echo "Start building the ${SUBMARINE_IMAGE_NAME} docker image ..." diff --git a/helm-charts/submarine/.helmignore b/helm-charts/submarine/.helmignore index b92c4a4..bd768d0 100644 --- a/helm-charts/submarine/.helmignore +++ b/helm-charts/submarine/.helmignore @@ -39,4 +39,3 @@ .vscode/ # ignore dolder ignore/* - diff --git a/helm-charts/submarine/Chart.yaml b/helm-charts/submarine/Chart.yaml index 3c595ec..f4feb97 100644 --- a/helm-charts/submarine/Chart.yaml +++ b/helm-charts/submarine/Chart.yaml @@ -22,12 +22,11 @@ name: submarine version: 0.7.0-SNAPSHOT icon: https://submarine.apache.org/assets/themes/submarine/img/submarine_white_logo.png dependencies: -- name: tfjob - version: "0.1.0" -- name: pytorchjob - version: "0.1.0" -- name: notebook-controller - version: "0.1.0" -- name: traefik - version: "9.1.0" - condition: submarine.traefik.enabled + - name: tfjob + version: "0.1.0" + - name: pytorchjob + version: "0.1.0" + - name: notebook-controller + version: "0.1.0" + - name: traefik + version: "9.1.0" diff --git a/submarine-cloud-v2/helm-charts/submarine-operator/crds/crd.yaml b/helm-charts/submarine/crds/crd.yaml similarity index 100% rename from submarine-cloud-v2/helm-charts/submarine-operator/crds/crd.yaml rename to helm-charts/submarine/crds/crd.yaml diff --git a/submarine-cloud-v2/helm-charts/submarine-operator/templates/_helpers.tpl b/helm-charts/submarine/templates/_helpers.tpl similarity index 100% rename from submarine-cloud-v2/helm-charts/submarine-operator/templates/_helpers.tpl rename to helm-charts/submarine/templates/_helpers.tpl diff --git a/helm-charts/submarine/templates/rbac.yaml b/helm-charts/submarine/templates/rbac.yaml index 0481c61..df95280 100644 --- a/helm-charts/submarine/templates/rbac.yaml +++ b/helm-charts/submarine/templates/rbac.yaml @@ -15,83 +15,94 @@ # limitations under the License. # +kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 -kind: Role metadata: - name: "{{ .Values.submarine.server.name }}" + name: submarine-operator rules: -- apiGroups: - - kubeflow.org - resources: - - tfjobs - - tfjobs/status - - pytorchjobs - - pytorchjobs/status - - notebooks - - notebooks/status - verbs: - - get - - list - - watch - - create - - delete - - deletecollection - - patch - - update -- apiGroups: - - traefik.containo.us - resources: - - ingressroutes - - middlewares - verbs: - - get - - list - - watch - - create - - delete - - deletecollection - - patch - - update -- apiGroups: - - machinelearning.seldon.io - resources: - - seldondeployments - verbs: - - get - - list - - watch - - create - - delete - - deletecollection - - patch - - update -- apiGroups: - - "" - resources: - - pods - - pods/log - - services - - persistentvolumeclaims - - events - verbs: - - '*' -- apiGroups: - - "apps" - resources: - - deployments - - deployments/status - verbs: - - '*' + - apiGroups: + - submarine.apache.org + resources: + - submarines + verbs: + - "*" + - apiGroups: + - traefik.containo.us + resources: + - ingressroutes + - ingressroutetcps + - ingressrouteudps + - middlewares + - tlsoptions + - tlsstores + - traefikservices + verbs: + - "*" + - apiGroups: + - kubeflow.org + resources: + - notebooks + - pytorchjobs + - tfjobs + verbs: + - "*" + - apiGroups: + - "" + resources: + - pods + - secrets + - configmaps + - services + - namespaces + - jobs + - serviceaccounts + - persistentvolumeclaims + - pods/portforward + - events + verbs: + - "*" + - apiGroups: + - "apps" + resources: + - deployments + - replicasets + verbs: + - "*" + - apiGroups: + - "extensions" + resources: + - ingresses + verbs: + - "*" + - apiGroups: + - "rbac.authorization.k8s.io" + resources: + - roles + - rolebindings + verbs: + - "*" + - apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - "*" +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: submarine-operator + namespace: default --- -kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding metadata: - name: "{{ .Values.submarine.server.name }}" -subjects: -- kind: ServiceAccount - namespace: {{ .Release.Namespace }} - name: "{{ .Values.submarine.server.name }}" + name: submarine-operator roleRef: - kind: Role - name: "{{ .Values.submarine.server.name }}" apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: submarine-operator +subjects: + - kind: ServiceAccount + name: submarine-operator + namespace: default diff --git a/helm-charts/submarine/templates/storageclass.yaml b/helm-charts/submarine/templates/storageclass.yaml index b439569..cc78b9c 100644 --- a/helm-charts/submarine/templates/storageclass.yaml +++ b/helm-charts/submarine/templates/storageclass.yaml @@ -14,11 +14,10 @@ # See the License for the specific language governing permissions and # limitations under the License. # + apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: submarine-storageclass - labels: - app: notebook-controller -provisioner: k8s.io/minikube-hostpath -reclaimPolicy: Delete +{{- template "storageClass.fields" . }} + diff --git a/helm-charts/submarine/templates/submarine-database.yaml b/helm-charts/submarine/templates/submarine-database.yaml deleted file mode 100644 index 530029b..0000000 --- a/helm-charts/submarine/templates/submarine-database.yaml +++ /dev/null @@ -1,97 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -apiVersion: v1 -kind: PersistentVolume -metadata: - name: submarine-database-pv -spec: - accessModes: - - ReadWriteMany - capacity: - storage: 1Gi -{{- with .Values.submarine.storage }} - {{- if eq (.type | lower) "nfs" }} - nfs: - server: {{ .nfs.ip }} - path: {{ .nfs.path }} - {{- else }} - hostPath: - path: "{{ .host.path }}" - type: DirectoryOrCreate - {{- end }} -{{- end}} ---- -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: submarine-database-pvc -spec: - accessModes: - - ReadWriteMany - storageClassName: "" - resources: - requests: - storage: 1Gi - volumeName: submarine-database-pv ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: "{{ .Values.submarine.database.name }}" -spec: - replicas: {{ .Values.submarine.database.replicas }} - selector: - matchLabels: - app: "{{ .Values.submarine.database.name }}" - template: - metadata: - labels: - app: "{{ .Values.submarine.database.name }}" - {{ if .Values.submarine.database.dev }} - annotations: - timestamp: {{ now | quote }} - {{ end }} - spec: - containers: - - name: "{{ .Values.submarine.database.name }}" - image: "{{ .Values.submarine.database.image }}" - imagePullPolicy: "{{ .Values.submarine.database.imagePullPolicy }}" - ports: - - containerPort: 3306 - env: - - name: MYSQL_ROOT_PASSWORD - value: "{{ .Values.submarine.database.mysqlRootPassword }}" - volumeMounts: - - mountPath: /var/lib/mysql - name: volume - subPath: submarine-database - volumes: - - name: volume - persistentVolumeClaim: - claimName: submarine-database-pvc ---- -apiVersion: v1 -kind: Service -metadata: - name: "{{ .Values.submarine.database.name }}" -spec: - ports: - - name: "{{ .Values.submarine.database.name }}" - port: 3306 - targetPort: {{ .Values.submarine.database.servicePort }} - selector: - app: "{{ .Values.submarine.database.name }}" diff --git a/helm-charts/submarine/templates/submarine-ingress.yaml b/helm-charts/submarine/templates/submarine-ingress.yaml deleted file mode 100644 index 6bddae3..0000000 --- a/helm-charts/submarine/templates/submarine-ingress.yaml +++ /dev/null @@ -1,16 +0,0 @@ -{{ if .Values.submarine.traefik.enabled }} -apiVersion: extensions/v1beta1 -kind: Ingress -metadata: - name: {{ .Values.submarine.server.name}}-ingress - namespace: {{ .Release.namespace }} - -spec: - rules: - - http: - paths: - - path: / - backend: - serviceName: {{ .Values.submarine.server.name }} - servicePort: {{ .Values.submarine.server.servicePort }} -{{ end }} diff --git a/helm-charts/submarine/templates/submarine-minio.yaml b/helm-charts/submarine/templates/submarine-minio.yaml deleted file mode 100644 index ed3d8ce..0000000 --- a/helm-charts/submarine/templates/submarine-minio.yaml +++ /dev/null @@ -1,114 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -apiVersion: v1 -kind: PersistentVolume -metadata: - name: submarine-minio-pv -spec: - accessModes: - - ReadWriteMany - capacity: - storage: "{{ .Values.submarine.minio.storage }}" -{{- with .Values.submarine.storage }} - {{- if eq (.type | lower) "nfs" }} - nfs: - server: {{ .nfs.ip }} - path: {{ .nfs.path }} - {{- else }} - hostPath: - path: "{{ .host.path }}" - type: DirectoryOrCreate - {{- end }} -{{- end}} ---- -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: submarine-minio-pvc -spec: - accessModes: - - ReadWriteMany - storageClassName: "" - resources: - requests: - storage: "{{ .Values.submarine.minio.storage }}" - volumeName: submarine-minio-pv ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: submarine-minio -spec: - selector: - matchLabels: - app: submarine-minio-pod - template: - metadata: - labels: - app: submarine-minio-pod - spec: - containers: - - name: submarine-minio-container - image: minio/minio:RELEASE.2021-02-14T04-01-33Z - imagePullPolicy: IfNotPresent - args: - - server - - /data - env: - - name: MINIO_ACCESS_KEY - value: "submarine_minio" - - name: MINIO_SECRET_KEY - value: "submarine_minio" - ports: - - containerPort: 9000 - volumeMounts: - - mountPath: "/data" - name: "volume" - subPath: "submarine-minio" - volumes: - - name: "volume" - persistentVolumeClaim: - claimName: "submarine-minio-pvc" ---- -apiVersion: v1 -kind: Service -metadata: - name: submarine-minio-service -spec: - type: ClusterIP - selector: - app: submarine-minio-pod - ports: - - protocol: TCP - port: 9000 - targetPort: 9000 ---- -apiVersion: traefik.containo.us/v1alpha1 -kind: IngressRoute -metadata: - name: submarine-minio-ingressroute -spec: - entryPoints: - - web - routes: - - kind: Rule - match: "PathPrefix(`{{ .Values.submarine.minio.ingressPath }}`)" - services: - - kind: Service - name: submarine-minio-service - port: 9000 diff --git a/helm-charts/submarine/templates/submarine-mlflow.yaml b/helm-charts/submarine/templates/submarine-mlflow.yaml deleted file mode 100644 index 606e8fb..0000000 --- a/helm-charts/submarine/templates/submarine-mlflow.yaml +++ /dev/null @@ -1,131 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -apiVersion: v1 -kind: PersistentVolume -metadata: - name: submarine-mlflow-pv -spec: - accessModes: - - ReadWriteMany - capacity: - storage: "{{ .Values.submarine.mlflow.storage }}" -{{- with .Values.submarine.storage }} - {{- if eq (.type | lower) "nfs" }} - nfs: - server: {{ .nfs.ip }} - path: {{ .nfs.path }} - {{- else }} - hostPath: - path: "{{ .host.path }}" - type: DirectoryOrCreate - {{- end }} -{{- end}} ---- -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: submarine-mlflow-pvc -spec: - accessModes: - - ReadWriteMany - storageClassName: "" - resources: - requests: - storage: "{{ .Values.submarine.mlflow.storage }}" - volumeName: submarine-mlflow-pv ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: submarine-mlflow -spec: - selector: - matchLabels: - app: submarine-mlflow-pod - template: - metadata: - labels: - app: submarine-mlflow-pod - spec: - initContainers: - - name: submarine-mlflow-initcontainer - image: "minio/mc" - command: ["/bin/bash", "-c", - "cnt=0; - while ! /bin/bash -c 'mc config host add minio http://submarine-minio-service:9000 - submarine_minio submarine_minio' 2>&1; - do - sleep 15; - ((cnt=cnt+1)); - if [ $cnt -eq 80 ];then - echo 'ERROR: wait too long for minio pod'; - exit 1; - fi; - done; - if /bin/bash -c 'mc ls minio/mlflow' >/dev/null 2>&1; then - echo 'Bucket minio/mlflow already exists, skipping creation.'; - else - /bin/bash -c 'mc mb minio/mlflow'; - fi;"] - containers: - - name: submarine-mlflow-container - image: apache/submarine:mlflow-0.7.0-SNAPSHOT - imagePullPolicy: IfNotPresent - ports: - - containerPort: 5000 - volumeMounts: - - mountPath: "/logs" - name: "volume" - subPath: "submarine-mlflow" - readinessProbe: - tcpSocket: - port: 5000 - initialDelaySeconds: 60 - periodSeconds: 10 - volumes: - - name: "volume" - persistentVolumeClaim: - claimName: "submarine-mlflow-pvc" ---- -apiVersion: v1 -kind: Service -metadata: - name: submarine-mlflow-service -spec: - type: ClusterIP - selector: - app: submarine-mlflow-pod - ports: - - protocol: TCP - port: 5000 - targetPort: 5000 ---- -apiVersion: traefik.containo.us/v1alpha1 -kind: IngressRoute -metadata: - name: submarine-mlflow-ingressroute -spec: - entryPoints: - - web - routes: - - kind: Rule - match: "PathPrefix(`{{ .Values.submarine.mlflow.ingressPath }}`)" - services: - - kind: Service - name: submarine-mlflow-service - port: 5000 diff --git a/submarine-cloud-v2/helm-charts/submarine-operator/templates/submarine-operator.yaml b/helm-charts/submarine/templates/submarine-operator.yaml similarity index 100% rename from submarine-cloud-v2/helm-charts/submarine-operator/templates/submarine-operator.yaml rename to helm-charts/submarine/templates/submarine-operator.yaml diff --git a/helm-charts/submarine/templates/submarine-server.yaml b/helm-charts/submarine/templates/submarine-server.yaml deleted file mode 100644 index 697f832..0000000 --- a/helm-charts/submarine/templates/submarine-server.yaml +++ /dev/null @@ -1,92 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -apiVersion: v1 -kind: Service -metadata: - name: "{{ .Values.submarine.server.name }}" - labels: - run: "{{ .Values.submarine.server.name }}" -spec: - ports: - - port: 8080 - targetPort: {{ .Values.submarine.server.servicePort }} - protocol: TCP - selector: - run: "{{ .Values.submarine.server.name }}" - ---- -apiVersion: v1 -kind: ServiceAccount -metadata: - name: "{{ .Values.submarine.server.name }}" ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: "{{ .Values.submarine.server.name }}" -spec: - selector: - matchLabels: - run: "{{ .Values.submarine.server.name }}" - replicas: {{ .Values.submarine.server.replicas }} - template: - metadata: - labels: - run: "{{ .Values.submarine.server.name }}" - {{ if .Values.submarine.server.dev }} - annotations: - timestamp: {{ now | quote }} - {{ end }} - spec: - serviceAccountName: "{{ .Values.submarine.server.name }}" - initContainers: - - name: submarine-server-initcontainer - image: "minio/mc" - command: ["/bin/bash", "-c", - "cnt=0; - while ! /bin/bash -c 'mc config host add minio http://submarine-minio-service:9000 - submarine_minio submarine_minio' 2>&1; - do - sleep 15; - ((cnt=cnt+1)); - if [ $cnt -eq 80 ];then - echo 'ERROR: wait too long for minio pod'; - exit 1; - fi; - done; - if /bin/bash -c 'mc ls minio/submarine' >/dev/null 2>&1; then - echo 'Bucket minio/submarine already exists, skipping creation.'; - else - /bin/bash -c 'mc mb minio/submarine'; - fi;"] - containers: - - name: "{{ .Values.submarine.server.name }}" - env: - - name: SUBMARINE_SERVER_PORT - value: "8080" - - name: SUBMARINE_SERVER_PORT_8080_TCP - value: "8080" - - name: SUBMARINE_SERVER_DNS_NAME - value: "{{ .Values.submarine.server.name }}.{{ .Release.Namespace }}" - - name: K8S_APISERVER_URL - value: "kubernetes.default.svc" - - image: "{{ .Values.submarine.server.image }}" - imagePullPolicy: {{ .Values.submarine.server.imagePullPolicy }} - ports: - - containerPort: 8080 \ No newline at end of file diff --git a/helm-charts/submarine/templates/submarine-tensorboard.yaml b/helm-charts/submarine/templates/submarine-tensorboard.yaml deleted file mode 100644 index f03490a..0000000 --- a/helm-charts/submarine/templates/submarine-tensorboard.yaml +++ /dev/null @@ -1,112 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -apiVersion: v1 -kind: PersistentVolume -metadata: - name: submarine-tensorboard-pv -spec: - accessModes: - - ReadWriteMany - capacity: - storage: "{{ .Values.submarine.tensorboard.storage }}" -{{- with .Values.submarine.storage }} - {{- if eq (.type | lower) "nfs" }} - nfs: - server: {{ .nfs.ip }} - path: {{ .nfs.path }} - {{- else }} - hostPath: - path: "{{ .host.path }}" - type: DirectoryOrCreate - {{- end }} -{{- end}} ---- -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: submarine-tensorboard-pvc -spec: - accessModes: - - ReadWriteMany - storageClassName: "" - resources: - requests: - storage: "{{ .Values.submarine.tensorboard.storage }}" - volumeName: submarine-tensorboard-pv # bind to specific pv ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: submarine-tensorboard -spec: - selector: - matchLabels: - app: submarine-tensorboard-pod - template: - metadata: - labels: - app: submarine-tensorboard-pod - spec: - containers: - - name: submarine-tensorboard-container - image: tensorflow/tensorflow:1.11.0 - command: - - "tensorboard" - - "--logdir=/logs" - - "--path_prefix={{ .Values.submarine.tensorboard.ingressPath }}" - imagePullPolicy: IfNotPresent - ports: - - containerPort: 6006 - volumeMounts: - - mountPath: "/logs" - name: "volume" - subPath: "submarine-tensorboard" - readinessProbe: - tcpSocket: - port: 6006 - periodSeconds: 10 - volumes: - - name: "volume" - persistentVolumeClaim: - claimName: "submarine-tensorboard-pvc" ---- -apiVersion: v1 -kind: Service -metadata: - name: submarine-tensorboard-service -spec: - selector: - app: submarine-tensorboard-pod - ports: - - protocol: TCP - port: 8080 - targetPort: 6006 ---- -apiVersion: traefik.containo.us/v1alpha1 -kind: IngressRoute -metadata: - name: submarine-tensorboard-ingressroute -spec: - entryPoints: - - web - routes: - - kind: Rule - match: "PathPrefix(`{{ .Values.submarine.tensorboard.ingressPath }}`)" - services: - - kind: Service - name: submarine-tensorboard-service - port: 8080 diff --git a/helm-charts/submarine/values.yaml b/helm-charts/submarine/values.yaml index 66e8a3f..76d1397 100644 --- a/helm-charts/submarine/values.yaml +++ b/helm-charts/submarine/values.yaml @@ -15,37 +15,18 @@ # limitations under the License. # -submarine: - server: - imagePullPolicy: IfNotPresent - replicas: 1 - name: submarine-server - image: apache/submarine:server-0.7.0-SNAPSHOT - servicePort: 8080 - dev: false # if true, restart server pod every time at helm upgrade - database: - imagePullPolicy: IfNotPresent - replicas: 1 - name: submarine-database - image: apache/submarine:database-0.7.0-SNAPSHOT - servicePort: 3306 - mysqlRootPassword: password - dev: false # if true, restart database pod every time at helm upgrade - traefik: - enabled: true - tensorboard: - storage: 10Gi - ingressPath: "/tensorboard" - mlflow: - storage: 10Gi - ingressPath: "/mlflow" - minio: - storage: 10Gi - ingressPath: "/minio" - storage: - type: host # "host" or "nfs" - host: - path: "/tmp/submarine/host" - nfs: - ip: 10.96.0.2 - path: "/" +name: submarine-operator +replicas: 1 +image: apache/submarine:operator-0.7.0-SNAPSHOT +# dev is to tell helm to install submarine-operator or not +dev: false +# storageClass is for dynamically creating persistent volumes +storageClass: + # reclaimPolicy is to determine the action after the persistent volume is released + reclaimPolicy: Delete + # volumeBindingMode controls when volume binding and dynamically provisioning should occur + volumeBindingMode: Immediate + # provisioner is to determine what volume plugin is used for provisioning PVs + provisioner: k8s.io/minikube-hostpath + # parameters describe volumes belonging to the storage class + parameters: diff --git a/submarine-cloud-v2/Makefile b/submarine-cloud-v2/Makefile index 58ee38d..0c86815 100644 --- a/submarine-cloud-v2/Makefile +++ b/submarine-cloud-v2/Makefile @@ -27,9 +27,7 @@ api: .PHONY: image image: - GOOS=linux go build -o submarine-operator - docker build -t "apache/submarine:operator-0.7.0-SNAPSHOT" -f dev.Dockerfile . - go build -o submarine-operator + cd ..; ./dev-support/docker-images/operator/build.sh .PHONY: test-unit test-unit: diff --git a/submarine-cloud-v2/README.md b/submarine-cloud-v2/README.md index e8f2b0f..17fc821 100644 --- a/submarine-cloud-v2/README.md +++ b/submarine-cloud-v2/README.md @@ -26,8 +26,6 @@ ## Initialization ```bash -# Add helm-chart dependencies -cp -r ../helm-charts/submarine/charts ./helm-charts/submarine-operator/ # Install dependencies go mod vendor # Run the cluster @@ -46,7 +44,7 @@ Documentation for storage class: https://kubernetes.io/docs/concepts/storage/sto ```bash # Step1: Install helm chart dependencies -helm install --set dev=true submarine-operator ./helm-charts/submarine-operator/ +helm install --set dev=true submarine ../helm-charts/submarine/ # Step2: Build & Run "submarine-operator" make @@ -61,7 +59,7 @@ kubectl apply -n submarine-user-test -f artifacts/examples/example-submarine.yam minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2 # Method2 -- use port-forwarding -kubectl port-forward --address 0.0.0.0 service/submarine-operator-traefik 32080:80 +kubectl port-forward --address 0.0.0.0 service/submarine-traefik 32080:80 # Step5: View Workbench # http://{minikube ip}:32080 (from Method 1), ex: http://192.168.49.2:32080 @@ -79,14 +77,14 @@ kubectl delete submarine example-submarine -n submarine-user-test # Press ctrl+c to stop the operator # Step7: Uninstall helm chart dependencies -helm delete submarine-operator +helm delete submarine ``` ## Run operator in-cluster ```bash # Step1: Install submarine-operator -helm install submarine-operator ./helm-charts/submarine-operator/ +helm install submarine ../helm-charts/submarine/ # Step2: Deploy a submarine kubectl create ns submarine-user-test @@ -100,7 +98,7 @@ kubectl logs -f $(kubectl get pods --output=name | grep submarine-operator) minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2 # Method2 -- use port-forwarding -kubectl port-forward --address 0.0.0.0 service/submarine-operator-traefik 32080:80 +kubectl port-forward --address 0.0.0.0 service/submarine-traefik 32080:80 # Step5: View Workbench # http://{minikube ip}:32080 (from Method 1), ex: http://192.168.49.2:32080 @@ -115,7 +113,7 @@ kubectl port-forward --address 0.0.0.0 service/submarine-operator-traefik 32080: kubectl delete submarine example-submarine -n submarine-user-test # Step7: Delete the submarine-operator -helm delete submarine-operator +helm delete submarine ``` # Development diff --git a/submarine-cloud-v2/dev.Dockerfile b/submarine-cloud-v2/dev.Dockerfile deleted file mode 100644 index 3fb3812..0000000 --- a/submarine-cloud-v2/dev.Dockerfile +++ /dev/null @@ -1,39 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM ubuntu:18.04 -MAINTAINER Apache Software Foundation <dev@submarine.apache.org> - -WORKDIR /usr/src - -RUN apt-get update &&\ - apt-get install -y wget vim git curl - -ENV GOROOT="/usr/local/go" -ENV GOPATH=$HOME/gocode -ENV GOBIN=$GOPATH/bin -ENV PATH=$PATH:$GOPATH:$GOBIN:$GOROOT/bin - -RUN wget https://dl.google.com/go/go1.16.2.linux-amd64.tar.gz &&\ - tar -C /usr/local -xzf go1.16.2.linux-amd64.tar.gz - -RUN curl -LO https://dl.k8s.io/release/v1.14.2/bin/linux/amd64/kubectl &&\ - install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl &&\ - kubectl version --client - -ADD submarine-operator /usr/src -COPY ["./artifacts", "/usr/src/artifacts"] - -CMD ["/usr/src/submarine-operator", "-incluster=true"] diff --git a/submarine-cloud-v2/docs/developer-guide.md b/submarine-cloud-v2/docs/developer-guide.md index f01a21d..8ad1c26 100644 --- a/submarine-cloud-v2/docs/developer-guide.md +++ b/submarine-cloud-v2/docs/developer-guide.md @@ -82,7 +82,7 @@ make image eval $(minikube docker-env -u) # Update the operator pod -helm upgrade submarine-operator ./helm-charts/submarine-operator +helm upgrade submarine ../helm-charts/submarine ``` ## Rebuild Images for Other Components diff --git a/submarine-cloud-v2/helm-charts/submarine-operator/.helmignore b/submarine-cloud-v2/helm-charts/submarine-operator/.helmignore deleted file mode 100644 index cab781a..0000000 --- a/submarine-cloud-v2/helm-charts/submarine-operator/.helmignore +++ /dev/null @@ -1,40 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -# Patterns to ignore when building packages. -# This supports shell glob matching, relative path matching, and -# negation (prefixed with !). Only one pattern per line. -.DS_Store -# Common VCS dirs -.git/ -.gitignore -.bzr/ -.bzrignore -.hg/ -.hgignore -.svn/ -# Common backup files -*.swp -*.bak -*.tmp -*.orig -*~ -# Various IDEs -.project -.idea/ -*.tmproj -.vscode/ diff --git a/submarine-cloud-v2/helm-charts/submarine-operator/Chart.yaml b/submarine-cloud-v2/helm-charts/submarine-operator/Chart.yaml deleted file mode 100644 index c583a5d..0000000 --- a/submarine-cloud-v2/helm-charts/submarine-operator/Chart.yaml +++ /dev/null @@ -1,32 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -apiVersion: v2 -name: submarine-operator -description: A Helm chart for Submarine operator -type: application -version: 0.1.0 -appVersion: "0.7.0-SNAPSHOT" -dependencies: - - name: tfjob - version: "0.1.0" - - name: pytorchjob - version: "0.1.0" - - name: notebook-controller - version: "0.1.0" - - name: traefik - version: "9.1.0" diff --git a/submarine-cloud-v2/helm-charts/submarine-operator/templates/rbac.yaml b/submarine-cloud-v2/helm-charts/submarine-operator/templates/rbac.yaml deleted file mode 100644 index df95280..0000000 --- a/submarine-cloud-v2/helm-charts/submarine-operator/templates/rbac.yaml +++ /dev/null @@ -1,108 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -kind: ClusterRole -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: submarine-operator -rules: - - apiGroups: - - submarine.apache.org - resources: - - submarines - verbs: - - "*" - - apiGroups: - - traefik.containo.us - resources: - - ingressroutes - - ingressroutetcps - - ingressrouteudps - - middlewares - - tlsoptions - - tlsstores - - traefikservices - verbs: - - "*" - - apiGroups: - - kubeflow.org - resources: - - notebooks - - pytorchjobs - - tfjobs - verbs: - - "*" - - apiGroups: - - "" - resources: - - pods - - secrets - - configmaps - - services - - namespaces - - jobs - - serviceaccounts - - persistentvolumeclaims - - pods/portforward - - events - verbs: - - "*" - - apiGroups: - - "apps" - resources: - - deployments - - replicasets - verbs: - - "*" - - apiGroups: - - "extensions" - resources: - - ingresses - verbs: - - "*" - - apiGroups: - - "rbac.authorization.k8s.io" - resources: - - roles - - rolebindings - verbs: - - "*" - - apiGroups: - - apiextensions.k8s.io - resources: - - customresourcedefinitions - verbs: - - "*" ---- -apiVersion: v1 -kind: ServiceAccount -metadata: - name: submarine-operator - namespace: default ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: submarine-operator -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: submarine-operator -subjects: - - kind: ServiceAccount - name: submarine-operator - namespace: default diff --git a/submarine-cloud-v2/helm-charts/submarine-operator/templates/storageclass.yaml b/submarine-cloud-v2/helm-charts/submarine-operator/templates/storageclass.yaml deleted file mode 100644 index cc78b9c..0000000 --- a/submarine-cloud-v2/helm-charts/submarine-operator/templates/storageclass.yaml +++ /dev/null @@ -1,23 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -apiVersion: storage.k8s.io/v1 -kind: StorageClass -metadata: - name: submarine-storageclass -{{- template "storageClass.fields" . }} - diff --git a/submarine-cloud-v2/helm-charts/submarine-operator/values.yaml b/submarine-cloud-v2/helm-charts/submarine-operator/values.yaml deleted file mode 100644 index 690eec6..0000000 --- a/submarine-cloud-v2/helm-charts/submarine-operator/values.yaml +++ /dev/null @@ -1,32 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -name: submarine-operator-demo -replicas: 1 -image: apache/submarine:operator-0.7.0-SNAPSHOT -# dev is to tell helm to install submarine-operator or not -dev: false -# storageClass is for dynamically creating persistent volumes -storageClass: - # reclaimPolicy is to determine the action after the persistent volume is released - reclaimPolicy: Delete - # volumeBindingMode controls when volume binding and dynamically provisioning should occur - volumeBindingMode: Immediate - # provisioner is to determine what volume plugin is used for provisioning PVs - provisioner: k8s.io/minikube-hostpath - # parameters describe volumes belonging to the storage class - parameters: diff --git a/website/docs/devDocs/Development.md b/website/docs/devDocs/Development.md index 5c2b4a6..6f8a83f 100644 --- a/website/docs/devDocs/Development.md +++ b/website/docs/devDocs/Development.md @@ -79,7 +79,7 @@ Checkstyle plugin may help to detect violations directly from the IDE. 1. Package the Submarine server into a new jar file ```bash - mvn package -DskipTests + mvn install -DskipTests ``` 2. Build the new server docker image in minikube @@ -95,14 +95,12 @@ Checkstyle plugin may help to detect violations directly from the IDE. eval $(minikube docker-env -u) ``` -3. Update server pod +3. Delete the server deployment and the operator will create a new one using the new image ```bash - helm upgrade --set submarine.server.dev=true submarine ./helm-charts/submarine + kubectl delete deployment submarine-server ``` - Set `submarine.server.dev` to `true`, enabling the server pod to be launched with the new docker image. - ## Develop workbench 1. Deploy the Submarine @@ -147,75 +145,9 @@ Checkstyle plugin may help to detect violations directly from the IDE. ```bash helm upgrade --set submarine.database.dev=true submarine ./helm-charts/submarine ``` - ## Develop operator - -- Before building - - 1. We assume the developer use **minikube** as a local kubernetes cluster. - 2. Make sure you have **NOT** installed the submarine helm-chart in the cluster. - -1. Start the minikube cluster - - ```bash - minikube start --vm-driver=docker --kubernetes-version v1.15.11 - ``` - -2. Install the dependencies - - ```bash - cd submarine-cloud-v2/ - cp -r ../helm-charts/submarine/charts ./helm-charts/submarine-operator/ - go mod vendor - helm install --set dev=true submarine-operator ./helm-charts/submarine-operator/ - ``` - -3. Run the operator out-of-cluster - - ```bash - make - ./submarine-operator - ``` - -4. Deploy a Submarine - - ```bash - kubectl apply -f artifacts/examples/crd.yaml - kubectl create ns submarine-user-test - kubectl apply -n submarine-user-test -f artifacts/examples/example-submarine.yaml - ``` - -5. Exposing service - - ```bash - # Method1 -- use minikube ip - minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2 - - # Method2 -- use port-forwarding - kubectl port-forward --address 0.0.0.0 -n submarine-user-test service/traefik 32080:80 - ``` - -6. View workbench - - If you use method 1 in step 5, please go to `http://{minikube ip}:32080`, ex: http://192.168.49.2:32080 - - If you use method 2 in step 5, please go to http://127.0.0.1:32080 - -7. Delete submarine - - ```bash - kubectl delete submarine example-submarine -n submarine-user-test - ``` - -8. Stop the operator - - Press ctrl+c to stop the operator. - -9. Uninstall helm chart dependencies - ```bash - helm delete submarine-operator - ``` +## Develop operator -For other details, please check out the [README](https://github.com/apache/submarine/blob/master/submarine-cloud-v2/README.md) and [Developer Guide](https://github.com/apache/submarine/blob/master/submarine-cloud-v2/docs/developer-guide.md) on GitHub. +For details, please check out the [README](https://github.com/apache/submarine/blob/master/submarine-cloud-v2/README.md) and [Developer Guide](https://github.com/apache/submarine/blob/master/submarine-cloud-v2/docs/developer-guide.md) on GitHub. ## Develop Submarine Website Submarine website is built using [Docusaurus 2](https://v2.docusaurus.io/), a modern static website generator. diff --git a/website/docs/gettingStarted/quickstart.md b/website/docs/gettingStarted/quickstart.md index 2912e24..1f4f2d4 100644 --- a/website/docs/gettingStarted/quickstart.md +++ b/website/docs/gettingStarted/quickstart.md @@ -36,54 +36,63 @@ This document gives you a quick view on the basic usage of Submarine platform. Y 2. Start minikube cluster ``` -$ minikube start --vm-driver=docker --cpus 8 --memory 4096 --kubernetes-version v1.15.11 +minikube start --vm-driver=docker --cpus 8 --memory 4096 --kubernetes-version v1.15.11 ``` ### Launch submarine in the cluster 1. Clone the project ``` -$ git clone https://github.com/apache/submarine.git +git clone https://github.com/apache/submarine.git ``` -2. Install the resources by helm chart +2. Install the submarine operator and dependencies by helm chart ``` -$ cd submarine -$ helm install submarine ./helm-charts/submarine +cd submarine +helm install submarine ./helm-charts/submarine ``` + +3. Create a Submarine custom resource and the operator will create the submarine server, database, etc. for us. +``` +kubectl apply -f submarine-cloud-v2/artifacts/examples/example-submarine.yaml +``` + ### Ensure submarine is ready 1. Use kubectl to query the status of pods ``` -$ kubectl get pods +kubectl get pods ``` 2. Make sure each pod is `Running` ``` NAME READY STATUS RESTARTS AGE -notebook-controller-deployment-5d4f5f874c-vwds8 1/1 Running 0 3h33m -pytorch-operator-844c866d54-q5ztd 1/1 Running 0 3h33m -submarine-database-674987ff7d-r8zqs 1/1 Running 0 3h33m -submarine-minio-5fdd957785-xd987 1/1 Running 0 3h33m -submarine-mlflow-76bbf5c7b-g2ntd 1/1 Running 0 3h33m -submarine-server-66f7b8658b-sfmv8 1/1 Running 0 3h33m -submarine-tensorboard-6c44944dfb-tvbr9 1/1 Running 0 3h33m -submarine-traefik-7cbcfd4bd9-4bczn 1/1 Running 0 3h33m -tf-job-operator-6bb69fd44-mc8ww 1/1 Running 0 3h33m +notebook-controller-deployment-5d4f5f874c-mnbc8 1/1 Running 0 61m +pytorch-operator-844c866d54-xm8nl 1/1 Running 2 61m +submarine-database-85bd68dbc5-qggtm 1/1 Running 0 11m +submarine-minio-76465444f6-hdgdp 1/1 Running 0 11m +submarine-mlflow-75f86d8f4d-rj2z7 1/1 Running 0 11m +submarine-operator-5dd79cdf86-gpm2p 1/1 Running 0 61m +submarine-server-68985b767-vjdvx 1/1 Running 0 11m +submarine-tensorboard-5df8499fd4-vnklf 1/1 Running 0 11m +submarine-traefik-7cbcfd4bd9-wbf8b 1/1 Running 0 61m +tf-job-operator-6bb69fd44-zmlmr 1/1 Running 1 61m ``` ### Connect to workbench -1. Port-forwarding - -``` -# using port-forwarding -$ kubectl port-forward --address 0.0.0.0 service/submarine-traefik 32080:80 -``` - -2. Open `http://0.0.0.0:32080` +1. Exposing service + ``` + # Method 1 -- use minikube ip + minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2 + + # Method 2 -- use port-forwarding + kubectl port-forward --address 0.0.0.0 service/submarine-traefik 32080:80 + ``` -![](/img/quickstart-worbench.png) +2. View workbench + If you use method 1, go to `http://{minikube ip}:32080`. For example, `http://192.168.49.2:32080`. If you use method 2, go to `http://0.0.0.0:32080`. + ![](/img/quickstart-worbench.png) ## Example: Submit a mnist distributed example @@ -173,7 +182,7 @@ if __name__ == '__main__': Build a docker image equipped with the requirement of the environment. ```bash -$ ./dev-support/examples/quickstart/build.sh +./dev-support/examples/quickstart/build.sh ``` ### 3. Submit the experiment --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org