This is an automated email from the ASF dual-hosted git repository.
kaihsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 65c21f2 SUBMARINE-917. Update adminDocs directory structure
65c21f2 is described below
commit 65c21f23a4f4af299e2bfa0988a4e14ea988c22b
Author: Kai-Hsun Chen <[email protected]>
AuthorDate: Sat Jul 10 22:53:20 2021 +0800
SUBMARINE-917. Update adminDocs directory structure
### What is this PR for?
Update adminDocs directory structure
https://github.com/apache/submarine/tree/master/website/docs/adminDocs
### What type of PR is it?
[Documentation]
### Todos
* List all development dependencies in ***dependencies.md***.
* ***helm.md***
* Screenshot
* helm chart configuration (values.yaml)
* ***kind***
* How to setup ***kind*** Kubernetes environment
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-917
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? No
Author: Kai-Hsun Chen <[email protected]>
Signed-off-by: Kai-Hsun Chen <[email protected]>
Closes #659 from kevin85421/SUBMARINE-917 and squashes the following
commits:
fb339652 [Kai-Hsun Chen] Fix website error
bd7e7f35 [Kai-Hsun Chen] Fix website error
46741cd4 [Kai-Hsun Chen] Update adminDocs directory structure
---
.../adminDocs/k8s/run-tensorflow-experiment-ui.md | 56 ----------------
website/docs/adminDocs/k8s/tensorflow.md | 54 ---------------
.../k8s/README.md => devDocs/Dependencies.md} | 25 +++----
.../docs/{adminDocs/k8s => gettingStarted}/helm.md | 78 +++++++---------------
.../docs/{adminDocs/k8s => gettingStarted}/kind.md | 0
website/sidebars.js | 7 --
6 files changed, 34 insertions(+), 186 deletions(-)
diff --git a/website/docs/adminDocs/k8s/run-tensorflow-experiment-ui.md
b/website/docs/adminDocs/k8s/run-tensorflow-experiment-ui.md
deleted file mode 100644
index f68a878..0000000
--- a/website/docs/adminDocs/k8s/run-tensorflow-experiment-ui.md
+++ /dev/null
@@ -1,56 +0,0 @@
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Run Tensorflow Experiment on Submarine UI
-
-## Steps to run Tensorflow Experiment
-
-- Click `+ New Experiment` on the "Experiment" page.
-
-- Click `Define your experiment`
-
-- Put a name to experiment, like "minst-example"
-
-- Command: `python /var/tf_mnist/mnist_with_summaries.py --log_dir=/train/log
--learning_rate=0.01 --batch_size=150`
-
-- Image you can put; `apache/submarine:tf-mnist-with-summaries-1.0`
-
-- Click `Next Step`
-
-- Choose `Distributed Tensorflow`
-
-- Click `Add new spec` twice to add two new specs (roles)
-
-- One is Worker, another one is PS, leave rest of the parameters unchanged
-
-- Click next step, you can review your parameters before submitting the job:
-
-
-
- It should look like below:
-
- | Name | mnist-example-111
| | |
- | --------------------- |
------------------------------------------------------------ | ---- | ---- |
- | Command | python /var/tf_mnist/mnist_with_summaries.py
--log_dir=/train/log --learning_rate=0.01 --batch_size=150 | | |
- | Image | apache/submarine:tf-mnist-with-summaries-1.0
| | |
- | Environment Variables |
| | |
- | Ps | cpu=1,nvidia.com/gpu=0,memory=1024M
| | |
- | Worker | cpu=1,nvidia.com/gpu=0,memory=1024M
| | |
-
-- Click `Submit` it will be submitted, you can see the new example running in
the `Experiment` list, you can get logs, etc. directly from the UI
\ No newline at end of file
diff --git a/website/docs/adminDocs/k8s/tensorflow.md
b/website/docs/adminDocs/k8s/tensorflow.md
deleted file mode 100644
index e3cd2ba..0000000
--- a/website/docs/adminDocs/k8s/tensorflow.md
+++ /dev/null
@@ -1,54 +0,0 @@
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Deploy Tensorflow Operator on Kubernetes
-
-## TFJob
-We support Tensorflow job on kubernetes by using the tf-operator as a runtime.
For more info about tf-operator see
[here](https://github.com/kubeflow/tf-operator).
-
-### Deploy tf-operator
-> If you don't have the `submarine` namespace on your K8s cluster, you should
create it first. Run command: `kubectl create namespace submarine`
-
-Running the follow commands:
-```
-kubectl apply -f ./dev-support/k8s/tfjob/crd.yaml
-kubectl kustomize ./dev-support/k8s/tfjob/operator | kubectl apply -f -
-```
-
-> Since K8s 1.14, Kubectl also supports the management of Kubernetes objects
using a kustomization file. For more info see
[kustomization](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/)
-
-Default namespace is `submarine`, if you want to modify the namespace, please
modify `./dev-support/k8s/tfjob/operator/kustomization.yaml`, such as modify
`${NAMESPACE}` as below:
-```yaml
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-namespace: ${NAMESPACE}
-resources:
-- cluster-role-binding.yaml
-- cluster-role.yaml
-- deployment.yaml
-- service-account.yaml
-- service.yaml
-commonLabels:
- kustomize.component: tf-job-operator
-images:
-- name: apache/submarine
- newName: apache/submarine
- newTag: tf_operator-v1.1.0-g92389064
-```
-
diff --git a/website/docs/adminDocs/k8s/README.md
b/website/docs/devDocs/Dependencies.md
similarity index 52%
rename from website/docs/adminDocs/k8s/README.md
rename to website/docs/devDocs/Dependencies.md
index 46e4ac5..cc4b500 100644
--- a/website/docs/adminDocs/k8s/README.md
+++ b/website/docs/devDocs/Dependencies.md
@@ -1,5 +1,5 @@
---
-title: Submarine on K8s
+title: Dependencies for Submarine
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
@@ -19,13 +19,11 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
-Submarine for K8s supports distributed TensorFlow and PyTorch.
+# Dependencies
+* These are the dependencies currently used by Apache Submarine.
-Submarine can run on K8s >= `1.14`, supports features like GPU isolation.
-
-We have validated Submarine on the following versions:
-
-| K8s Version | Support? |
+# Kubernetes
+| Kubernetes Version | Support? |
| ------------- |:-------------:|
| 1.13.x (or earlier) | X |
| 1.14.x | √ |
@@ -34,12 +32,11 @@ We have validated Submarine on the following versions:
| 1.17.x | To be verified |
| 1.18.x | To be verified |
-## Install Submarine
-
-### Setup Kubernetes
-Submarine can be deployed on any K8s environment if version matches. If you
don't have a running K8s, you can set up a K8s using [Docker
Desktop](https://www.docker.com/products/docker-desktop),
[MiniKube](https://kubernetes.io/docs/tasks/tools/install-minikube/), or [kind,
Kubernetes-in-Docker](https://kind.sigs.k8s.io/).
+# Java
+* TODO
-From our experiences, Docker Desktop is an easier choice.
+# Maven
+* TODO
-### Install Submarine Use Helm Charts
-After you have an up-and-running K8s, you can follow [Submarine Helm Charts
Guide](helm) to deploy Submarine services on K8s cluster in minutes.
+# Docker
+* TODO
\ No newline at end of file
diff --git a/website/docs/adminDocs/k8s/helm.md
b/website/docs/gettingStarted/helm.md
similarity index 61%
rename from website/docs/adminDocs/k8s/helm.md
rename to website/docs/gettingStarted/helm.md
index 6a00471..a33cefa 100644
--- a/website/docs/adminDocs/k8s/helm.md
+++ b/website/docs/gettingStarted/helm.md
@@ -1,5 +1,5 @@
---
-title: Deploy Submarine On K8s
+title: Deploy Submarine with Helm
---
<!--
@@ -20,66 +20,40 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
+# Deploy Submarine with Helm
+With the help of [Helm](https://helm.sh/), users can deploy Submarine service
to Kubernetes in one command.
+Check
[helm-charts/submarine](https://github.com/apache/submarine/tree/master/helm-charts/submarine)
for more details.
-## Deploy Submarine Using Helm Chart (Recommended)
+## Prerequisite
+* Install Helm v3: https://helm.sh/docs/intro/install/
+* A Kubernetes environment (ex: minikube or kind)
-Submarine's Helm Chart will deploy Submarine Server, TF/PyTorch Operator,
Notebook controller
-and Traefik. We use the TF/PyTorch operator to run tf/pytorch job, the
notebook controller to
-manage jupyter notebook and Traefik as reverse-proxy.
-
-
-### Install Helm
-
-Helm v3 is minimum requirement.
-See here for installation: https://helm.sh/docs/intro/install/
-
-### Install Submarine
-
-The Submarine helm charts is released with the source code for now.
-Please go to `http://submarine.apache.org/download.html` to download
-
-- Install Helm charts from source code
+## Deploy Submarine to Kubernetes
```bash
-cd <PathTo>/submarine
+git clone https://github.com/apache/submarine.git
+cd submarine
helm install submarine ./helm-charts/submarine
```
-This will install submarine in the "default" namespace.
-The images are from Docker hub `apache/submarine`. See
`./helm-charts/submarine/values.yaml` for more details
+* With these commands, the Submarine service will be deployed to the "default"
namespace.
+* The first time installation will take about 10 mins because the docker
images are pulled from
[apache/submarine](https://hub.docker.com/r/apache/submarine) on DockerHub.
-If we'd like use a different namespace like "submarine"
-```bash
-kubectl create namespace submarine
-helm install submarine ./helm-charts/submarine -n submarine
-```
-> Note that if you encounter below issue when installation:
-```bash
-Error: rendered manifests contain a resource that already exists.
-Unable to continue with install: existing resource conflict: namespace: ,
name: podgroups.scheduling.incubator.k8s.io, existing_kind:
apiextensions.k8s.io/v1beta1, Kind=CustomResourceDefinition, new_kind:
apiextensions.k8s.io/v1beta1, Kind=CustomResourceDefinition
-```
-It might be caused by the previous installed submarine charts. Fix it by
running:
+## Verify installation
```bash
-kubectl delete crd/tfjobs.kubeflow.org && kubectl delete
crd/podgroups.scheduling.incubator.k8s.io && kubectl delete
crd/pytorchjobs.kubeflow.org
+kubectl get all
```
+* TODO: screenshot
-- Verify installation
-
-Once you got it installed, check with below commands and you should see
similar outputs:
+## Uninstall Submarine
```bash
-kubectl get pods
-```
+helm uninstall submarine
-```bash
-NAME READY STATUS RESTARTS
AGE
-notebook-controller-deployment-5db8b6cbf7-k65jm 1/1 Running 0
5s
-pytorch-operator-7ff5d96d59-gx7f5 1/1 Running 0
5s
-submarine-database-8d95d74f7-ntvqp 1/1 Running 0
5s
-submarine-server-b6cd4787b-7bvr7 1/1 Running 0
5s
-submarine-traefik-9bb6f8577-66sx6 1/1 Running 0
5s
-tf-job-operator-7844656dd-lfgmd 1/1 Running 0
5s
-```
+# Check
+helm ls
+```
-### Configure volume type
+# Helm chart configuation (values.yaml)
+## Volume Type
Submarine can support various [volume
types](https://kubernetes.io/docs/concepts/storage/volumes/#nfs), currently
including hostPath (default) and NFS. It can be easily configured in the
`./helm-charts/submarine/values.yaml`, or you can override the default values
in `values.yaml` by [helm CLI](https://helm.sh/docs/helm/helm_install/).
#### hostPath
@@ -171,10 +145,4 @@ http://127.0.0.1:32080
If minikube is installed, use the following command to find the URL to the
Submarine server.
```
$ minikube service submarine-traefik --url
-```
-
-
-### Uninstall Submarine
-```bash
-helm delete submarine
-```
+```
\ No newline at end of file
diff --git a/website/docs/adminDocs/k8s/kind.md
b/website/docs/gettingStarted/kind.md
similarity index 100%
rename from website/docs/adminDocs/k8s/kind.md
rename to website/docs/gettingStarted/kind.md
diff --git a/website/sidebars.js b/website/sidebars.js
index 7a7ca4a..9364a9e 100644
--- a/website/sidebars.js
+++ b/website/sidebars.js
@@ -60,13 +60,6 @@ module.exports = {
],
"Administrator Docs": [
{
- "Submarine on Kubernetes": [
- "adminDocs/k8s/README",
- "adminDocs/k8s/kind",
- "adminDocs/k8s/helm",
- ],
- },
- {
"Submarine on Yarn": ["adminDocs/yarn/README"],
},
],
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
