This is an automated email from the ASF dual-hosted git repository.
kaihsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 6797efb SUBMARINE-931. Update README of submarine-cloud-v2 and add
developer guide
6797efb is described below
commit 6797efb9a881322c57a3dfb79c1c283489a7f039
Author: MortalHappiness <b07901...@ntu.edu.tw>
AuthorDate: Tue Jul 13 23:31:03 2021 +0800
SUBMARINE-931. Update README of submarine-cloud-v2 and add developer guide
### What is this PR for?
Update README of submarine-cloud-v2 and add developer guide.
### What type of PR is it?
[Documentation]
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-931
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? No
Author: MortalHappiness <b07901...@ntu.edu.tw>
Author: Chi-Sheng Liu <47914085+mortalhappin...@users.noreply.github.com>
Signed-off-by: Kai-Hsun Chen <kaih...@apache.org>
Closes #670 from MortalHappiness/SUBMARINE-931 and squashes the following
commits:
9a8ef66d [Chi-Sheng Liu] SUBMARINE-931. Update README.md
4c721e20 [MortalHappiness] SUBMARINE-931. Add license header
753ec5d6 [MortalHappiness] SUBMARINE-931. Update README of
submarine-cloud-v2 and add developer guide
---
submarine-cloud-v2/README.md | 199 ++++++++---------------------
submarine-cloud-v2/docs/developer-guide.md | 129 +++++++++++++++++++
2 files changed, 181 insertions(+), 147 deletions(-)
diff --git a/submarine-cloud-v2/README.md b/submarine-cloud-v2/README.md
index 48f6886..d95f51f 100644
--- a/submarine-cloud-v2/README.md
+++ b/submarine-cloud-v2/README.md
@@ -1,90 +1,80 @@
-# Why submarine-cloud-v2?
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
-- Because `submarine-cloud` is outdated, `submarine-cloud-v2` is the
refactored version of `submarine-cloud`. In addition, after
`submarine-cloud-v2` finishes, we will replace `submarine-cloud` with
`submarine-cloud-v2`.
+ http://www.apache.org/licenses/LICENSE-2.0
-# Formatting the code
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
-For `go` files, please use [gofmt](https://golang.org/pkg/cmd/gofmt/) to
format the code.
+# submarine-cloud-v2
-For `yaml` files, please use [prettier](https://prettier.io/) to format the
code.
+`submarine-cloud-v2` implements the operator for Submarine application.
-# Initialization
+## Why submarine-cloud-v2?
-```bash
-minikube start --vm-driver=docker --kubernetes-version v1.15.11
-go mod vendor
-```
-
-# Generate API
-
-We use the generators in
[k8s.io/code-generator](https://github.com/kubernetes/code-generator) to
generate a typed client, informers, listers and deep-copy functions.
-
-**Important**: You **MUST** put this repository in a folder named
`github.com/apache/`, otherwise the code will be generated into wrong folder.
Therefore the full path of this `README.md` should be like
`SOMEWHERE_IN_FILESYSTEM/github.com/apache/submarine/submarine-cloud-v2/README.md`.
+Because `submarine-cloud` is outdated, `submarine-cloud-v2` is the refactored
version of `submarine-cloud`. In addition, after `submarine-cloud-v2` finishes,
we will replace `submarine-cloud` with `submarine-cloud-v2`.
-Everytime when you change the codes in `pkg/apis`, you must run `make api` to
re-generate the API.
-
-# Add new dependencies
+## Initialization
```bash
-# Step1: Add the dependency to go.mod
-go get ${new_dependency} # Example: go get k8s.io/code-generator
-
-# Step2: Download the dependency to vendor/
+# Install dependencies
go mod vendor
+# Run the cluster
+minikube start --vm-driver=docker --kubernetes-version v1.15.11
```
-# Run Unit Test
-
-```bash
-# Step1: Register Custom Resource Definition
-kubectl apply -f artifacts/examples/crd.yaml
-
-# Step2: Create a Custom Resource
-kubectl apply -f artifacts/examples/example-submarine.yaml
-
-# Step3: Run unit test
-make test-unit
-```
-
-# Run submarine-operator out-of-cluster
+## Run operator out-of-cluster
```bash
# Step1: Build & Run "submarine-operator"
-go build -o submarine-operator
+make
./submarine-operator
-# Step2: Deploy a submarine
+# Step2: Deploy a Submarine
kubectl apply -f artifacts/examples/crd.yaml
kubectl create ns submarine-user-test
kubectl apply -n submarine-user-test -f
artifacts/examples/example-submarine.yaml
# Step3: Exposing Service
-# Method1 -- using minikube ip + NodePort
-$ minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2
+# Method1 -- use minikube ip
+minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2
-# Method2 -- using port-forwarding
-$ kubectl port-forward --address 0.0.0.0 -n submarine-user-test
service/traefik 32080:80
+# Method2 -- use port-forwarding
+kubectl port-forward --address 0.0.0.0 -n submarine-user-test service/traefik
32080:80
-# Step4: View workbench
-# http://{minikube ip}:32080(from Method1), ex: http://192.168.49.2:32080
+# Step4: View Workbench
+# http://{minikube ip}:32080 (from Method 1), ex: http://192.168.49.2:32080
# or http://127.0.0.1:32080 (from Method 2).
-# Step5: Delete:
+# Step5: Delete Submarine
+# By deleting the submarine custom resource, the operator will do the
following things:
# (1) Remove all relevant Helm chart releases
# (2) Remove all resources in the namespace "submariner-user-test"
# (3) Remove all non-namespaced resources (Ex: PersistentVolume) created by
client-go API
# (4) **Note:** The namespace "submarine-user-test" will not be deleted
kubectl delete submarine example-submarine -n submarine-user-test
+
+# Step6: Stop the operator
+# Press ctrl+c to stop the operator
```
-# Run operator in-cluster
+## Run operator in-cluster
```bash
# Step1: Build image "submarine-operator" to minikube's Docker
eval $(minikube docker-env)
make image
-# Step2: RBAC (ClusterRole, ClusterRoleBinding, and ServiceAccount)
+# Step2: Setup RBAC (ClusterRole, ClusterRoleBinding, and ServiceAccount)
kubectl apply -f artifacts/examples/submarine-operator-service-account.yaml
# Step3: Deploy a submarine-operator
@@ -95,117 +85,32 @@ kubectl apply -f artifacts/examples/crd.yaml
kubectl create ns submarine-user-test
kubectl apply -n submarine-user-test -f
artifacts/examples/example-submarine.yaml
-# Step5: Inspect submarine-operator POD logs
-kubectl logs -f ${submarine-operator POD}
+# Step5: Inspect the logs of submarine-operator
+kubectl logs -f $(kubectl get pods --output=name | grep submarine-operator)
-# Step6: The operator will create a new namespace "submarine-user-test"
-kubectl get all -n submarine-user-test
+# Step6: Exposing Service
+# Method1 -- use minikube ip
+minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2
-# Step7: Exposing Service
-# Method1 -- using minikube ip + NodePort
-$ minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2
+# Method2 -- use port-forwarding
+kubectl port-forward --address 0.0.0.0 -n submarine-user-test service/traefik
32080:80
-# Method2 -- using port-forwarding
-$ kubectl port-forward --address 0.0.0.0 -n submarine-user-test
service/traefik 32080:80
-
-# Step8: View workbench
-# http://{minikube ip}:32080(from Method1), ex: http://192.168.49.2:32080
+# Step7: View Workbench
+# http://{minikube ip}:32080 (from Method 1), ex: http://192.168.49.2:32080
# or http://127.0.0.1:32080 (from Method 2).
-# Step9: Delete:
+# Step8: Delete Submarine
+# By deleting the submarine custom resource, the operator will do the
following things:
# (1) Remove all relevant Helm chart releases
# (2) Remove all resources in the namespace "submariner-user-test"
# (3) Remove all non-namespaced resources (Ex: PersistentVolume) created by
client-go API
# (4) **Note:** The namespace "submarine-user-test" will not be deleted
kubectl delete submarine example-submarine -n submarine-user-test
-# Step10: Delete "submarine-operator"
+# Step9: Delete the submarine-operator
kubectl delete deployment submarine-operator-demo
```
-# Helm Golang API
-
-- Function `HelmInstall` is defined in pkg/helm/helm.go.
-- Example: (You can see this example in controller.go:123.)
-
-```go
-// Example: HelmInstall
-// This is equal to:
-// helm repo add k8s-as-helm https://ameijer.github.io/k8s-as-helm/
-// . helm repo update
-// helm install helm-install-example-release k8s-as-helm/svc --set
ports[0].protocol=TCP,ports[0].port=80,ports[0].targetPort=9376
-// Useful Links:
-// (1) https://github.com/PrasadG193/helm-clientgo-example
-// . (2) https://github.com/ameijer/k8s-as-helm/tree/master/charts/svc
-helmActionConfig := helm.HelmInstall(
- "https://ameijer.github.io/k8s-as-helm/";,
- "k8s-as-helm",
- "svc",
- "helm-install-example-release",
- "default",
- map[string]string {
- "set":
"ports[0].protocol=TCP,ports[0].port=80,ports[0].targetPort=9376",
- },
-)
-// Example: HelmUninstall
-// This is equal to:
-// helm uninstall helm-install-example-release
-helm.HelmUninstall("helm-install-example-release", helmActionConfig)
-
-```
-
-- Troubleshooting:
- - If the release name exists, Helm will report the error "cannot re-use a
name that is still in use".
-
-```
-helm ls
-helm uninstall helm-install-example-release
-```
-
-# Build custom images when development
-
-Use the following helper script to build images and update the images used by
running pods.
+## Development
-```
-./hack/build_image.sh [all|server|database|jupyter|jupyter-gpu|mlflow]
-```
-
-Examples:
-
-```
-./hack/build_image.sh all # build all images
-./hack/build_image.sh server # only build the server image
-```
-
-# Run frontend E2E tests
-
-Use the following helper script to run frontend E2E tests.
-
-```
-# Prerequisite: Make sure Workbench is running on $URL:$WORKBENCH_PORT.
-./hack/run_frontend_e2e.sh [testcase]
-```
-
-- [testcase]: Check the directory
[integration](../submarine-test/test-e2e/src/test/java/org/apache/submarine/integration/).
-
-Examples:
-
-```
-./hack/run_frontend_e2e.sh loginIT
-```
-
-# Run Operator E2E tests
-
-Reference: [spark-on-k8s-operator e2e
test](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/tree/master/test/e2e)
-
-```bash
-# Step1: Build image "submarine-operator" to minikube's Docker
-eval $(minikube docker-env)
-make image
-
-# Step2: Register Custom Resource Definition
-kubectl apply -f artifacts/examples/crd.yaml
-
-# Step3: Run Test
-go ./test/e2e
-```
+Please check out the [Developer Guide](./docs/developer-guide.md).
diff --git a/submarine-cloud-v2/docs/developer-guide.md
b/submarine-cloud-v2/docs/developer-guide.md
new file mode 100644
index 0000000..bb8bf16
--- /dev/null
+++ b/submarine-cloud-v2/docs/developer-guide.md
@@ -0,0 +1,129 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Developer Guide
+
+## Coding Style
+
+For `go` files, please use [gofmt](https://golang.org/pkg/cmd/gofmt/) to
format the code.
+
+For `yaml` files, please use [prettier](https://prettier.io/) to format the
code.
+
+## Generate API
+
+**Important**: You **MUST** put this repository in a folder named
`github.com/apache/`, otherwise the code will be generated into wrong folder.
Therefore the full path of this `developer-guide.md` should be like
`SOMEWHERE_IN_FILESYSTEM/github.com/apache/submarine/submarine-cloud-v2/docs/developer-guide.md`.
+
+We use [code-generator](https://github.com/kubernetes/code-generator) to
generate a typed client, informers, listers and deep-copy functions.
+
+Everytime when you change the codes in `submarine-cloud-v2/pkg/apis`, you must
run `make api` to re-generate the API.
+
+## Add New Dependencies
+
+```bash
+# Step1: Add the dependency to go.mod
+go get ${new_dependency} # Example: go get k8s.io/code-generator
+
+# Step2: Download the dependency to vendor/
+go mod vendor
+```
+
+## Run Operator End-to-end Tests
+
+Reference: [spark-on-k8s-operator e2e
test](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/tree/master/test/e2e)
+
+```bash
+# Step1: Build image "submarine-operator" to minikube's Docker
+eval $(minikube docker-env)
+make image
+
+# Step2: Register Custom Resource Definition
+kubectl apply -f artifacts/examples/crd.yaml
+
+# Step3: Run Tests
+go test ./test/e2e
+```
+
+## Run Frontend End-to-end Tests
+
+Use the following helper script to run frontend E2E tests.
+
+```bash
+# Prerequisite: Make sure Workbench is running on $URL:$WORKBENCH_PORT.
+./hack/run_frontend_e2e.sh [testcase]
+```
+
+- `[testcase]`: Check the directory
[integration](../../submarine-test/test-e2e/src/test/java/org/apache/submarine/integration/).
+
+Example:
+
+```bash
+./hack/run_frontend_e2e.sh loginIT
+```
+
+## Build Custom Images
+
+Use the following helper script to build images and update the images used by
running pods.
+
+```
+./hack/build_image.sh [all|server|database|jupyter|jupyter-gpu|mlflow]
+```
+
+Examples:
+
+```
+./hack/build_image.sh all # build all images
+./hack/build_image.sh server # only build the server image
+```
+
+## Helm Golang API
+
+- Function `HelmInstall` is defined in pkg/helm/helm.go.
+- Example: (You can see this example in controller.go:123.)
+
+```go
+// Example: HelmInstall
+// This is equal to:
+// helm repo add k8s-as-helm https://ameijer.github.io/k8s-as-helm/
+// . helm repo update
+// helm install helm-install-example-release k8s-as-helm/svc --set
ports[0].protocol=TCP,ports[0].port=80,ports[0].targetPort=9376
+// Useful Links:
+// (1) https://github.com/PrasadG193/helm-clientgo-example
+// . (2) https://github.com/ameijer/k8s-as-helm/tree/master/charts/svc
+helmActionConfig := helm.HelmInstall(
+ "https://ameijer.github.io/k8s-as-helm/";,
+ "k8s-as-helm",
+ "svc",
+ "helm-install-example-release",
+ "default",
+ map[string]string {
+ "set":
"ports[0].protocol=TCP,ports[0].port=80,ports[0].targetPort=9376",
+ },
+)
+// Example: HelmUninstall
+// This is equal to:
+// helm uninstall helm-install-example-release
+helm.HelmUninstall("helm-install-example-release", helmActionConfig)
+
+```
+
+- Troubleshooting:
+ - If the release name exists, Helm will report the error "cannot re-use a
name that is still in use".
+
+```
+helm ls
+helm uninstall helm-install-example-release
+```
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
