This is an automated email from the ASF dual-hosted git repository. kaihsun pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push: new adc083c SUBMARINE-941. Improve documents about operator and integration tests adc083c is described below commit adc083c8c13d6ac6d382d4fb333b5ca3bcdd2187 Author: Kai-Hsun Chen <b03901...@ntu.edu.tw> AuthorDate: Thu Jul 15 19:44:38 2021 +0800 SUBMARINE-941. Improve documents about operator and integration tests ### What is this PR for? Improve documents about operator and integration tests ### What type of PR is it? [Documentation] ### Todos ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-941 ### How should this be tested? ### Screenshots (if appropriate) ### Questions: * Do the license files need updating? No * Are there breaking changes for older versions? No * Does this need new documentation? No Author: Kai-Hsun Chen <b03901...@ntu.edu.tw> Signed-off-by: Kai-Hsun Chen <kaih...@apache.org> Closes #682 from kevin85421/SUBMARINE-941 and squashes the following commits: 57c63558 [Kai-Hsun Chen] Fix bug 6dedc0d7 [Kai-Hsun Chen] SUBMARINE-941. Improve documents about submarine-operator and integration tests --- website/docs/devDocs/Development.md | 106 ++++++++++++++--------------- website/docs/devDocs/IntegrationTest.md | 95 -------------------------- website/docs/devDocs/IntegrationTestE2E.md | 37 +++++++--- website/docs/devDocs/IntegrationTestK8s.md | 12 ++-- website/docs/gettingStarted/kind.md | 13 ++-- website/sidebars.js | 1 - 6 files changed, 90 insertions(+), 174 deletions(-) diff --git a/website/docs/devDocs/Development.md b/website/docs/devDocs/Development.md index 0934c1d..dfb82d0 100644 --- a/website/docs/devDocs/Development.md +++ b/website/docs/devDocs/Development.md @@ -17,30 +17,28 @@ title: Development Guide limitations under the License. --> -## Overview - -From [Getting Started/Submarine Local Deployment](../gettingStarted/localDeployment.md), you already know that Submarine is installed and uninstalled by Helm. As you can see by `kubectl get pods`, there are nine major components in Submarine, including `notebook-controller`, `pytorch-operator`, `submarine-database`, `submarine-minio`, `submarine-mlflow`, `submarine-server`, `submarine-tensorboard`, `submarine-traefik`, and `tf-job-operator`. They are launched as pods in kubernetes from t [...] - -Some of the components are borrowed from other projects (kubeflow, traefik), including `notebook-controller`, `pytorch-operator`, `submarine-traefik` and `tf-job-operator`. The rest of them are built by ourselves, including `submarine-database`, `submarine-minio`, `submarine-mlflow`, `submarine-tensorboard`, and `submarine-server`. - -The purpose of the components are as the following: - -1. `tf-job-operator`: manage the operation of tensorflow jobs -2. `pytorch-operator`: manage the operation of pytorch jobs -3. `notebook-controller`: manage the operation of notebook instances -4. `submarine-traefik`: manage the ingress service - -5. `submarine-database`: store metadata in mysql database -6. `submarine-minio`: store machine learning artifacts in minio object storage -7. `submarine-mlflow`: platform for managing the end-to-end machine learning lifecycle -8. `submarine-tensorboard`: tool for providing the measurements and visualizations during ml workflow. -9. `submarine-server`: handle api request, submit job to container orchestration, and connect with database. - -In this document, we focus on `submarine-server` and `submarine-database`. You can learn how to develop server, database, and workbench here. - +# Project Overview +The document [Submarine Local Deployment](../gettingStarted/localDeployment.md) shows how to deploy the Submarine service to your Kubernetes cluster. The Submarine service consists mainly of nine components, and you can check them with the following command: + +``` +kubectl get pods -n ${your_namespace} +``` + +A brief introduction about these components: + +1. **tf-operator**: Enable users to run TensorFlow jobs distributedly +2. **pytorch-operator**: Enable users to run PyTorch jobs distributedly +3. **notebook-controller**: Jupyter Notebook controller +4. **submarine-traefik**: Kubernetes Ingress controller +5. **submarine-database**: A MySQL database to store metadata +6. **submarine-minio**: An object store for machine learning artifacts +7. **submarine-mlflow**: A platform for model management +8. **submarine-tensorboard**: A visualization tool for distributed training experiments +9. **submarine-server**: Handle API requests, and submit distributed training experiments to Kubernetes. + +# Submarine Development ## Video - -From [This Video](https://youtu.be/32Na2k6Alv4), you will know how to deal with the configuration of Submarine and be able to contribute to it via Github. +* From this [Video](https://youtu.be/32Na2k6Alv4), you will know how to deal with the configuration of Submarine and be able to contribute to it via Github. ## Develop server @@ -69,9 +67,7 @@ Checkstyle plugin may help to detect violations directly from the IDE. For each class, there is a corresponding testClass. For example, `SubmarineServerTest` is used for testing `SubmarineServer`. Whenever you add a funtion in classes, you must write a unit test to test it. -- Integration Test - - See [IntegrationTest.md](./IntegrationTest.md) +- Integration Test: [IntegrationTestK8s.md](./IntegrationTestK8s.md) ### Build from source @@ -129,6 +125,8 @@ Checkstyle plugin may help to detect violations directly from the IDE. 1. The request sent to `http://localhost:4200` will be redirected to `http://localhost:32080`. 2. Open `http://localhost:4200` in browser to see the real-time change of workbench. +4. Frontend E2E test: [IntegrationTestE2E.md](./IntegrationTestE2E.md) + ## Develop database 1. Build the docker image @@ -159,56 +157,56 @@ Checkstyle plugin may help to detect violations directly from the IDE. 1. Start the minikube cluster - ```bash - minikube start --vm-driver=docker --kubernetes-version v1.15.11 - ``` + ```bash + minikube start --vm-driver=docker --kubernetes-version v1.15.11 + ``` 2. Install the dependencies - ```bash - cd submarine-cloud-v2/ - go mod vendor - ``` + ```bash + cd submarine-cloud-v2/ + go mod vendor + ``` 3. Run the operator out-of-cluster - ```bash - make - ./submarine-operator - ``` + ```bash + make + ./submarine-operator + ``` 4. Deploy a Submarine - ```bash - kubectl apply -f artifacts/examples/crd.yaml - kubectl create ns submarine-user-test - kubectl apply -n submarine-user-test -f artifacts/examples/example-submarine.yaml - ``` + ```bash + kubectl apply -f artifacts/examples/crd.yaml + kubectl create ns submarine-user-test + kubectl apply -n submarine-user-test -f artifacts/examples/example-submarine.yaml + ``` 5. Exposing service - ```bash - # Method1 -- use minikube ip - minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2 + ```bash + # Method1 -- use minikube ip + minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2 - # Method2 -- use port-forwarding - kubectl port-forward --address 0.0.0.0 -n submarine-user-test service/traefik 32080:80 - ``` + # Method2 -- use port-forwarding + kubectl port-forward --address 0.0.0.0 -n submarine-user-test service/traefik 32080:80 + ``` 6. View workbench - If you use method 1 in step 5, please go to `http://{minikube ip}:32080`, ex: http://192.168.49.2:32080 + If you use method 1 in step 5, please go to `http://{minikube ip}:32080`, ex: http://192.168.49.2:32080 - If you use method 2 in step 5, please go to http://127.0.0.1:32080 + If you use method 2 in step 5, please go to http://127.0.0.1:32080 7. Delete submarine - ```bash - kubectl delete submarine example-submarine -n submarine-user-test - ``` + ```bash + kubectl delete submarine example-submarine -n submarine-user-test + ``` 8. Stop the operator - Press ctrl+c to stop the operator. + Press ctrl+c to stop the operator. For other details, please check out the [README](https://github.com/apache/submarine/blob/master/submarine-cloud-v2/README.md) and [Developer Guide](https://github.com/apache/submarine/blob/master/submarine-cloud-v2/docs/developer-guide.md) on GitHub. diff --git a/website/docs/devDocs/IntegrationTest.md b/website/docs/devDocs/IntegrationTest.md deleted file mode 100644 index dd94141..0000000 --- a/website/docs/devDocs/IntegrationTest.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -title: How to Run Integration Test ---- - -<!--- - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. See accompanying LICENSE file. ---> - -## Introduction -1. Now, Apache Submarine supports two kinds of integration test: `test-e2e` and `test-k8s`. These two modules can be found in the [submarine/submarine-test](https://github.com/apache/submarine/tree/master/submarine-test) directory. - -2. Currently, there are some differences between `test-e2e` and `test-k8s` in operation mode. To elaborate, `test-e2e` needs to deploy Apache Submarine locally, while `test-k8s` deploys Apache Submarine via k8s. - -3. These two test modules can be applied to different test scenarios. (In the future, these two test modules may be combined or adjusted) - -## E2E test - -### E2E tests can be executed both locally and in Travis (For workbench developer) -* Run E2E tests locally: - * Step1: Follow [HowToRun.md](https://github.com/apache/submarine/blob/master/website/docs/adminDocs/yarn/workbench/HowToRun.md) to launch the submarine-server and database. - * Step2: Run workbench (Angular version) locally - ``` - cd submarine/submarine-workbench/workbench-web - npm start - // Check 127.0.0.1:4200 - ``` - * Step3: Modify the port from 8080 to 4200 - * [WebDriverManager.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/WebDriverManager.java): `url = "http://localhost:8080";` --> `url = "http://localhost:4200";` - * [Your Unit test case](https://github.com/apache/submarine/tree/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration): `8080` --> `4200` - * Step4: Comment the `headless` option - * [ChromeWebDriverProvider.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/ChromeWebDriverProvider.java): `chromeOptions.addArguments("--headless");` --> `//chromeOptions.addArguments("--headless");` - * With the `headless` option, the selenium will be executed in background. - * Step5: Run E2E test cases (Please check the following section **Run the existing tests**) -* Run E2E tests in Travis: - * Step1: Make sure that the port must be 8080 rather than in [WebDriverManager.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/WebDriverManager.java) and [all test cases](https://github.com/apache/submarine/tree/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration). - * Step2: Make sure that the `headless` option is not commented in [ChromeWebDriverProvider.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/ChromeWebDriverProvider.java). - * Step3: If you push the commit to Github, the Travis CI will execute automatically and you can check it in `https://travis-ci.com/${your_github_account}/${your_repo_name}`. -### Run the existing tests. -##### Move to the working directory. -``` -cd submarine/submarine-test/test-e2e -``` -##### Compile & Run. - -> Following command will compile all files and run **all** files ending with "IT" in the [directory](https://github.com/apache/submarine/tree/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration). -* For linux - ``` - mvn verify - ``` -* For MacOS -``` -mvn clean install -U -``` -> Run a specific testcase -``` -mvn -Dtest=${your_test_case_file_name} test //ex: mvn -Dtest=loginIT test -``` - -##### Result -If all of the function under test are succeeded, it will show. -``` -BUILD SUCCESS -``` -Otherwise, it will show. -``` -BUILD FAILURE -``` - -### Add your own integration test -1. Create a new file ending with "IT" under "submarine/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration/". -2. Your public class is recommended to extend AbstractSubmarineIT. The class AbstractSubmarineIT contains some commonly used functions. -```java - WebElement pollingWait(final By locator, final long timeWait); // Find element on the website. - void clickAndWait(final By locator); // Click element and wait for 1 second. - void sleep(long millis, boolean logOutput); // Let system sleep a period of time. -``` -3. There are also some commonly used functions except in AbstractSubmarineIT.java. -```java - // In WebDriverManager.java: - public static WebDriver getWebDriver(); // This return a firefox webdriver which has been set to your workbench website. -``` -4. Add [JUnit](https://junit.org/junit5/docs/current/user-guide/) annotation before your testing function, e.g., @Beforeclass, @Test, and @AfterClass. You can refer to [loginIT.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration/loginIT.java). -5. Use command mentioned above to compile and run to test whether it works as your anticipation. - - diff --git a/website/docs/devDocs/IntegrationTestE2E.md b/website/docs/devDocs/IntegrationTestE2E.md index adc7c9a..46317dd 100644 --- a/website/docs/devDocs/IntegrationTestE2E.md +++ b/website/docs/devDocs/IntegrationTestE2E.md @@ -1,5 +1,5 @@ --- -title: How to Run integration E2E Test +title: How to Run Frontend Integration Test --- <!--- @@ -15,10 +15,9 @@ title: How to Run integration E2E Test --> ## Introduction +* The test cases under the directory [test-e2e](https://github.com/apache/submarine/tree/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration) are integration tests to ensure the correctness of the Submarine Workbench. -* It checks the components in the website works correctly. - -* You can run the test-e2e either locally or on GitHub Actions. +* These test cases can be run either locally or on GitHub Actions. ## Run E2E test locally @@ -26,16 +25,16 @@ title: How to Run integration E2E Test 2. Forward port - ```bash - kubectl port-forward --address 0.0.0.0 service/submarine-traefik 32080:80 - ``` + ```bash + kubectl port-forward --address 0.0.0.0 service/submarine-traefik 32080:80 + ``` 3. Modify run_frontend_e2e.sh You need to modify the port and the URL in this script to where you run the workbench on. > Example: - > If you just finished developing the workbench appearance and the workbench is running on localhost:4200, then you should modify the WORKBENCH_PORT to 4200 + > If your Submarine workbench is running on 127.0.0.1:4200, you should modify the **WORKBENCH_PORT** to 4200. ```bash # at submarine-test/test_e2e/run_frontend_e2e.sh @@ -49,7 +48,7 @@ title: How to Run integration E2E Test ... ``` -4. Run run_frontend_e2e.sh +4. Run run_frontend_e2e.sh (Run a specific test case) This script will check whether the port can be accessed or not, and run the test case. ```bash @@ -58,6 +57,24 @@ title: How to Run integration E2E Test # TESTCASE is the IT you want to run, ex: loginIT, experimentIT... ``` +5. Run all test cases +* Following commands will compile all files and run all files ending with "IT" in the [directory](https://github.com/apache/submarine/tree/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/integration). + ```bash + # Make sure the Submarine workbench is running on 127.0.0.1:8080 + cd submarine/submarine-test/test-e2e + # Method 1: + mvn verify + + # Method 2: + mvn clean install -U + ``` + ## Run E2E test in GitHub Actions -Each time a code is submitted, GitHub Actions is automatically triggered for testing. \ No newline at end of file +* Each time a commit is pushed, GitHub Actions will be triggered automatically. + +## Add a new frontend E2E test case +* **WARNING** + * You **MUST** read the [document](https://www.selenium.dev/documentation/en/webdriver/waits/) carefully, and understand the difference between **explicit wait**, **implicit wait**, and **fluent wait**. + * **Do not mix implicit and explicit waits.** Doing so can cause unpredictable wait times. +* We define many useful functions in [AbstractSubmarineIT.java](https://github.com/apache/submarine/blob/master/submarine-test/test-e2e/src/test/java/org/apache/submarine/AbstractSubmarineIT.java). \ No newline at end of file diff --git a/website/docs/devDocs/IntegrationTestK8s.md b/website/docs/devDocs/IntegrationTestK8s.md index 75c2802..8f98e3f 100644 --- a/website/docs/devDocs/IntegrationTestK8s.md +++ b/website/docs/devDocs/IntegrationTestK8s.md @@ -18,13 +18,12 @@ title: How to Run Integration K8s Test ## Introduction -* It checks the API on each page works correctly. +* The test cases under the directory `test-k8s` are integration tests to ensure the correctness of the Submarine RESTful API. -* You can run the test-k8s either locally or on GitHub Actions. - * Before running the test-k8s, the minikube (KinD) cluster must be created. +* You can run these tests either locally or on GitHub Actions. + * Before running the tests, the minikube (KinD) cluster must be created. * Then, compile and package the submarine project in `submarine-dist` directory for building a docker image. - * Otherwise, the 8080 port in submarine-traefik should be forward. - * Finally, the test case under the `test-k8s` directory is ready to run. + * In addition, the 8080 port in submarine-traefik should be forwarded. ## Run k8s test locally @@ -45,5 +44,4 @@ title: How to Run Integration K8s Test ``` ## Run k8s test in GitHub Actions - -Each time a code is submitted, GitHub Actions is automatically triggered for testing. +* Each time a code is submitted, GitHub Actions is triggered automatically. diff --git a/website/docs/gettingStarted/kind.md b/website/docs/gettingStarted/kind.md index 8fbb18e..5c18432 100644 --- a/website/docs/gettingStarted/kind.md +++ b/website/docs/gettingStarted/kind.md @@ -21,10 +21,10 @@ specific language governing permissions and limitations under the License. --> -## Create K8s cluster -We recommend using [`KinD`](https://kind.sigs.k8s.io/) to setup a Kubernetes cluster on a local machine. +## Create Kubernetes cluster with KinD +We recommend users developing Submarine with minikube. However, [`KinD`](https://kind.sigs.k8s.io/) is also an option to setup a Kubernetes cluster on your local machine. -Running the following command, and specify the KinD version and Kubernetes version [`here`](../devDocs/Dependencies). +Run the following command, and specify the KinD version and Kubernetes version [`here`](../devDocs/Dependencies). ```bash # Download the specific version of KinD (must >= v0.6.0) export KIND_VERSION=v0.11.1 @@ -41,20 +41,20 @@ kind create cluster --image kindest/node:${KUBE_VERSION} ## Kubernetes Dashboard (optional) ### Deploy -To deploy Dashboard, execute following command: +To deploy Dashboard, execute the following command: ``` kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml ``` ### Create RBAC -Ensure to grant the cluster access permission of dashboard, run the following command: +Run the following commands to grant the cluster access permission of dashboard: ``` kubectl create serviceaccount dashboard-admin-sa kubectl create clusterrolebinding dashboard-admin-sa --clusterrole=cluster-admin --serviceaccount=default:dashboard-admin-sa ``` ### Get access token (optional) -If you want to use the token to login the dashboard, run the following command to get key: +If you want to use the token to login the dashboard, run the following commands to get key: ``` kubectl get secrets # select the right dashboard-admin-sa-token to describe the secret @@ -62,7 +62,6 @@ kubectl describe secret dashboard-admin-sa-token-6nhkx ``` ### Start dashboard service -To start the dashboard service, we can run the following command: ``` kubectl proxy ``` diff --git a/website/sidebars.js b/website/sidebars.js index cf4693b..80fd24d 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -66,7 +66,6 @@ module.exports = { "devDocs/Dependencies", "devDocs/BuildFromCode", "devDocs/Development", - "devDocs/IntegrationTest", "devDocs/IntegrationTestK8s", "devDocs/IntegrationTestE2E", ], --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org For additional commands, e-mail: dev-h...@submarine.apache.org