This is an automated email from the ASF dual-hosted git repository.
byronhsu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 7d2de63 SUBMARINE-724. Build failed "submarine-cicd" image
7d2de63 is described below
commit 7d2de63bc7866fe38ed3c57753fb22062619b60c
Author: ByronHsu <[email protected]>
AuthorDate: Sat Jun 26 15:51:23 2021 +0800
SUBMARINE-724. Build failed "submarine-cicd" image
### What is this PR for?
When running cicd image, we'll encounter the following error.
```
docker.io/library/ubuntu:18.04sha256:122f506735a26c0a1aff2363335412cfc4f84de38326356d31ee00c2cbe52171
0.0s
=> CACHED [2/6] RUN apt-get update && apt-get install -y wget vim git
python-pip zlib1g-dev libssl-dev && wget ht 0.0s
=> CACHED [3/6] RUN git clone
https://gitbox.apache.org/repos/asf/submarine.git && cd submarine && git remote
ren 0.0s
=> CACHED [4/6] ADD entry.sh /entry.sh 0.0s
=> CACHED [5/6] RUN apt-get update && apt-get -y install bundler 0.0s
=> ERROR [6/6] RUN cd / && git clone
https://gitbox.apache.org/repos/asf/submarine-site.git && cd /submarine-sit
18.0s
------
> [6/6] RUN cd / && git clone
https://gitbox.apache.org/repos/asf/submarine-site.git && cd /submarine-site &&
bundle update:
#10 0.437 Cloning into 'submarine-site'...
#10 17.75 Could not locate Gemfile
------
executor failed running [/bin/sh -c cd / && git clone
https://gitbox.apache.org/repos/asf/submarine-site.git && cd /submarine-site &&
bundle update]:
```
The cause is that we have rewritten submarine site, and it does not contain
Gemfile now.
Furthermore, the submarine site is auto-updated by GitHub action now, so we
do not need to push it manually. Therefore, we can remove the part of submarine
site updating in cicd-image.
### What type of PR is it?
[Bug Fix]
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/projects/SUBMARINE/issues/SUBMARINE-798
### How should this be tested?
Use new image to successfully merge SUBMARINE-815
```
~/Desktop/submarine cicd *3 !4 >
./dev-support/cicd/build_and_start_cicd_image.sh
3m 0s py base byronhsuneihu 15:42:09
Building Submarine CI/CD Image.
Sending build context to Docker daemon 50.69kB
Step 1/6 : FROM ubuntu:18.04
---> 7d0d8fa37224
Step 2/6 : RUN apt-get update && apt-get install -y wget vim git
python-pip zlib1g-dev libssl-dev && wget
https://www.python.org/ftp/python/2.7.13/Python-2.7.13.tgz && tar xzf
Python-2.7.13.tgz && cd Python-2.7.13 && ./configure --with-zlib && make
install && make altinstall && cd /root && wget
https://files.pythonhosted.org/packages/59/38/55dd25a965990bd93f77eb765b189e72cf581ce1c2de651cb7b1dea74ed1/virtualenv-16.2.0.tar.gz
&& tar xf virtualenv-16.2.0.tar.gz && python vi [...]
---> Using cache
---> 58d77231ab03
Step 3/6 : RUN git clone
https://gitbox.apache.org/repos/asf/submarine.git && cd submarine && git
remote rename origin apache && git remote add apache-github
https://github.com/apache/submarine.git
---> Using cache
---> 26cc55ff0d28
Step 4/6 : ADD entry.sh /entry.sh
---> Using cache
---> fe39e53df011
Step 5/6 : ENV SUBMARINE_HOME=/submarine PYTHON_VENV_PATH=/root
---> Using cache
---> 93594bd2d37f
Step 6/6 : CMD /entry.sh
---> Using cache
---> 4fcba0d09e5b
Successfully built 4fcba0d09e5b
Successfully tagged submarine-cicd:latest
Start Submarine CI/CD.
==== Merge PR Begin ====
Enter Your Apache JIRA User name: ByronHsu
Got JIRA name: ByronHsu
Enter Your Apache JIRA User passwd:
Enter Your Apache committer ID: byronhsu
Got Apache ID: byronhsu
Enter Your Apache committer name: byronhsu
Got Apache name: byronhsu
Already on 'master'
Your branch is up to date with 'apache/master'.
remote: Counting objects: 33, done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 33 (delta 17), reused 0 (delta 0)
Unpacking objects: 100% (33/33), done.
From https://gitbox.apache.org/repos/asf/submarine
f24e25b9..e88af3e6 master -> apache/master
Updating f24e25b9..e88af3e6
Fast-forward
.github/workflows/master.yml
| 2 +-
.gitignore
| 6 +++-
dev-support/examples/tracking/Dockerfile
| 22 +++++++++++++++
dev-support/examples/tracking/build.sh
| 44 +++++++++++++++++++++++++++++
dev-support/examples/tracking/post.sh
| 42 +++++++++++++++++++++++++++
dev-support/examples/tracking/readme.md
| 22 +++++++++++++++
dev-support/examples/tracking/tracking.py
| 29 +++++++++++++++++++
submarine-sdk/pysubmarine/submarine/models/client.py
| 45 +++++++++++++++++++++++++++--
submarine-sdk/pysubmarine/submarine/models/utils.py
| 94
++++++++++++++++++++++++++++++++++++++++++++++++++-----------
submarine-server/server-core/src/test/java/org/apache/submarine/server/rest/ClusterRestApiTest.java
| 6 ++--
10 files changed, 289 insertions(+), 23 deletions(-)
create mode 100644 dev-support/examples/tracking/Dockerfile
create mode 100755 dev-support/examples/tracking/build.sh
create mode 100755 dev-support/examples/tracking/post.sh
create mode 100644 dev-support/examples/tracking/readme.md
create mode 100644 dev-support/examples/tracking/tracking.py
git rev-parse HEAD
Which pull request would you like to merge? (e.g. 23): 610
=== Pull Request #610 ===
title SUBMARINE-815. Export ModelsClient from pysubmarine init
source jeff-901/SUBMARINE-815
target master
url https://api.github.com/repos/apache/submarine/pulls/610
Proceed with merging pull request #610? (y/n): y
git fetch apache-github pull/610/head:PR_TOOL_MERGE_PR_610
remote: Enumerating objects: 23, done.
remote: Counting objects: 100% (23/23), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 23 (delta 14), reused 23 (delta 14), pack-reused 0
Unpacking objects: 100% (23/23), done.
From https://github.com/apache/submarine
* [new ref] refs/pull/610/head -> PR_TOOL_MERGE_PR_610
git fetch apache master:PR_TOOL_MERGE_PR_610_MASTER
From https://gitbox.apache.org/repos/asf/submarine
* [new branch] master -> PR_TOOL_MERGE_PR_610_MASTER
git checkout PR_TOOL_MERGE_PR_610_MASTER
Switched to branch 'PR_TOOL_MERGE_PR_610_MASTER'
['git', 'merge', 'PR_TOOL_MERGE_PR_610', '--squash']
['git', 'log', 'HEAD..PR_TOOL_MERGE_PR_610', '--pretty=format:%an <%ae>']
['git', 'log', 'PR_TOOL_MERGE_PR_610', '-1', '--pretty=format:%ad']
['git', 'log', 'HEAD..PR_TOOL_MERGE_PR_610', '--pretty=format:%h [%an] %s']
git config --get user.name
git config --get user.email
['git', 'commit', '--author="jeff-901 <b07901052ntu.edu.com>"',
'--date="Fri Jun 25 11:12:05 2021 +0800"', '-m', u'SUBMARINE-815. Export
ModelsClient from pysubmarine init', '-m', '### What is this PR for?\r\nExport
ModelsClient from pysubmarine/submarine __init__.py\r\nExample:\r\n```\r\nfrom
submarine import *\r\nclient = ModelsClient() \r\n```\r\nbefore:
failed\r\nafter: success\r\n\r\n### What type of PR is it?\r\nBug
Fix\r\n\r\n### Todos\r\n\r\n### What is the Jira issue?\r\nhttp [...]
Merge complete (local ref PR_TOOL_MERGE_PR_610_MASTER). Push to apache?
(y/n): y
git push apache PR_TOOL_MERGE_PR_610_MASTER:master
Username for 'https://gitbox.apache.org': byronhsu
Password for 'https://byronhsugitbox.apache.org':
Counting objects: 6, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 1009 bytes | 1009.00 KiB/s, done.
Total 6 (delta 4), reused 0 (delta 0)
remote: remote:
remote: remote: GitHub found 103 vulnerabilities on apache/submarine's
default branch (4 high, 7 moderate, 92 low). To find out more, visit:
remote: remote: https://github.com/apache/submarine/security/dependabot
remote: remote:
remote: To gitgithub:apache/submarine.git
remote: e88af3e..85a87e8 85a87e89fbe339cd82bc502dfe139184794540e6 ->
master
remote: Syncing refs/heads/master...
remote: Sending notification emails to: [u'"devsubmarine.apache.org"
<devsubmarine.apache.org>']
To https://gitbox.apache.org/repos/asf/submarine.git
e88af3e6..85a87e89 PR_TOOL_MERGE_PR_610_MASTER -> master
git rev-parse PR_TOOL_MERGE_PR_610_MASTER
Restoring head pointer to e88af3e6
git checkout e88af3e6
Note: checking out 'e88af3e6'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at e88af3e6 SUBMARINE-845. Fix submarine-server
test->ClusterRestApiTest->testGetClusterNode
git branch
Deleting local branch PR_TOOL_MERGE_PR_610
git branch -D PR_TOOL_MERGE_PR_610
Deleting local branch PR_TOOL_MERGE_PR_610_MASTER
git branch -D PR_TOOL_MERGE_PR_610_MASTER
Pull request #610 merged!
Merge hash: 85a87e89
Would you like to pick 85a87e89 into another branch? (y/n): n
Would you like to update an associated JIRA? (y/n): y
Enter a JIRA id [SUBMARINE-815]: SUBMARINE-815
=== JIRA SUBMARINE-815 ===
summary Export ModelsClient from submarine init
assignee Jhan-Shuo, Liu
status Open
url https://issues.apache.org/jira/browse/SUBMARINE-815
Enter comma-separated fix version(s) [0.6.0]:
Successfully resolved SUBMARINE-815 with fixVersions=[u'0.6.0']!
==== Merge PR END ====
```
### Screenshots (if appropriate)
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? No
Author: ByronHsu <[email protected]>
Signed-off-by: byronhsu <[email protected]>
Closes #612 from ByronHsu/cicd and squashes the following commits:
9a6e0e19 [ByronHsu] remove manually update site part
---
dev-support/cicd/Dockerfile | 10 ---
dev-support/cicd/README.md | 10 +--
dev-support/cicd/build_and_start_cicd_image.sh | 12 ++++
dev-support/cicd/entry.sh | 95 +-------------------------
4 files changed, 16 insertions(+), 111 deletions(-)
diff --git a/dev-support/cicd/Dockerfile b/dev-support/cicd/Dockerfile
index edf0402..21411d1 100644
--- a/dev-support/cicd/Dockerfile
+++ b/dev-support/cicd/Dockerfile
@@ -37,14 +37,4 @@ ENV \
SUBMARINE_HOME=/submarine \
PYTHON_VENV_PATH=/root
-RUN apt-get update && apt-get -y install bundler
-
-RUN \
- cd / && \
- git clone https://gitbox.apache.org/repos/asf/submarine-site.git && \
- cd /submarine-site && bundle update
-
-ENV \
- SUBMARINE_SITE=/submarine-site
-
CMD /entry.sh
diff --git a/dev-support/cicd/README.md b/dev-support/cicd/README.md
index 4ed4c3a..1a327a1 100644
--- a/dev-support/cicd/README.md
+++ b/dev-support/cicd/README.md
@@ -52,14 +52,10 @@ $ docker run -it -e JIRA_USERNAME="${JIRA_USERNAME}" -e
JIRA_PASSWORD="${JIRA_PA
The screen outputs the following information:
```
-Menu:
- 1. Merge PR
- 2. Update Submarine Website
-Enter Menu ID:
+Start Submarine CI/CD.
+==== Merge PR Begin ====
+Enter Your Apache JIRA User name:
```
-
-As you can see, the Docker mode support several features like merging PR and
updating the web site. Choose the task you need to do and follow the popup tip
to go on.
-
## Manual mode (Not Recommended)
First, You need install `python 2.7.13` and `pip insall jira`
diff --git a/dev-support/cicd/build_and_start_cicd_image.sh
b/dev-support/cicd/build_and_start_cicd_image.sh
index a43600f..b35ca53 100755
--- a/dev-support/cicd/build_and_start_cicd_image.sh
+++ b/dev-support/cicd/build_and_start_cicd_image.sh
@@ -15,6 +15,18 @@
# limitations under the License.
set -euo pipefail
+
+if [ -L ${BASH_SOURCE-$0} ]; then
+ PWD=$(dirname $(readlink "${BASH_SOURCE-$0}"))
+else
+ PWD=$(dirname ${BASH_SOURCE-$0})
+fi
+CURRENT_PATH=$(cd "${PWD}">/dev/null; pwd)
+
+# So it can executed from any directory
+# e.g. We can run ./dev-support/cicd/build_and_start_cicd_image.sh from the
root of submarine project
+cd ${CURRENT_PATH}
+
printf "Building Submarine CI/CD Image.\n"
docker build -t submarine-cicd .
printf "Start Submarine CI/CD.\n"
diff --git a/dev-support/cicd/entry.sh b/dev-support/cicd/entry.sh
index 811683f..4ccdb43 100755
--- a/dev-support/cicd/entry.sh
+++ b/dev-support/cicd/entry.sh
@@ -17,23 +17,6 @@ set -euo pipefail
# activate python 2.7.13 environment
. ${PYTHON_VENV_PATH}/venv2.7/bin/activate
-function start_menu(){
- printf "Menu:\n"
- printf "\t1. Merge PR\n"
- printf "\t2. Update Submarine Website\n"
- read -p "Enter Menu ID:" menu_id
- case $menu_id in
- "1")
- merge_pr
- ;;
- "2")
- update_submarine_site
- ;;
- "*")
- printf "unknown. Exiting."
- ;;
- esac
-}
function merge_pr(){
printf "==== Merge PR Begin ====\n"
@@ -81,81 +64,5 @@ function merge_pr(){
printf "==== Merge PR END ====\n"
}
-function update_submarine_site(){
- printf "==== Update Submarine Site Begin ====\n"
- apache_id="id"
- apache_name="name"
-
- if [ -z "${APACHE_ID:-}" ]; then
- printf "\n"
- read -p "Enter Your Apache committer ID: " apache_id
- else
- apache_id=$APACHE_ID
- fi
- echo "Got Apache ID: ${apache_id}"
-
- if [ -z "${APACHE_NAME:-}" ]; then
- read -p "Enter Your Apache committer name: " apache_name
- else
- apache_name=$APACHE_NAME
- fi
- echo "Got Apache name: ${apache_name}"
-
- cd $SUBMARINE_SITE
- git checkout master
- git pull
- git config user.name "${apache_name}"
- git config user.email "${apache_id}@apache.org"
- git config credential.helper store
- bundle update
- bundle exec jekyll serve --watch --host=0.0.0.0 > /tmp/jekyll.log 2>&1 &
- echo "==== Please use vim to edit md files and check http://localhost:4000/
for the update ===="
- while true; do
- echo "==== Edit Mode: Type 'exit' when you finish the changes (you don't
need to perform git commit/push). ===="
- bash
- read -p "Have you finished updating the MD files? y/n/quit " commit
- case $commit in
- "y")
- echo "Start committing changes.."
- cd $SUBMARINE_SITE
- git add .
- git status
- read -p "Please input the commit message: " message
- git commit -m "${message} (master branch)"
- git push origin master
- cp -r _site /_site
- git checkout asf-site
- cp -r /_site/* ./
- git add .
- git status
- git commit -m "${message} (asf-site branch)"
- git push origin asf-site
- echo "Exiting edit mode.."
- break
- ;;
- "n")
- continue
- ;;
- "quit")
- printf "Exiting edit mode.."
- break
- ;;
- "q")
- printf "Exiting edit mode.."
- break
- ;;
- "*")
- printf "Unknown. Exiting edit mode.."
- break
- ;;
- esac
- done
- printf "\n"
- printf "==== Update Submarine Site END ====\n"
- echo "==== Enter shell again incase any unexpected error happens ===="
- bash
- echo "Exiting CICD.."
-}
-
-start_menu
+merge_pr
deactivate
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
