This is an automated email from the ASF dual-hosted git repository.
liuhaopeng pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/bigtop-manager.git
The following commit(s) were added to refs/heads/main by this push:
new 95e539ba BIGTOP-4415: Fix docker dev environment (#215)
95e539ba is described below
commit 95e539baaa3ab70728214ef189013d303b74bd2c
Author: Zhiguo Wu <[email protected]>
AuthorDate: Thu Apr 24 14:33:05 2025 +0800
BIGTOP-4415: Fix docker dev environment (#215)
---
dev-support/README.md | 13 +++----
dev-support/docker/containers/build.sh | 43 ++++++++++------------
...{Dockerfile.template => Dockerfile.openeuler24} | 24 +++++-------
.../{Dockerfile.template => Dockerfile.rocky8} | 8 +---
dev-support/docker/image/build.sh | 33 +++++------------
docs/en/cluster.md | 2 +
docs/en/release.md | 2 +-
docs/zh/cluster.md | 2 +
docs/zh/release.md | 2 +-
9 files changed, 53 insertions(+), 76 deletions(-)
diff --git a/dev-support/README.md b/dev-support/README.md
index 98222b94..4c45fbac 100644
--- a/dev-support/README.md
+++ b/dev-support/README.md
@@ -32,25 +32,24 @@ yum install -y git docker
```shell
git clone https://github.com/apache/bigtop-manager.git
```
-> You need to change the `node.version` in the `pom.xml` file under the
`bigtop-manager-ui` module to `16.x` if you are using CentOS-7.
### **Step 3**: Build develop basic image
-Run the setup command, you will get `bigtop-manager/develop:trunk-rocky-8`
image. It has the environment needed to compile Bigtop-Manager and run servers
such as Bigtop-Manager Server, Bigtop-Manager Agent, Mysql, etc.
+Run the setup command, you will get `bigtop-manager/develop:rocky-8` image. It
has the environment needed to compile Bigtop-Manager and run servers such as
Bigtop-Manager Server, Bigtop-Manager Agent, Mysql, etc.
**RHEL (Rocky 8) :**
```shell
-/bin/bash dev-support/docker/image/build.sh trunk-rocky-8
+/bin/bash dev-support/docker/image/build.sh rocky-8
```
### **Step 4**: Build source & create cluster
-* Bigtop Manager UI、Bigtop Manager Server Debug Port、MariaDB Server are also
exposed to local ports: 8080、5005、3306.
-* Docker hostnames are:
bigtop-manager-server、bigtop-manager-agent-01、bigtop-manager-agent-02.
+* Bigtop Manager UI、Bigtop Manager Server Debug Port、MariaDB Server are also
exposed to local ports: 18080、15005、13306.
+* Docker hostnames are: bm-1、bm-2、bm-3 and etc.
**RHEL (Rocky 8) :**
```shell
-/bin/bash dev-support/docker/containers/build.sh -e postgres -c 3 -o
trunk-rocky-8 [--skip-compile]
+/bin/bash dev-support/docker/containers/build.sh -e postgres -c 3 -o rocky-8
[--skip-compile]
```
### **Step 5**: Access Web UI
-Now you can access Web UI which exposes on `http://localhost:8080`. Log in
with username `admin` and password `admin`.
+Now you can access Web UI which exposes on `http://localhost:18080`. Log in
with username `admin` and password `admin`.
### **Step 6**: Clear cluster
Clean up the containers when you are done developing or testing.
diff --git a/dev-support/docker/containers/build.sh
b/dev-support/docker/containers/build.sh
index 4fc0f4cc..5cfce3ac 100644
--- a/dev-support/docker/containers/build.sh
+++ b/dev-support/docker/containers/build.sh
@@ -20,7 +20,7 @@ usage() {
echo " commands:"
echo " -c NUM_INSTANCES, --create NUM_INSTANCES - Create Docker
containers based bigtop-manager cluster, defaults to 3"
echo " -e, --database - The specified
database, defaults to postgres"
- echo " -o, --os - Specify the
operating system, default is trunk-rocky-8"
+ echo " -o, --os - Specify the
operating system, default is rocky-8"
echo " --skip-compile - Skip Compile"
echo " -d, --destroy - Destroy all
containers"
echo " -h, --help"
@@ -32,8 +32,8 @@ log() {
}
build() {
- log "Build on docker $SKIP_BUILD"
- if ! $SKIP_BUILD; then
+ log "Build on docker: $SKIP_COMPILE"
+ if ! $SKIP_COMPILE; then
log "Compiling bigtop-manager"
docker run -it --rm -u $(id -u):$(id -g) \
-v $PWD/../../../:/opt/develop/bigtop-manager/ \
@@ -67,21 +67,20 @@ create_container() {
log "Create ${container_name}"
if [ $i -eq 1 ]; then
docker run -itd -p 15005:5005 -p 15006:5006 -p 18080:8080 --name
${container_name} --hostname ${container_name} --network bigtop-manager
--cap-add=SYS_TIME bigtop-manager/develop:${OS}
- docker cp
../../../bigtop-manager-server/target/bigtop-manager-server.tar.gz
${container_name}:/opt/
+ docker cp
../../../bigtop-manager-dist/target/apache-bigtop-manager-*-server.tar.gz
${container_name}:/opt/bigtop-manager-server.tar.gz
docker exec ${container_name} bash -c "cd /opt && tar -zxvf
bigtop-manager-server.tar.gz"
+ docker exec ${container_name} bash -c "ssh-keygen -f '/root/.ssh/id_rsa'
-N '' -t rsa"
SERVER_PUB_KEY=`docker exec ${container_name} /bin/cat
/root/.ssh/id_rsa.pub`
else
docker run -itd --name ${container_name} --hostname ${container_name}
--network bigtop-manager --cap-add=SYS_TIME bigtop-manager/develop:${OS}
fi
- docker cp
../../../bigtop-manager-agent/target/bigtop-manager-agent.tar.gz
${container_name}:/opt/
- docker exec ${container_name} bash -c "cd /opt && tar -zxvf
bigtop-manager-agent.tar.gz"
- docker exec ${container_name} bash -c "echo '$SERVER_PUB_KEY' >
/root/.ssh/authorized_keys"
- docker exec ${container_name} ssh-keygen -N '' -t rsa -b 2048 -f
/etc/ssh/ssh_host_rsa_key
- docker exec ${container_name} ssh-keygen -N '' -t ecdsa -b 256 -f
/etc/ssh/ssh_host_ecdsa_key
- docker exec ${container_name} ssh-keygen -N '' -t ed25519 -b 256 -f
/etc/ssh/ssh_host_ed25519_key
- docker exec ${container_name} /bin/systemctl start sshd
-
- docker exec ${container_name} bash -c "systemctl start chronyd &&
chronyc tracking"
+
+ docker exec ${container_name} bash -c "echo '$SERVER_PUB_KEY' >
/root/.ssh/authorized_keys"
+ docker exec ${container_name} ssh-keygen -N '' -t rsa -b 2048 -f
/etc/ssh/ssh_host_rsa_key
+ docker exec ${container_name} ssh-keygen -N '' -t ecdsa -b 256 -f
/etc/ssh/ssh_host_ecdsa_key
+ docker exec ${container_name} ssh-keygen -N '' -t ed25519 -b 256 -f
/etc/ssh/ssh_host_ed25519_key
+ docker exec ${container_name} /bin/systemctl start sshd
+ docker exec ${container_name} bash -c "systemctl start chronyd && chronyc
tracking"
done
containers=($(docker network inspect bigtop-manager -f '{{range
.Containers}}{{.Name}}{{" "}}{{end}}'))
@@ -109,14 +108,12 @@ create_container() {
docker exec ${container} bash -c "sed -i
's/localhost:5432/bm-mysql:3306/'
/opt/bigtop-manager-server/conf/application.yml"
docker exec ${container} bash -c "sed -i 's/postgres/root/'
/opt/bigtop-manager-server/conf/application.yml"
elif [ $DATABASE == "postgres" ]; then
- docker exec ${container} bash -c "yum install postgresql -y"
docker exec ${container} bash -c "PGPASSWORD=postgres psql -h
bm-postgres -p5432 -U postgres -c 'create database bigtop_manager'"
docker exec ${container} bash -c "PGPASSWORD=postgres psql -h
bm-postgres -p5432 -U postgres -d bigtop_manager -f
/opt/bigtop-manager-server/ddl/PostgreSQL-DDL-CREATE.sql"
docker exec ${container} bash -c "sed -i
's/localhost:5432/bm-postgres:5432/'
/opt/bigtop-manager-server/conf/application.yml"
fi
docker exec ${container} bash -c "nohup /bin/bash
/opt/bigtop-manager-server/bin/start.sh --debug > /dev/null 2>&1 &"
fi
- docker exec ${container} bash -c "nohup /bin/bash
/opt/bigtop-manager-agent/bin/start.sh --debug > /dev/null 2>&1 &"
log "All Service Started!!!"
done
@@ -125,7 +122,7 @@ create_container() {
create_db() {
if [ $DATABASE == "mysql" ]; then
docker run --restart=always -it -d \
- -p 3306:3306 \
+ -p 13306:3306 \
--cap-add=SYS_TIME \
--network bigtop-manager \
--name bm-mysql \
@@ -150,7 +147,7 @@ create_db() {
done
elif [ $DATABASE == "postgres" ]; then
docker run --restart=always -d \
- -p 5432:5432 \
+ -p 15432:5432 \
--network bigtop-manager \
--name bm-postgres \
--hostname bm-postgres \
@@ -167,9 +164,9 @@ echo $PWD
PROG=`basename $0`
DATABASE=postgres
-OS=trunk-rocky-8
+OS=rocky-8
NUM_INSTANCES=3
-SKIP_BUILD=false
+SKIP_COMPILE=false
while [ $# -gt 0 ]; do
case "$1" in
@@ -189,8 +186,8 @@ while [ $# -gt 0 ]; do
echo "Requires a os" 1>&2
usage
fi
- if [ $2 != "trunk-rocky-8" ] && [ $2 != "trunk-centos-7" ] && [ $2 !=
"trunk-openeuler-22" ]; then
- echo "The OS should be [trunk-rocky-8], [trunk-centos-7], or
[trunk-openeuler-22]" 1>&2
+ if [ $2 != "rocky-8" ] && [ $2 != "openeuler-24" ]; then
+ echo "The OS should be [rocky-8] or [openeuler-24]" 1>&2
usage
fi
OS=$2
@@ -207,7 +204,7 @@ while [ $# -gt 0 ]; do
NUM_INSTANCES=$2
shift 2;;
--skip-compile)
- SKIP_BUILD=true
+ SKIP_COMPILE=true
shift;;
-d|--destroy)
destroy
@@ -221,7 +218,7 @@ while [ $# -gt 0 ]; do
esac
done
-log "DATABASE: $DATABASE; OS: $OS; NUM_INSTANCES: $NUM_INSTANCES; SKIP_BUILD:
$SKIP_BUILD; "
+log "DATABASE: $DATABASE; OS: $OS; NUM_INSTANCES: $NUM_INSTANCES;
SKIP_COMPILE: $SKIP_COMPILE; "
build
diff --git a/dev-support/docker/image/Dockerfile.template
b/dev-support/docker/image/Dockerfile.openeuler24
similarity index 51%
copy from dev-support/docker/image/Dockerfile.template
copy to dev-support/docker/image/Dockerfile.openeuler24
index 1b365948..f8d8b68a 100644
--- a/dev-support/docker/image/Dockerfile.template
+++ b/dev-support/docker/image/Dockerfile.openeuler24
@@ -13,21 +13,17 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-FROM OS:VERSION
+FROM openeuler/openeuler:24.03
-MAINTAINER [email protected]
+RUN dnf install -y sudo wget openssh-clients openssh-server mariadb
mariadb-server net-tools chrony krb5-server krb5-libs krb5-workstation git
python3 procps-ng
+RUN dnf install -y postgresql java-17-openjdk-devel vim
-RUN yum -y install sudo wget openssh-clients openssh-server vim mariadb
mariadb-server net-tools chrony krb5-server krb5-libs krb5-workstation git
rpm-build python3 procps-ng
-RUN wget
https://download.java.net/java/GA/jdk17.0.2/dfd4a8d0985749f896bed50d7138ee7f/8/GPL/openjdk-17.0.2_linux-x64_bin.tar.gz
-O /tmp/jdk.tar.gz --no-check-certificate \
- && mkdir -p /usr/local/java && tar -xzf /tmp/jdk.tar.gz -C /usr/local/java
--strip-components=1 \
- && rm -f /tmp/jdk.tar.gz \
- && ln -s /usr/local/java/bin/java /usr/bin/java
-RUN wget
https://archive.apache.org/dist/maven/maven-3/3.8.8/binaries/apache-maven-3.8.8-bin.tar.gz
-O /tmp/apache-maven.tar.gz --no-check-certificate \
- && mkdir -p /usr/share/maven && tar -xzf /tmp/apache-maven.tar.gz -C
/usr/share/maven --strip-components=1 \
- && rm -f /tmp/apache-maven.tar.gz \
- && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
-RUN wget
https://raw.githubusercontent.com/gdraheim/docker-systemctl-replacement/master/files/docker/systemctl3.py
-O /usr/bin/systemctl
+ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk
+ENV PATH=$JAVA_HOME/bin:$PATH
+RUN wget --no-check-certificate
https://downloads.apache.org/maven/maven-3/3.8.8/binaries/apache-maven-3.8.8-bin.tar.gz
&& \
+ tar -xvf apache-maven-3.8.8-bin.tar.gz && \
+ mv apache-maven-3.8.8 /usr/lib/maven
+ENV PATH=${PATH}:/usr/lib/maven/bin
+RUN wget
https://raw.githubusercontent.com/gdraheim/docker-systemctl-replacement/master/files/docker/systemctl3.py
-O /usr/bin/systemctl
RUN /bin/sed -i 's,# StrictHostKeyChecking ask,StrictHostKeyChecking no,g'
/etc/ssh/ssh_config
-
-RUN ssh-keygen -f "/root/.ssh/id_rsa" -N ""
diff --git a/dev-support/docker/image/Dockerfile.template
b/dev-support/docker/image/Dockerfile.rocky8
similarity index 89%
rename from dev-support/docker/image/Dockerfile.template
rename to dev-support/docker/image/Dockerfile.rocky8
index 1b365948..978189a6 100644
--- a/dev-support/docker/image/Dockerfile.template
+++ b/dev-support/docker/image/Dockerfile.rocky8
@@ -13,11 +13,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-FROM OS:VERSION
+FROM rockylinux:8
-MAINTAINER [email protected]
-
-RUN yum -y install sudo wget openssh-clients openssh-server vim mariadb
mariadb-server net-tools chrony krb5-server krb5-libs krb5-workstation git
rpm-build python3 procps-ng
+RUN yum -y install sudo wget openssh-clients openssh-server vim postgresql
mariadb mariadb-server net-tools chrony krb5-server krb5-libs krb5-workstation
git rpm-build python3 procps-ng
RUN wget
https://download.java.net/java/GA/jdk17.0.2/dfd4a8d0985749f896bed50d7138ee7f/8/GPL/openjdk-17.0.2_linux-x64_bin.tar.gz
-O /tmp/jdk.tar.gz --no-check-certificate \
&& mkdir -p /usr/local/java && tar -xzf /tmp/jdk.tar.gz -C /usr/local/java
--strip-components=1 \
&& rm -f /tmp/jdk.tar.gz \
@@ -29,5 +27,3 @@ RUN wget
https://archive.apache.org/dist/maven/maven-3/3.8.8/binaries/apache-mav
RUN wget
https://raw.githubusercontent.com/gdraheim/docker-systemctl-replacement/master/files/docker/systemctl3.py
-O /usr/bin/systemctl
RUN /bin/sed -i 's,# StrictHostKeyChecking ask,StrictHostKeyChecking no,g'
/etc/ssh/ssh_config
-
-RUN ssh-keygen -f "/root/.ssh/id_rsa" -N ""
diff --git a/dev-support/docker/image/build.sh
b/dev-support/docker/image/build.sh
index cc0d380d..6f1238ad 100644
--- a/dev-support/docker/image/build.sh
+++ b/dev-support/docker/image/build.sh
@@ -33,46 +33,31 @@ echo $PWD
if [ $# != 1 ]; then
echo "Creates bigtop-manager/develop image"
echo
- echo "Usage: build.sh <PREFIX-OS-VERSION>"
+ echo "Usage: build.sh <OS-VERSION>"
echo
- echo "Example: build.sh trunk-rocky-8"
- echo " : build.sh 1.0.0-rocky-8"
+ echo "Example: build.sh rocky-8"
exit 1
fi
-PREFIX=$(echo "$1" | cut -d '-' -f 1)
-OS=$(echo "$1" | cut -d '-' -f 2)
-VERSION=$(echo "$1" | cut -d '-' -f 3-)
+OS=$(echo "$1" | cut -d '-' -f 1)
+VERSION=$(echo "$1" | cut -d '-' -f 2-)
# Decimals are not supported. Either use integers only
# e.g. 16.04 -> 16
VERSION_INT=$(echo "$VERSION" | cut -d '.' -f 1)
-log "PREFIX: ${PREFIX}; OS: ${OS}; VERSION: ${VERSION}; VERSION_INT:
${VERSION_INT}"
-DOCKER_OS=${OS}
-DOCKER_VERSION=${VERSION}
-CUSTOM_REPO=""
+log "OS: ${OS}; VERSION: ${VERSION}; VERSION_INT: ${VERSION_INT}"
case ${OS}-${VERSION_INT} in
- centos-7)
- CUSTOM_REPO="RUN mv /etc/yum.repos.d/CentOS-Base.repo
/etc/yum.repos.d/CentOS-Base.repo.back \
- && curl -o /etc/yum.repos.d/CentOS-Base.repo
http://mirrors.aliyun.com/repo/Centos-7.repo";
- ;;
rocky-8)
- DOCKER_OS=rockylinux
+ DOCKERFILE="Dockerfile.rocky8"
;;
- openeuler-22)
- DOCKER_OS=openeuler/openeuler
- DOCKER_VERSION=22.03
+ openeuler-24)
+ DOCKERFILE="Dockerfile.openeuler24"
;;
*)
echo "Unsupported OS ${OS}-${VERSION}."
exit 1
esac
-# generate Dockerfile for build
-sed -e "s|OS|${DOCKER_OS}|;s|VERSION|${DOCKER_VERSION}|" Dockerfile.template |
- sed -e "/MAINTAINER [email protected]/a\\$CUSTOM_REPO" > Dockerfile
-
-docker build --rm --no-cache -t
bigtop-manager/develop:${PREFIX}-${OS}-${VERSION} -f Dockerfile ../..
-rm -f Dockerfile
+docker build --rm --no-cache -t bigtop-manager/develop:${OS}-${VERSION} -f
${DOCKERFILE} ../..
diff --git a/docs/en/cluster.md b/docs/en/cluster.md
index da0b6440..19a32195 100644
--- a/docs/en/cluster.md
+++ b/docs/en/cluster.md
@@ -17,6 +17,8 @@ When creating a cluster, users need to fill in the following
information:
### Stack
The stack page is a display page that mainly shows which optional services are
available for subsequent installation.
+Currently, the Bigtop stack provides tarball files generated by Apache Bigtop
instead of rpm/deb packages.
+
Expect for our official repository, users can also set their own repository
which can configured here(due to license issue, MySQL/Grafana in Infra Stack
are downloaded from the official website instead of the repository).
diff --git a/docs/en/release.md b/docs/en/release.md
index a83e60cb..e91a88e1 100644
--- a/docs/en/release.md
+++ b/docs/en/release.md
@@ -353,7 +353,7 @@ Hello Community,
I am glad to announce that Apache Bigtop Manager 1.0.0 has been released.
The source release can be downloaded here:
-https://www.apache.org/dyn/closer.cgi/bigtop/bigtop-manager-1.0.0
+https://www.apache.org/dyn/closer.cgi/bigtop/bigtop-manager-1.0.0/
Detailed release notes can be checked here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12354831
diff --git a/docs/zh/cluster.md b/docs/zh/cluster.md
index 3dab536c..f13d0508 100644
--- a/docs/zh/cluster.md
+++ b/docs/zh/cluster.md
@@ -15,6 +15,8 @@
### 组件栈
组件栈页面为展示页面，主要显示后续安装服务时有哪些可选服务
+目前 Bigtop 组件栈提供的是 Apache Bigtop 生成的 tarball 文件而非 rpm/deb 包
+
除了官方 Repository 外，用户也可以搭建自己的 Repository 并在此处进行配置（注意 Infra 中的 MySQL/Grafana 由于
License 原因，是从官方网站下载而不是从 Repository 下载）
diff --git a/docs/zh/release.md b/docs/zh/release.md
index 67cfe751..e2493be9 100644
--- a/docs/zh/release.md
+++ b/docs/zh/release.md
@@ -353,7 +353,7 @@ Hello Community,
I am glad to announce that Apache Bigtop Manager 1.0.0 has been released.
The source release can be downloaded here:
-https://www.apache.org/dyn/closer.cgi/bigtop/bigtop-manager-1.0.0
+https://www.apache.org/dyn/closer.cgi/bigtop/bigtop-manager-1.0.0/
Detailed release notes can be checked here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12354831
