leborchuk commented on PR #1292: URL: https://github.com/apache/cloudberry/pull/1292#issuecomment-3191745413
> Hey @leborchuk, thanks for working on this! It’s great to see the devops-release scripts being brought into the main repository — this will definitely help improve maintenance and visibility moving forward. > > The following is my feedback for your reference: > > 1. Concern about Missing Commit History > > We see that this is a complex work. It looks like the original commit history from the old repository has been lost during the migration. While it’s fine to move the code over, that would be better to keep the original commit history. > > It will be a lot of noise for the main repo if we keep the full commit history. So I'd suggest squashing them into an initial import commit, which helps trace the origin of the scripts, like: > > ``` > Initial import from cloudberry-devops-release > > (original commit <sha>) > > xxxx > xxxx > ``` > > 2. For the tweaks to the old files, we can create new commits following the initial commits, which will make the changes obviously and the review work will be easier. > 3. Dir reorganization > > Now it's a good time to reorganzation files for a better management, also taking the future feature files into consideration. > > Here is a new file tree: > > ``` > apache/cloudberry/ > ├── .github/workflows/ # GitHub Actions > ├── devops/ # New: Unified DevOps directory > │ ├── README.md # DevOps overview documentation > │ ├── build/ # Build-related files > │ │ ├── automation/ # Build automation scripts > │ │ │ └── cloudberry/ > │ │ │ └── scripts/* > │ │ ├── packaging/ # Packaging-related files > │ │ │ ├── rpm/ # RPM packaging > │ │ │ │ ├── apache-cloudberry-db-incubating.spec > │ │ │ │ ├── apache-cloudberry-hll-incubating.spec > │ │ │ │ ├── apache-cloudberry-pgvector-incubating.spec > │ │ │ │ ├── cloudberry-dev-repo.spec > │ │ │ │ ├── build-rpm.sh > │ │ │ │ └── README.md > │ │ │ ├── deb/* # Future DEB packaging (reserved) > │ │ │ └── README.md # Packaging documentation > │ │ └── README.md # Build documentation > │ ├── deploy/ # Deployment-related files > │ │ ├── docker/ # Docker images > │ │ │ ├── build/ > │ │ │ │ ├── rocky8/ > │ │ │ │ ├── rocky9/ > │ │ │ │ └── ubuntu22.04/ # Future addition > │ │ │ ├── test/ > │ │ │ │ ├── rocky8/ > │ │ │ │ ├── rocky9/ > │ │ │ │ └── ubuntu22.04/ # Future addition > │ │ │ └── README.md > │ │ └── README.md # Deployment documentation > │ ├── release/ # Release management > │ │ ├── cloudberry-release.sh > │ │ └── README.md # Release process documentation > │ └── tools/ # Utility scripts > │ ├── elf_rockylinux_dependency_analyzer.py > │ ├── elf_ubuntu_dependency_analyzer.py > │ ├── s3-repo-sync-and-sign.sh > │ └── README.md # Tools documentation > ├── .asf.yaml # Remove the old one > ├── README.md # Remove the old one > ├── CONTRIBUTING.md # Remove the old one > ├── LICENSE # Remove the old one > ``` > > Directory Migration Mapping: > > ``` > # Build automation > build_automation/ → devops/build/automation/ > build_automation/cloudberry/ → devops/build/automation/cloudberry/ > build_automation/cloudberry/scripts/ → devops/build/automation/cloudberry/scripts/ > > # Packaging > packaging/ → devops/build/packaging/ > packaging/rpm/ → devops/build/packaging/rpm/ > packaging/rpm/el/SPECS/ → devops/build/packaging/rpm/ > packaging/rpm/repo-el/SPECS/ → devops/build/packaging/rpm/ > > # Docker images > images/ → devops/deploy/ > images/docker/ → devops/deploy/docker/ > images/docker/cbdb/ → devops/deploy/docker/ > images/docker/cbdb/build/ → devops/deploy/docker/build/ > images/docker/cbdb/test/ → devops/deploy/docker/test/ > images/docker/cbdb/build/rocky8/ → devops/deploy/docker/build/rocky8/ > images/docker/cbdb/build/rocky9/ → devops/deploy/docker/build/rocky9/ > images/docker/cbdb/test/rocky8/ → devops/deploy/docker/test/rocky8/ > images/docker/cbdb/test/rocky9/ → devops/deploy/docker/test/rocky9/ > > # Script files > scripts/ → # Distributed to different directories > scripts/release/ → devops/release/ > scripts/build-rpm.sh → devops/build/packaging/rpm/ > scripts/elf_rockylinux_dependency_analyzer.py → devops/tools/ > scripts/elf_ubuntu_dependency_analyzer.py → devops/tools/ > scripts/s3-repo-sync-and-sign.sh → devops/tools/ > ``` > > Files migration mapping: > > ``` > # Build automation scripts > build_automation/cloudberry/scripts/analyze_core_dumps.sh → devops/build/automation/cloudberry/scripts/analyze_core_dumps.sh > build_automation/cloudberry/scripts/build-cloudberry.sh → devops/build/automation/cloudberry/scripts/build-cloudberry.sh > build_automation/cloudberry/scripts/cloudberry-utils.sh → devops/build/automation/cloudberry/scripts/cloudberry-utils.sh > build_automation/cloudberry/scripts/configure-cloudberry.sh → devops/build/automation/cloudberry/scripts/configure-cloudberry.sh > build_automation/cloudberry/scripts/create-cloudberry-demo-cluster.sh → devops/build/automation/cloudberry/scripts/create-cloudberry-demo-cluster.sh > build_automation/cloudberry/scripts/destroy-cloudberry-demo-cluster.sh → devops/build/automation/cloudberry/scripts/destroy-cloudberry-demo-cluster.sh > build_automation/cloudberry/scripts/parse-results.pl → devops/build/automation/cloudberry/scripts/parse-results.pl > build_automation/cloudberry/scripts/parse-test-results.sh → devops/build/automation/cloudberry/scripts/parse-test-results.sh > build_automation/cloudberry/scripts/test-cloudberry.sh → devops/build/automation/cloudberry/scripts/test-cloudberry.sh > build_automation/cloudberry/scripts/unittest-cloudberry.sh → devops/build/automation/cloudberry/scripts/unittest-cloudberry.sh > > # RPM packaging files > packaging/rpm/el/SPECS/apache-cloudberry-db-incubating.spec → devops/build/packaging/rpm/apache-cloudberry-db-incubating.spec > packaging/rpm/el/SPECS/apache-cloudberry-hll-incubating.spec → devops/build/packaging/rpm/apache-cloudberry-hll-incubating.spec > packaging/rpm/el/SPECS/apache-cloudberry-pgvector-incubating.spec → devops/build/packaging/rpm/apache-cloudberry-pgvector-incubating.spec > packaging/rpm/repo-el/SPECS/cloudberry-dev-repo.spec → devops/build/packaging/rpm/cloudberry-dev-repo.spec > > # Script files > scripts/build-rpm.sh → devops/build/packaging/rpm/build-rpm.sh > scripts/release/cloudberry-release.sh → devops/release/cloudberry-release.sh > scripts/elf_rockylinux_dependency_analyzer.py → devops/tools/elf_rockylinux_dependency_analyzer.py > scripts/elf_ubuntu_dependency_analyzer.py → devops/tools/elf_ubuntu_dependency_analyzer.py > scripts/s3-repo-sync-and-sign.sh → devops/tools/s3-repo-sync-and-sign.sh > > # Docker related files (keep internal structure unchanged) > images/docker/cbdb/build/rocky8/* → devops/deploy/docker/build/rocky8/* > images/docker/cbdb/build/rocky9/* → devops/deploy/docker/build/rocky9/* > images/docker/cbdb/test/rocky8/* → devops/deploy/docker/test/rocky8/* > images/docker/cbdb/test/rocky9/* → devops/deploy/docker/test/rocky9/* > > # GitHub Actions workflows (file location unchanged, but need to update internal path references) > .github/workflows/docker-cbdb-build-containers.yml → .github/workflows/docker-cbdb-build-containers.yml > .github/workflows/docker-cbdb-test-containers.yml → .github/workflows/docker-cbdb-test-containers.yml > ``` > > Note: > > * For the files under `build/automation/cloudberry/scripts/*`, if no more future plans for them, we can directly place them in `build/automation/*`, making them easier to access. > * For the old `deploy` dir under `apache/cloudberry`, there are no files in it now. So we can make it as a subdir under devops. For the `cloudberry-bootcamp/sandbox`, I plan to migrate it to `devops/sandbox`. > * For the new README.md files, there is no need to create them all at once in this PR. We can do that in the future. > > Love to have your thoughts! @leborchuk @edespino Hi, @tuhaihe All done, could you launch the tests in order to check if everything is fine here in github actions? I do not have rights do it by myself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
