Hi Leonid, Thanks for driving this work!
Once it's completed, we can ask the ASF Infra team to archive the old `apache/cloudberry-devops-release` repo. All new CICD-related work will be conducted in the main repo. For the `cloudberry-bootcamp/sandbox`, I also plan to migrate it to `devops/sandbox` based on the new file structure. Best, Dianjin Wang On Fri, Aug 29, 2025 at 8:09 PM Leonid Borchuk <le.borc...@gmail.com> wrote: > > Hi, developers! > > As we discussed here I have created PR to move cloudberry-devops-release to > the main repository. This should simplify the process of developing new > features - one can add packages to build, change configure scripts and use it > in a code - all in one PR. > > The new file tree is > > apache/cloudberry/ > ├── .github/workflows/ # GitHub Actions > ├── devops/ # New: Unified DevOps directory > │ ├── README.md # DevOps overview documentation > │ ├── build/ # Build-related files > │ │ ├── automation/ # Build automation scripts > │ │ │ └── cloudberry/ > │ │ │ └── scripts/* > │ │ ├── packaging/ # Packaging-related files > │ │ │ ├── rpm/ # RPM packaging > │ │ │ │ ├── apache-cloudberry-db-incubating.spec > │ │ │ │ ├── apache-cloudberry-hll-incubating.spec > │ │ │ │ ├── apache-cloudberry-pgvector-incubating.spec > │ │ │ │ ├── cloudberry-dev-repo.spec > │ │ │ │ ├── build-rpm.sh > │ │ │ │ └── README.md > │ │ │ ├── deb/* # Future DEB packaging (reserved) > │ │ │ └── README.md # Packaging documentation > │ │ └── README.md # Build documentation > │ ├── deploy/ # Deployment-related files > │ │ ├── docker/ # Docker images > │ │ │ ├── build/ > │ │ │ │ ├── rocky8/ > │ │ │ │ ├── rocky9/ > │ │ │ │ └── ubuntu22.04/ # Future addition > │ │ │ ├── test/ > │ │ │ │ ├── rocky8/ > │ │ │ │ ├── rocky9/ > │ │ │ │ └── ubuntu22.04/ # Future addition > │ │ │ └── README.md > │ │ └── README.md # Deployment documentation > │ ├── release/ # Release management > │ │ ├── cloudberry-release.sh > │ │ └── README.md # Release process documentation > │ └── tools/ # Utility scripts > │ ├── elf_rockylinux_dependency_analyzer.py > │ ├── elf_ubuntu_dependency_analyzer.py > │ ├── s3-repo-sync-and-sign.sh > │ └── README.md # Tools documentation > > What I need - please review PR for moving files > https://github.com/apache/cloudberry/pull/1292 > > I with @Dianjin Wang have been discussing changes but since the changes could > affect everyone, additional review and approval are strictly recommended. > > > On Tue, Jul 15, 2025 at 2:00 PM Dianjin Wang <wangdian...@gmail.com> wrote: >> >> Hi all, >> >> Regarding the sandbox files in the bootcamp, I plan to move them to >> the `deploy/sandbox` directory. I also plan to refactor them to use >> the Cloudberry build image as the builder. This will help us avoid >> maintaining duplicate base image Docker files across multiple >> directories. >> >> BTW, do we have a plan for moving `cloudberry-release-devops` to the >> main repository? There's currently a PR that requires introducing >> extra dependencies, and developers are having difficulty finding them >> to edit. >> >> Best, >> Dianjin Wang >> >> >> On Mon, Jun 9, 2025 at 5:52 PM Dianjin Wang <wangdian...@gmail.com> wrote: >> > >> > Cool! Then I need to rethink the way how to reorganize the >> > `cloudberry-bootcamp` better after our `cloudberry-devops-release` >> > move to the main repo. >> > >> > Best, >> > Dianjin Wang >> > >> > >> > On Mon, Jun 9, 2025 at 5:25 PM Ed Espino <esp...@apache.org> wrote: >> > > >> > > Hi Leonid, >> > > >> > > Totally agree — let’s move the build scripts into the main repo. >> > > >> > > Having everything in one place will simplify development, especially for >> > > larger changes like PAX support. It’ll streamline testing, reduce >> > > cross-repo friction, and make iteration much faster. We can always >> > > revisit >> > > a split setup later if our release needs grow. >> > > >> > > Thanks for raising this — fully on board. >> > > >> > > -=e >> > > >> > > >> > > On Mon, Jun 9, 2025 at 1:47 AM Leonid Borchuk <le.borc...@gmail.com> >> > > wrote: >> > > >> > > > I totally agree. It would be great to have a single library where all >> > > > of >> > > > our >> > > > assets are stored. >> > > > >> > > > But until we start large-scale reconstruction work, I would like to >> > > > ask a >> > > > slightly off-topic question: Why do we even need a >> > > > cloudberry-release-devops? >> > > > Why can't all our scripts be stored in the main repository? >> > > > >> > > > It happens to be rather tedious to use two repos, especially when >> > > > changing >> > > > something big. Such as adding PAX support. For such a big change you >> > > > need >> > > > to commit something breaking the building to the main repo. And at that >> > > > point, your tests will fail. Then you have to fix the >> > > > cloudberry-release-devops (add additional packages), but the tests may >> > > > still fail. And now you could fix the tests and make a final commit. >> > > > And >> > > > only now can one check if everything is OK. We could store all our >> > > > scripts >> > > > together with the code. >> > > > >> > > > Best regards, Leonid >> > > > >> > > > On Mon, Jun 9, 2025 at 9:05 AM Dianjin Wang <wangdian...@gmail.com> >> > > > wrote: >> > > > >> > > > > Hi all, >> > > > > >> > > > > Over the past two years, the bootcamp repo has grown to include >> > > > > various types of content, such as sandbox Dockerfiles, tutorials, and >> > > > > learning materials. I’d like to start a discussion around >> > > > > reorganizing >> > > > > the `apache/cloudberry-bootcamp` repo to improve clarity and >> > > > > maintainability across our community resources. >> > > > > >> > > > > The primary motivation is to consolidate content by type and reduce >> > > > > redundancy, which would: >> > > > > * Make it easier for contributors to find and update documentation >> > > > > * Reduce ongoing maintenance overhead >> > > > > >> > > > > ## Proposal (for discussion) >> > > > > >> > > > > I propose reorganizing this bootcamp repo as follows: >> > > > > >> > > > > * Move sandbox-related files to `apache/cloudberry-devops-release`, >> > > > > where we maintain tools related to deployment and release automation >> > > > > -- a new dir called `sandbox` will be created to store them. >> > > > > * For the benchmark test files -- a new directory called `benchmark` >> > > > > will be created under `apache/cloudberry-devops-release`, but we can >> > > > > do this until these files are ready. >> > > > > * Migrate tutorials, crash courses, and learning content to the >> > > > > `apache/cloudberry-site` repo, which now serves as the single source >> > > > > for end-user documentation. All of these materials have been copied >> > > > > to >> > > > > our website repo [1]. >> > > > > >> > > > > Once the content has been fully migrated and verified, plan to: >> > > > > >> > > > > * Archive the cloudberry-bootcamp repository, with a notice in the >> > > > > README indicating that it is no longer actively maintained and that >> > > > > the relevant content has been moved. >> > > > > >> > > > > ## Request for Feedback >> > > > > >> > > > > This is just a proposal at this stage. I’d love to hear your thoughts >> > > > > and suggestions. If most of the members agree, I would like to help >> > > > > take action on this plan. >> > > > > >> > > > > Thanks, and looking forward to your feedback! >> > > > > >> > > > > [1] https://cloudberry.apache.org/bootcamp#cloudberry-sandbox >> > > > > >> > > > > Best, >> > > > > Dianjin Wang >> > > > > >> > > > > --------------------------------------------------------------------- >> > > > > To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org >> > > > > For additional commands, e-mail: dev-h...@cloudberry.apache.org >> > > > > >> > > > > >> > > > >> > > >> > > >> > > -- >> > > Ed Espino >> > > Apache Cloudberry (Incubating) & MADlib >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org >> For additional commands, e-mail: dev-h...@cloudberry.apache.org >> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org For additional commands, e-mail: dev-h...@cloudberry.apache.org
