edespino commented on code in PR #1379: URL: https://github.com/apache/cloudberry/pull/1379#discussion_r2410067083
########## .github/workflows/README.md: ########## @@ -0,0 +1,255 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# GitHub Actions Workflows + +This directory contains GitHub Actions workflows for Apache Cloudberry CI/CD. + +## Table of Contents + +- [Available Workflows](#available-workflows) +- [Manual Workflow Triggers](#manual-workflow-triggers) +- [Artifact Reuse for Faster Testing](#artifact-reuse-for-faster-testing) +- [Running Workflows in Forked Repositories](#running-workflows-in-forked-repositories) + +## Available Workflows + +| Workflow | Purpose | Trigger | +|----------|---------|---------| +| `build-cloudberry.yml` | Main CI: build, test, create RPMs | Push, PR, Manual | +| `build-dbg-cloudberry.yml` | Debug build with assertions enabled | Push, PR, Manual | +| `apache-rat-audit.yml` | License header compliance check | Push, PR | +| `coverity.yml` | Static code analysis with Coverity | Weekly, Manual | +| `sonarqube.yml` | Code quality analysis with SonarQube | Push to main | +| `docker-cbdb-build-containers.yml` | Build Docker images for CI | Manual | +| `docker-cbdb-test-containers.yml` | Build test Docker images | Manual | + +## Manual Workflow Triggers + +Many workflows support manual triggering via `workflow_dispatch`, allowing developers to run CI jobs on-demand. + +### How to Manually Trigger a Workflow + +1. Navigate to the **Actions** tab in GitHub +2. Select the workflow from the left sidebar (e.g., "Build and Test Cloudberry") +3. Click **Run workflow** button (top right) +4. Select your branch +5. Configure input parameters (if available) +6. Click **Run workflow** + +### Workflow Input Parameters + +#### `build-cloudberry.yml` - Main CI + +| Parameter | Description | Default | Example | +|-----------|-------------|---------|---------| +| `test_selection` | Comma-separated list of tests to run, or "all" | `all` | `ic-good-opt-off,ic-contrib` | +| `reuse_artifacts_from_run_id` | Run ID to reuse build artifacts from (see below) | _(empty)_ | `12345678901` | + +**Available test selections:** +- `all` - Run all test suites +- `ic-good-opt-off` - Installcheck with optimizer off +- `ic-good-opt-on` - Installcheck with optimizer on +- `ic-contrib` - Contrib extension tests +- `ic-resgroup` - Resource group tests +- `ic-resgroup-v2` - Resource group v2 tests +- `ic-resgroup-v2-memory-accounting` - Resource group memory tests +- `ic-singlenode` - Single-node mode tests +- `make-installcheck-world` - Full test suite +- And more... (see workflow for complete list) + +## Artifact Reuse for Faster Testing + +When debugging test failures, rebuilding Cloudberry (~50-70 minutes) on every iteration is inefficient. The artifact reuse feature allows you to reuse build artifacts from a previous successful run. + +### How It Works + +1. Build artifacts (RPMs, source tarballs) from a previous workflow run are downloaded +2. Build job is skipped (saves ~45-60 minutes) +3. RPM installation test is skipped (saves ~5-10 minutes) +4. Test jobs run with the reused artifacts +5. You can iterate on test configurations without rebuilding + +### Step-by-Step Guide + +#### 1. Find the Run ID + +After a successful build (even if tests failed), get the run ID: + +**Option A: From GitHub Actions UI** +- Go to **Actions** tab → Click on a completed workflow run +- The URL will be: `https://github.com/apache/cloudberry/actions/runs/12345678901` +- The run ID is `12345678901` + +**Option B: From GitHub API** +```bash +# List recent workflow runs +gh run list --workflow=build-cloudberry.yml --limit 5 + +# Get run ID from specific branch +gh run list --workflow=build-cloudberry.yml --branch=my-feature --limit 1 +``` + +#### 2. Trigger New Run with Artifact Reuse + +**Via GitHub UI:** +1. Go to **Actions** → **Build and Test Cloudberry** +2. Click **Run workflow** +3. Enter the run ID in **"Reuse build artifacts from a previous run ID"** +4. Optionally customize **test_selection** +5. Click **Run workflow** + +**Via GitHub CLI:** +```bash +# Reuse artifacts from run 12345678901, run only specific tests +gh workflow run build-cloudberry.yml \ + --field reuse_artifacts_from_run_id=12345678901 \ + --field test_selection=ic-good-opt-off +``` + +#### 3. Monitor Test Execution + +- Build job will be skipped (shows as "Skipped" in Actions UI) +- RPM Install Test will be skipped +- Test jobs will run with artifacts from the specified run ID +- Total time: ~15-30 minutes (vs ~65-100 minutes for full build+test) + +### Use Cases + +**Debugging a specific test failure:** +```bash +# Run 1: Full build + all tests (finds test failure in ic-good-opt-off) +gh workflow run build-cloudberry.yml + +# Get the run ID from output +RUN_ID=$(gh run list --workflow=build-cloudberry.yml --limit 1 --json databaseId --jq '.[0].databaseId') + +# Run 2: Reuse artifacts, run only failing test +gh workflow run build-cloudberry.yml \ + --field reuse_artifacts_from_run_id=$RUN_ID \ + --field test_selection=ic-good-opt-off +``` + +**Testing different configurations:** +```bash +# Test with optimizer off, then on, using same build +gh workflow run build-cloudberry.yml \ + --field reuse_artifacts_from_run_id=$RUN_ID \ + --field test_selection=ic-good-opt-off + +gh workflow run build-cloudberry.yml \ + --field reuse_artifacts_from_run_id=$RUN_ID \ + --field test_selection=ic-good-opt-on +``` + +### Limitations + +- Artifacts expire after 90 days (GitHub default retention) +- Run ID must be from the same repository (or accessible fork) +- Artifacts must include both RPM and source build artifacts +- Cannot reuse artifacts across different OS/architecture combinations +- Changes to source code require a fresh build + +## Running Workflows in Forked Repositories + +GitHub Actions workflows are enabled in forks, allowing you to validate changes before submitting a Pull Request. + +### Initial Setup (One-Time) + +1. **Fork the repository** to your GitHub account + +2. **Enable GitHub Actions** in your fork: + - Go to your fork's **Actions** tab + - Click **"I understand my workflows, go ahead and enable them"** + +3. **Configure secrets** (if needed for advanced features): + - Go to fork **Settings** → **Secrets and variables** → **Actions** Review Comment: I updated the section. Thank you for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
