edespino commented on PR #11:
URL: https://github.com/apache/cloudberry-pxf/pull/11#issuecomment-3424066100
PXF CI Workflow Refactoring - Work in Progress
I'm currently testing a new CI architecture that separates Cloudberry builds
from PXF testing to improve efficiency and enable test matrices.
Current PR Approach (Original)
- Builds Cloudberry from source on every PXF CI run (~20 minutes)
- Single workflow, monolithic approach
- Works, but slow and not reusable
New Approach Being Tested (branch: PXF-ci)
Two workflows:
1. build-cloudberry-rpm.yml (runs once, manually or on-demand)
- Builds Cloudberry RPM using official build scripts
- Uploads RPM + source as artifacts (30-day retention)
- ~20 minutes, but only runs when needed
2. pxf-ci.yml (runs on every PXF change)
- Job 1: Downloads pre-built artifacts from build-cloudberry-rpm.yml
- Job 2: Installs RPM, creates cluster, tests PXF
- ~15-20 minutes total (vs 35-40 minutes original)
Benefits:
- 50% faster CI when reusing RPMs
- RPM-based installation matches production deployment
- Sets up test matrix support (test against multiple Cloudberry
versions/configs)
- Cleaner separation: Cloudberry build vs PXF testing
Status: Actively debugging artifact paths and cluster setup. Hope to have it
working soon, but the original approach in this PR remains valid and functional.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
