GitHub user Yicong-Huang created a discussion: Adopt a UCI-sponsored self-hosted GitHub Actions runner
Hi everyone, I'd like to start a discussion on whether Apache Texera (Incubating) should adopt a self-hosted GitHub Actions runner sponsored and hosted by UC Irvine (UCI). ## Motivation The motivation of adopting self-hosted runners are: - **CI queueing.** Apache projects share the same pool of GitHub-hosted runners, so our jobs frequently wait in the queue during busy periods despite our relatively low usage. - **Benchmark reproducibility.** GitHub-hosted runners vary in hardware and share resources with unrelated workloads, making benchmark results noisy. We do **not** have a GitHub Actions quota issue. Texera currently uses about **240k minutes/month** (~60k/week), only **24%** of ASF's weekly allocation. <details> <summary><strong>Existing practice in ASF</strong></summary> - Apache Airflow uses self-hosted GitHub Actions runners. - Apache Flink runs CI on sponsored Azure Pipelines. - Apache Arrow maintains dedicated benchmark machines. I do **not** recommend Spark's fork-CI model, where contributors run CI from their own forks. Besides the contributor experience, it does not solve benchmark reproducibility. </details> ## Proposal Use a **UCI-sponsored self-hosted runner** for trusted workflows. **Proposed Machine Spec** (Up to revisions) - 64 CPU cores - 256 GB RAM - Ephemeral GitHub Actions runners **Routing** - Fork PRs → `ubuntu-latest` - `merge_group` / trusted branches → self-hosted This follows the same general approach as Apache Airflow. Compared to GitHub-hosted runners, this removes shared-runner contention while also enabling warm build caches and a consistent benchmark environment. <details> <summary><strong>Risks & ownership</strong></summary> **Benchmark isolation.** CPU pinning alone may not be sufficient. One option is to reserve dedicated benchmark windows. **Host failure.** CI can fall back to GitHub-hosted runners by changing the runner label. **Security.** Self-hosted runners remain ephemeral and execute only trusted workflows. Fork PRs continue to use GitHub-hosted runners. **Ownership.** The machine would be hosted and maintained by UC Irvine. ASF Infra would only manage the runner registration. </details> ## Discussion I'd appreciate feedback on: - whether this direction makes sense; - benchmark isolation; - operational ownership; - whether a single machine is sufficient. If there is consensus, I'll follow up with a **[VOTE]** on the dev mailing list and an ASF INFRA request. GitHub link: https://github.com/apache/texera/discussions/6973 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
