sandeepgupta12 commented on issue #43817:
URL: https://github.com/apache/arrow/issues/43817#issuecomment-2619171798
Hi team,
I wanted to share how the ephemeral self-hosted runner solution works for
ppc64le builds, inspired by the approach used for s390x in Pytorch. Here’s a
detailed overview:
**1. Infrastructure Setup:**
○ A self-hosted runner is deployed on an OSU-provided VM
specifically for the ppc64le architecture.
○ The runner is configured to remain idle and is triggered only
for specific events (e.g., commits to the main branch, pull requests affecting
ppc64le-related code).
**2. Ephemeral Runner Inside a Container:**
○ The runner runs inside a container, ensuring complete
isolation and consistency.
**○ Key Behavior:**
§ The runner is registered as an ephemeral runner in
GitHub Actions.
§ Once a job is triggered, the runner performs the task.
§ After the job is completed, the container stops, the
runner unregisters itself automatically, and the container is deleted.
§ For subsequent jobs, a new container is created with
a clean environment, avoiding any conflicts from previous runs.
§ To manage the runner, the command systemctl enable
--now actions-runner@$NAME ensures that the runner is started.
**3. Workflow Integration:**
○ A dedicated job is added to the workflow YAML to target the
ppc64le architecture.
○ The job utilizes Docker containers to maintain a clean and
reproducible environment for building and testing.
**4. Execution Flow:**
○ When a relevant event occurs (e.g., a new commit or pull
request), a GitHub webhook triggers the job.
○ The ephemeral runner (inside the container) picks up the job,
executes the build and tests, unregisters itself, and the container is deleted
after completion.
This containerized ephemeral runner setup has proven effective for managing
ppc64le builds and tests efficiently while ensuring scalability, isolation, and
optimal resource utilization. If you have any questions or need more details,
feel free to ask!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
