[
https://issues.apache.org/jira/browse/SPARK-35607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang updated SPARK-35607:
--------------------------------
Issue Type: Umbrella (was: Test)
> Migrate Spark Arm Job from Jenkins to GitHub Actions
> ----------------------------------------------------
>
> Key: SPARK-35607
> URL: https://issues.apache.org/jira/browse/SPARK-35607
> Project: Spark
> Issue Type: Umbrella
> Components: Project Infra
> Affects Versions: 3.1.2
> Reporter: Yikun Jiang
> Priority: Major
>
> There were two Arm CI jobs in AMP lab:
> [https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-python-arm/]
> [https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-arm/]
> As note from ML[1], AMPLab jenkins had been shut down at the end of 2021.
> We should consider to migrate the Arm job from jenkins to somewhere like
> github action. Unfortunately, the github action doen't support native arm64
> vitualenv yet.
>
> There are some two possible way to complete migrations:
> * Github Action self hosted
> ** Running jobs in ARM VM.
> ** Potential risk mentioned in [2], but can solve
> ** Reuse many actions and build script
> ** Easy to migrate when official GA arm support ready
> * Github Action + 3rd Party Kubernetes Cluster
> ** Running jobs in the pod of ARM k8s cluster and print log in github action.
> ** We have to pack separate build/script in dockerfile.
> ** We can't easy to use existing actions
> ** Bring many pontential cost of maintainance
> Looks like 2nd way, bring many pontential cost of maintainance and not easy
> to migrate when native arm supported in GA, so I think Github Action self
> hosted with strict allow list would be best choice in here.
>
> [1] [https://www.mail-archive.com/dev@spark.apache.org/msg28480.html]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
