[
https://issues.apache.org/jira/browse/SPARK-39522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558067#comment-17558067
]
Yikun Jiang commented on SPARK-39522:
-------------------------------------
I have some questions here, the cache seems to just reduce the build time
(docker build), and the current infra uses static sha images (docker run/pull),
so in theory there won't be any improvement.
So, what you mean is that we want to migrate
dongjoon-hyun/ApacheSparkGitHubActionImage to apache/spark, and then in each
pyspark workflow, do a full docker build in every pr triggered job, and speed
it up by caching?
> Uses Docker image cache over a custom image
> -------------------------------------------
>
> Key: SPARK-39522
> URL: https://issues.apache.org/jira/browse/SPARK-39522
> Project: Spark
> Issue Type: Sub-task
> Components: Project Infra
> Affects Versions: 3.4.0
> Reporter: Hyukjin Kwon
> Priority: Major
>
> We should probably replace the base image
> (https://github.com/apache/spark/blob/master/.github/workflows/build_and_test.yml#L302,
> https://github.com/dongjoon-hyun/ApacheSparkGitHubActionImage) to plain
> ubunto image w/ Docker image cache. See also
> https://github.com/docker/build-push-action/blob/master/docs/advanced/cache.md
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
