Re: [DISCUSS] Unified Apache Spark Docker image tag?

As I said in my last mail, I am very sorry if any mislead. As you know, it is a little bit complicated to take into account K8s, base image, standalone, Docker official image, etc., as well as various docker image requirements such as java version and docker image tag. Of course, it's not an

Re: [DISCUSS] Unified Apache Spark Docker image tag?

Hi, This has already been discussed a few times notably in August 2022 under the topic Time to start publishing Spark Docker Images? Having said that, building a docker image is a trivial job not taking a few minutes. Beside most cloud vendors they have their own specific tags. For example see

Re: [DISCUSS] Unified Apache Spark Docker image tag?

> 1. The size regression: `apache/spark:3.4.0` tag which is claimed to be a replacement of the existing `apache/spark:v3.4.0`. However, 3.4.0 is 500MB while the original v3.4.0 is 405MB. 25% is huge in terms of the size. > 2. Accidental overwrite: `apache/spark:latest` was accidentally

Re: [DISCUSS] Unified Apache Spark Docker image tag?

The whole content of SPIP (Support Docker Official Image for Spark) aims to add (1) newly, not to corrupt or destroy the existing (2). (1) https://hub.docker.com/_/spark (2) https://hub.docker.com/r/apache/spark/tags The reference model repos were also documented like the followings.

Re: [DISCUSS] Unified Apache Spark Docker image tag?

May I ask why you think that sentence, "might need to deprecate ..." of SPIP, decided anything at that time? >From my perspective, - `might need to` suggested only a possible necessity at some point in the future. - `deprecation` means no breaking change. Dongjoon On Tue, May 9, 2023 at

Re: [DISCUSS] Unified Apache Spark Docker image tag?

> It seems that your reply (the following) didn't reach out to the mailing list correctly. Thanks! I'm not sure what happened before, thanks for your forward > Let me add my opinion. IIUC, the whole content of SPIP (Support Docker Official Image for Spark) aims to add (1) newly, not to corrupt

Re: [DISCUSS] Unified Apache Spark Docker image tag?

To Yikun, It seems that your reply (the following) didn't reach out to the mailing list correctly. > Just FYI, we also had a discussion about tag policy (latest/3.4.0) and also rough size estimation [1] in "SPIP: Support Docker Official Image for Spark". >

Re: [DISCUSS] Unified Apache Spark Docker image tag?

Thank you for initiating the discussion in the community. Yes, we need to give more context in the dev mailing list. This root cause is not about SPARK-40941 or SPARK-40513. Technically, this situation started 16 days ago due to SPARK-43148 because it made some breaking changes.

[DISCUSS] Unified Apache Spark Docker image tag?

This is a call for discussion for how we can unified Apache Spark Docker image tag fluently. As you might know, there is an apache/spark-docker repo to store the dockerfiles and help to publish the docker images, also intended to replace the original