Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

On Fri, Feb 25, 2022 at 8:31 AM Ismaël Mejía wrote: > The ready to use docker images are great news. I have been waiting for > this for so long! Extra kudos for including ARM64 versions too! > > I am curious, what are the non-ASF artifacts included in them (or you > refer to the OS specific

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

The ready to use docker images are great news. I have been waiting for this for so long! Extra kudos for including ARM64 versions too! I am curious, what are the non-ASF artifacts included in them (or you refer to the OS specific elements with other licenses?), and what consequences might be for

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

I understand that, and we do so, but news about official images were breaking me Ok, I will follow you on those activities. Thanks for the quick response. On Tue, 22 Feb 2022 at 22:03, Holden Karau wrote: > So your more than welcome to still build your own Spark docker containers > with

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

So your more than welcome to still build your own Spark docker containers with the docker image tool, these are provided to make it easier for folks without specific needs. In the future well hopefully have published Spark containers tagged for different JDKs but that work has not yet been done.

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

Hello Holden, Could you please provide more details and plan for docker images support? So far I see that there are only two tags, I get from them spark version, but there is no information about java, hadoop, scala versions. Also there is no description on docker hub, probably it would be nice

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

Well that is just a recommendation. The onus is on me the user to download and go through dev and test running suite of batch jobs to ensure that all work ok, especially on the edge, sign the release off and roll it in out into production. It won’t be prudent otherwise. HHH On Tue, 22 Feb 2022

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

"Spark 3.1.3 is a maintenance release containing stability fixes. This release is based on the branch-3.1 maintenance branch of Spark. We strongly recommend all 3.1.3 users to upgrade to this stable release." https://spark.apache.org/releases/spark-release-3-1-3.html Do we have another 3.13 or do

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

Hi, seems - [SPARK-35391] : Memory leak in ExecutorAllocationListener breaks dynamic allocation under high load Links to wrong jira ticket? Mich Talebzadeh 于2022年2月22日周二 15:49写道： > Well, that is pretty easy to do. > > However, a

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

Well, that is pretty easy to do. However, a quick fix for now could be to retag the image created. It is a small volume which can be done manually for now. For example, I just downloaded v3.1.3 docker image ls REPOSITORY TAG IMAGE ID CREATED

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

Apologies, please ignore my previous message On Mon, Feb 21, 2022 at 5:56 PM Prasad Paravatha wrote: > FYI, I am getting 404 for https://hub.docker.com/apache/spark > > On Mon, Feb 21, 2022 at 5:51 PM Holden Karau wrote: > >> Yeah I think we should still adopt that naming convention, however

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

FYI, I am getting 404 for https://hub.docker.com/apache/spark On Mon, Feb 21, 2022 at 5:51 PM Holden Karau wrote: > Yeah I think we should still adopt that naming convention, however no one > has taken the time submit write a script to do it yet so until we get that > script merged I think

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

Yeah I think we should still adopt that naming convention, however no one has taken the time submit write a script to do it yet so until we get that script merged I think we'll just have one build. I can try and do that for the next release but it would be a great 2nd issue for someone getting

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

Ok thanks for the correction. The docker pull line shows as follows: docker pull apache/spark:v3.2.1 So this only tells me the version of Spark 3.2.1 I thought we discussed deciding on the docker naming conventions in detail, and broadly agreed on what needs to be in the naming convention.

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

My bad, the correct link is: https://hub.docker.com/r/apache/spark/tags On Mon, Feb 21, 2022 at 1:17 PM Mich Talebzadeh wrote: > well that docker link is not found! may be permission issue > > [image: image.png] > > > > >view my Linkedin profile >

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

well that docker link is not found! may be permission issue [image: image.png] view my Linkedin profile https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any

[ANNOUNCE] Apache Spark 3.1.3 released + Docker images

We are happy to announce the availability of Spark 3.1.3! Spark 3.1.3 is a maintenance release containing stability fixes. This release is based on the branch-3.1 maintenance branch of Spark. We strongly recommend all 3.1 users to upgrade to this stable release. To download Spark 3.1.3, head