"Spark 3.1.3 is a maintenance release containing stability fixes. This release is based on the branch-3.1 maintenance branch of Spark. We strongly recommend all 3.1.3 users to upgrade to this stable release." https://spark.apache.org/releases/spark-release-3-1-3.html
Do we have another 3.13 or do we strongly recommend all 3.1.2 users to upgrade to this stable release ? tir. 22. feb. 2022 kl. 09:50 skrev angers zhu <angers....@gmail.com>: > Hi, seems > > - [SPARK-35391] <https://issues.apache.org/jira/browse/SPARK-36339>: > Memory leak in ExecutorAllocationListener breaks dynamic allocation under > high load > > Links to wrong jira ticket? > > Mich Talebzadeh <mich.talebza...@gmail.com> 于2022年2月22日周二 15:49写道： > >> Well, that is pretty easy to do. >> >> However, a quick fix for now could be to retag the image created. It is a >> small volume which can be done manually for now. For example, I just >> downloaded v3.1.3 >> >> >> docker image ls >> >> REPOSITORY TAG >> IMAGE ID CREATED SIZE >> >> apache/spark v3.1.3 >> 31ed15daa2bf 12 hours ago 531MB >> >> Retag it with >> >> >> docker tag 31ed15daa2bf >> apache/spark/tags/spark-3.1.3-scala_2.12-8-jre-slim-buster >> >> docker image ls >> >> REPOSITORY TAG >> IMAGE ID CREATED SIZE >> >> apache/spark/tags/spark-3.1.3-scala_2.12-8-jre-slim-buster latest >> 31ed15daa2bf 12 hours ago >> 531MB >> >> Then push it with (example) >> >> docker push apache/spark/tags/spark-3.1.3-scala_2.12-8-jre-slim-buster >> >> >> HTH >> >> >> view my Linkedin profile >> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >> >> >> https://en.everybodywiki.com/Mich_Talebzadeh >> >> >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any loss, damage or destruction of data or any other property which may >> arise from relying on this email's technical content is explicitly >> disclaimed. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> >> >> On Mon, 21 Feb 2022 at 23:51, Holden Karau <hol...@pigscanfly.ca> wrote: >> >>> Yeah I think we should still adopt that naming convention, however no >>> one has taken the time submit write a script to do it yet so until we get >>> that script merged I think we'll just have one build. I can try and do that >>> for the next release but it would be a great 2nd issue for someone getting >>> more familiar with the release tooling. >>> >>> On Mon, Feb 21, 2022 at 2:18 PM Mich Talebzadeh < >>> mich.talebza...@gmail.com> wrote: >>> >>>> Ok thanks for the correction. >>>> >>>> The docker pull line shows as follows: >>>> >>>> docker pull apache/spark:v3.2.1 >>>> >>>> >>>> So this only tells me the version of Spark 3.2.1 >>>> >>>> >>>> I thought we discussed deciding on the docker naming conventions in >>>> detail, and broadly agreed on what needs to be in the naming convention. >>>> For example, in this thread: >>>> >>>> >>>> Time to start publishing Spark Docker Images? - >>>> mich.talebza...@gmail.com - Gmail (google.com) >>>> <https://mail.google.com/mail/u/0/?hl=en-GB#search/publishing/FMfcgzGkZQSzbXWQDWfddGDNRDQfPCpg> >>>> dated >>>> 22nd July 2021 >>>> >>>> >>>> Referring to that, I think the broad agreement was that the docker >>>> image name should be of the form: >>>> >>>> >>>> The name of the file provides: >>>> >>>> - Built for spark or spark-py (PySpark) spark-r >>>> - Spark version: 3.1.1, 3.1.2, 3.2.1 etc. >>>> - Scala version; 2.1.2 >>>> - The OS version based on JAVA: 8-jre-slim-buster, >>>> 11-jre-slim-buster meaning JAVA 8 and JAVA 11 respectively >>>> >>>> I believe it is a good thing and we ought to adopt that convention. For >>>> example: >>>> >>>> >>>> spark-py-3.2.1-scala_2.12-11-jre-slim-buster >>>> >>>> >>>> HTH >>>> >>>> >>>> >>>> view my Linkedin profile >>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>> >>>> >>>> https://en.everybodywiki.com/Mich_Talebzadeh >>>> >>>> >>>> >>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>>> any loss, damage or destruction of data or any other property which may >>>> arise from relying on this email's technical content is explicitly >>>> disclaimed. The author will in no case be liable for any monetary damages >>>> arising from such loss, damage or destruction. >>>> >>>> >>>> >>>> >>>> On Mon, 21 Feb 2022 at 21:58, Holden Karau <hol...@pigscanfly.ca> >>>> wrote: >>>> >>>>> My bad, the correct link is: >>>>> >>>>> https://hub.docker.com/r/apache/spark/tags >>>>> >>>>> On Mon, Feb 21, 2022 at 1:17 PM Mich Talebzadeh < >>>>> mich.talebza...@gmail.com> wrote: >>>>> >>>>>> well that docker link is not found! may be permission issue >>>>>> >>>>>> [image: image.png] >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> view my Linkedin profile >>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>>>> >>>>>> >>>>>> https://en.everybodywiki.com/Mich_Talebzadeh >>>>>> >>>>>> >>>>>> >>>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility >>>>>> for any loss, damage or destruction of data or any other property which >>>>>> may >>>>>> arise from relying on this email's technical content is explicitly >>>>>> disclaimed. The author will in no case be liable for any monetary damages >>>>>> arising from such loss, damage or destruction. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Mon, 21 Feb 2022 at 21:09, Holden Karau <hol...@pigscanfly.ca> >>>>>> wrote: >>>>>> >>>>>>> We are happy to announce the availability of Spark 3.1.3! >>>>>>> >>>>>>> Spark 3.1.3 is a maintenance release containing stability fixes. This >>>>>>> release is based on the branch-3.1 maintenance branch of Spark. We >>>>>>> strongly >>>>>>> recommend all 3.1 users to upgrade to this stable release. >>>>>>> >>>>>>> To download Spark 3.1.3, head over to the download page: >>>>>>> https://spark.apache.org/downloads.html >>>>>>> >>>>>>> To view the release notes: >>>>>>> https://spark.apache.org/releases/spark-release-3-1-3.html >>>>>>> >>>>>>> We would like to acknowledge all community members for contributing >>>>>>> to this >>>>>>> release. This release would not have been possible without you. >>>>>>> >>>>>>> *New Dockerhub magic in this release:* >>>>>>> >>>>>>> We've also started publishing docker containers to the Apache >>>>>>> Dockerhub, >>>>>>> these contain non-ASF artifacts that are subject to different >>>>>>> license terms than the >>>>>>> Spark release. The docker containers are built for Linux x86 and >>>>>>> ARM64 since that's >>>>>>> what I have access to (thanks to NV for the ARM64 machines). >>>>>>> >>>>>>> You can get them from https://hub.docker.com/apache/spark (and >>>>>>> spark-r and spark-py) :) >>>>>>> (And version 3.2.1 is also now published on Dockerhub). >>>>>>> >>>>>>> Holden >>>>>>> >>>>>>> -- >>>>>>> Twitter: https://twitter.com/holdenkarau >>>>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>>>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> Twitter: https://twitter.com/holdenkarau >>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>>> >>>> >>> >>> -- >>> Twitter: https://twitter.com/holdenkarau >>> Books (Learning Spark, High Performance Spark, etc.): >>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>> >> -- Bjørn Jørgensen Vestre Aspehaug 4, 6010 Ålesund Norge +47 480 94 297
