Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

On Fri, Feb 25, 2022 at 8:31 AM Ismaël Mejía wrote: > The ready to use docker images are great news. I have been waiting for > this for so long! Extra kudos for including ARM64 versions too! > > I am curious, what are the non-ASF artifacts included in them (or you > refer to the OS specific eleme

Re: [ANNOUNCE] Apache Spark 3.1.3 released + Docker images

The ready to use docker images are great news. I have been waiting for this for so long! Extra kudos for including ARM64 versions too! I am curious, what are the non-ASF artifacts included in them (or you refer to the OS specific elements with other licenses?), and what consequences might be for e

Re: Recap on current status of "SPIP: Support Customized Kubernetes Schedulers"

spreadsheet with the actual size [image: 26243d69-ac3a-43f1-b2cc-903f9744237b.png] Also the spec for GKE cluster build gcloud beta container \ --project "xxx" clusters create "spark-on-gke" \ --zone "europe-west2-c" \ --no-enable-basic-auth \ --cluster-version "1.21.6-gke.1500" \ --r

Re: Recap on current status of "SPIP: Support Customized Kubernetes Schedulers"

Hi Yikun, GKE is Google's Kubernetes engine first in the market and pretty stable. The cluster deployed is a 3 node GKE with 4 Vcores and 16GB of RAM each. Autoscaling is on to take nodes from 3 to 6. So it is pretty robust. I did 15 sequences of tests