Hi ,
Yes I know by setting shuffle tracking property enabled we can use DRA.
But , it is marked as experimental. Is it advised to use ?
Also , regarding HPA. We do not have HPA differently as such for Spark.
Right?
Kind Regards,
Sachit Murarka
On Mon, Jan 11, 2021 at 2:17 AM Sandish Kumar HN
Sachit,
K8S based spark dynamic allocation is only available on Spark 3.0.X+ and
that too without External Shuffling Service.
https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/master/docs/user-guide.md#dynamic-allocation
Hi All,
I have read about HPA Horizontal Pod Autoscaling(for pod scaling).
I understand it can be achieved by setting the request and limit for
resources in yaml:
kubectl autoscale deploy/application-cpu --cpu-percent=95 --min=1 --max=10
// example command.
But does Kubernetes actually work
