Re: Dynamic allocation on K8

Hi Nikhil, Spark on Kubernetes supports dynamic allocation using shuffle tracking feature instead of the external shuffle service. In order to enable dynamic allocation, you should set these two configs as true: spark.dynamicAllocation.enabled and spark.dynamicAllocation.shuffleTracking.enabled

Dynamic allocation on K8

Hi folks, When running spark on Kubernetes is it possible to use dynamic allocation? Some blog posts mentioned that dynamic allocation is available, however I am not sure how it works. Spark official docs