@dongjoon-hyun @yangwwei Thanks!
@Mich Thanks for testing it, I'm not very professional with GKE,
I'm also not quite sure if it is different in configurations, internal
network, scheduler implementations
itself VS upstream K8S. As far as I know, different K8S vendors also
maintain their own
I did some preliminary tests without volcana and with volcano addition to
spark-submit.
*setup*
The K8s cluster used was a Google Kubernetes standard cluster with three
nodes with autoscale up to 6 nodes. It runs *spark 3.1.1* with spark-py
dockers also using *spark 3.1.1 with Java 8*. In
Hi,
what do expect the performance gain to be by using volcano versus standard
scheduler.
Just to be sure there are two aspects here.
1. Procuring the Kubernetes cluster
2. Running the job through spark-submit
Item 1 is left untouched and we should see improvements in item 2 with
