Spark on tin boxes like Google Dataproc or AWS EC2 often utilise YARN
resource manager. YARN is the most widely used resource manager not just
for Spark but for other artefacts as well. On-premise YARN is used
extensively. In Cloud it is also used widely in Infrastructure as a Service
such as
Hi all,
I am learning about the performance difference of Spark when performing a
JOIN problem on Serverless (K8S) and Serverful (Traditional server)
environments.
Through experiment, Spark on K8s tends to run slower than Serverful.
Through understanding the architecture, I know that Spark runs
