Can I simply understand Spark Connect this way: The client process is now the
Spark driver?
From: Brian Huynh
Sent: Thursday, August 10, 2023 10:15 PM
To: Kezhi Xiong
Cc: user@spark.apache.org
Subject: Re: Spark Connect, Master, and Workers
Hi Kezhi,
Yes
Hi Kezhi,
Yes, you no longer need to start a master to make the client work. Please
see the quickstart.
https://spark.apache.org/docs/latest/api/python/getting_started/quickstart_connect.html
You can think of Spark Connect as an API on top of Master so workers can be
added to the cluster same
Hi,
I'm recently learning Spark Connect but have some questions regarding the
connect server's relation with master or workers: so when I'm using the
connect server, I don't have to start a master alone side to make clients
work. Is the connect server simply using "local[*]" as master? Then, if I
