Hi,
My understanding is there is only one driver/spark context for all user
sessions. When you run the bin/start-connect-server script, you are
submitting one long standing spark job / application. Every time a new user
request comes in, a new user session is created under that. Please correct
me
Hi,
>From Spark Connect's official site's image, it mentions the "Multi-tenant
Application Gateway" on driver. Are there any more documents about it? Can
I know how users can utilize such a feature?
Thanks,
Kezhi
;
> On Wed, Sep 20, 2023, 3:00 PM Kezhi Xiong
> wrote:
>
>> Hi,
>>
>> Are there any plans to upload PySpark 3.5.0 to PyPI (
>> https://pypi.org/project/pyspark/)? It's still 3.4.1.
>>
>> Thanks,
>> Kezhi
>>
>>
>>
Hi,
Are there any plans to upload PySpark 3.5.0 to PyPI (
https://pypi.org/project/pyspark/)? It's still 3.4.1.
Thanks,
Kezhi
Hi,
I'm recently learning Spark Connect but have some questions regarding the
connect server's relation with master or workers: so when I'm using the
connect server, I don't have to start a master alone side to make clients
work. Is the connect server simply using "local[*]" as master? Then, if I
