Cores and memory setting of driver ?
On Wed, 23 Nov 2022, 12:56 Pralabh Kumar, wrote:
> How many cores and u are running driver with?
>
> On Tue, 22 Nov 2022, 21:00 Nikhil Goyal, wrote:
>
>> Hi folks,
>> We are running a job on our on prem cluster on K8s but writing the output
>> to S3. We
How many cores and u are running driver with?
On Tue, 22 Nov 2022, 21:00 Nikhil Goyal, wrote:
> Hi folks,
> We are running a job on our on prem cluster on K8s but writing the output
> to S3. We noticed that all the executors finish in < 1h but the driver
> takes another 5h to finish. Logs:
>
>
2 9:00 PM
To: user @spark/'user @spark'/spark users/user@spark
Subject: EXT: Driver takes long time to finish once job ends
EXTERNAL: Report suspicious emails to Email Abuse.
Hi folks,
We are running a job on our on prem cluster on K8s but writing the output to
S3. We noticed that all the exe
Hi folks,
We are running a job on our on prem cluster on K8s but writing the output
to S3. We noticed that all the executors finish in < 1h but the driver
takes another 5h to finish. Logs:
22/11/22 02:08:29 INFO BlockManagerInfo: Removed broadcast_3_piece0 on
10.42.145.11:39001 in memory (size:
