Re: Non-Partition based Workload Distribution

2022-02-25 Thread Gourav Sengupta
Hi, not quite sure here, but can you please share your code? Regards, Gourav Sengupta On Thu, Feb 24, 2022 at 8:25 PM Artemis User wrote: > We got a Spark program that iterates through a while loop on the same > input DataFrame and produces different results per iteration. I see > through

Non-Partition based Workload Distribution

2022-02-24 Thread Artemis User
We got a Spark program that iterates through a while loop on the same input DataFrame and produces different results per iteration. I see through Spark UI that the workload is concentrated on a single core of the same worker.  Is there anyway to distribute the workload to different