High GC is relatively hard to debug in general but I can give you a few
pointers. This basically means that the time spent cleaning up unused
objects is high which usually means memory is be used and thrown away
rapidly. It can also mean that GC is ineffective, and is being run many
times in an
Please help on this.
Thanks
Amit
On Fri, Jul 17, 2020 at 2:34 PM Amit Sharma wrote:
> Hi All, i am running the same batch job in my two separate spark clusters.
> In one of the clusters it is showing GC warning on spark -ui under
> executer tag. Garbage collection is taking longer time
What is your heap size, and JVM vendor/version? Generally, G1 only
outperforms CMS on large heap sizes (ex: 31GB or larger).
On Mon, Jul 20, 2020 at 1:22 PM Amit Sharma wrote:
> Please help on this.
>
>
> Thanks
> Amit
>
> On Fri, Jul 17, 2020 at 2:34 PM Amit Sharma wrote:
>
>> Hi All, i am
Hi All, i am running the same batch job in my two separate spark clusters.
In one of the clusters it is showing GC warning on spark -ui under
executer tag. Garbage collection is taking longer time around 20 % while
in another cluster it is under 10 %. I am using the same configuration in
my
