[ https://issues.apache.org/jira/browse/SPARK-27704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-27704. ------------------------------- Resolution: Not A Problem Do you mean garbage collector? the user can already choose the GC by setting JVM options. There is no default in Spark. I don't think there's anything to do here. > Change default class loader to ParallelGC > ----------------------------------------- > > Key: SPARK-27704 > URL: https://issues.apache.org/jira/browse/SPARK-27704 > Project: Spark > Issue Type: Sub-task > Components: Build > Affects Versions: 3.0.0 > Reporter: Mihaly Toth > Priority: Major > > In JDK 11 the default class loader changed from ParallelGC to G1GC. Even > though this gc performs better on pause times and interactivity, most of the > tasks that need to be processed are more sensitive to throughput and the to > the amount of memory. G1 sacrifices these to some extend to avoid the big > pauses. As a result the user may perceive a regression compared to JDK 8. > Even worse, the regression may not be limited to performance only but some > jobs may start failing in case they do not fit into the memory they used to > be happy with when running with previous JDK. > Some other kind of apps, like streaming ones, may rather use G1 because of > their more interactive, more realtime needs. > With this jira it is proposed to have a configurable default GC for all spark > applications. This may be overridable by the user through command line > parameters. The default value of the default GC (in case it is not provided > in spark-defaults.conf) could be ParallelGC. > I do not see this change required but I think it would benefit to the user > experience. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org