[
https://issues.apache.org/jira/browse/SPARK-54753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xihuan updated SPARK-54753:
---------------------------
Description:
For Apache Spark *4.0.1* local mode, the memory is not released after long time
running, while downgrade the spark version {*}v3.5.6{*}, there is no issue.
We got the eclipse memory analyzer to check the heap dump.
h3. Problem Suspect 1
9,109 instances of {*}org.apache.hadoop.conf.Configuration{*}, loaded by
*org.springframework.boot.loader.launch.LaunchedClassLoader @ 0x80098888*
occupy *972,232,520 (61.58%)* bytes.
Most of these instances are referenced from one instance of
{*}jdk.internal.ref.CleanerImpl$PhantomCleanableRef{*}, loaded by {*}<system
class loader>{*}, which occupies *48 (0.00%)* bytes. The instance is referenced
by {*}org.apache.spark.sql.artifact.ArtifactManager @ 0x80a14948{*}, loaded by
{*}org.springframework.boot.loader.launch.LaunchedClassLoader @ 0x80098888{*}.
Thread *jdk.internal.misc.InnocuousThread @ 0x866c1710 Cleaner-1* has a local
variable or reference to *jdk.internal.ref.CleanerImpl @ 0x866c1898* which is
on the shortest path to {*}jdk.internal.ref.CleanerImpl$PhantomCleanableRef @
0x861b7638{*}. The thread *jdk.internal.misc.InnocuousThread @ 0x866c1710
Cleaner-1* keeps local variables with total size *184 (0.00%)* bytes.
Significant stack frames and local variables
* jdk.internal.ref.CleanerImpl.run()V (CleanerImpl.java:140)
** jdk.internal.ref.CleanerImpl @ 0x866c1898 retains 24 (0.00%) bytes
The stacktrace of this Thread is available. [See
stacktrace|file:///C:/Users/xihuan/AppData/Local/Temp/2/report8526993352425331840/pages/27.html].
[See stacktrace with involved local
variables|file:///C:/Users/xihuan/AppData/Local/Temp/2/report8526993352425331840/pages/28.html].
*Keywords*
* org.apache.hadoop.conf.Configuration
* org.springframework.boot.loader.launch.LaunchedClassLoader
* jdk.internal.ref.CleanerImpl$PhantomCleanableRef
* org.apache.spark.sql.artifact.ArtifactManager
* jdk.internal.ref.CleanerImpl.run()V
* CleanerImpl.java:140
h3.
h3.
h3. Problem Suspect 2
12,291 instances of {*}jdk.internal.ref.CleanerImpl$PhantomCleanableRef{*},
loaded by *<system class loader>* occupy *432,466,008 (27.39%)* bytes.
*Keywords*
* jdk.internal.ref.CleanerImpl$PhantomCleanableRef
Could anyone help provide any insights on the issue? Thanks
was:
for spark 4.0.1 local mode, the memory is not released after long time running,
while downgrade the spark version v3.5.6, there is no issue.
we got the eclipse memory analyzer to check the heap dump.
h3. Problem Suspect 1
9,109 instances of {*}org.apache.hadoop.conf.Configuration{*}, loaded by
*org.springframework.boot.loader.launch.LaunchedClassLoader @ 0x80098888*
occupy *972,232,520 (61.58%)* bytes.
Most of these instances are referenced from one instance of
{*}jdk.internal.ref.CleanerImpl$PhantomCleanableRef{*}, loaded by {*}<system
class loader>{*}, which occupies *48 (0.00%)* bytes. The instance is referenced
by {*}org.apache.spark.sql.artifact.ArtifactManager @ 0x80a14948{*}, loaded by
{*}org.springframework.boot.loader.launch.LaunchedClassLoader @ 0x80098888{*}.
Thread *jdk.internal.misc.InnocuousThread @ 0x866c1710 Cleaner-1* has a local
variable or reference to *jdk.internal.ref.CleanerImpl @ 0x866c1898* which is
on the shortest path to {*}jdk.internal.ref.CleanerImpl$PhantomCleanableRef @
0x861b7638{*}. The thread *jdk.internal.misc.InnocuousThread @ 0x866c1710
Cleaner-1* keeps local variables with total size *184 (0.00%)* bytes.
Significant stack frames and local variables
* jdk.internal.ref.CleanerImpl.run()V (CleanerImpl.java:140)
** jdk.internal.ref.CleanerImpl @ 0x866c1898 retains 24 (0.00%) bytes
The stacktrace of this Thread is available. [See
stacktrace|file:///C:/Users/xihuan/AppData/Local/Temp/2/report8526993352425331840/pages/27.html].
[See stacktrace with involved local
variables|file:///C:/Users/xihuan/AppData/Local/Temp/2/report8526993352425331840/pages/28.html].
*Keywords*
* org.apache.hadoop.conf.Configuration
* org.springframework.boot.loader.launch.LaunchedClassLoader
* jdk.internal.ref.CleanerImpl$PhantomCleanableRef
* org.apache.spark.sql.artifact.ArtifactManager
* jdk.internal.ref.CleanerImpl.run()V
* CleanerImpl.java:140
h3.
h3.
h3. Problem Suspect 2
12,291 instances of {*}jdk.internal.ref.CleanerImpl$PhantomCleanableRef{*},
loaded by *<system class loader>* occupy *432,466,008 (27.39%)* bytes.
*Keywords*
* jdk.internal.ref.CleanerImpl$PhantomCleanableRef
Could anyone help provide any insights on the issue? Thanks
> possible memory leak in spark 4.0.1
> -----------------------------------
>
> Key: SPARK-54753
> URL: https://issues.apache.org/jira/browse/SPARK-54753
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 4.0.1
> Reporter: xihuan
> Priority: Major
>
> For Apache Spark *4.0.1* local mode, the memory is not released after long
> time running, while downgrade the spark version {*}v3.5.6{*}, there is no
> issue.
> We got the eclipse memory analyzer to check the heap dump.
> h3. Problem Suspect 1
> 9,109 instances of {*}org.apache.hadoop.conf.Configuration{*}, loaded by
> *org.springframework.boot.loader.launch.LaunchedClassLoader @ 0x80098888*
> occupy *972,232,520 (61.58%)* bytes.
> Most of these instances are referenced from one instance of
> {*}jdk.internal.ref.CleanerImpl$PhantomCleanableRef{*}, loaded by {*}<system
> class loader>{*}, which occupies *48 (0.00%)* bytes. The instance is
> referenced by {*}org.apache.spark.sql.artifact.ArtifactManager @
> 0x80a14948{*}, loaded by
> {*}org.springframework.boot.loader.launch.LaunchedClassLoader @ 0x80098888{*}.
> Thread *jdk.internal.misc.InnocuousThread @ 0x866c1710 Cleaner-1* has a local
> variable or reference to *jdk.internal.ref.CleanerImpl @ 0x866c1898* which is
> on the shortest path to {*}jdk.internal.ref.CleanerImpl$PhantomCleanableRef @
> 0x861b7638{*}. The thread *jdk.internal.misc.InnocuousThread @ 0x866c1710
> Cleaner-1* keeps local variables with total size *184 (0.00%)* bytes.
> Significant stack frames and local variables
> * jdk.internal.ref.CleanerImpl.run()V (CleanerImpl.java:140)
> ** jdk.internal.ref.CleanerImpl @ 0x866c1898 retains 24 (0.00%) bytes
> The stacktrace of this Thread is available. [See
> stacktrace|file:///C:/Users/xihuan/AppData/Local/Temp/2/report8526993352425331840/pages/27.html].
> [See stacktrace with involved local
> variables|file:///C:/Users/xihuan/AppData/Local/Temp/2/report8526993352425331840/pages/28.html].
> *Keywords*
> * org.apache.hadoop.conf.Configuration
> * org.springframework.boot.loader.launch.LaunchedClassLoader
> * jdk.internal.ref.CleanerImpl$PhantomCleanableRef
> * org.apache.spark.sql.artifact.ArtifactManager
> * jdk.internal.ref.CleanerImpl.run()V
> * CleanerImpl.java:140
>
> h3.
> h3.
> h3. Problem Suspect 2
> 12,291 instances of {*}jdk.internal.ref.CleanerImpl$PhantomCleanableRef{*},
> loaded by *<system class loader>* occupy *432,466,008 (27.39%)* bytes.
> *Keywords*
> * jdk.internal.ref.CleanerImpl$PhantomCleanableRef
> Could anyone help provide any insights on the issue? Thanks
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
