[jira] [Updated] (FLINK-35737) Prevent Memory Leak by Closing MemoryExecutionGraphInfoStore on MiniCluster Shutdown
[
https://issues.apache.org/jira/browse/FLINK-35737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weijie Guo updated FLINK-35737:
---
Fix Version/s: 2.0.0
(was: 1.18.2)
(was: 1.20.0)
(was: 1.19.2)
> Prevent Memory Leak by Closing MemoryExecutionGraphInfoStore on MiniCluster
> Shutdown
>
>
> Key: FLINK-35737
> URL: https://issues.apache.org/jira/browse/FLINK-35737
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.18.1
>Reporter: Feng Jiajie
>Priority: Critical
> Labels: pull-request-available
> Fix For: 2.0.0
>
>
> MemoryExecutionGraphInfoStore registers a ShutdownHook upon construction and
> deregisters it within its close() method.
> {code:java}
> public MemoryExecutionGraphInfoStore(...) {
> ...
> this.shutdownHook = ShutdownHookUtil.addShutdownHook(this,
> getClass().getSimpleName(), LOG);
> }
> @Override
> public void close() throws IOException {
> ...
> // Remove shutdown hook to prevent resource leaks
> ShutdownHookUtil.removeShutdownHook(shutdownHook,
> getClass().getSimpleName(), LOG);
> }{code}
> Currently, MiniCluster instantiates a MemoryExecutionGraphInfoStore object
> but doesn't retain a reference to it, nor does it call close() during its own
> shutdown process.
> {code:java}
> final DispatcherResourceManagerComponent
> dispatcherResourceManagerComponent =
> dispatcherResourceManagerComponentFactory.create(
> ...
> new MemoryExecutionGraphInfoStore(), // -> new
> ...); {code}
> This behavior leads to an accumulation of ShutdownHooks when running multiple
> Flink jobs within the same local JVM. These accumulating hooks, along with
> their associated references, contribute to a memory leak.
> This patch addresses the issue by ensuring that
> MemoryExecutionGraphInfoStore's close() method is invoked during MiniCluster
> shutdown.
> https://github.com/apache/flink/pull/25009
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-35737) Prevent Memory Leak by Closing MemoryExecutionGraphInfoStore on MiniCluster Shutdown
[
https://issues.apache.org/jira/browse/FLINK-35737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-35737:
---
Labels: pull-request-available (was: )
> Prevent Memory Leak by Closing MemoryExecutionGraphInfoStore on MiniCluster
> Shutdown
>
>
> Key: FLINK-35737
> URL: https://issues.apache.org/jira/browse/FLINK-35737
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.18.1
>Reporter: Feng Jiajie
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.18.2, 1.20.0, 1.19.2
>
>
> MemoryExecutionGraphInfoStore registers a ShutdownHook upon construction and
> deregisters it within its close() method.
> {code:java}
> public MemoryExecutionGraphInfoStore(...) {
> ...
> this.shutdownHook = ShutdownHookUtil.addShutdownHook(this,
> getClass().getSimpleName(), LOG);
> }
> @Override
> public void close() throws IOException {
> ...
> // Remove shutdown hook to prevent resource leaks
> ShutdownHookUtil.removeShutdownHook(shutdownHook,
> getClass().getSimpleName(), LOG);
> }{code}
> Currently, MiniCluster instantiates a MemoryExecutionGraphInfoStore object
> but doesn't retain a reference to it, nor does it call close() during its own
> shutdown process.
> {code:java}
> final DispatcherResourceManagerComponent
> dispatcherResourceManagerComponent =
> dispatcherResourceManagerComponentFactory.create(
> ...
> new MemoryExecutionGraphInfoStore(), // -> new
> ...); {code}
> This behavior leads to an accumulation of ShutdownHooks when running multiple
> Flink jobs within the same local JVM. These accumulating hooks, along with
> their associated references, contribute to a memory leak.
> This patch addresses the issue by ensuring that
> MemoryExecutionGraphInfoStore's close() method is invoked during MiniCluster
> shutdown.
> https://github.com/apache/flink/pull/25009
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-35737) Prevent Memory Leak by Closing MemoryExecutionGraphInfoStore on MiniCluster Shutdown
[
https://issues.apache.org/jira/browse/FLINK-35737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Jiajie updated FLINK-35737:
Description:
MemoryExecutionGraphInfoStore registers a ShutdownHook upon construction and
deregisters it within its close() method.
{code:java}
public MemoryExecutionGraphInfoStore(...) {
...
this.shutdownHook = ShutdownHookUtil.addShutdownHook(this,
getClass().getSimpleName(), LOG);
}
@Override
public void close() throws IOException {
...
// Remove shutdown hook to prevent resource leaks
ShutdownHookUtil.removeShutdownHook(shutdownHook,
getClass().getSimpleName(), LOG);
}{code}
Currently, MiniCluster instantiates a MemoryExecutionGraphInfoStore object but
doesn't retain a reference to it, nor does it call close() during its own
shutdown process.
{code:java}
final DispatcherResourceManagerComponent
dispatcherResourceManagerComponent =
dispatcherResourceManagerComponentFactory.create(
...
new MemoryExecutionGraphInfoStore(), // -> new
...); {code}
This behavior leads to an accumulation of ShutdownHooks when running multiple
Flink jobs within the same local JVM. These accumulating hooks, along with
their associated references, contribute to a memory leak.
This patch addresses the issue by ensuring that MemoryExecutionGraphInfoStore's
close() method is invoked during MiniCluster shutdown.
https://github.com/apache/flink/pull/25009
was:
MemoryExecutionGraphInfoStore registers a ShutdownHook upon construction and
deregisters it within its close() method.
{code:java}
public MemoryExecutionGraphInfoStore(...) {
...
this.shutdownHook = ShutdownHookUtil.addShutdownHook(this,
getClass().getSimpleName(), LOG);
}
@Override
public void close() throws IOException {
...
// Remove shutdown hook to prevent resource leaks
ShutdownHookUtil.removeShutdownHook(shutdownHook,
getClass().getSimpleName(), LOG);
}{code}
Currently, MiniCluster instantiates a MemoryExecutionGraphInfoStore object but
doesn't retain a reference to it, nor does it call close() during its own
shutdown process.
{code:java}
final DispatcherResourceManagerComponent
dispatcherResourceManagerComponent =
dispatcherResourceManagerComponentFactory.create(
...
new MemoryExecutionGraphInfoStore(), // -> new
...); {code}
This behavior leads to an accumulation of ShutdownHooks when running multiple
Flink jobs within the same local JVM. These accumulating hooks, along with
their associated references, contribute to a memory leak.
This patch addresses the issue by ensuring that MemoryExecutionGraphInfoStore's
close() method is invoked during MiniCluster shutdown.
> Prevent Memory Leak by Closing MemoryExecutionGraphInfoStore on MiniCluster
> Shutdown
>
>
> Key: FLINK-35737
> URL: https://issues.apache.org/jira/browse/FLINK-35737
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.18.1
>Reporter: Feng Jiajie
>Priority: Critical
> Fix For: 1.18.2, 1.20.0, 1.19.2
>
>
> MemoryExecutionGraphInfoStore registers a ShutdownHook upon construction and
> deregisters it within its close() method.
> {code:java}
> public MemoryExecutionGraphInfoStore(...) {
> ...
> this.shutdownHook = ShutdownHookUtil.addShutdownHook(this,
> getClass().getSimpleName(), LOG);
> }
> @Override
> public void close() throws IOException {
> ...
> // Remove shutdown hook to prevent resource leaks
> ShutdownHookUtil.removeShutdownHook(shutdownHook,
> getClass().getSimpleName(), LOG);
> }{code}
> Currently, MiniCluster instantiates a MemoryExecutionGraphInfoStore object
> but doesn't retain a reference to it, nor does it call close() during its own
> shutdown process.
> {code:java}
> final DispatcherResourceManagerComponent
> dispatcherResourceManagerComponent =
> dispatcherResourceManagerComponentFactory.create(
> ...
> new MemoryExecutionGraphInfoStore(), // -> new
> ...); {code}
> This behavior leads to an accumulation of ShutdownHooks when running multiple
> Flink jobs within the same local JVM. These accumulating hooks, along with
> their associated references, contribute to a memory leak.
> This patch addresses the issue by ensuring that
> MemoryExecutionGraphInfoStore's close() method is invoked during MiniCluster
> shutdown.
> https://github.com/apache/flink/pull/25009
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
