[ https://issues.apache.org/jira/browse/FLINK-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17190276#comment-17190276 ]
Kezhu Wang commented on FLINK-18815: ------------------------------------ Seems that recently two cases are leaking while previous cases are duplicated closing. I think these two cases are caused by {{SafetyNetCloseableRegistry.close}} which interrupt reaper thread. Suppose that: 1. A closeable became phantom reachable and queued in {{CloseableReaperThread.referenceQueue}} but did not get a chance to close. 2. {{SafetyNetCloseableRegistry.close}} calls {{CloseableReaperThread.interrupt}} which set {{CloseableReaperThread.running}} to false and interrupt that java thread. 3. {{CloseableReaperThread}} terminates due to false {{CloseableReaperThread.running}} or {{InterruptedException}}. 4. That enqueued closeable leaks. I think there are two different approaches to fix this issue: * Use at most one {{CloseableReaperThread}}, and don't close it. This may cause leaking if Flink is embedded as guest in other host application. * Count registered phantom references, and close reaper thread only if all registered phantom references are popped and {{CloseableReaperThread}} is dropped by caller. Since Flink is not an end stop application, I think the counting approach maybe more appropriate ? As a analogy, {{java.lang.ref.Cleaner}} has no close-like method, it [tracks all registered referents|https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/master/src/java.base/share/classes/jdk/internal/ref/PhantomCleanable.java#L65], its underlying thread will terminate after [itself|https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/master/src/java.base/share/classes/jdk/internal/ref/CleanerImpl.java#L101] and [all registered references|https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/master/src/java.base/share/classes/jdk/internal/ref/CleanerImpl.java#L133] are cleaned. [~kevin.cyj] [~dian.fu] [~trohrmann] Any thoughts ? > AbstractCloseableRegistryTest.testClose unstable > ------------------------------------------------ > > Key: FLINK-18815 > URL: https://issues.apache.org/jira/browse/FLINK-18815 > Project: Flink > Issue Type: Bug > Components: FileSystems, Tests > Affects Versions: 1.10.1, 1.12.0, 1.11.1 > Reporter: Robert Metzger > Assignee: Kezhu Wang > Priority: Critical > Labels: pull-request-available, test-stability > Fix For: 1.10.2, 1.12.0, 1.11.2 > > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=5164&view=logs&j=0da23115-68bb-5dcd-192c-bd4c8adebde1&t=05b74a19-4ee4-5036-c46f-ada307df6cf0 > {code} > [ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.509 > s <<< FAILURE! - in org.apache.flink.core.fs.SafetyNetCloseableRegistryTest > [ERROR] testClose(org.apache.flink.core.fs.SafetyNetCloseableRegistryTest) > Time elapsed: 1.15 s <<< FAILURE! > java.lang.AssertionError: expected:<0> but was:<-1> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.flink.core.fs.AbstractCloseableRegistryTest.testClose(AbstractCloseableRegistryTest.java:93) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)