Robert Metzger created FLINK-1455:
-------------------------------------
Summary:
ExternalSortLargeRecordsITCase.testSortWithShortMediumAndLargeRecords:
Potential Memory leak
Key: FLINK-1455
URL: https://issues.apache.org/jira/browse/FLINK-1455
Project: Flink
Issue Type: Bug
Components: Local Runtime
Affects Versions: 0.9
Reporter: Robert Metzger
Priority: Minor
This error occurred in one of my Travis jobs:
https://travis-ci.org/rmetzger/flink/jobs/48343022
Would be cool if somebody who knows the sorter better could verify/invalidate
the issue.
{code}
Running org.apache.flink.runtime.operators.sort.ExternalSortLargeRecordsITCase
java.lang.RuntimeException: Error obtaining the sorted input: Thread
'SortMerger spilling thread' terminated due to an exception: Cannot allocate
memory
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:593)
at
org.apache.flink.runtime.operators.sort.ExternalSortLargeRecordsITCase.testSortWithShortMediumAndLargeRecords(ExternalSortLargeRecordsITCase.java:285)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
Caused by: java.io.IOException: Thread 'SortMerger spilling thread' terminated
due to an exception: Cannot allocate memory
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:770)
Caused by: java.io.IOException: Cannot allocate memory
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:205)
at
org.apache.flink.runtime.io.disk.iomanager.SegmentWriteRequest.write(AsynchronousFileIOChannel.java:267)
at
org.apache.flink.runtime.io.disk.iomanager.IOManagerAsync$WriterThread.run(IOManagerAsync.java:440)
Tests run: 5, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 56.089 sec <<<
FAILURE! - in
org.apache.flink.runtime.operators.sort.ExternalSortLargeRecordsITCase
testSortWithShortMediumAndLargeRecords(org.apache.flink.runtime.operators.sort.ExternalSortLargeRecordsITCase)
Time elapsed: 25.84 sec <<< FAILURE!
java.lang.AssertionError: Error obtaining the sorted input: Thread 'SortMerger
spilling thread' terminated due to an exception: Cannot allocate memory
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.flink.runtime.operators.sort.ExternalSortLargeRecordsITCase.testSortWithShortMediumAndLargeRecords(ExternalSortLargeRecordsITCase.java:304)
testSortWithShortMediumAndLargeRecords(org.apache.flink.runtime.operators.sort.ExternalSortLargeRecordsITCase)
Time elapsed: 25.84 sec <<< FAILURE!
java.lang.AssertionError: Memory leak: not all segments have been returned to
the memory manager.
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.flink.runtime.operators.sort.ExternalSortLargeRecordsITCase.afterTest(ExternalSortLargeRecordsITCase.java:78)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
