[ https://issues.apache.org/jira/browse/SPARK-11617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003879#comment-15003879 ]
Jacek Lewandowski commented on SPARK-11617: ------------------------------------------- This also happens in standalone mode, Netty based RPC - I've seen this in Master and Worker logs. > MEMORY LEAK: ByteBuf.release() was not called before it's garbage-collected > --------------------------------------------------------------------------- > > Key: SPARK-11617 > URL: https://issues.apache.org/jira/browse/SPARK-11617 > Project: Spark > Issue Type: Bug > Components: Spark Core, YARN > Affects Versions: 1.6.0 > Reporter: LingZhou > > The problem may be related to > [SPARK-11235][NETWORK] Add ability to stream data using network lib. > while running on yarn-client mode, there are error messages: > 15/11/09 10:23:55 ERROR util.ResourceLeakDetector: LEAK: ByteBuf.release() > was not called before it's garbage-collected. Enable advanced leak reporting > to find out where the leak occurred. To enable advanced leak reporting, > specify the JVM option '-Dio.netty.leakDetectionLevel=advanced' or call > ResourceLeakDetector.setLevel() See > http://netty.io/wiki/reference-counted-objects.html for more information. > and then it will cause > cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Container killed by YARN > for exceeding memory limits. 9.0 GB of 9 GB physical memory used. Consider > boosting spark.yarn.executor.memoryOverhead. > and WARN scheduler.TaskSetManager: Lost task 105.0 in stage 1.0 (TID 2616, > gsr489): java.lang.IndexOutOfBoundsException: index: 130828, length: 16833 > (expected: range(0, 524288)). -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org