I've put some information about my situation in the ticket https://issues.apache.org/jira/browse/FLINK-16142?focusedCommentId=17049679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17049679
On Mon, Mar 2, 2020 at 2:55 PM Arvid Heise <ar...@ververica.com> wrote: > Hi Niels, > > to add to Yang. 96m is plenty of space and was heavily tested by Alibaba. > > The most likely reason and the motivation for the change is that you > probably have a classloader leak in your pipeline, quite possibly by one of > our connectors. For example, see FLINK-16142 [1]. > If you could give us more details about your pipeline, we can try to ping > it down. > > However, if the error already occurs during the first start on the > cluster, then the limit is too narrow and increasing it will already help. > In that case, please leave us a quick comment with your used dependencies, > so that we could potentially increase the default setting for future > releases. > > [1] https://issues.apache.org/jira/browse/FLINK-16142 > > On Mon, Mar 2, 2020 at 1:27 PM Yang Wang <danrtsey...@gmail.com> wrote: > >> From 1.10, Flink will enable the metaspace limit via >> "-XX:MaxMetaspaceSize" >> by default. The default value is 96m, loading too many classes will cause >> "OutOfMemoryError: Metaspace"[1]. So you need to increase the configured >> value. >> >> >> [1]. >> https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/memory/mem_trouble.html#outofmemoryerror-metaspace >> >> >> Best, >> Yang >> >> Niels Basjes <ni...@basjes.nl> 于2020年3月2日周一 下午7:16写道： >> >>> Hi, >>> >>> I'm running a lot of batch jobs on Kubernetes once in a while I get this >>> exception. >>> What is causing this? >>> How can I fix this? >>> >>> Niels Basjes >>> >>> java.lang.OutOfMemoryError: Metaspace >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:757) >>> at java.security.SecureClassLoader.defineClass(SecureClassLoader >>> .java:142) >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) >>> at java.net.URLClassLoader.access$100(URLClassLoader.java:74) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:369) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:363) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:362) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:419) >>> at org.apache.flink.util.ChildFirstClassLoader.loadClass( >>> ChildFirstClassLoader.java:60) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:352) >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:757) >>> at java.security.SecureClassLoader.defineClass(SecureClassLoader >>> .java:142) >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) >>> at java.net.URLClassLoader.access$100(URLClassLoader.java:74) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:369) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:363) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:362) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:419) >>> at org.apache.flink.util.ChildFirstClassLoader.loadClass( >>> ChildFirstClassLoader.java:60) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:352) >>> at org.apache.logging.log4j.LogManager.<clinit>(LogManager.java:60) >>> at org.elasticsearch.common.logging.ESLoggerFactory.getLogger( >>> ESLoggerFactory.java:45) >>> at org.elasticsearch.common.logging.ESLoggerFactory.getLogger( >>> ESLoggerFactory.java:53) >>> at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java: >>> 104) >>> at org.elasticsearch.common.unit.ByteSizeValue.<clinit>( >>> ByteSizeValue.java:39) >>> at org.elasticsearch.action.bulk.BulkProcessor$Builder.<init>( >>> BulkProcessor.java:88) >>> at org.elasticsearch.action.bulk.BulkProcessor$Builder.<init>( >>> BulkProcessor.java:80) >>> at org.elasticsearch.action.bulk.BulkProcessor.builder(BulkProcessor >>> .java:174) >>> >>> -- >>> Best regards / Met vriendelijke groeten, >>> >>> Niels Basjes >>> >> -- Best regards / Met vriendelijke groeten, Niels Basjes
