[ https://issues.apache.org/jira/browse/IMPALA-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Laszlo Gaal resolved IMPALA-9900. --------------------------------- Resolution: Duplicate Although IMPALA-10057 is a more recent ticket, it contains a plausible root cause and a plan for a fix. > High rate log spam during FE tests > ---------------------------------- > > Key: IMPALA-9900 > URL: https://issues.apache.org/jira/browse/IMPALA-9900 > Project: IMPALA > Issue Type: Bug > Components: Frontend > Affects Versions: Impala 4.0 > Reporter: Laszlo Gaal > Priority: Critical > > During recent runs of the gerrit-verify-dryrun job FE tests exhibited a > disturbing tendency: all impalad instances emitted almost 1 GB's worth of > Java exception traces into their ERROR logfiles in a short, roughly 2 minute > period. This bloats the archived logs pretty badly, because the contents of > the ERROR log is replicated in the WARNING and in the INFO log as well, > resulting in about 6 GB's of wasted space on the build server. This also > makes it hard to extract meaningful information from those logs. > The latest example is > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/11102/ (the build is > currently locked to prevent it from being cleaned up); the affected files are > archive/Impala/logs_static/logs/ee_tests/impalad{,node1,node2}.ERROR. The > spam runs are present between log timestamps 21:05 to 21:07, the first error > is > {code} > E0624 21:05:28.938853 19419 TransactionKeepalive.java:137] Unexpected > exception thrown > Java exception follows: > java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: > org/apache/impala/common/TransactionKeepalive$HeartbeatContext > at > org.apache.impala.common.TransactionKeepalive$DaemonThread.run(TransactionKeepalive.java:114) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.NoClassDefFoundError: > org/apache/impala/common/TransactionKeepalive$HeartbeatContext > ... 2 more > Caused by: java.lang.ClassNotFoundException: > org.apache.impala.common.TransactionKeepalive$HeartbeatContext > at java.net.URLClassLoader.findClass(URLClassLoader.java:382) > at java.lang.ClassLoader.loadClass(ClassLoader.java:418) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) > at java.lang.ClassLoader.loadClass(ClassLoader.java:351) > ... 2 more > {code} then the repeated pattern is > {code} > E0624 21:05:28.939031 19419 TransactionKeepalive.java:137] Unexpected > exception thrown > Java exception follows: > java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: > at > org.apache.impala.common.TransactionKeepalive$DaemonThread.run(TransactionKeepalive.java:114) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.NoClassDefFoundError: > ... 2 more > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)