Laszlo Gaal created IMPALA-9900:
-----------------------------------
Summary: High rate log spam during FE tests
Key: IMPALA-9900
URL: https://issues.apache.org/jira/browse/IMPALA-9900
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 4.0
Reporter: Laszlo Gaal
During recent runs of the gerrit-verify-dryrun job FE tests exhibited a
disturbing tendency: all impalad instances emitted almost 1 GB's worth of Java
exception traces into their ERROR logfiles in a short, roughly 2 minute period.
This bloats the archived logs pretty badly, because the contents of the ERROR
log is replicated in the WARNING and in the INFO log as well, resulting in
about 6 GB's of wasted space on the build server. This also makes it hard to
extract meaningful information from those logs.
The latest example is
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/11102/ (the build is
currently locked to prevent it from being cleaned up); the affected files are
archive/Impala/logs_static/logs/ee_tests/impalad{,node1,node2}.ERROR. The spam
runs are present between log timestamps 21:05 to 21:07, the first error is
{code}
E0624 21:05:28.938853 19419 TransactionKeepalive.java:137] Unexpected exception
thrown
Java exception follows:
java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
org/apache/impala/common/TransactionKeepalive$HeartbeatContext
at
org.apache.impala.common.TransactionKeepalive$DaemonThread.run(TransactionKeepalive.java:114)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError:
org/apache/impala/common/TransactionKeepalive$HeartbeatContext
... 2 more
Caused by: java.lang.ClassNotFoundException:
org.apache.impala.common.TransactionKeepalive$HeartbeatContext
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 2 more
{code} then the repeated pattern is
{code}
E0624 21:05:28.939031 19419 TransactionKeepalive.java:137] Unexpected exception
thrown
Java exception follows:
java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
at
org.apache.impala.common.TransactionKeepalive$DaemonThread.run(TransactionKeepalive.java:114)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError:
... 2 more
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
