[ https://issues.apache.org/jira/browse/HDFS-15124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046803#comment-17046803 ]
Ayush Saxena commented on HDFS-15124: ------------------------------------- Committed to trunk. Thanx [~ctest.team] for the contribution and [~elgoiri] for the review!!! > Crashing bugs in NameNode when using a valid configuration for > `dfs.namenode.audit.loggers` > ------------------------------------------------------------------------------------------- > > Key: HDFS-15124 > URL: https://issues.apache.org/jira/browse/HDFS-15124 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Affects Versions: 2.10.0 > Reporter: Ctest > Assignee: Ctest > Priority: Critical > Attachments: HDFS-15124.000.patch, HDFS-15124.001.patch, > HDFS-15124.002.patch, HDFS-15124.003.patch, HDFS-15124.004.patch, > HDFS-15124.005.patch, HDFS-15124.006.patch > > > I am using Hadoop-2.10.0. > The configuration parameter `dfs.namenode.audit.loggers` allows `default` > (which is the default value) and > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`. > When I use `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, > namenode will not be started successfully because of an > `InstantiationException` thrown from > `org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers`. > The root cause is that while initializing namenode, `initAuditLoggers` will > be called and it will try to call the default constructor of > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` which doesn't > have a default constructor. Thus the `InstantiationException` exception is > thrown. > > *Symptom* > *$ ./start-dfs.sh* > {code:java} > 2019-12-18 14:05:20,670 ERROR > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem > initialization failed.java.lang.RuntimeException: > java.lang.InstantiationException: > org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1024) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:858) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:677) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:674) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:736) > at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:961) > at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:940) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1714) > at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1782) > Caused by: java.lang.InstantiationException: > org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger > at java.lang.Class.newInstance(Class.java:427) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1017)... > 8 more > Caused by: java.lang.NoSuchMethodException: > org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger.<init>() > at java.lang.Class.getConstructor0(Class.java:3082) > at java.lang.Class.newInstance(Class.java:412) > ... 9 more{code} > > > *Detailed Root Cause* > There is no default constructor in > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`: > {code:java} > /** > * An {@link AuditLogger} that sends logged data directly to the metrics > * systems. It is used when the top service is used directly by the name node > */ > @InterfaceAudience.Private > public class TopAuditLogger implements AuditLogger { > public static finalLogger LOG = > LoggerFactory.getLogger(TopAuditLogger.class); > private final TopMetrics topMetrics; > public TopAuditLogger(TopMetrics topMetrics) { > Preconditions.checkNotNull(topMetrics, "Cannot init with a null " + > "TopMetrics"); > this.topMetrics = topMetrics; > } > @Override > public void initialize(Configuration conf) { > } > {code} > As long as the configuration parameter `dfs.namenode.audit.loggers` is set to > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, > `initAuditLoggers` will try to call its default constructor to make a new > instance: > {code:java} > private List<AuditLogger> initAuditLoggers(Configuration conf) { > // Initialize the custom access loggers if configured. > Collection<String> alClasses = > conf.getTrimmedStringCollection(DFS_NAMENODE_AUDIT_LOGGERS_KEY); > List<AuditLogger> auditLoggers = Lists.newArrayList(); > if (alClasses != null && !alClasses.isEmpty()) { > for (String className : alClasses) { > try { > AuditLogger logger; > if (DFS_NAMENODE_DEFAULT_AUDIT_LOGGER_NAME.equals(className)) { > logger = new DefaultAuditLogger(); > } else { > logger = (AuditLogger) Class.forName(className).newInstance(); > } > logger.initialize(conf); > auditLoggers.add(logger); > } catch (RuntimeException re) { > throw re; > } catch (Exception e) { > throw new RuntimeException(e); > } > } > } > {code} > `initAuditLoggers` tries to call the default constructor to make a new > instance in: > {code:java} > logger = (AuditLogger) Class.forName(className).newInstance(); > {code} > This is very different from the default configuration, `default`, which > implements a default constructor so the default is fine. > > *How To Reproduce* > The version of Hadoop: 2.10.0 > # Set the value of configuration parameter `dfs.namenode.audit.loggers` to > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` in > "hdfs-site.xml"(the default value is `default`) > # Start the namenode by running "start-dfs.sh" > # The namenode will not be started successfully. > {code:java} > <property> > <name>dfs.namenode.audit.loggers</name> > <value>org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger</value> > <description> > List of classes implementing audit loggers that will receive audit events. > These should be implementations of > org.apache.hadoop.hdfs.server.namenode.AuditLogger. > The special value "default" can be used to reference the default audit > logger, which uses the configured log system. Installing custom audit > loggers > may affect the performance and stability of the NameNode. Refer to the > custom > logger's documentation for more details. > </description> > </property> > {code} > > *How To Fix* > Add a default constructor for > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`. > I wrote a patch to add a default constructor for TopAuditLogger. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org