[ https://issues.apache.org/jira/browse/HDFS-15124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ctest updated HDFS-15124: ------------------------- Description: I am using Hadoop-2.10.0. The configuration parameter `dfs.namenode.audit.loggers` allows `default` (which is the default value) and `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`. When I use `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, namenode will not be started successfully because of an `InstantiationException` thrown from `org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers`. The root cause is that while initializing namenode, `initAuditLoggers` will be called and it will try to call the default constructor of `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` which doesn't have a default constructor. Thus the `InstantiationException` exception is thrown. *Symptom* *$ ./start-dfs.sh* {code:java} 2019-12-18 14:05:20,670 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.java.lang.RuntimeException: java.lang.InstantiationException: org.apache.hadoop.hdfs.server.namenode.top.TopAuditLoggerat org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1024)at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:858)at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:677)at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:674)at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:736)at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:961)at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:940)at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1714)at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1782)Caused by: java.lang.InstantiationException: org.apache.hadoop.hdfs.server.namenode.top.TopAuditLoggerat java.lang.Class.newInstance(Class.java:427)at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1017)... 8 moreCaused by: java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger.<init>()at java.lang.Class.getConstructor0(Class.java:3082)at java.lang.Class.newInstance(Class.java:412) ... 9 more{code} *Detailed Root Cause* There is no default constructor in `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`: {code:java} /** * An {@link AuditLogger} that sends logged data directly to the metrics * systems. It is used when the top service is used directly by the name node */ @InterfaceAudience.Private public class TopAuditLogger implements AuditLogger { public static finalLogger LOG = LoggerFactory.getLogger(TopAuditLogger.class); private final TopMetrics topMetrics; public TopAuditLogger(TopMetrics topMetrics) { Preconditions.checkNotNull(topMetrics, "Cannot init with a null " + "TopMetrics"); this.topMetrics = topMetrics; } @Override public void initialize(Configuration conf) { } {code} As long as the configuration parameter `dfs.namenode.audit.loggers` is set to `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, `initAuditLoggers` will try to call its default constructor to make a new instance: {code:java} private List<AuditLogger> initAuditLoggers(Configuration conf) { // Initialize the custom access loggers if configured. Collection<String> alClasses = conf.getTrimmedStringCollection(DFS_NAMENODE_AUDIT_LOGGERS_KEY); List<AuditLogger> auditLoggers = Lists.newArrayList(); if (alClasses != null && !alClasses.isEmpty()) { for (String className : alClasses) { try { AuditLogger logger; if (DFS_NAMENODE_DEFAULT_AUDIT_LOGGER_NAME.equals(className)) { logger = new DefaultAuditLogger(); } else { logger = (AuditLogger) Class.forName(className).newInstance(); } logger.initialize(conf); auditLoggers.add(logger); } catch (RuntimeException re) { throw re; } catch (Exception e) { throw new RuntimeException(e); } } } {code} `initAuditLoggers` tries to call the default constructor to make a new instance in: {code:java} logger = (AuditLogger) Class.forName(className).newInstance(); {code} This is very different from the default configuration, `default`, which implements a default constructor so the default is fine. *How To Reproduce* The version of Hadoop: 2.10.0 # Set the value of configuration parameter `dfs.namenode.audit.loggers` to `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` in "hdfs-site.xml"(the default value is `default`) # Start the namenode by running "start-dfs.sh" # The namenode will not be started successfully. {code:java} <property> <name>dfs.namenode.audit.loggers</name> <value>org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger</value> <description> List of classes implementing audit loggers that will receive audit events. These should be implementations of org.apache.hadoop.hdfs.server.namenode.AuditLogger. The special value "default" can be used to reference the default audit logger, which uses the configured log system. Installing custom audit loggers may affect the performance and stability of the NameNode. Refer to the custom logger's documentation for more details. </description> </property> {code} *How To Fix* Add a default constructor for `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` was: I am using Hadoop-2.10.0. The configuration parameter `dfs.namenode.audit.loggers` allows `default` (which is the default value) and `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`. When I use `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, namenode will not be started successfully because of an `InstantiationException` thrown from `org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers`. The root cause is that while initializing namenode, `initAuditLoggers` will be called and it will try to call the default constructor of `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` which doesn't have a default constructor. Thus the `InstantiationException` exception is thrown. *Symptom* *$ ./start-dfs.sh* {code:java} 2019-12-18 14:05:20,670 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.java.lang.RuntimeException: java.lang.InstantiationException: org.apache.hadoop.hdfs.server.namenode.top.TopAuditLoggerat org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1024)at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:858)at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:677)at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:674)at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:736)at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:961)at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:940)at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1714)at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1782)Caused by: java.lang.InstantiationException: org.apache.hadoop.hdfs.server.namenode.top.TopAuditLoggerat java.lang.Class.newInstance(Class.java:427)at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1017)... 8 moreCaused by: java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger.<init>()at java.lang.Class.getConstructor0(Class.java:3082)at java.lang.Class.newInstance(Class.java:412) ... 9 more{code} *Detailed Root Cause* There is no default constructor in `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`: {code:java} /** * An {@link AuditLogger} that sends logged data directly to the metrics * systems. It is used when the top service is used directly by the name node */ @InterfaceAudience.Private public class TopAuditLogger implements AuditLogger { public static finalLogger LOG = LoggerFactory.getLogger(TopAuditLogger.class); private final TopMetrics topMetrics; public TopAuditLogger(TopMetrics topMetrics) { Preconditions.checkNotNull(topMetrics, "Cannot init with a null " + "TopMetrics"); this.topMetrics = topMetrics; } @Override public void initialize(Configuration conf) { } {code} As long as the configuration parameter `dfs.namenode.audit.loggers` is set to `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, `initAuditLoggers` will try to call its default constructor to make a new instance: {code:java} private List<AuditLogger> initAuditLoggers(Configuration conf) { // Initialize the custom access loggers if configured. Collection<String> alClasses = conf.getTrimmedStringCollection(DFS_NAMENODE_AUDIT_LOGGERS_KEY); List<AuditLogger> auditLoggers = Lists.newArrayList(); if (alClasses != null && !alClasses.isEmpty()) { for (String className : alClasses) { try { AuditLogger logger; if (DFS_NAMENODE_DEFAULT_AUDIT_LOGGER_NAME.equals(className)) { logger = new DefaultAuditLogger(); } else { logger = (AuditLogger) Class.forName(className).newInstance(); } logger.initialize(conf); auditLoggers.add(logger); } catch (RuntimeException re) { throw re; } catch (Exception e) { throw new RuntimeException(e); } } } {code} `initAuditLoggers` tries to call the default constructor to make a new instance in: {code:java} logger = (AuditLogger) Class.forName(className).newInstance(); {code} This is very different from the default configuration, `default`, which implements a default constructor so the default is fine. *How To Reproduce* The version of Hadoop: 2.10.0 # Set the value of configuration parameter `dfs.namenode.audit.loggers` to `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` in "hdfs-site.xml"(the default value is `default`) # Start the namenode by running "start-dfs.sh" # The namenode will not be started successfully. {code:java} <property> <name>dfs.namenode.audit.loggers</name> <value>org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger</value> <description> List of classes implementing audit loggers that will receive audit events. These should be implementations of org.apache.hadoop.hdfs.server.namenode.AuditLogger. The special value "default" can be used to reference the default audit logger, which uses the configured log system. Installing custom audit loggers may affect the performance and stability of the NameNode. Refer to the custom logger's documentation for more details. </description> </property> {code} *How To Fix* Add a default constructor for `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` > Crashing bugs in NameNode when using a valid configuration for > `dfs.namenode.audit.loggers` > ------------------------------------------------------------------------------------------- > > Key: HDFS-15124 > URL: https://issues.apache.org/jira/browse/HDFS-15124 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Affects Versions: 2.10.0 > Reporter: Ctest > Priority: Critical > > I am using Hadoop-2.10.0. > The configuration parameter `dfs.namenode.audit.loggers` allows `default` > (which is the default value) and > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`. > When I use `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, > namenode will not be started successfully because of an > `InstantiationException` thrown from > `org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers`. > The root cause is that while initializing namenode, `initAuditLoggers` will > be called and it will try to call the default constructor of > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` which doesn't > have a default constructor. Thus the `InstantiationException` exception is > thrown. > > *Symptom* > *$ ./start-dfs.sh* > {code:java} > 2019-12-18 14:05:20,670 ERROR > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem > initialization failed.java.lang.RuntimeException: > java.lang.InstantiationException: > org.apache.hadoop.hdfs.server.namenode.top.TopAuditLoggerat > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1024)at > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:858)at > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:677)at > > org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:674)at > > org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:736)at > org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:961)at > org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:940)at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1714)at > > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1782)Caused > by: java.lang.InstantiationException: > org.apache.hadoop.hdfs.server.namenode.top.TopAuditLoggerat > java.lang.Class.newInstance(Class.java:427)at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1017)... > 8 moreCaused by: java.lang.NoSuchMethodException: > org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger.<init>()at > java.lang.Class.getConstructor0(Class.java:3082)at > java.lang.Class.newInstance(Class.java:412) > ... 9 more{code} > > > *Detailed Root Cause* > There is no default constructor in > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`: > {code:java} > /** > * An {@link AuditLogger} that sends logged data directly to the metrics > * systems. It is used when the top service is used directly by the name node > */ > @InterfaceAudience.Private > public class TopAuditLogger implements AuditLogger { > public static finalLogger LOG = > LoggerFactory.getLogger(TopAuditLogger.class); > private final TopMetrics topMetrics; > public TopAuditLogger(TopMetrics topMetrics) { > Preconditions.checkNotNull(topMetrics, "Cannot init with a null " + > "TopMetrics"); > this.topMetrics = topMetrics; > } > @Override > public void initialize(Configuration conf) { > } > {code} > As long as the configuration parameter `dfs.namenode.audit.loggers` is set to > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`, > `initAuditLoggers` will try to call its default constructor to make a new > instance: > {code:java} > private List<AuditLogger> initAuditLoggers(Configuration conf) { > // Initialize the custom access loggers if configured. > Collection<String> alClasses = > conf.getTrimmedStringCollection(DFS_NAMENODE_AUDIT_LOGGERS_KEY); > List<AuditLogger> auditLoggers = Lists.newArrayList(); > if (alClasses != null && !alClasses.isEmpty()) { > for (String className : alClasses) { > try { > AuditLogger logger; > if (DFS_NAMENODE_DEFAULT_AUDIT_LOGGER_NAME.equals(className)) { > logger = new DefaultAuditLogger(); > } else { > logger = (AuditLogger) Class.forName(className).newInstance(); > } > logger.initialize(conf); > auditLoggers.add(logger); > } catch (RuntimeException re) { > throw re; > } catch (Exception e) { > throw new RuntimeException(e); > } > } > } > {code} > `initAuditLoggers` tries to call the default constructor to make a new > instance in: > {code:java} > logger = (AuditLogger) Class.forName(className).newInstance(); > {code} > This is very different from the default configuration, `default`, which > implements a default constructor so the default is fine. > > *How To Reproduce* > The version of Hadoop: 2.10.0 > # Set the value of configuration parameter `dfs.namenode.audit.loggers` to > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` in > "hdfs-site.xml"(the default value is `default`) > # Start the namenode by running "start-dfs.sh" > # The namenode will not be started successfully. > {code:java} > <property> > <name>dfs.namenode.audit.loggers</name> > <value>org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger</value> > <description> > List of classes implementing audit loggers that will receive audit events. > These should be implementations of > org.apache.hadoop.hdfs.server.namenode.AuditLogger. > The special value "default" can be used to reference the default audit > logger, which uses the configured log system. Installing custom audit > loggers > may affect the performance and stability of the NameNode. Refer to the > custom > logger's documentation for more details. > </description> > </property> > {code} > > *How To Fix* > Add a default constructor for > `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org