[
https://issues.apache.org/jira/browse/HDFS-15124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017653#comment-17017653
]
Hadoop QA commented on HDFS-15124:
----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 11s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 41s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 1 new + 166 unchanged - 0 fixed = 167 total (was 166) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 25s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}123m 13s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}191m 22s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDeadNodeDetection |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.namenode.TestRedudantBlocks |
| | hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | HDFS-15124 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12991181/HDFS-15124.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 0fb99bc6d2d0 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 263413e |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_232 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/28687/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/28687/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/28687/testReport/ |
| Max. process+thread count | 2570 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/28687/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Crashing bugs in NameNode when using a valid configuration for
> `dfs.namenode.audit.loggers`
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-15124
> URL: https://issues.apache.org/jira/browse/HDFS-15124
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.10.0
> Reporter: Ctest
> Priority: Critical
> Attachments: HDFS-15124.000.patch, HDFS-15124.001.patch,
> HDFS-15124.002.patch
>
>
> I am using Hadoop-2.10.0.
> The configuration parameter `dfs.namenode.audit.loggers` allows `default`
> (which is the default value) and
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`.
> When I use `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`,
> namenode will not be started successfully because of an
> `InstantiationException` thrown from
> `org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers`.
> The root cause is that while initializing namenode, `initAuditLoggers` will
> be called and it will try to call the default constructor of
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` which doesn't
> have a default constructor. Thus the `InstantiationException` exception is
> thrown.
>
> *Symptom*
> *$ ./start-dfs.sh*
> {code:java}
> 2019-12-18 14:05:20,670 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.java.lang.RuntimeException:
> java.lang.InstantiationException:
> org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1024)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:858)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:677)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:674)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:736)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:961)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:940)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1714)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1782)
> Caused by: java.lang.InstantiationException:
> org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger
> at java.lang.Class.newInstance(Class.java:427)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initAuditLoggers(FSNamesystem.java:1017)...
> 8 more
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger.<init>()
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.newInstance(Class.java:412)
> ... 9 more{code}
>
>
> *Detailed Root Cause*
> There is no default constructor in
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`:
> {code:java}
> /**
> * An {@link AuditLogger} that sends logged data directly to the metrics
> * systems. It is used when the top service is used directly by the name node
> */
> @InterfaceAudience.Private
> public class TopAuditLogger implements AuditLogger {
> public static finalLogger LOG =
> LoggerFactory.getLogger(TopAuditLogger.class);
> private final TopMetrics topMetrics;
> public TopAuditLogger(TopMetrics topMetrics) {
> Preconditions.checkNotNull(topMetrics, "Cannot init with a null " +
> "TopMetrics");
> this.topMetrics = topMetrics;
> }
> @Override
> public void initialize(Configuration conf) {
> }
> {code}
> As long as the configuration parameter `dfs.namenode.audit.loggers` is set to
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`,
> `initAuditLoggers` will try to call its default constructor to make a new
> instance:
> {code:java}
> private List<AuditLogger> initAuditLoggers(Configuration conf) {
> // Initialize the custom access loggers if configured.
> Collection<String> alClasses =
> conf.getTrimmedStringCollection(DFS_NAMENODE_AUDIT_LOGGERS_KEY);
> List<AuditLogger> auditLoggers = Lists.newArrayList();
> if (alClasses != null && !alClasses.isEmpty()) {
> for (String className : alClasses) {
> try {
> AuditLogger logger;
> if (DFS_NAMENODE_DEFAULT_AUDIT_LOGGER_NAME.equals(className)) {
> logger = new DefaultAuditLogger();
> } else {
> logger = (AuditLogger) Class.forName(className).newInstance();
> }
> logger.initialize(conf);
> auditLoggers.add(logger);
> } catch (RuntimeException re) {
> throw re;
> } catch (Exception e) {
> throw new RuntimeException(e);
> }
> }
> }
> {code}
> `initAuditLoggers` tries to call the default constructor to make a new
> instance in:
> {code:java}
> logger = (AuditLogger) Class.forName(className).newInstance();
> {code}
> This is very different from the default configuration, `default`, which
> implements a default constructor so the default is fine.
>
> *How To Reproduce*
> The version of Hadoop: 2.10.0
> # Set the value of configuration parameter `dfs.namenode.audit.loggers` to
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger` in
> "hdfs-site.xml"(the default value is `default`)
> # Start the namenode by running "start-dfs.sh"
> # The namenode will not be started successfully.
> {code:java}
> <property>
> <name>dfs.namenode.audit.loggers</name>
> <value>org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger</value>
> <description>
> List of classes implementing audit loggers that will receive audit events.
> These should be implementations of
> org.apache.hadoop.hdfs.server.namenode.AuditLogger.
> The special value "default" can be used to reference the default audit
> logger, which uses the configured log system. Installing custom audit
> loggers
> may affect the performance and stability of the NameNode. Refer to the
> custom
> logger's documentation for more details.
> </description>
> </property>
> {code}
>
> *How To Fix*
> Add a default constructor for
> `org.apache.hadoop.hdfs.server.namenode.top.TopAuditLogger`.
> I wrote a patch to add a default constructor for TopAuditLogger.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
