[jira] [Updated] (HDFS-15919) BlockPoolManager should log stack trace if unable to get Namenode addresses
[
https://issues.apache.org/jira/browse/HDFS-15919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen O'Donnell updated HDFS-15919:
-
Attachment: HDFS-15919.001.patch
> BlockPoolManager should log stack trace if unable to get Namenode addresses
> ---
>
> Key: HDFS-15919
> URL: https://issues.apache.org/jira/browse/HDFS-15919
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Affects Versions: 3.4.0
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Attachments: HDFS-15919.001.patch
>
>
> If the hdfs config is badly configured, the datanode can fail to start with
> this stack trace:
> {code}
> 2021-03-24 05:58:27,026 INFO datanode.DataNode
> (BlockPoolManager.java:refreshNamenodes(149)) - Refresh request received for
> nameservices: null
> 2021-03-24 05:58:27,033 WARN datanode.DataNode
> (BlockPoolManager.java:refreshNamenodes(161)) - Unable to get NameNode
> addresses.
> ...
> 2021-03-24 05:58:27,077 ERROR datanode.DataNode
> (DataNode.java:secureMain(2883)) - Exception in secureMain
> java.io.IOException: No services to connect, missing NameNode address.
> at
> org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.refreshNamenodes(BlockPoolManager.Java:165)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1440)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:500)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2782)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2690)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2732)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2876)
> at
> org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.start(SecureDataNodeStarter.java:100)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
> {code}
> In this case, the issue was an exception thrown in
> DFSUtil.getNNServiceRpcAddressesForCluster(...) but there are a couple of
> scenarios within it which can cause an exception, so its difficult to figure
> out what is wrong with the config.
> We should simple add the exception onto the existing log message when an
> error occurs so it is clear what caused it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15919) BlockPoolManager should log stack trace if unable to get Namenode addresses
[
https://issues.apache.org/jira/browse/HDFS-15919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HDFS-15919:
Status: Patch Available (was: Open)
> BlockPoolManager should log stack trace if unable to get Namenode addresses
> ---
>
> Key: HDFS-15919
> URL: https://issues.apache.org/jira/browse/HDFS-15919
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Affects Versions: 3.4.0
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Attachments: HDFS-15919.001.patch
>
>
> If the hdfs config is badly configured, the datanode can fail to start with
> this stack trace:
> {code}
> 2021-03-24 05:58:27,026 INFO datanode.DataNode
> (BlockPoolManager.java:refreshNamenodes(149)) - Refresh request received for
> nameservices: null
> 2021-03-24 05:58:27,033 WARN datanode.DataNode
> (BlockPoolManager.java:refreshNamenodes(161)) - Unable to get NameNode
> addresses.
> ...
> 2021-03-24 05:58:27,077 ERROR datanode.DataNode
> (DataNode.java:secureMain(2883)) - Exception in secureMain
> java.io.IOException: No services to connect, missing NameNode address.
> at
> org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.refreshNamenodes(BlockPoolManager.Java:165)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1440)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:500)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2782)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2690)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2732)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2876)
> at
> org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.start(SecureDataNodeStarter.java:100)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
> {code}
> In this case, the issue was an exception thrown in
> DFSUtil.getNNServiceRpcAddressesForCluster(...) but there are a couple of
> scenarios within it which can cause an exception, so its difficult to figure
> out what is wrong with the config.
> We should simple add the exception onto the existing log message when an
> error occurs so it is clear what caused it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15919) BlockPoolManager should log stack trace if unable to get Namenode addresses
[
https://issues.apache.org/jira/browse/HDFS-15919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoqiao He updated HDFS-15919:
---
Fix Version/s: 3.2.3
3.4.0
3.3.1
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
LGTM, +1. Committed to trunk and cherry-pick clean to branch-3.3 and branch-3.2.
Thanks [~sodonnell] for your contributions! Thanks [~vjasani] and [~ayushtkn]
for your reviews!
> BlockPoolManager should log stack trace if unable to get Namenode addresses
> ---
>
> Key: HDFS-15919
> URL: https://issues.apache.org/jira/browse/HDFS-15919
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Affects Versions: 3.4.0
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Fix For: 3.3.1, 3.4.0, 3.2.3
>
> Attachments: HDFS-15919.001.patch
>
>
> If the hdfs config is badly configured, the datanode can fail to start with
> this stack trace:
> {code}
> 2021-03-24 05:58:27,026 INFO datanode.DataNode
> (BlockPoolManager.java:refreshNamenodes(149)) - Refresh request received for
> nameservices: null
> 2021-03-24 05:58:27,033 WARN datanode.DataNode
> (BlockPoolManager.java:refreshNamenodes(161)) - Unable to get NameNode
> addresses.
> ...
> 2021-03-24 05:58:27,077 ERROR datanode.DataNode
> (DataNode.java:secureMain(2883)) - Exception in secureMain
> java.io.IOException: No services to connect, missing NameNode address.
> at
> org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.refreshNamenodes(BlockPoolManager.Java:165)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1440)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:500)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2782)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2690)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2732)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2876)
> at
> org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.start(SecureDataNodeStarter.java:100)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
> {code}
> In this case, the issue was an exception thrown in
> DFSUtil.getNNServiceRpcAddressesForCluster(...) but there are a couple of
> scenarios within it which can cause an exception, so its difficult to figure
> out what is wrong with the config.
> We should simple add the exception onto the existing log message when an
> error occurs so it is clear what caused it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15919) BlockPoolManager should log stack trace if unable to get Namenode addresses
[
https://issues.apache.org/jira/browse/HDFS-15919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoqiao He updated HDFS-15919:
---
Fix Version/s: 3.1.5
> BlockPoolManager should log stack trace if unable to get Namenode addresses
> ---
>
> Key: HDFS-15919
> URL: https://issues.apache.org/jira/browse/HDFS-15919
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Affects Versions: 3.4.0
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Fix For: 3.3.1, 3.4.0, 3.1.5, 3.2.3
>
> Attachments: HDFS-15919.001.patch
>
>
> If the hdfs config is badly configured, the datanode can fail to start with
> this stack trace:
> {code}
> 2021-03-24 05:58:27,026 INFO datanode.DataNode
> (BlockPoolManager.java:refreshNamenodes(149)) - Refresh request received for
> nameservices: null
> 2021-03-24 05:58:27,033 WARN datanode.DataNode
> (BlockPoolManager.java:refreshNamenodes(161)) - Unable to get NameNode
> addresses.
> ...
> 2021-03-24 05:58:27,077 ERROR datanode.DataNode
> (DataNode.java:secureMain(2883)) - Exception in secureMain
> java.io.IOException: No services to connect, missing NameNode address.
> at
> org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.refreshNamenodes(BlockPoolManager.Java:165)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1440)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.(DataNode.java:500)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2782)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2690)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2732)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2876)
> at
> org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.start(SecureDataNodeStarter.java:100)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
> {code}
> In this case, the issue was an exception thrown in
> DFSUtil.getNNServiceRpcAddressesForCluster(...) but there are a couple of
> scenarios within it which can cause an exception, so its difficult to figure
> out what is wrong with the config.
> We should simple add the exception onto the existing log message when an
> error occurs so it is clear what caused it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
