Hi Ravi,
The two datanodes are on different Machines. At the time when these error were generating I can see that DN1 was replicating under-replicating blocks on DN2. Can this be related to properties I added for increasing replication rate? Regards Om Prakash From: Ravi Prakash [mailto:ravihad...@gmail.com] Sent: 27 July 2017 01:26 To: omprakash <ompraka...@cdac.in> Cc: user <user@hadoop.apache.org> Subject: Re: Lots of Exception for "cannot assign requested address" in datanode logs Hi Omprakash! DatanodeRegistration happens when the Datanode first hearbeats to the Namenode. In your case, it seems some other application has acquired the port 50010 . You can check this with the command "netstat -anp | grep 50010" . Are you trying to run 2 datanode processes on the same machine? HTH Ravi On Wed, Jul 26, 2017 at 5:46 AM, omprakash <ompraka...@cdac.in <mailto:ompraka...@cdac.in> > wrote: Hi all, I am running a 4 node cluster with 2 Master node( NN1, NN2 with HA using QJM) and 2 Slave nodes(DN1, DN2). I am receiving lots of Exceptions in Datanode logs as shown below 2017-07-26 17:56:00,703 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(192.168.9.132:50010 <http://192.168.9.132:50010> , datanodeUuid=5a2e6721-3a9a-43f1-94cc-f58f24b5a15b, infoPort=50075, infoSecurePort=0, ipcPort=50020, storageInfo=lv=-57;cid=CID-7aa9fcd4-36fc-4e7b-87cd-d20594774b85;nsid=1753301932;c=1500696043365):Failed to transfer BP-1085904515-192.168.9.116-1500696043365:blk_1078544770_4804082 to 192.168.9.116:50010 <http://192.168.9.116:50010> got java.net.BindException: Cannot assign requested address at sun.nio.ch.Net.connect0(Native Method) at sun.nio.ch.Net.connect(Net.java:465) at sun.nio.ch.Net.connect(Net.java:457) at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) at org.apache.hadoop.net <http://org.apache.hadoop.net> .SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) at org.apache.hadoop.net <http://org.apache.hadoop.net> .NetUtils.connect(NetUtils.java:531) at org.apache.hadoop.net <http://org.apache.hadoop.net> .NetUtils.connect(NetUtils.java:495) at org.apache.hadoop.hdfs.server.datanode.DataNode$DataTransfer.run(DataNode.java:2312) at java.lang.Thread.run(Thread.java:745) I have 10 million files in hdfs. All the nodes have same configurations. Above Exception started occurring when I changed the below parameters in hdfs-site.xml file. I made these changes to increase replication rate for under-replicated blocks. dfs.namenode.handler.count=5000 dfs.namenode.replication.work <http://dfs.namenode.replication.work> .multiplier.per.iteration=1000 dfs.namenode.replication.max-streams=2000 --> not documented in hdfs.site.xml dfs.namenode.replication.max-streams-hard-limit=4000 ---> not documented in hdfs.site.xml The rate of replication of blocks increased but suddenly the Exception started to appear. Can anybody explain this behavior? Regards Omprakash Paliwal ------------------------------------------------------------------------------------------------------------------------------- [ C-DAC is on Social-Media too. Kindly follow us at: Facebook: https://www.facebook.com/CDACINDIA & Twitter: @cdacindia ] This e-mail is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies and the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email is strictly prohibited and appropriate legal action will be taken. ------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------- [ C-DAC is on Social-Media too. Kindly follow us at: Facebook: https://www.facebook.com/CDACINDIA & Twitter: @cdacindia ] This e-mail is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies and the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email is strictly prohibited and appropriate legal action will be taken. -------------------------------------------------------------------------------------------------------------------------------
