[
https://issues.apache.org/jira/browse/HADOOP-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657031#action_12657031
]
Brian Bockelman commented on HADOOP-4866:
-----------------------------------------
If I filter out these commitBlockSync messages, I also see the following error
which may or may not be related. This is from the datanode side, but the first
portion is printed out at the NameNode.
Brian
2008-12-16 09:52:19,176 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
Client calls recoverBlock(block=blk_-2476019851465028189_97924,
targets=[172.16.1.99:50010])
2008-12-16 09:52:19,179 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3
on 50020, call recoverBlock(blk_-2476019851465028189_97924, false,
[Lorg.apache.hadoop.hdfs.protocol.DatanodeInfo;@f749036) from
129.93.239.151:58727: error: org.apache.hadoop.ipc.RemoteException:
java.io.IOException: blk_-2476019851465028189_97924 is already commited,
storedBlock == null.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:4556)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:402)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:892)
org.apache.hadoop.ipc.RemoteException: java.io.IOException:
blk_-2476019851465028189_97924 is already commited, storedBlock == null.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.nextGenerationStampForBlock(FSNamesystem.java:4556)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.nextGenerationStamp(NameNode.java:402)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:892)
at org.apache.hadoop.ipc.Client.call(Client.java:696)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
at $Proxy4.nextGenerationStamp(Unknown Source)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.syncBlock(DataNode.java:1468)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1442)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.recoverBlock(DataNode.java:1508)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:892)
> NameNode error in commitBlockSynchronization
> --------------------------------------------
>
> Key: HADOOP-4866
> URL: https://issues.apache.org/jira/browse/HADOOP-4866
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Brian Bockelman
> Attachments: 4866_20081215.patch
>
>
> The NameNode continuously has an error in the commitBlockSynchronization.
> This happens for ~5 blocks at a rate of 5-10Hz. I have no idea when this
> started happening because this has been going on for days, well past the
> start of our current logs.
> This appears to be a new symptom in 0.19.0, but I have no idea what could be
> causing it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
