[
https://issues.apache.org/jira/browse/HADOOP-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dhruba borthakur updated HADOOP-1520:
-------------------------------------
Attachment: 1520.patch
I added appropriate synchronization to FSEditLog.java. can you pl see if you
started seeing these problems since HADOOP-1003 was applied? Also, if possible,
can you apply this patch and rerun the benchmark on the 100 node cluster to see
if it fixes your problem? Thanks.
> IndexOutOfBoundsException in FSEditLog.processIOError
> -----------------------------------------------------
>
> Key: HADOOP-1520
> URL: https://issues.apache.org/jira/browse/HADOOP-1520
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.14.0
> Reporter: Nigel Daley
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: 1520.patch
>
>
> Running NNBench I saw these exceptions in the NameNode logs:
> NAMENODE:
> 2007-06-21 04:02:53,587 INFO org.apache.hadoop.fs.FSNamesystem: Roll FSImage
> 2007-06-21 04:02:53,594 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 0 on 8020, call complete(/user/hadoopqa/nameNode100Benchmark/output/.214.crc,
> DFSClient_-1646448212) from 72.30.51.75:56245: error: java.io.IOException:
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
> java.io.IOException: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
> at java.util.ArrayList.RangeCheck(ArrayList.java:547)
> at java.util.ArrayList.remove(ArrayList.java:387)
> at org.apache.hadoop.dfs.FSEditLog.processIOError(FSEditLog.java:169)
> at org.apache.hadoop.dfs.FSEditLog.logSync(FSEditLog.java:407)
> at
> org.apache.hadoop.dfs.FSNamesystem.completeFile(FSNamesystem.java:878)
> at org.apache.hadoop.dfs.NameNode.complete(NameNode.java:346)
> at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566)
> ...
> 2007-06-21 04:02:53,823 WARN org.apache.hadoop.dfs.StateChange: DIR*
> NameSystem.completeFile: failed to complete
> /user/hadoopqa/nameNode100Benchmark/output/.214.crc because
> dir.getFileBlocks() is non-null and pendingFile is null
> 2007-06-21 04:02:53,828 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 14 on 8020, call
> complete(/user/hadoopqa/nameNode100Benchmark/output/.214.crc,
> DFSClient_-1646448212) from 72.30.51.75:56245: error: java.io.IOException:
> Could not complete write to file
> /user/hadoopqa/nameNode100Benchmark/output/.214.crc by DFSClient_-1646448212
> java.io.IOException: Could not complete write to file
> /user/hadoopqa/nameNode100Benchmark/output/.214.crc by DFSClient_-1646448212
> at org.apache.hadoop.dfs.NameNode.complete(NameNode.java:352)
> at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566)
> This last exception occurs over and over again forever (this could be a
> result of the way NNBench is written).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
