[jira] Resolved: (HDFS-1549) ArrayIndexOutOfBoundsException throwed from BlockLocation
[ https://issues.apache.org/jira/browse/HDFS-1549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhou resolved HDFS-1549. Resolution: Not A Problem Since the bug has been fixed in trunk, I should close this issue. > ArrayIndexOutOfBoundsException throwed from BlockLocation > -- > > Key: HDFS-1549 > URL: https://issues.apache.org/jira/browse/HDFS-1549 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Min Zhou > Fix For: 0.22.0 > > > BlockLocation object created through the default constructor has a hosts > array with its length of zero. It will apparently throw an > ArrayIndexOutOfBoundsException when reading fields from DataInput if not > resized the array length. > Exception in thread "IPC Client (47) connection to nn151/192.168.201.151:9020 > from zhoumin" java.lang.ArrayIndexOutOfBoundsException: 0 > at > org.apache.hadoop.fs.BlockLocation.readFields(BlockLocation.java:177) > at > org.apache.hadoop.fs.LocatedFileStatus.readFields(LocatedFileStatus.java:85) > at > org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:237) > at > org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:171) > at > org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:219) > at > org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:66) > at > org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:509) > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:439) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1549) ArrayIndexOutOfBoundsException throwed from BlockLocation
ArrayIndexOutOfBoundsException throwed from BlockLocation -- Key: HDFS-1549 URL: https://issues.apache.org/jira/browse/HDFS-1549 Project: Hadoop HDFS Issue Type: Bug Reporter: Min Zhou Fix For: 0.22.0 BlockLocation object created through the default constructor has a hosts array with its length of zero. It will apparently throw an ArrayIndexOutOfBoundsException when reading fields from DataInput if not resized the array length. Exception in thread "IPC Client (47) connection to nn151/192.168.201.151:9020 from zhoumin" java.lang.ArrayIndexOutOfBoundsException: 0 at org.apache.hadoop.fs.BlockLocation.readFields(BlockLocation.java:177) at org.apache.hadoop.fs.LocatedFileStatus.readFields(LocatedFileStatus.java:85) at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:237) at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:171) at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:219) at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:66) at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:509) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:439) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1052) HDFS scalability with multiple namenodes
[ https://issues.apache.org/jira/browse/HDFS-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870569#action_12870569 ] Min Zhou commented on HDFS-1052: I don't think multiple namespaces is a good solution for this issue. The datasets stored on our cluster are shared by many departments of our company. If these datasets are seperated by a number of namespaces, there is no data sharing; If we put them in one namespace managed by a single NameNode, however, the scalability is limited by NameNode's memory . Why don't we employ some distributed metadata management approaches like dynamic subtree patitioning(ceph) or hash-based partitioning(Lustre) ? Min > HDFS scalability with multiple namenodes > > > Key: HDFS-1052 > URL: https://issues.apache.org/jira/browse/HDFS-1052 > Project: Hadoop HDFS > Issue Type: New Feature > Components: name-node >Affects Versions: 0.22.0 >Reporter: Suresh Srinivas >Assignee: Suresh Srinivas > Attachments: Block pool proposal.pdf, Mulitple Namespaces5.pdf > > > HDFS currently uses a single namenode that limits scalability of the cluster. > This jira proposes an architecture to scale the nameservice horizontally > using multiple namenodes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-265) Revisit append
[ https://issues.apache.org/jira/browse/HDFS-265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805818#action_12805818 ] Min Zhou commented on HDFS-265: --- @Hairong Kuang Which algorithm did you pick for read operation of hdfs? How can I find it from the trunk ? btw, I think "A DataNode changes its (BA, BR) to be (a+b, a+b) right after step 2.a " should be ".. 3.a" on Page 5 of your appendDesign2.pdf Min > Revisit append > -- > > Key: HDFS-265 > URL: https://issues.apache.org/jira/browse/HDFS-265 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 0.21.0 >Reporter: Hairong Kuang >Assignee: Hairong Kuang > Fix For: 0.21.0 > > Attachments: a.sh, appendDesign.pdf, appendDesign.pdf, > appendDesign1.pdf, appendDesign2.pdf, AppendSpec.pdf, AppendTestPlan.html, > AppendTestPlan.html, AppendTestPlan.html, AppendTestPlan.html, > AppendTestPlan.html, AppendTestPlan.html, AppendTestPlan.html, > AppendTestPlan.html, AppendTestPlan.html, TestPlanAppend.html > > > HADOOP-1700 and related issues have put a lot of efforts to provide the first > implementation of append. However, append is such a complex feature. It turns > out that there are issues that were initially seemed trivial but needs a > careful design. This jira revisits append, aiming for a design and > implementation supporting a semantics that are acceptable to its users. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
