[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901234#comment-13901234 ] Hudson commented on HDFS-5285: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5169 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5169/]) Move Flatten INode hierarchy jiras (HDFS-5531, HDFS-5285, HDFS-5286, HDFS-5537, HDFS-5554, HDFS-5647, HDFS-5632, HDFS-5715, HDFS-5726) to 2.4.0 section in CHANGES.txt (jing9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1568218) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Flatten INodeFile hierarchy: Add UnderContruction Feature - Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Jing Zhao Fix For: 3.0.0, 2.4.0 Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch, HDFS-5285.003.patch, HDFS-5285.b2.patch, h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of classes is exponential to the number of features. Currently, there are only two features, UnderConstruction and WithSnapshot. The number of classes is 2^2 = 4. It is hard to add one more feature since the number of classes will become 2^3 = 8. As a first step, we implement an Under-Construction feature to replace INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot in this jira. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901321#comment-13901321 ] Hudson commented on HDFS-5285: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #481 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/481/]) Move Flatten INode hierarchy jiras (HDFS-5531, HDFS-5285, HDFS-5286, HDFS-5537, HDFS-5554, HDFS-5647, HDFS-5632, HDFS-5715, HDFS-5726) to 2.4.0 section in CHANGES.txt (jing9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1568218) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Flatten INodeFile hierarchy: Add UnderContruction Feature - Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Jing Zhao Fix For: 3.0.0, 2.4.0 Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch, HDFS-5285.003.patch, HDFS-5285.b2.patch, h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of classes is exponential to the number of features. Currently, there are only two features, UnderConstruction and WithSnapshot. The number of classes is 2^2 = 4. It is hard to add one more feature since the number of classes will become 2^3 = 8. As a first step, we implement an Under-Construction feature to replace INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot in this jira. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901416#comment-13901416 ] Hudson commented on HDFS-5285: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1673 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1673/]) Move Flatten INode hierarchy jiras (HDFS-5531, HDFS-5285, HDFS-5286, HDFS-5537, HDFS-5554, HDFS-5647, HDFS-5632, HDFS-5715, HDFS-5726) to 2.4.0 section in CHANGES.txt (jing9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1568218) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Flatten INodeFile hierarchy: Add UnderContruction Feature - Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Jing Zhao Fix For: 3.0.0, 2.4.0 Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch, HDFS-5285.003.patch, HDFS-5285.b2.patch, h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of classes is exponential to the number of features. Currently, there are only two features, UnderConstruction and WithSnapshot. The number of classes is 2^2 = 4. It is hard to add one more feature since the number of classes will become 2^3 = 8. As a first step, we implement an Under-Construction feature to replace INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot in this jira. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901493#comment-13901493 ] Hudson commented on HDFS-5285: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1698 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1698/]) Move Flatten INode hierarchy jiras (HDFS-5531, HDFS-5285, HDFS-5286, HDFS-5537, HDFS-5554, HDFS-5647, HDFS-5632, HDFS-5715, HDFS-5726) to 2.4.0 section in CHANGES.txt (jing9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1568218) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Flatten INodeFile hierarchy: Add UnderContruction Feature - Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Jing Zhao Fix For: 3.0.0, 2.4.0 Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch, HDFS-5285.003.patch, HDFS-5285.b2.patch, h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of classes is exponential to the number of features. Currently, there are only two features, UnderConstruction and WithSnapshot. The number of classes is 2^2 = 4. It is hard to add one more feature since the number of classes will become 2^3 = 8. As a first step, we implement an Under-Construction feature to replace INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot in this jira. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901162#comment-13901162 ] Tsz Wo (Nicholas), SZE commented on HDFS-5285: -- +1 the branch-2 patch looks good. Flatten INodeFile hierarchy: Add UnderContruction Feature - Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Jing Zhao Fix For: 3.0.0 Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch, HDFS-5285.003.patch, HDFS-5285.b2.patch, h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of classes is exponential to the number of features. Currently, there are only two features, UnderConstruction and WithSnapshot. The number of classes is 2^2 = 4. It is hard to add one more feature since the number of classes will become 2^3 = 8. As a first step, we implement an Under-Construction feature to replace INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot in this jira. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13829871#comment-13829871 ] Hudson commented on HDFS-5285: -- FAILURE: Integrated in Hadoop-Yarn-trunk #399 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/399/]) HDFS-5285. Flatten INodeFile hierarchy: Replace INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot with FileUnderContructionFeature. Contributed by jing9 (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1544389) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockCollection.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileUnderConstructionFeature.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileUnderConstruction.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeDirectoryWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCommitBlockSynchronization.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestRetryCacheWithHA.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotBlocksMap.java Flatten INodeFile hierarchy: Add UnderContruction Feature - Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Jing Zhao Fix For: 3.0.0 Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch, HDFS-5285.003.patch, h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13829947#comment-13829947 ] Hudson commented on HDFS-5285: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1590 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1590/]) HDFS-5285. Flatten INodeFile hierarchy: Replace INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot with FileUnderContructionFeature. Contributed by jing9 (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1544389) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockCollection.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileUnderConstructionFeature.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileUnderConstruction.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeDirectoryWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCommitBlockSynchronization.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestRetryCacheWithHA.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotBlocksMap.java Flatten INodeFile hierarchy: Add UnderContruction Feature - Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Jing Zhao Fix For: 3.0.0 Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch, HDFS-5285.003.patch, h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13829958#comment-13829958 ] Hudson commented on HDFS-5285: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1616 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1616/]) HDFS-5285. Flatten INodeFile hierarchy: Replace INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot with FileUnderContructionFeature. Contributed by jing9 (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1544389) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockCollection.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileUnderConstructionFeature.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileUnderConstruction.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeDirectoryWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCommitBlockSynchronization.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestRetryCacheWithHA.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotBlocksMap.java Flatten INodeFile hierarchy: Add UnderContruction Feature - Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Jing Zhao Fix For: 3.0.0 Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch, HDFS-5285.003.patch, h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13829557#comment-13829557 ] Vinay commented on HDFS-5285: - +1, patch looks good Jing. Flatten INodeFile hierarchy: Add UnderContruction Feature - Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Jing Zhao Fix For: 3.0.0 Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch, HDFS-5285.003.patch, h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of classes is exponential to the number of features. Currently, there are only two features, UnderConstruction and WithSnapshot. The number of classes is 2^2 = 4. It is hard to add one more feature since the number of classes will become 2^3 = 8. As a first step, we implement an Under-Construction feature to replace INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot in this jira. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13829566#comment-13829566 ] Hudson commented on HDFS-5285: -- SUCCESS: Integrated in Hadoop-trunk-Commit #4785 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/4785/]) HDFS-5285. Flatten INodeFile hierarchy: Replace INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot with FileUnderContructionFeature. Contributed by jing9 (szetszwo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1544389) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockCollection.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileUnderConstructionFeature.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileUnderConstruction.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeDirectoryWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithSnapshot.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCommitBlockSynchronization.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestRetryCacheWithHA.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestRenameWithSnapshots.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotBlocksMap.java Flatten INodeFile hierarchy: Add UnderContruction Feature - Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Jing Zhao Fix For: 3.0.0 Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch, HDFS-5285.003.patch, h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[
https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13829389#comment-13829389
]
Hadoop QA commented on HDFS-5285:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12615172/HDFS-5285.003.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5530//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5530//console
This message is automatically generated.
Flatten INodeFile hierarchy: Add UnderContruction Feature
-
Key: HDFS-5285
URL: https://issues.apache.org/jira/browse/HDFS-5285
Project: Hadoop HDFS
Issue Type: Sub-task
Components: namenode
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Jing Zhao
Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch,
HDFS-5285.003.patch, h5285_20131001.patch, h5285_20131002.patch,
h5285_20131118.patch
For files, there are INodeFile, INodeFileUnderConstruction,
INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for
representing whether a file is under construction or whether it is in some
snapshot. The following are two major problems of the current approach:
- Java class does not support multiple inheritances so that
INodeFileUnderConstructionWithSnapshot cannot extend both
INodeFileUnderConstruction and INodeFileWithSnapshot.
- The number of classes is exponential to the number of features. Currently,
there are only two features, UnderConstruction and WithSnapshot. The number
of classes is 2^2 = 4. It is hard to add one more feature since the number
of classes will become 2^3 = 8.
As a first step, we implement an Under-Construction feature to replace
INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot in this
jira.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[
https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13828284#comment-13828284
]
Hadoop QA commented on HDFS-5285:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12614961/HDFS-5285.001.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReplication
org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement
org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus
org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader
org.apache.hadoop.hdfs.server.datanode.TestDiskError
org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace
org.apache.hadoop.hdfs.server.balancer.TestBalancer
org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
org.apache.hadoop.hdfs.server.datanode.TestBlockReport
org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
org.apache.hadoop.hdfs.server.namenode.TestCorruptFilesJsp
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
org.apache.hadoop.hdfs.server.blockmanagement.TestNodeCount
org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
org.apache.hadoop.hdfs.server.namenode.TestCreateEditsLog
org.apache.hadoop.hdfs.server.namenode.TestHostsFiles
org.apache.hadoop.hdfs.TestSafeMode
org.apache.hadoop.hdfs.TestFileAppend4
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer
org.apache.hadoop.hdfs.TestDatanodeBlockScanner
org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool
org.apache.hadoop.hdfs.server.namenode.TestEditLog
The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestDecommission
org.apache.hadoop.hdfs.TestReplication
org.apache.hadoop.hdfs.TestInjectionForSimulatedStorage
org.apache.hadoop.hdfs.TestMissingBlocksAlert
org.apache.hadoop.hdfs.TestSetrepIncreasing
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
org.apache.hadoop.hdfs.TestDatanodeDeath
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5506//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5506//console
This message is automatically generated.
Flatten INodeFile hierarchy: Add UnderContruction Feature
-
Key: HDFS-5285
URL: https://issues.apache.org/jira/browse/HDFS-5285
Project: Hadoop HDFS
Issue Type: Sub-task
Components: namenode
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Jing Zhao
Attachments: HDFS-5285.001.patch, h5285_20131001.patch,
h5285_20131002.patch, h5285_20131118.patch
For files, there are INodeFile, INodeFileUnderConstruction,
INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for
representing whether a file is under
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[
https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13828510#comment-13828510
]
Vinay commented on HDFS-5285:
-
Overall patch looks good.
Following are some of the minor nits
1. Better to add an error message to these checks wherever necessary ( May not
be necessary at all places)
{code} Preconditions.checkState(fileINode.isUnderConstruction());{code}
{code} Preconditions.checkArgument(file.isUnderConstruction());{code}
2. Better update {{INodeFile.toINodeFile()}} to some suitable name like
{{INodeFile.completeFile()}}
3. in {{FSNameSystem#getAdditionalBlock}} can add Precondition check for
underconstruction ( 2nd time)
{code} final INodeFile pendingFile = inodes[inodes.length -
1].asFile();{code}
4. There are many blank lines with trailing whitespaces. Its better to clean
these. (May be can correct eclipse formatter)
5. Since {{MutableBlockCollection}} is removed we can remove in comments as well
{code}
@Override // MutableBlockCollection
public void setBlock(int index, BlockInfo blk) {
this.blocks[index] = blk;
}{code}
6. In TestRenameWithSnapshots additional assertion of isUnderConstruction is
required.
{code} INode fooNode = fooRef.asFile();
assertTrue(fooNode instanceof INodeFileWithSnapshot);{code}
Lets wait and see for jenkins report, how many tests are OK..?
Flatten INodeFile hierarchy: Add UnderContruction Feature
-
Key: HDFS-5285
URL: https://issues.apache.org/jira/browse/HDFS-5285
Project: Hadoop HDFS
Issue Type: Sub-task
Components: namenode
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Jing Zhao
Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch,
h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch
For files, there are INodeFile, INodeFileUnderConstruction,
INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for
representing whether a file is under construction or whether it is in some
snapshot. The following are two major problems of the current approach:
- Java class does not support multiple inheritances so that
INodeFileUnderConstructionWithSnapshot cannot extend both
INodeFileUnderConstruction and INodeFileWithSnapshot.
- The number of classes is exponential to the number of features. Currently,
there are only two features, UnderConstruction and WithSnapshot. The number
of classes is 2^2 = 4. It is hard to add one more feature since the number
of classes will become 2^3 = 8.
As a first step, we implement an Under-Construction feature to replace
INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot in this
jira.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy: Add UnderContruction Feature
[
https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13828550#comment-13828550
]
Hadoop QA commented on HDFS-5285:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12615061/HDFS-5285.002.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5517//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5517//console
This message is automatically generated.
Flatten INodeFile hierarchy: Add UnderContruction Feature
-
Key: HDFS-5285
URL: https://issues.apache.org/jira/browse/HDFS-5285
Project: Hadoop HDFS
Issue Type: Sub-task
Components: namenode
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Jing Zhao
Attachments: HDFS-5285.001.patch, HDFS-5285.002.patch,
h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch
For files, there are INodeFile, INodeFileUnderConstruction,
INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for
representing whether a file is under construction or whether it is in some
snapshot. The following are two major problems of the current approach:
- Java class does not support multiple inheritances so that
INodeFileUnderConstructionWithSnapshot cannot extend both
INodeFileUnderConstruction and INodeFileWithSnapshot.
- The number of classes is exponential to the number of features. Currently,
there are only two features, UnderConstruction and WithSnapshot. The number
of classes is 2^2 = 4. It is hard to add one more feature since the number
of classes will become 2^3 = 8.
As a first step, we implement an Under-Construction feature to replace
INodeFileUnderConstruction and INodeFileUnderConstructionWithSnapshot in this
jira.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826112#comment-13826112 ] Vinay commented on HDFS-5285: - Thanks Nicholas for the Work. INode hierarchy definitely needs optimization for supporting more features. Current looks good. It will be more clear once all compilation errors are addressed. I can see INodeFileWithSnapshot still in the hierarchy which extends INodeFile. You intentionally left it as it is ..? Flatten INodeFile hierarchy --- Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of classes is exponential to the number of features. Currently, there are only two features, UnderConstruction and WithSnapshot. The number of classes is 2^2 = 4. It is hard to add one more feature since the number of classes will become 2^3 = 8. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy
[ https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13826124#comment-13826124 ] Tsz Wo (Nicholas), SZE commented on HDFS-5285: -- Vinay, thanks for taking a look. I can see INodeFileWithSnapshot still in the hierarchy which extends INodeFile. You intentionally left it as it is ..? My patch is not completed yet. Will remove INodeFileWithSnapshot and the other INodeFile subclasses. Flatten INodeFile hierarchy --- Key: HDFS-5285 URL: https://issues.apache.org/jira/browse/HDFS-5285 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Attachments: h5285_20131001.patch, h5285_20131002.patch, h5285_20131118.patch For files, there are INodeFile, INodeFileUnderConstruction, INodeFileWithSnapshot and INodeFileUnderConstructionWithSnapshot for representing whether a file is under construction or whether it is in some snapshot. The following are two major problems of the current approach: - Java class does not support multiple inheritances so that INodeFileUnderConstructionWithSnapshot cannot extend both INodeFileUnderConstruction and INodeFileWithSnapshot. - The number of classes is exponential to the number of features. Currently, there are only two features, UnderConstruction and WithSnapshot. The number of classes is 2^2 = 4. It is hard to add one more feature since the number of classes will become 2^3 = 8. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HDFS-5285) Flatten INodeFile hierarchy
[
https://issues.apache.org/jira/browse/HDFS-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13785350#comment-13785350
]
Hadoop QA commented on HDFS-5285:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12606594/h5285_20131002.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
org.apache.hadoop.hdfs.TestClose
org.apache.hadoop.hdfs.TestShortCircuitLocalRead
org.apache.hadoop.hdfs.web.TestFSMainOperationsWebHdfs
org.apache.hadoop.hdfs.TestFSInputChecker
org.apache.hadoop.hdfs.server.namenode.TestBackupNode
org.apache.hadoop.hdfs.TestDataTransferProtocol
org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol
org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
org.apache.hadoop.hdfs.TestDFSClientFailover
org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
org.apache.hadoop.tools.TestJMXGet
org.apache.hadoop.hdfs.TestDFSShell
org.apache.hadoop.hdfs.security.TestDelegationToken
org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
org.apache.hadoop.fs.permission.TestStickyBit
org.apache.hadoop.hdfs.TestFileConcurrentReader
org.apache.hadoop.hdfs.server.datanode.TestCachingStrategy
org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
org.apache.hadoop.fs.TestFcHdfsCreateMkdir
org.apache.hadoop.hdfs.TestCrcCorruption
org.apache.hadoop.hdfs.TestAppendDifferentChecksum
org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
org.apache.hadoop.fs.viewfs.TestViewFileSystemHdfs
org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser
org.apache.hadoop.hdfs.server.namenode.TestSequentialBlockId
org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
org.apache.hadoop.hdfs.TestDFSPermission
org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
org.apache.hadoop.hdfs.TestListFilesInFileContext
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
org.apache.hadoop.fs.viewfs.TestViewFsDefaultValue
org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics
org.apache.hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot
org.apache.hadoop.hdfs.TestParallelShortCircuitReadNoChecksum
org.apache.hadoop.hdfs.TestDFSRemove
org.apache.hadoop.hdfs.TestRestartDFS
org.apache.hadoop.hdfs.TestFSOutputSummer
org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
org.apache.hadoop.hdfs.TestHDFSTrash
org.apache.hadoop.hdfs.TestDFSRollback
org.apache.hadoop.hdfs.server.namenode.TestFSDirectory
org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
org.apache.hadoop.hdfs.TestClientReportBadBlock
org.apache.hadoop.hdfs.TestDecommission
org.apache.hadoop.hdfs.TestSmallBlock
