Auto-Re: [jira] [Commented] (HDFS-8659) Block scanner INFO message is spamming logs
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8669) Erasure Coding: handle missing internal block locations in DFSStripedInputStream
[ https://issues.apache.org/jira/browse/HDFS-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602593#comment-14602593 ] Walter Su commented on HDFS-8669: - Patch looks good. It passed simple local test. some minor issues: 1. should be {{>}} {code} if (alignedStripe.missingChunksNum >= parityBlkNum) { //checkMissingBlocks() {code} 2.should check missing blocks after first round of read. And you add checkMissingBlocks() at the end of readDataForDecoding(), how about add it at the end of readParityChunks(int) either? {code} void readStripe() throws IOException { for (int i = 0; i < dataBlkNum; i++) {...} + checkMissingBlocks(); // There are missing block locations at this stage. Thus we need to read {code} > Erasure Coding: handle missing internal block locations in > DFSStripedInputStream > > > Key: HDFS-8669 > URL: https://issues.apache.org/jira/browse/HDFS-8669 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Jing Zhao >Assignee: Jing Zhao > Attachments: HDFS-8669.000.patch > > > Currently DFSStripedInputStream assumes we always have complete internal > block location information, i.e., we can always get all the DataNodes for a > striped block group. In a lot of scenarios the client cannot get complete > block location info, e.g., some internal blocks are missing and the NameNode > has not finished the recovery yet. We should add functionality to handle > missing block locations in DFSStripedInputStream. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8669) Erasure Coding: handle missing internal block locations in DFSStripedInputStream
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8653) Code cleanup for DatanodeManager, DatanodeDescriptor and DatanodeStorageInfo
[ https://issues.apache.org/jira/browse/HDFS-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602598#comment-14602598 ] Walter Su commented on HDFS-8653: - Patch LGTM,+1 ( non-binding). Hi, [~andrew.wang]! Could you take a look at this? > Code cleanup for DatanodeManager, DatanodeDescriptor and DatanodeStorageInfo > > > Key: HDFS-8653 > URL: https://issues.apache.org/jira/browse/HDFS-8653 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 2.7.0 >Reporter: Zhe Zhang >Assignee: Zhe Zhang > Attachments: HDFS-8653.00.patch > > > While updating the {{blockmanagement}} module to distribute erasure coding > recovery work to Datanode, the HDFS-7285 branch also did some code cleanup > that should be merged into trunk independently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8653) Code cleanup for DatanodeManager, DatanodeDescriptor and DatanodeStorageInfo
您的邮件已收到!谢谢!
[jira] [Updated] (HDFS-8672) Erasure Coding: Add EC-related Metrics to NN (seperate striped blocks count from UnderReplicatedBlocks count)
[ https://issues.apache.org/jira/browse/HDFS-8672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8672: Summary: Erasure Coding: Add EC-related Metrics to NN (seperate striped blocks count from UnderReplicatedBlocks count) (was: Erasure Coding: Add EC-related Metrics on NN (seperate striped blocks count from UnderReplicatedBlocks count)) > Erasure Coding: Add EC-related Metrics to NN (seperate striped blocks count > from UnderReplicatedBlocks count) > - > > Key: HDFS-8672 > URL: https://issues.apache.org/jira/browse/HDFS-8672 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Walter Su >Assignee: Walter Su >Priority: Minor > > 1. {{MissingBlocks}} metric is updated in HDFS-8461 so it includes striped > blocks. > 2. {{CorruptBlocks}} metric is updated in HDFS-8619 so it includes striped > blocks. > 3. {{UnderReplicatedBlocks}} and {{PendingReplicationBlocks}} includes > striped blocks (HDFS-7912). > This jira aims to seperate striped blocks count from > {{UnderReplicatedBlocks}} count. > EC file recovery need coding. It's more expensive than block duplication. > It's necessary to seperate striped blocks count from UnderReplicatedBlocks > count. So user can know what's going on. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Updated] (HDFS-8672) Erasure Coding: Add EC-related Metrics to NN (seperate striped blocks count from UnderReplicatedBlocks count)
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8598) Add and optimize for get LocatedFileStatus in DFSClient
[ https://issues.apache.org/jira/browse/HDFS-8598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602633#comment-14602633 ] Hadoop QA commented on HDFS-8598: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 19m 22s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 3 new or modified test files. | | {color:green}+1{color} | javac | 7m 27s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 2m 12s | The applied patch generated 1 new checkstyle issues (total was 142, now 142). | | {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 36s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 5m 9s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | common tests | 22m 22s | Tests passed in hadoop-common. | | {color:red}-1{color} | hdfs tests | 162m 55s | Tests failed in hadoop-hdfs. | | | | 232m 4s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.server.namenode.TestFsck | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12742042/HDFS-8598.002.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 8ef07f7 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/11495/artifact/patchprocess/diffcheckstylehadoop-common.txt | | hadoop-common test log | https://builds.apache.org/job/PreCommit-HDFS-Build/11495/artifact/patchprocess/testrun_hadoop-common.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/11495/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/11495/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/11495/console | This message was automatically generated. > Add and optimize for get LocatedFileStatus in DFSClient > > > Key: HDFS-8598 > URL: https://issues.apache.org/jira/browse/HDFS-8598 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Yong Zhang >Assignee: Yong Zhang > Attachments: HDFS-8598.001.patch, HDFS-8598.002.patch > > > If we want to get all files block locations in one directory, we have to call > getFileBlockLocations for each file, it will take long time because of too > many request. > LocatedFileStatus has block location, but we can find it also call > getFileBlockLocations for each file in DFSClient. this jira is trying to > optimize with only one RPC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8598) Add and optimize for get LocatedFileStatus in DFSClient
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8659) Block scanner INFO message is spamming logs
[ https://issues.apache.org/jira/browse/HDFS-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602717#comment-14602717 ] Hadoop QA commented on HDFS-8659: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 18m 26s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | | {color:green}+1{color} | javac | 7m 53s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 0s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | checkstyle | 2m 26s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 38s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 3m 24s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 21s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 162m 30s | Tests failed in hadoop-hdfs. | | | | 210m 39s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.server.namenode.TestCacheDirectives | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12742066/HDFS-8659.002.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 8ef07f7 | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/11496/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/11496/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/11496/console | This message was automatically generated. > Block scanner INFO message is spamming logs > --- > > Key: HDFS-8659 > URL: https://issues.apache.org/jira/browse/HDFS-8659 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.1 >Reporter: Yongjun Zhang >Assignee: Yongjun Zhang > Labels: supportability > Attachments: HDFS-8659.001.patch, HDFS-8659.002.patch > > > We are seeing the following message spam the DN log: > {quote} > 2015-06-16 08:51:10,566 INFO > org.apache.hadoop.hdfs.server.datanode.BlockScanner: Not scanning suspicious > block BP-943360218-10.106.148.16-1416571803827:blk_1083076388_9372245 on > DS-2ec89056-afb0-459e-b4e0-ac5eaececda3, because the block scanner is > disabled. > {quote} > Create this jira to change this and other relevant messages to debug level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8659) Block scanner INFO message is spamming logs
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
[ https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602749#comment-14602749 ] Hudson commented on HDFS-8546: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #240 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/240/]) HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 1403b84b122fb76ef2b085a728b5402c32499c1f) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java > Use try with resources in DataStorage and Storage > - > > Key: HDFS-8546 > URL: https://issues.apache.org/jira/browse/HDFS-8546 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Minor > Fix For: 2.8.0 > > Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, > HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, > HDFS-8546.006.patch > > > We have some old-style try/finally to close files in DataStorage and Storage, > let's update them. > Also a few small cleanups: > * Actually check that tryLock returns a FileLock in isPreUpgradableLayout > * Remove unused parameter from writeProperties > * Add braces for one-line if statements per coding style -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
[ https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602746#comment-14602746 ] Hudson commented on HDFS-8462: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #240 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/240/]) HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. Contributed by Jagadesh Kiran N. (aajisaka: rev bc433908d35758ff0a7225cd6f5662959ef4d294) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java > Implement GETXATTRS and LISTXATTRS operations for WebImageViewer > > > Key: HDFS-8462 > URL: https://issues.apache.org/jira/browse/HDFS-8462 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Akira AJISAKA >Assignee: Jagadesh Kiran N > Fix For: 2.8.0 > > Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, > HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, > HDFS-8462-05.patch, HDFS-8462-06.patch > > > In Hadoop 2.7.0, WebImageViewer supports the following operations: > * {{GETFILESTATUS}} > * {{LISTSTATUS}} > * {{GETACLSTATUS}} > I'm thinking it would be better for administrators if {{GETXATTRS}} and > {{LISTXATTRS}} are supported. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
[ https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602748#comment-14602748 ] Hudson commented on HDFS-8640: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #240 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/240/]) HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java > Make reserved RBW space visible through JMX > --- > > Key: HDFS-8640 > URL: https://issues.apache.org/jira/browse/HDFS-8640 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: kanaka kumar avvaru >Assignee: kanaka kumar avvaru > Fix For: 2.8.0 > > Attachments: HDFS-8640-00.patch > > > At present there is no way to trace the reserved space for RBW in DataNode > to identify any leaks like HDFS-8626 > Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX > bean can be referred to know the present reserved space -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
[ https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602745#comment-14602745 ] Hudson commented on HDFS-8665: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #240 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/240/]) HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev ff0e5e572f5dcf7b49381cbe901360f6e171d423) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix replication check in DFSTestUtils#waitForReplication > > > Key: HDFS-8665 > URL: https://issues.apache.org/jira/browse/HDFS-8665 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.8.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: hdfs-8665.001.patch > > > The check looks at the repl factor set on the file rather than reported # of > replica locations. Let's do the latter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
[ https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602763#comment-14602763 ] Hudson commented on HDFS-8640: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #970 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/970/]) HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Make reserved RBW space visible through JMX > --- > > Key: HDFS-8640 > URL: https://issues.apache.org/jira/browse/HDFS-8640 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: kanaka kumar avvaru >Assignee: kanaka kumar avvaru > Fix For: 2.8.0 > > Attachments: HDFS-8640-00.patch > > > At present there is no way to trace the reserved space for RBW in DataNode > to identify any leaks like HDFS-8626 > Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX > bean can be referred to know the present reserved space -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
[ https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602760#comment-14602760 ] Hudson commented on HDFS-8665: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #970 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/970/]) HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev ff0e5e572f5dcf7b49381cbe901360f6e171d423) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix replication check in DFSTestUtils#waitForReplication > > > Key: HDFS-8665 > URL: https://issues.apache.org/jira/browse/HDFS-8665 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.8.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: hdfs-8665.001.patch > > > The check looks at the repl factor set on the file rather than reported # of > replica locations. Let's do the latter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
[ https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602764#comment-14602764 ] Hudson commented on HDFS-8546: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #970 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/970/]) HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 1403b84b122fb76ef2b085a728b5402c32499c1f) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Use try with resources in DataStorage and Storage > - > > Key: HDFS-8546 > URL: https://issues.apache.org/jira/browse/HDFS-8546 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Minor > Fix For: 2.8.0 > > Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, > HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, > HDFS-8546.006.patch > > > We have some old-style try/finally to close files in DataStorage and Storage, > let's update them. > Also a few small cleanups: > * Actually check that tryLock returns a FileLock in isPreUpgradableLayout > * Remove unused parameter from writeProperties > * Add braces for one-line if statements per coding style -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
[ https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602761#comment-14602761 ] Hudson commented on HDFS-8462: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #970 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/970/]) HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. Contributed by Jagadesh Kiran N. (aajisaka: rev bc433908d35758ff0a7225cd6f5662959ef4d294) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Implement GETXATTRS and LISTXATTRS operations for WebImageViewer > > > Key: HDFS-8462 > URL: https://issues.apache.org/jira/browse/HDFS-8462 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Akira AJISAKA >Assignee: Jagadesh Kiran N > Fix For: 2.8.0 > > Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, > HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, > HDFS-8462-05.patch, HDFS-8462-06.patch > > > In Hadoop 2.7.0, WebImageViewer supports the following operations: > * {{GETFILESTATUS}} > * {{LISTSTATUS}} > * {{GETACLSTATUS}} > I'm thinking it would be better for administrators if {{GETXATTRS}} and > {{LISTXATTRS}} are supported. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
您的邮件已收到!谢谢!
[jira] [Updated] (HDFS-8627) NPE thrown if unable to fetch token from Namenode
[ https://issues.apache.org/jira/browse/HDFS-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated HDFS-8627: - Status: Patch Available (was: Open) > NPE thrown if unable to fetch token from Namenode > - > > Key: HDFS-8627 > URL: https://issues.apache.org/jira/browse/HDFS-8627 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-8627.1.patch > > > DelegationTokenFetcher#saveDelegationToken > Missed to check if token is null. > {code} > Token token = fs.getDelegationToken(renewer); > Credentials cred = new Credentials(); > cred.addToken(token.getKind(), token); > {code} > {noformat} > XX:~/hadoop/namenode/bin> ./hdfs fetchdt --renewer Rex > /home/REX/file1 > Exception in thread "main" java.lang.NullPointerException > at > org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.saveDelegationToken(DelegationTokenFetcher.java:181) > at > org.apache.hadoop.hdfs.tools.DelegationTokenFetcher$1.run(DelegationTokenFetcher.java:126) > at > java.security.AccessController.doPrivileged(AccessController.java:314) > at javax.security.auth.Subject.doAs(Subject.java:572) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1666) > at > org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.main(DelegationTokenFetcher.java:114) > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8627) NPE thrown if unable to fetch token from Namenode
[ https://issues.apache.org/jira/browse/HDFS-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated HDFS-8627: - Attachment: HDFS-8627.1.patch Attached an initial patch. Please review. > NPE thrown if unable to fetch token from Namenode > - > > Key: HDFS-8627 > URL: https://issues.apache.org/jira/browse/HDFS-8627 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-8627.1.patch > > > DelegationTokenFetcher#saveDelegationToken > Missed to check if token is null. > {code} > Token token = fs.getDelegationToken(renewer); > Credentials cred = new Credentials(); > cred.addToken(token.getKind(), token); > {code} > {noformat} > XX:~/hadoop/namenode/bin> ./hdfs fetchdt --renewer Rex > /home/REX/file1 > Exception in thread "main" java.lang.NullPointerException > at > org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.saveDelegationToken(DelegationTokenFetcher.java:181) > at > org.apache.hadoop.hdfs.tools.DelegationTokenFetcher$1.run(DelegationTokenFetcher.java:126) > at > java.security.AccessController.doPrivileged(AccessController.java:314) > at javax.security.auth.Subject.doAs(Subject.java:572) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1666) > at > org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.main(DelegationTokenFetcher.java:114) > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8628) Update missing command option for fetchdt
[ https://issues.apache.org/jira/browse/HDFS-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated HDFS-8628: - Attachment: HDFS-8628.1.patch Attached an initial patch Please review. > Update missing command option for fetchdt > - > > Key: HDFS-8628 > URL: https://issues.apache.org/jira/browse/HDFS-8628 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-8628.1.patch > > > Update missing command option for fetchdt > *Expected:* > {noformat} > fetchdt > Options: > --webservice Url to contact NN on (starts with http:// or https://) > --renewer Name of the delegation token renewer > --cancelCancel the delegation token > --renew Renew the delegation token. Delegation token must have > been fetched using the --renewer option. > --print Print the delegation token > {noformat} > *Actual:* > {noformat} > Usage: hdfs fetchdt [--webservice ] > COMMAND_OPTIONDescription > --webservice https_address use http protocol instead of RPC > fileName File name to store > the token into. > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8628) Update missing command option for fetchdt
[ https://issues.apache.org/jira/browse/HDFS-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated HDFS-8628: - Status: Patch Available (was: Open) > Update missing command option for fetchdt > - > > Key: HDFS-8628 > URL: https://issues.apache.org/jira/browse/HDFS-8628 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-8628.1.patch > > > Update missing command option for fetchdt > *Expected:* > {noformat} > fetchdt > Options: > --webservice Url to contact NN on (starts with http:// or https://) > --renewer Name of the delegation token renewer > --cancelCancel the delegation token > --renew Renew the delegation token. Delegation token must have > been fetched using the --renewer option. > --print Print the delegation token > {noformat} > *Actual:* > {noformat} > Usage: hdfs fetchdt [--webservice ] > COMMAND_OPTIONDescription > --webservice https_address use http protocol instead of RPC > fileName File name to store > the token into. > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Updated] (HDFS-8627) NPE thrown if unable to fetch token from Namenode
您的邮件已收到!谢谢!
Auto-Re: [jira] [Updated] (HDFS-8627) NPE thrown if unable to fetch token from Namenode
您的邮件已收到!谢谢!
Auto-Re: [jira] [Updated] (HDFS-8628) Update missing command option for fetchdt
您的邮件已收到!谢谢!
[jira] [Updated] (HDFS-8579) Update HDFS usage with missing options
[ https://issues.apache.org/jira/browse/HDFS-8579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated HDFS-8579: - Status: Patch Available (was: Open) > Update HDFS usage with missing options > -- > > Key: HDFS-8579 > URL: https://issues.apache.org/jira/browse/HDFS-8579 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina >Priority: Minor > Attachments: HDFS-8579-branch-2.7-1.patch, HDFS-8579-trunk-1.patch > > > Update hdfs usage with missing options (fetchdt and debug) > {noformat} > 1 > ./hdfs fetchdt > fetchdt > Options: > --webservice Url to contact NN on > --renewer Name of the delegation token renewer > --cancelCancel the delegation token > --renew Renew the delegation token. Delegation token must have > been fetched using the --renewer option. > --print Print the delegation token > 2 > ./hdfs debug > Usage: hdfs debug [arguments] > verify [-meta ] [-block ] > recoverLease [-path ] [-retries ] > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Updated] (HDFS-8628) Update missing command option for fetchdt
您的邮件已收到!谢谢!
[jira] [Updated] (HDFS-8579) Update HDFS usage with missing options
[ https://issues.apache.org/jira/browse/HDFS-8579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated HDFS-8579: - Attachment: HDFS-8579-trunk-1.patch HDFS-8579-branch-2.7-1.patch Please review the patch. > Update HDFS usage with missing options > -- > > Key: HDFS-8579 > URL: https://issues.apache.org/jira/browse/HDFS-8579 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina >Priority: Minor > Attachments: HDFS-8579-branch-2.7-1.patch, HDFS-8579-trunk-1.patch > > > Update hdfs usage with missing options (fetchdt and debug) > {noformat} > 1 > ./hdfs fetchdt > fetchdt > Options: > --webservice Url to contact NN on > --renewer Name of the delegation token renewer > --cancelCancel the delegation token > --renew Renew the delegation token. Delegation token must have > been fetched using the --renewer option. > --print Print the delegation token > 2 > ./hdfs debug > Usage: hdfs debug [arguments] > verify [-meta ] [-block ] > recoverLease [-path ] [-retries ] > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Updated] (HDFS-8579) Update HDFS usage with missing options
您的邮件已收到!谢谢!
Auto-Re: [jira] [Updated] (HDFS-8579) Update HDFS usage with missing options
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8628) Update missing command option for fetchdt
[ https://issues.apache.org/jira/browse/HDFS-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602841#comment-14602841 ] Hadoop QA commented on HDFS-8628: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 3m 3s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | release audit | 0m 20s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | site | 1m 15s | Site compilation is broken. | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | | | 4m 41s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12742111/HDFS-8628.1.patch | | Optional Tests | site | | git revision | trunk / 8ef07f7 | | site | https://builds.apache.org/job/PreCommit-HDFS-Build/11497/artifact/patchprocess/patchSiteWarnings.txt | | Java | 1.7.0_55 | | uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/11497/console | This message was automatically generated. > Update missing command option for fetchdt > - > > Key: HDFS-8628 > URL: https://issues.apache.org/jira/browse/HDFS-8628 > Project: Hadoop HDFS > Issue Type: Bug > Components: documentation >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-8628.1.patch > > > Update missing command option for fetchdt > *Expected:* > {noformat} > fetchdt > Options: > --webservice Url to contact NN on (starts with http:// or https://) > --renewer Name of the delegation token renewer > --cancelCancel the delegation token > --renew Renew the delegation token. Delegation token must have > been fetched using the --renewer option. > --print Print the delegation token > {noformat} > *Actual:* > {noformat} > Usage: hdfs fetchdt [--webservice ] > COMMAND_OPTIONDescription > --webservice https_address use http protocol instead of RPC > fileName File name to store > the token into. > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8628) Update missing command option for fetchdt
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX
[ https://issues.apache.org/jira/browse/HDFS-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602885#comment-14602885 ] Gabor Liptak commented on HDFS-8670: I also made a patch before seeing [~andreina]'s message. > Better to exclude decommissioned nodes for namenode NodeUsage JMX > - > > Key: HDFS-8670 > URL: https://issues.apache.org/jira/browse/HDFS-8670 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ming Ma >Assignee: J.Andreina > > The namenode NodeUsage JMX has Max, Median, Min and Standard Deviation of > DataNodes usage, it currently includes decommissioned nodes for the > calculation. However, given balancer doesn't work on decommissioned nodes and > sometimes we could have nodes stay in decommissioned states for a long time; > it might be better to exclude decommissioned nodes for the metrics > calculation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
[ https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602938#comment-14602938 ] Hudson commented on HDFS-8665: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #229 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/229/]) HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev ff0e5e572f5dcf7b49381cbe901360f6e171d423) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix replication check in DFSTestUtils#waitForReplication > > > Key: HDFS-8665 > URL: https://issues.apache.org/jira/browse/HDFS-8665 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.8.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: hdfs-8665.001.patch > > > The check looks at the repl factor set on the file rather than reported # of > replica locations. Let's do the latter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
[ https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602940#comment-14602940 ] Hudson commented on HDFS-8640: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #229 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/229/]) HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java > Make reserved RBW space visible through JMX > --- > > Key: HDFS-8640 > URL: https://issues.apache.org/jira/browse/HDFS-8640 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: kanaka kumar avvaru >Assignee: kanaka kumar avvaru > Fix For: 2.8.0 > > Attachments: HDFS-8640-00.patch > > > At present there is no way to trace the reserved space for RBW in DataNode > to identify any leaks like HDFS-8626 > Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX > bean can be referred to know the present reserved space -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
[ https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602941#comment-14602941 ] Hudson commented on HDFS-8546: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #229 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/229/]) HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 1403b84b122fb76ef2b085a728b5402c32499c1f) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java > Use try with resources in DataStorage and Storage > - > > Key: HDFS-8546 > URL: https://issues.apache.org/jira/browse/HDFS-8546 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Minor > Fix For: 2.8.0 > > Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, > HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, > HDFS-8546.006.patch > > > We have some old-style try/finally to close files in DataStorage and Storage, > let's update them. > Also a few small cleanups: > * Actually check that tryLock returns a FileLock in isPreUpgradableLayout > * Remove unused parameter from writeProperties > * Add braces for one-line if statements per coding style -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
[ https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602939#comment-14602939 ] Hudson commented on HDFS-8462: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #229 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/229/]) HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. Contributed by Jagadesh Kiran N. (aajisaka: rev bc433908d35758ff0a7225cd6f5662959ef4d294) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java > Implement GETXATTRS and LISTXATTRS operations for WebImageViewer > > > Key: HDFS-8462 > URL: https://issues.apache.org/jira/browse/HDFS-8462 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Akira AJISAKA >Assignee: Jagadesh Kiran N > Fix For: 2.8.0 > > Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, > HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, > HDFS-8462-05.patch, HDFS-8462-06.patch > > > In Hadoop 2.7.0, WebImageViewer supports the following operations: > * {{GETFILESTATUS}} > * {{LISTSTATUS}} > * {{GETACLSTATUS}} > I'm thinking it would be better for administrators if {{GETXATTRS}} and > {{LISTXATTRS}} are supported. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
[ https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602952#comment-14602952 ] Hudson commented on HDFS-8546: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #238 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/238/]) HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 1403b84b122fb76ef2b085a728b5402c32499c1f) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java > Use try with resources in DataStorage and Storage > - > > Key: HDFS-8546 > URL: https://issues.apache.org/jira/browse/HDFS-8546 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Minor > Fix For: 2.8.0 > > Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, > HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, > HDFS-8546.006.patch > > > We have some old-style try/finally to close files in DataStorage and Storage, > let's update them. > Also a few small cleanups: > * Actually check that tryLock returns a FileLock in isPreUpgradableLayout > * Remove unused parameter from writeProperties > * Add braces for one-line if statements per coding style -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
[ https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602950#comment-14602950 ] Hudson commented on HDFS-8462: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #238 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/238/]) HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. Contributed by Jagadesh Kiran N. (aajisaka: rev bc433908d35758ff0a7225cd6f5662959ef4d294) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md > Implement GETXATTRS and LISTXATTRS operations for WebImageViewer > > > Key: HDFS-8462 > URL: https://issues.apache.org/jira/browse/HDFS-8462 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Akira AJISAKA >Assignee: Jagadesh Kiran N > Fix For: 2.8.0 > > Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, > HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, > HDFS-8462-05.patch, HDFS-8462-06.patch > > > In Hadoop 2.7.0, WebImageViewer supports the following operations: > * {{GETFILESTATUS}} > * {{LISTSTATUS}} > * {{GETACLSTATUS}} > I'm thinking it would be better for administrators if {{GETXATTRS}} and > {{LISTXATTRS}} are supported. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
[ https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602951#comment-14602951 ] Hudson commented on HDFS-8640: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #238 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/238/]) HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Make reserved RBW space visible through JMX > --- > > Key: HDFS-8640 > URL: https://issues.apache.org/jira/browse/HDFS-8640 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: kanaka kumar avvaru >Assignee: kanaka kumar avvaru > Fix For: 2.8.0 > > Attachments: HDFS-8640-00.patch > > > At present there is no way to trace the reserved space for RBW in DataNode > to identify any leaks like HDFS-8626 > Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX > bean can be referred to know the present reserved space -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
[ https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602962#comment-14602962 ] Hudson commented on HDFS-8665: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2186 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2186/]) HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev ff0e5e572f5dcf7b49381cbe901360f6e171d423) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix replication check in DFSTestUtils#waitForReplication > > > Key: HDFS-8665 > URL: https://issues.apache.org/jira/browse/HDFS-8665 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.8.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: hdfs-8665.001.patch > > > The check looks at the repl factor set on the file rather than reported # of > replica locations. Let's do the latter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
[ https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602957#comment-14602957 ] Hudson commented on HDFS-8462: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2168 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2168/]) HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. Contributed by Jagadesh Kiran N. (aajisaka: rev bc433908d35758ff0a7225cd6f5662959ef4d294) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md > Implement GETXATTRS and LISTXATTRS operations for WebImageViewer > > > Key: HDFS-8462 > URL: https://issues.apache.org/jira/browse/HDFS-8462 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Akira AJISAKA >Assignee: Jagadesh Kiran N > Fix For: 2.8.0 > > Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, > HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, > HDFS-8462-05.patch, HDFS-8462-06.patch > > > In Hadoop 2.7.0, WebImageViewer supports the following operations: > * {{GETFILESTATUS}} > * {{LISTSTATUS}} > * {{GETACLSTATUS}} > I'm thinking it would be better for administrators if {{GETXATTRS}} and > {{LISTXATTRS}} are supported. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
[ https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602949#comment-14602949 ] Hudson commented on HDFS-8665: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #238 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/238/]) HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev ff0e5e572f5dcf7b49381cbe901360f6e171d423) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java > Fix replication check in DFSTestUtils#waitForReplication > > > Key: HDFS-8665 > URL: https://issues.apache.org/jira/browse/HDFS-8665 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.8.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: hdfs-8665.001.patch > > > The check looks at the repl factor set on the file rather than reported # of > replica locations. Let's do the latter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
[ https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602959#comment-14602959 ] Hudson commented on HDFS-8546: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2168 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2168/]) HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 1403b84b122fb76ef2b085a728b5402c32499c1f) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Use try with resources in DataStorage and Storage > - > > Key: HDFS-8546 > URL: https://issues.apache.org/jira/browse/HDFS-8546 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Minor > Fix For: 2.8.0 > > Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, > HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, > HDFS-8546.006.patch > > > We have some old-style try/finally to close files in DataStorage and Storage, > let's update them. > Also a few small cleanups: > * Actually check that tryLock returns a FileLock in isPreUpgradableLayout > * Remove unused parameter from writeProperties > * Add braces for one-line if statements per coding style -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
[ https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602958#comment-14602958 ] Hudson commented on HDFS-8640: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2168 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2168/]) HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java > Make reserved RBW space visible through JMX > --- > > Key: HDFS-8640 > URL: https://issues.apache.org/jira/browse/HDFS-8640 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: kanaka kumar avvaru >Assignee: kanaka kumar avvaru > Fix For: 2.8.0 > > Attachments: HDFS-8640-00.patch > > > At present there is no way to trace the reserved space for RBW in DataNode > to identify any leaks like HDFS-8626 > Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX > bean can be referred to know the present reserved space -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX
[ https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602964#comment-14602964 ] Hudson commented on HDFS-8640: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2186 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2186/]) HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java > Make reserved RBW space visible through JMX > --- > > Key: HDFS-8640 > URL: https://issues.apache.org/jira/browse/HDFS-8640 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: kanaka kumar avvaru >Assignee: kanaka kumar avvaru > Fix For: 2.8.0 > > Attachments: HDFS-8640-00.patch > > > At present there is no way to trace the reserved space for RBW in DataNode > to identify any leaks like HDFS-8626 > Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX > bean can be referred to know the present reserved space -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
[ https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602956#comment-14602956 ] Hudson commented on HDFS-8665: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2168 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2168/]) HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev ff0e5e572f5dcf7b49381cbe901360f6e171d423) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java > Fix replication check in DFSTestUtils#waitForReplication > > > Key: HDFS-8665 > URL: https://issues.apache.org/jira/browse/HDFS-8665 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.8.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: hdfs-8665.001.patch > > > The check looks at the repl factor set on the file rather than reported # of > replica locations. Let's do the latter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage
[ https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602965#comment-14602965 ] Hudson commented on HDFS-8546: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2186 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2186/]) HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 1403b84b122fb76ef2b085a728b5402c32499c1f) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java > Use try with resources in DataStorage and Storage > - > > Key: HDFS-8546 > URL: https://issues.apache.org/jira/browse/HDFS-8546 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Minor > Fix For: 2.8.0 > > Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, > HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, > HDFS-8546.006.patch > > > We have some old-style try/finally to close files in DataStorage and Storage, > let's update them. > Also a few small cleanups: > * Actually check that tryLock returns a FileLock in isPreUpgradableLayout > * Remove unused parameter from writeProperties > * Add braces for one-line if statements per coding style -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
[ https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602963#comment-14602963 ] Hudson commented on HDFS-8462: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2186 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2186/]) HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. Contributed by Jagadesh Kiran N. (aajisaka: rev bc433908d35758ff0a7225cd6f5662959ef4d294) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java * hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Implement GETXATTRS and LISTXATTRS operations for WebImageViewer > > > Key: HDFS-8462 > URL: https://issues.apache.org/jira/browse/HDFS-8462 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Akira AJISAKA >Assignee: Jagadesh Kiran N > Fix For: 2.8.0 > > Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, > HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, > HDFS-8462-05.patch, HDFS-8462-06.patch > > > In Hadoop 2.7.0, WebImageViewer supports the following operations: > * {{GETFILESTATUS}} > * {{LISTSTATUS}} > * {{GETACLSTATUS}} > I'm thinking it would be better for administrators if {{GETXATTRS}} and > {{LISTXATTRS}} are supported. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX
[ https://issues.apache.org/jira/browse/HDFS-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated HDFS-8670: - Status: Patch Available (was: Open) > Better to exclude decommissioned nodes for namenode NodeUsage JMX > - > > Key: HDFS-8670 > URL: https://issues.apache.org/jira/browse/HDFS-8670 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ming Ma >Assignee: J.Andreina > Attachments: HDFS-8670.1.patch > > > The namenode NodeUsage JMX has Max, Median, Min and Standard Deviation of > DataNodes usage, it currently includes decommissioned nodes for the > calculation. However, given balancer doesn't work on decommissioned nodes and > sometimes we could have nodes stay in decommissioned states for a long time; > it might be better to exclude decommissioned nodes for the metrics > calculation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX
[ https://issues.apache.org/jira/browse/HDFS-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated HDFS-8670: - Attachment: HDFS-8670.1.patch Attached code patch along with a testcase. Please review. > Better to exclude decommissioned nodes for namenode NodeUsage JMX > - > > Key: HDFS-8670 > URL: https://issues.apache.org/jira/browse/HDFS-8670 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ming Ma >Assignee: J.Andreina > Attachments: HDFS-8670.1.patch > > > The namenode NodeUsage JMX has Max, Median, Min and Standard Deviation of > DataNodes usage, it currently includes decommissioned nodes for the > calculation. However, given balancer doesn't work on decommissioned nodes and > sometimes we could have nodes stay in decommissioned states for a long time; > it might be better to exclude decommissioned nodes for the metrics > calculation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Updated] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX
您的邮件已收到!谢谢!
Auto-Re: [jira] [Updated] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8579) Update HDFS usage with missing options
[ https://issues.apache.org/jira/browse/HDFS-8579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603126#comment-14603126 ] Hadoop QA commented on HDFS-8579: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 15m 3s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | | {color:green}+1{color} | javac | 7m 40s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 46s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | shellcheck | 0m 6s | There were no new shellcheck (v0.3.3) issues. | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 32s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | native | 3m 17s | Pre-build of native portion | | {color:green}+1{color} | hdfs tests | 159m 26s | Tests passed in hadoop-hdfs. | | | | 197m 47s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12742113/HDFS-8579-trunk-1.patch | | Optional Tests | javadoc javac unit shellcheck | | git revision | trunk / 8ef07f7 | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/11498/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/11498/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/11498/console | This message was automatically generated. > Update HDFS usage with missing options > -- > > Key: HDFS-8579 > URL: https://issues.apache.org/jira/browse/HDFS-8579 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina >Priority: Minor > Attachments: HDFS-8579-branch-2.7-1.patch, HDFS-8579-trunk-1.patch > > > Update hdfs usage with missing options (fetchdt and debug) > {noformat} > 1 > ./hdfs fetchdt > fetchdt > Options: > --webservice Url to contact NN on > --renewer Name of the delegation token renewer > --cancelCancel the delegation token > --renew Renew the delegation token. Delegation token must have > been fetched using the --renewer option. > --print Print the delegation token > 2 > ./hdfs debug > Usage: hdfs debug [arguments] > verify [-meta ] [-block ] > recoverLease [-path ] [-retries ] > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8579) Update HDFS usage with missing options
您的邮件已收到!谢谢!
[jira] [Created] (HDFS-8674) Improve performance of postponed block scans
Daryn Sharp created HDFS-8674: - Summary: Improve performance of postponed block scans Key: HDFS-8674 URL: https://issues.apache.org/jira/browse/HDFS-8674 Project: Hadoop HDFS Issue Type: Improvement Components: HDFS Affects Versions: 2.6.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Priority: Critical When a standby goes active, it marks all nodes as "stale" which will cause block invalidations for over-replicated blocks to be queued until full block reports are received from the nodes with the block. The replication monitor scans the queue with O(N) runtime. It picks a random offset and iterates through the set to randomize blocks scanned. The result is devastating when a cluster loses multiple nodes during a rolling upgrade. Re-replication occurs, the nodes come back, the excess block invalidations are postponed. Rescanning just 2k blocks out of millions of postponed blocks may take multiple seconds. During the scan, the write lock is held which stalls all other processing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Created] (HDFS-8674) Improve performance of postponed block scans
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8656) Preserve compatibility of ClientProtocol#rollingUpgrade after finalization
[ https://issues.apache.org/jira/browse/HDFS-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603138#comment-14603138 ] Ming Ma commented on HDFS-8656: --- +1. Thanks Andrew. The unit test failures are unrelated. > Preserve compatibility of ClientProtocol#rollingUpgrade after finalization > -- > > Key: HDFS-8656 > URL: https://issues.apache.org/jira/browse/HDFS-8656 > Project: Hadoop HDFS > Issue Type: Bug > Components: rolling upgrades >Affects Versions: 2.8.0 >Reporter: Andrew Wang >Assignee: Andrew Wang >Priority: Critical > Attachments: hdfs-8656.001.patch, hdfs-8656.002.patch, > hdfs-8656.003.patch, hdfs-8656.004.patch > > > HDFS-7645 changed rollingUpgradeInfo to still return an RUInfo after > finalization, so the DNs can differentiate between rollback and a > finalization. However, this breaks compatibility for the user facing APIs, > which always expect a null after finalization. Let's fix this and edify it in > unit tests. > As an additional improvement, isFinalized and isStarted are part of the Java > API, but not in the JMX output of RollingUpgradeInfo. It'd be nice to expose > these booleans so JMX users don't need to do the != 0 check that possibly > exposes our implementation details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8656) Preserve compatibility of ClientProtocol#rollingUpgrade after finalization
您的邮件已收到!谢谢!
Auto-Re: [jira] [Commented] (HDFS-8627) NPE thrown if unable to fetch token from Namenode
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8627) NPE thrown if unable to fetch token from Namenode
[ https://issues.apache.org/jira/browse/HDFS-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603146#comment-14603146 ] Hadoop QA commented on HDFS-8627: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 17m 32s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 26s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 36s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | checkstyle | 2m 17s | There were no new checkstyle issues. | | {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 36s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 3m 15s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 14s | Pre-build of native portion | | {color:green}+1{color} | hdfs tests | 161m 42s | Tests passed in hadoop-hdfs. | | | | 207m 34s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12742110/HDFS-8627.1.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 8ef07f7 | | whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/11499/artifact/patchprocess/whitespace.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/11499/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/11499/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/11499/console | This message was automatically generated. > NPE thrown if unable to fetch token from Namenode > - > > Key: HDFS-8627 > URL: https://issues.apache.org/jira/browse/HDFS-8627 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-8627.1.patch > > > DelegationTokenFetcher#saveDelegationToken > Missed to check if token is null. > {code} > Token token = fs.getDelegationToken(renewer); > Credentials cred = new Credentials(); > cred.addToken(token.getKind(), token); > {code} > {noformat} > XX:~/hadoop/namenode/bin> ./hdfs fetchdt --renewer Rex > /home/REX/file1 > Exception in thread "main" java.lang.NullPointerException > at > org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.saveDelegationToken(DelegationTokenFetcher.java:181) > at > org.apache.hadoop.hdfs.tools.DelegationTokenFetcher$1.run(DelegationTokenFetcher.java:126) > at > java.security.AccessController.doPrivileged(AccessController.java:314) > at javax.security.auth.Subject.doAs(Subject.java:572) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1666) > at > org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.main(DelegationTokenFetcher.java:114) > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HDFS-8675) IBRs from dead DNs go into infinite loop
Daryn Sharp created HDFS-8675: - Summary: IBRs from dead DNs go into infinite loop Key: HDFS-8675 URL: https://issues.apache.org/jira/browse/HDFS-8675 Project: Hadoop HDFS Issue Type: Bug Components: datanode Affects Versions: 2.6.0 Reporter: Daryn Sharp If the DN sends an IBR after the NN declares it dead, the NN returns an IOE of unregistered or dead. The DN catches the IOE, ignores it, and infinitely loops spamming the NN with retries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Created] (HDFS-8675) IBRs from dead DNs go into infinite loop
您的邮件已收到!谢谢!
Auto-Re: [jira] [Created] (HDFS-8676) Delayed rolling upgrade finalization can cause heartbeat expiration
您的邮件已收到!谢谢!
[jira] [Created] (HDFS-8676) Delayed rolling upgrade finalization can cause heartbeat expiration
Kihwal Lee created HDFS-8676: Summary: Delayed rolling upgrade finalization can cause heartbeat expiration Key: HDFS-8676 URL: https://issues.apache.org/jira/browse/HDFS-8676 Project: Hadoop HDFS Issue Type: Bug Reporter: Kihwal Lee Priority: Critical In big busy clusters where the deletion rate is also high, a lot of blocks can pile up in the datanode trash directories until an upgrade is finalized. When it is finally finalized, the deletion of trash is done in the service actor thread's context synchronously. This blocks the heartbeat and can cause heartbeat expiration. We have seen a namenode losing hundreds of nodes after a delayed upgrade finalization. The deletion of trash directories should be made asynchronous. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8675) IBRs from dead DNs go into infinite loop
[ https://issues.apache.org/jira/browse/HDFS-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603154#comment-14603154 ] Daryn Sharp commented on HDFS-8675: --- The solution isn't as simple as replacing the IOE with UnregisteredNodeException. The DN will commit suicide. Handling the exception will require buffering the IBR (or any other calls this may affect), registering, resending the rejected message. Issue might be related to network issues or defects that stall BPOfferService like the synchronous clearing of the trash after RU finalize. > IBRs from dead DNs go into infinite loop > > > Key: HDFS-8675 > URL: https://issues.apache.org/jira/browse/HDFS-8675 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Affects Versions: 2.6.0 >Reporter: Daryn Sharp > > If the DN sends an IBR after the NN declares it dead, the NN returns an IOE > of unregistered or dead. The DN catches the IOE, ignores it, and infinitely > loops spamming the NN with retries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8675) IBRs from dead DNs go into infinite loop
您的邮件已收到!谢谢!
Auto-Re: [jira] [Updated] (HDFS-8673) HDFS reports file already exists if there is a file/dir name end with ._COPYING_
您的邮件已收到!谢谢!
[jira] [Updated] (HDFS-8673) HDFS reports file already exists if there is a file/dir name end with ._COPYING_
[ https://issues.apache.org/jira/browse/HDFS-8673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He updated HDFS-8673: -- Assignee: (was: Chen He) > HDFS reports file already exists if there is a file/dir name end with > ._COPYING_ > > > Key: HDFS-8673 > URL: https://issues.apache.org/jira/browse/HDFS-8673 > Project: Hadoop HDFS > Issue Type: Bug > Components: fs >Affects Versions: 2.7.0 >Reporter: Chen He > > Because CLI is using CommandWithDestination.java which add "._COPYING_" to > the tail of file name when it does the copy. It will cause problem if there > is a file/dir already called *._COPYING_ on HDFS. > For file: > -bash-4.1$ hadoop fs -put 5M /user/occ/ > -bash-4.1$ hadoop fs -mv /user/occ/5M /user/occ/5M._COPYING_ > -bash-4.1$ hadoop fs -ls /user/occ/ > Found 1 items > -rw-r--r-- 1 occ supergroup5242880 2015-06-26 05:16 > /user/occ/5M._COPYING_ > -bash-4.1$ hadoop fs -put 128K /user/occ/5M > -bash-4.1$ hadoop fs -ls /user/occ/ > Found 1 items > -rw-r--r-- 1 occ supergroup 131072 2015-06-26 05:19 /user/occ/5M > For dir: > -bash-4.1$ hadoop fs -mkdir /user/occ/5M._COPYING_ > -bash-4.1$ hadoop fs -ls /user/occ/ > Found 1 items > drwxr-xr-x - occ supergroup 0 2015-06-26 05:24 > /user/occ/5M._COPYING_ > -bash-4.1$ hadoop fs -put 128K /user/occ/5M > put: /user/occ/5M._COPYING_ already exists as a directory > -bash-4.1$ hadoop fs -ls /user/occ/ > (/user/occ/5M._COPYING_ is gone) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8661) DataNode should filter the set of NameSpaceInfos passed to Datasets
[ https://issues.apache.org/jira/browse/HDFS-8661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8661: Attachment: HDFS-8661-HDFS-7240.03.patch v3 patch fixes checkstyle issues. > DataNode should filter the set of NameSpaceInfos passed to Datasets > --- > > Key: HDFS-8661 > URL: https://issues.apache.org/jira/browse/HDFS-8661 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Affects Versions: HDFS-7240 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Attachments: HDFS-8661-HDFS-7240.01.patch, > HDFS-8661-HDFS-7240.02.patch, HDFS-8661-HDFS-7240.03.patch > > > {{DataNode#refreshVolumes}} passes the list of NamespaceInfos to each dataset > when adding new volumes. > This list should be filtered by the correct NodeType(s) for each dataset. > e.g. in a shared HDFS+Ozone cluster, FsDatasets would be notified of NN block > pools and Ozone datasets would be notified of Ozone block pool(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Updated] (HDFS-8661) DataNode should filter the set of NameSpaceInfos passed to Datasets
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8664) Allow wildcards in dfs.datanode.data.dir
[ https://issues.apache.org/jira/browse/HDFS-8664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603236#comment-14603236 ] Patrick White commented on HDFS-8664: - interesting, lemme go over those tests. > Allow wildcards in dfs.datanode.data.dir > > > Key: HDFS-8664 > URL: https://issues.apache.org/jira/browse/HDFS-8664 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, HDFS >Affects Versions: 3.0.0 >Reporter: Patrick White >Assignee: Patrick White > Attachments: HDFS-8664.001.patch > > > We have many disks per machine (12+) that don't always have the same > numbering when they come back from provisioning, but they're always in the > same tree following the same pattern. > It would greatly reduce our config complexity to be able to specify a > wildcard for all the data directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8664) Allow wildcards in dfs.datanode.data.dir
您的邮件已收到!谢谢!
[jira] [Updated] (HDFS-8651) Make hadoop-hdfs-project Native code -Wall-clean
[ https://issues.apache.org/jira/browse/HDFS-8651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8651: --- Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) Committed to 2.8. Thanks, Alan. > Make hadoop-hdfs-project Native code -Wall-clean > > > Key: HDFS-8651 > URL: https://issues.apache.org/jira/browse/HDFS-8651 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: native >Affects Versions: 2.7.0 >Reporter: Alan Burlison >Assignee: Alan Burlison > Fix For: 2.8.0 > > Attachments: HDFS-8651.001.patch > > > As we specify -Wall as a default compilation flag, it would be helpful if the > Native code was -Wall-clean -- This message was sent by Atlassian JIRA (v6.3.4#6332)