Auto-Re: [jira] [Commented] (HDFS-8659) Block scanner INFO message is spamming logs

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8669) Erasure Coding: handle missing internal block locations in DFSStripedInputStream

2015-06-26 Thread Walter Su (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602593#comment-14602593
 ] 

Walter Su commented on HDFS-8669:
-

Patch looks good. It passed simple local test. some minor issues:
1. should be {{>}}
{code}
if (alignedStripe.missingChunksNum >= parityBlkNum) { //checkMissingBlocks()
{code}
2.should check missing blocks after first round of read. And you add 
checkMissingBlocks() at the end of readDataForDecoding(), how about add it at 
the end of readParityChunks(int) either?
{code}
void readStripe() throws IOException {
  for (int i = 0; i < dataBlkNum; i++) {...}
+ checkMissingBlocks();
  // There are missing block locations at this stage. Thus we need to read
{code}

> Erasure Coding: handle missing internal block locations in 
> DFSStripedInputStream
> 
>
> Key: HDFS-8669
> URL: https://issues.apache.org/jira/browse/HDFS-8669
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Jing Zhao
>Assignee: Jing Zhao
> Attachments: HDFS-8669.000.patch
>
>
> Currently DFSStripedInputStream assumes we always have complete internal 
> block location information, i.e., we can always get all the DataNodes for a 
> striped block group. In a lot of scenarios the client cannot get complete 
> block location info, e.g., some internal blocks are missing and the NameNode 
> has not finished the recovery yet. We should add functionality to handle 
> missing block locations in DFSStripedInputStream.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8669) Erasure Coding: handle missing internal block locations in DFSStripedInputStream

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8653) Code cleanup for DatanodeManager, DatanodeDescriptor and DatanodeStorageInfo

2015-06-26 Thread Walter Su (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602598#comment-14602598
 ] 

Walter Su commented on HDFS-8653:
-

Patch LGTM,+1 ( non-binding). Hi, [~andrew.wang]! Could you take a look at this?

> Code cleanup for DatanodeManager, DatanodeDescriptor and DatanodeStorageInfo
> 
>
> Key: HDFS-8653
> URL: https://issues.apache.org/jira/browse/HDFS-8653
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
> Attachments: HDFS-8653.00.patch
>
>
> While updating the {{blockmanagement}} module to distribute erasure coding 
> recovery work to Datanode, the HDFS-7285 branch also did some code cleanup 
> that should be merged into trunk independently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8653) Code cleanup for DatanodeManager, DatanodeDescriptor and DatanodeStorageInfo

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Updated] (HDFS-8672) Erasure Coding: Add EC-related Metrics to NN (seperate striped blocks count from UnderReplicatedBlocks count)

2015-06-26 Thread Walter Su (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Walter Su updated HDFS-8672:

Summary: Erasure Coding: Add EC-related Metrics to NN (seperate striped 
blocks count from UnderReplicatedBlocks count)  (was: Erasure Coding: Add 
EC-related Metrics on NN (seperate striped blocks count from 
UnderReplicatedBlocks count))

> Erasure Coding: Add EC-related Metrics to NN (seperate striped blocks count 
> from UnderReplicatedBlocks count)
> -
>
> Key: HDFS-8672
> URL: https://issues.apache.org/jira/browse/HDFS-8672
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Walter Su
>Assignee: Walter Su
>Priority: Minor
>
> 1. {{MissingBlocks}} metric is updated in HDFS-8461 so it includes striped 
> blocks.
> 2. {{CorruptBlocks}} metric is updated in HDFS-8619 so it includes striped 
> blocks.
> 3. {{UnderReplicatedBlocks}} and {{PendingReplicationBlocks}} includes 
> striped blocks (HDFS-7912).
> This jira aims to seperate striped blocks count from 
> {{UnderReplicatedBlocks}} count.
> EC file recovery need coding. It's more expensive than block duplication.
> It's necessary to seperate striped blocks count from UnderReplicatedBlocks 
> count. So user can know what's going on.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Updated] (HDFS-8672) Erasure Coding: Add EC-related Metrics to NN (seperate striped blocks count from UnderReplicatedBlocks count)

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8598) Add and optimize for get LocatedFileStatus in DFSClient

2015-06-26 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602633#comment-14602633
 ] 

Hadoop QA commented on HDFS-8598:
-

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  19m 22s | Pre-patch trunk compilation is 
healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any 
@author tags. |
| {color:green}+1{color} | tests included |   0m  0s | The patch appears to 
include 3 new or modified test files. |
| {color:green}+1{color} | javac |   7m 27s | There were no new javac warning 
messages. |
| {color:green}+1{color} | javadoc |   9m 37s | There were no new javadoc 
warning messages. |
| {color:green}+1{color} | release audit |   0m 22s | The applied patch does 
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle |   2m 12s | The applied patch generated  1 
new checkstyle issues (total was 142, now 142). |
| {color:green}+1{color} | whitespace |   0m  1s | The patch has no lines that 
end in whitespace. |
| {color:green}+1{color} | install |   1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 33s | The patch built with 
eclipse:eclipse. |
| {color:green}+1{color} | findbugs |   5m  9s | The patch does not introduce 
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | common tests |  22m 22s | Tests passed in 
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 162m 55s | Tests failed in hadoop-hdfs. |
| | | 232m  4s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.namenode.TestFsck |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12742042/HDFS-8598.002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 8ef07f7 |
| checkstyle |  
https://builds.apache.org/job/PreCommit-HDFS-Build/11495/artifact/patchprocess/diffcheckstylehadoop-common.txt
 |
| hadoop-common test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11495/artifact/patchprocess/testrun_hadoop-common.txt
 |
| hadoop-hdfs test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11495/artifact/patchprocess/testrun_hadoop-hdfs.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11495/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11495/console |


This message was automatically generated.

> Add and optimize for get LocatedFileStatus  in DFSClient
> 
>
> Key: HDFS-8598
> URL: https://issues.apache.org/jira/browse/HDFS-8598
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Yong Zhang
>Assignee: Yong Zhang
> Attachments: HDFS-8598.001.patch, HDFS-8598.002.patch
>
>
> If we want to get all files block locations in one directory, we have to call 
> getFileBlockLocations for each file, it will take long time because of too 
> many request. 
> LocatedFileStatus has block location, but we can find it also call 
> getFileBlockLocations  for each file in DFSClient. this jira is trying to 
> optimize with only one RPC. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8598) Add and optimize for get LocatedFileStatus in DFSClient

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8659) Block scanner INFO message is spamming logs

2015-06-26 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602717#comment-14602717
 ] 

Hadoop QA commented on HDFS-8659:
-

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  18m 26s | Pre-patch trunk compilation is 
healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any 
@author tags. |
| {color:red}-1{color} | tests included |   0m  0s | The patch doesn't appear 
to include any new or modified tests.  Please justify why no new tests are 
needed for this patch. Also please list what manual steps were performed to 
verify this patch. |
| {color:green}+1{color} | javac |   7m 53s | There were no new javac warning 
messages. |
| {color:green}+1{color} | javadoc |  10m  0s | There were no new javadoc 
warning messages. |
| {color:green}+1{color} | release audit |   0m 24s | The applied patch does 
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle |   2m 26s | There were no new checkstyle 
issues. |
| {color:green}+1{color} | whitespace |   0m  0s | The patch has no lines that 
end in whitespace. |
| {color:green}+1{color} | install |   1m 38s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 34s | The patch built with 
eclipse:eclipse. |
| {color:green}+1{color} | findbugs |   3m 24s | The patch does not introduce 
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native |   3m 21s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 162m 30s | Tests failed in hadoop-hdfs. |
| | | 210m 39s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.namenode.TestCacheDirectives |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12742066/HDFS-8659.002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 8ef07f7 |
| hadoop-hdfs test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11496/artifact/patchprocess/testrun_hadoop-hdfs.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11496/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11496/console |


This message was automatically generated.

> Block scanner INFO message is spamming logs
> ---
>
> Key: HDFS-8659
> URL: https://issues.apache.org/jira/browse/HDFS-8659
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.7.1
>Reporter: Yongjun Zhang
>Assignee: Yongjun Zhang
>  Labels: supportability
> Attachments: HDFS-8659.001.patch, HDFS-8659.002.patch
>
>
> We are seeing the following message spam the DN log:
> {quote}
> 2015-06-16 08:51:10,566 INFO 
> org.apache.hadoop.hdfs.server.datanode.BlockScanner: Not scanning suspicious 
> block BP-943360218-10.106.148.16-1416571803827:blk_1083076388_9372245 on 
> DS-2ec89056-afb0-459e-b4e0-ac5eaececda3, because the block scanner is 
> disabled.
> {quote}
> Create this jira to change this and other relevant messages to debug level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8659) Block scanner INFO message is spamming logs

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602749#comment-14602749
 ] 

Hudson commented on HDFS-8546:
--

FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #240 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/240/])
HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 
1403b84b122fb76ef2b085a728b5402c32499c1f)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java


> Use try with resources in DataStorage and Storage
> -
>
> Key: HDFS-8546
> URL: https://issues.apache.org/jira/browse/HDFS-8546
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.7.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, 
> HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, 
> HDFS-8546.006.patch
>
>
> We have some old-style try/finally to close files in DataStorage and Storage, 
> let's update them.
> Also a few small cleanups:
> * Actually check that tryLock returns a FileLock in isPreUpgradableLayout
> * Remove unused parameter from writeProperties
> * Add braces for one-line if statements per coding style



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602746#comment-14602746
 ] 

Hudson commented on HDFS-8462:
--

FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #240 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/240/])
HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. 
Contributed by Jagadesh Kiran N. (aajisaka: rev 
bc433908d35758ff0a7225cd6f5662959ef4d294)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java


> Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
> 
>
> Key: HDFS-8462
> URL: https://issues.apache.org/jira/browse/HDFS-8462
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Akira AJISAKA
>Assignee: Jagadesh Kiran N
> Fix For: 2.8.0
>
> Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, 
> HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, 
> HDFS-8462-05.patch, HDFS-8462-06.patch
>
>
> In Hadoop 2.7.0, WebImageViewer supports the following operations:
> * {{GETFILESTATUS}}
> * {{LISTSTATUS}}
> * {{GETACLSTATUS}}
> I'm thinking it would be better for administrators if {{GETXATTRS}} and 
> {{LISTXATTRS}} are supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602748#comment-14602748
 ] 

Hudson commented on HDFS-8640:
--

FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #240 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/240/])
HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka 
kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java


> Make reserved RBW space visible through JMX
> ---
>
> Key: HDFS-8640
> URL: https://issues.apache.org/jira/browse/HDFS-8640
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: kanaka kumar avvaru
>Assignee: kanaka kumar avvaru
> Fix For: 2.8.0
>
> Attachments: HDFS-8640-00.patch
>
>
> At present there is no way to trace the reserved space for RBW  in DataNode 
> to identify any leaks like HDFS-8626
> Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX 
> bean can be referred to know the present reserved space



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602745#comment-14602745
 ] 

Hudson commented on HDFS-8665:
--

FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #240 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/240/])
HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev 
ff0e5e572f5dcf7b49381cbe901360f6e171d423)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Fix replication check in DFSTestUtils#waitForReplication
> 
>
> Key: HDFS-8665
> URL: https://issues.apache.org/jira/browse/HDFS-8665
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Trivial
> Fix For: 2.8.0
>
> Attachments: hdfs-8665.001.patch
>
>
> The check looks at the repl factor set on the file rather than reported # of 
> replica locations. Let's do the latter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602763#comment-14602763
 ] 

Hudson commented on HDFS-8640:
--

SUCCESS: Integrated in Hadoop-Yarn-trunk #970 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/970/])
HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka 
kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Make reserved RBW space visible through JMX
> ---
>
> Key: HDFS-8640
> URL: https://issues.apache.org/jira/browse/HDFS-8640
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: kanaka kumar avvaru
>Assignee: kanaka kumar avvaru
> Fix For: 2.8.0
>
> Attachments: HDFS-8640-00.patch
>
>
> At present there is no way to trace the reserved space for RBW  in DataNode 
> to identify any leaks like HDFS-8626
> Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX 
> bean can be referred to know the present reserved space



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602760#comment-14602760
 ] 

Hudson commented on HDFS-8665:
--

SUCCESS: Integrated in Hadoop-Yarn-trunk #970 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/970/])
HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev 
ff0e5e572f5dcf7b49381cbe901360f6e171d423)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Fix replication check in DFSTestUtils#waitForReplication
> 
>
> Key: HDFS-8665
> URL: https://issues.apache.org/jira/browse/HDFS-8665
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Trivial
> Fix For: 2.8.0
>
> Attachments: hdfs-8665.001.patch
>
>
> The check looks at the repl factor set on the file rather than reported # of 
> replica locations. Let's do the latter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602764#comment-14602764
 ] 

Hudson commented on HDFS-8546:
--

SUCCESS: Integrated in Hadoop-Yarn-trunk #970 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/970/])
HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 
1403b84b122fb76ef2b085a728b5402c32499c1f)
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Use try with resources in DataStorage and Storage
> -
>
> Key: HDFS-8546
> URL: https://issues.apache.org/jira/browse/HDFS-8546
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.7.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, 
> HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, 
> HDFS-8546.006.patch
>
>
> We have some old-style try/finally to close files in DataStorage and Storage, 
> let's update them.
> Also a few small cleanups:
> * Actually check that tryLock returns a FileLock in isPreUpgradableLayout
> * Remove unused parameter from writeProperties
> * Add braces for one-line if statements per coding style



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602761#comment-14602761
 ] 

Hudson commented on HDFS-8462:
--

SUCCESS: Integrated in Hadoop-Yarn-trunk #970 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/970/])
HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. 
Contributed by Jagadesh Kiran N. (aajisaka: rev 
bc433908d35758ff0a7225cd6f5662959ef4d294)
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
> 
>
> Key: HDFS-8462
> URL: https://issues.apache.org/jira/browse/HDFS-8462
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Akira AJISAKA
>Assignee: Jagadesh Kiran N
> Fix For: 2.8.0
>
> Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, 
> HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, 
> HDFS-8462-05.patch, HDFS-8462-06.patch
>
>
> In Hadoop 2.7.0, WebImageViewer supports the following operations:
> * {{GETFILESTATUS}}
> * {{LISTSTATUS}}
> * {{GETACLSTATUS}}
> I'm thinking it would be better for administrators if {{GETXATTRS}} and 
> {{LISTXATTRS}} are supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Updated] (HDFS-8627) NPE thrown if unable to fetch token from Namenode

2015-06-26 Thread J.Andreina (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.Andreina updated HDFS-8627:
-
Status: Patch Available  (was: Open)

> NPE thrown if unable to fetch token from Namenode
> -
>
> Key: HDFS-8627
> URL: https://issues.apache.org/jira/browse/HDFS-8627
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: J.Andreina
>Assignee: J.Andreina
> Attachments: HDFS-8627.1.patch
>
>
> DelegationTokenFetcher#saveDelegationToken
> Missed to check if token is null.
> {code}
> Token token = fs.getDelegationToken(renewer);
> Credentials cred = new Credentials();
> cred.addToken(token.getKind(), token);
> {code}
> {noformat}
> XX:~/hadoop/namenode/bin> ./hdfs fetchdt --renewer Rex 
> /home/REX/file1
> Exception in thread "main" java.lang.NullPointerException
> at 
> org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.saveDelegationToken(DelegationTokenFetcher.java:181)
> at 
> org.apache.hadoop.hdfs.tools.DelegationTokenFetcher$1.run(DelegationTokenFetcher.java:126)
> at 
> java.security.AccessController.doPrivileged(AccessController.java:314)
> at javax.security.auth.Subject.doAs(Subject.java:572)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1666)
> at 
> org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.main(DelegationTokenFetcher.java:114)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HDFS-8627) NPE thrown if unable to fetch token from Namenode

2015-06-26 Thread J.Andreina (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.Andreina updated HDFS-8627:
-
Attachment: HDFS-8627.1.patch

Attached an initial patch.
Please review.

> NPE thrown if unable to fetch token from Namenode
> -
>
> Key: HDFS-8627
> URL: https://issues.apache.org/jira/browse/HDFS-8627
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: J.Andreina
>Assignee: J.Andreina
> Attachments: HDFS-8627.1.patch
>
>
> DelegationTokenFetcher#saveDelegationToken
> Missed to check if token is null.
> {code}
> Token token = fs.getDelegationToken(renewer);
> Credentials cred = new Credentials();
> cred.addToken(token.getKind(), token);
> {code}
> {noformat}
> XX:~/hadoop/namenode/bin> ./hdfs fetchdt --renewer Rex 
> /home/REX/file1
> Exception in thread "main" java.lang.NullPointerException
> at 
> org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.saveDelegationToken(DelegationTokenFetcher.java:181)
> at 
> org.apache.hadoop.hdfs.tools.DelegationTokenFetcher$1.run(DelegationTokenFetcher.java:126)
> at 
> java.security.AccessController.doPrivileged(AccessController.java:314)
> at javax.security.auth.Subject.doAs(Subject.java:572)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1666)
> at 
> org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.main(DelegationTokenFetcher.java:114)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HDFS-8628) Update missing command option for fetchdt

2015-06-26 Thread J.Andreina (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.Andreina updated HDFS-8628:
-
Attachment: HDFS-8628.1.patch

Attached an initial patch
Please review.

> Update missing command option for fetchdt
> -
>
> Key: HDFS-8628
> URL: https://issues.apache.org/jira/browse/HDFS-8628
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: documentation
>Reporter: J.Andreina
>Assignee: J.Andreina
> Attachments: HDFS-8628.1.patch
>
>
> Update missing command option for fetchdt
> *Expected:*
> {noformat}
> fetchdt  
> Options:
>   --webservice   Url to contact NN on (starts with http:// or https://)
>   --renewer Name of the delegation token renewer
>   --cancelCancel the delegation token
>   --renew Renew the delegation token.  Delegation token must have 
> been fetched using the --renewer  option.
>   --print Print the delegation token
> {noformat}
> *Actual:*
> {noformat}
> Usage: hdfs fetchdt [--webservice ] 
> COMMAND_OPTIONDescription
> --webservice https_address   use http protocol instead of RPC
> fileName File name to store 
> the token into.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HDFS-8628) Update missing command option for fetchdt

2015-06-26 Thread J.Andreina (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.Andreina updated HDFS-8628:
-
Status: Patch Available  (was: Open)

> Update missing command option for fetchdt
> -
>
> Key: HDFS-8628
> URL: https://issues.apache.org/jira/browse/HDFS-8628
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: documentation
>Reporter: J.Andreina
>Assignee: J.Andreina
> Attachments: HDFS-8628.1.patch
>
>
> Update missing command option for fetchdt
> *Expected:*
> {noformat}
> fetchdt  
> Options:
>   --webservice   Url to contact NN on (starts with http:// or https://)
>   --renewer Name of the delegation token renewer
>   --cancelCancel the delegation token
>   --renew Renew the delegation token.  Delegation token must have 
> been fetched using the --renewer  option.
>   --print Print the delegation token
> {noformat}
> *Actual:*
> {noformat}
> Usage: hdfs fetchdt [--webservice ] 
> COMMAND_OPTIONDescription
> --webservice https_address   use http protocol instead of RPC
> fileName File name to store 
> the token into.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Updated] (HDFS-8627) NPE thrown if unable to fetch token from Namenode

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Updated] (HDFS-8627) NPE thrown if unable to fetch token from Namenode

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Updated] (HDFS-8628) Update missing command option for fetchdt

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Updated] (HDFS-8579) Update HDFS usage with missing options

2015-06-26 Thread J.Andreina (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.Andreina updated HDFS-8579:
-
Status: Patch Available  (was: Open)

> Update HDFS usage with missing options
> --
>
> Key: HDFS-8579
> URL: https://issues.apache.org/jira/browse/HDFS-8579
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: J.Andreina
>Assignee: J.Andreina
>Priority: Minor
> Attachments: HDFS-8579-branch-2.7-1.patch, HDFS-8579-trunk-1.patch
>
>
> Update hdfs usage with missing options (fetchdt and debug)
> {noformat}
>  1
> ./hdfs fetchdt
> fetchdt  
> Options:
>   --webservice   Url to contact NN on
>   --renewer Name of the delegation token renewer
>   --cancelCancel the delegation token
>   --renew Renew the delegation token.  Delegation token must have 
> been fetched using the --renewer  option.
>   --print Print the delegation token
> 2
>  ./hdfs debug
> Usage: hdfs debug  [arguments]
> verify [-meta ] [-block ]
> recoverLease [-path ] [-retries ]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Updated] (HDFS-8628) Update missing command option for fetchdt

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Updated] (HDFS-8579) Update HDFS usage with missing options

2015-06-26 Thread J.Andreina (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.Andreina updated HDFS-8579:
-
Attachment: HDFS-8579-trunk-1.patch
HDFS-8579-branch-2.7-1.patch

Please review the patch.

> Update HDFS usage with missing options
> --
>
> Key: HDFS-8579
> URL: https://issues.apache.org/jira/browse/HDFS-8579
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: J.Andreina
>Assignee: J.Andreina
>Priority: Minor
> Attachments: HDFS-8579-branch-2.7-1.patch, HDFS-8579-trunk-1.patch
>
>
> Update hdfs usage with missing options (fetchdt and debug)
> {noformat}
>  1
> ./hdfs fetchdt
> fetchdt  
> Options:
>   --webservice   Url to contact NN on
>   --renewer Name of the delegation token renewer
>   --cancelCancel the delegation token
>   --renew Renew the delegation token.  Delegation token must have 
> been fetched using the --renewer  option.
>   --print Print the delegation token
> 2
>  ./hdfs debug
> Usage: hdfs debug  [arguments]
> verify [-meta ] [-block ]
> recoverLease [-path ] [-retries ]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Updated] (HDFS-8579) Update HDFS usage with missing options

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Updated] (HDFS-8579) Update HDFS usage with missing options

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8628) Update missing command option for fetchdt

2015-06-26 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602841#comment-14602841
 ] 

Hadoop QA commented on HDFS-8628:
-

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |   3m  3s | Pre-patch trunk compilation is 
healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any 
@author tags. |
| {color:green}+1{color} | release audit |   0m 20s | The applied patch does 
not increase the total number of release audit warnings. |
| {color:red}-1{color} | site |   1m 15s | Site compilation is broken. |
| {color:green}+1{color} | whitespace |   0m  0s | The patch has no lines that 
end in whitespace. |
| | |   4m 41s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12742111/HDFS-8628.1.patch |
| Optional Tests | site |
| git revision | trunk / 8ef07f7 |
| site | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11497/artifact/patchprocess/patchSiteWarnings.txt
 |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11497/console |


This message was automatically generated.

> Update missing command option for fetchdt
> -
>
> Key: HDFS-8628
> URL: https://issues.apache.org/jira/browse/HDFS-8628
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: documentation
>Reporter: J.Andreina
>Assignee: J.Andreina
> Attachments: HDFS-8628.1.patch
>
>
> Update missing command option for fetchdt
> *Expected:*
> {noformat}
> fetchdt  
> Options:
>   --webservice   Url to contact NN on (starts with http:// or https://)
>   --renewer Name of the delegation token renewer
>   --cancelCancel the delegation token
>   --renew Renew the delegation token.  Delegation token must have 
> been fetched using the --renewer  option.
>   --print Print the delegation token
> {noformat}
> *Actual:*
> {noformat}
> Usage: hdfs fetchdt [--webservice ] 
> COMMAND_OPTIONDescription
> --webservice https_address   use http protocol instead of RPC
> fileName File name to store 
> the token into.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8628) Update missing command option for fetchdt

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX

2015-06-26 Thread Gabor Liptak (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602885#comment-14602885
 ] 

Gabor Liptak commented on HDFS-8670:


I also made a patch before seeing [~andreina]'s message.

> Better to exclude decommissioned nodes for namenode NodeUsage JMX
> -
>
> Key: HDFS-8670
> URL: https://issues.apache.org/jira/browse/HDFS-8670
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: Ming Ma
>Assignee: J.Andreina
>
> The namenode NodeUsage JMX has Max, Median, Min and Standard Deviation of 
> DataNodes usage, it currently includes decommissioned nodes for the 
> calculation. However, given balancer doesn't work on decommissioned nodes and 
> sometimes we could have nodes stay in decommissioned states for a long time; 
> it might be better to exclude decommissioned nodes for the metrics 
> calculation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602938#comment-14602938
 ] 

Hudson commented on HDFS-8665:
--

FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #229 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/229/])
HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev 
ff0e5e572f5dcf7b49381cbe901360f6e171d423)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Fix replication check in DFSTestUtils#waitForReplication
> 
>
> Key: HDFS-8665
> URL: https://issues.apache.org/jira/browse/HDFS-8665
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Trivial
> Fix For: 2.8.0
>
> Attachments: hdfs-8665.001.patch
>
>
> The check looks at the repl factor set on the file rather than reported # of 
> replica locations. Let's do the latter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602940#comment-14602940
 ] 

Hudson commented on HDFS-8640:
--

FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #229 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/229/])
HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka 
kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java


> Make reserved RBW space visible through JMX
> ---
>
> Key: HDFS-8640
> URL: https://issues.apache.org/jira/browse/HDFS-8640
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: kanaka kumar avvaru
>Assignee: kanaka kumar avvaru
> Fix For: 2.8.0
>
> Attachments: HDFS-8640-00.patch
>
>
> At present there is no way to trace the reserved space for RBW  in DataNode 
> to identify any leaks like HDFS-8626
> Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX 
> bean can be referred to know the present reserved space



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602941#comment-14602941
 ] 

Hudson commented on HDFS-8546:
--

FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #229 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/229/])
HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 
1403b84b122fb76ef2b085a728b5402c32499c1f)
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java


> Use try with resources in DataStorage and Storage
> -
>
> Key: HDFS-8546
> URL: https://issues.apache.org/jira/browse/HDFS-8546
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.7.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, 
> HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, 
> HDFS-8546.006.patch
>
>
> We have some old-style try/finally to close files in DataStorage and Storage, 
> let's update them.
> Also a few small cleanups:
> * Actually check that tryLock returns a FileLock in isPreUpgradableLayout
> * Remove unused parameter from writeProperties
> * Add braces for one-line if statements per coding style



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602939#comment-14602939
 ] 

Hudson commented on HDFS-8462:
--

FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #229 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/229/])
HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. 
Contributed by Jagadesh Kiran N. (aajisaka: rev 
bc433908d35758ff0a7225cd6f5662959ef4d294)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java


> Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
> 
>
> Key: HDFS-8462
> URL: https://issues.apache.org/jira/browse/HDFS-8462
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Akira AJISAKA
>Assignee: Jagadesh Kiran N
> Fix For: 2.8.0
>
> Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, 
> HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, 
> HDFS-8462-05.patch, HDFS-8462-06.patch
>
>
> In Hadoop 2.7.0, WebImageViewer supports the following operations:
> * {{GETFILESTATUS}}
> * {{LISTSTATUS}}
> * {{GETACLSTATUS}}
> I'm thinking it would be better for administrators if {{GETXATTRS}} and 
> {{LISTXATTRS}} are supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602952#comment-14602952
 ] 

Hudson commented on HDFS-8546:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #238 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/238/])
HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 
1403b84b122fb76ef2b085a728b5402c32499c1f)
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java


> Use try with resources in DataStorage and Storage
> -
>
> Key: HDFS-8546
> URL: https://issues.apache.org/jira/browse/HDFS-8546
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.7.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, 
> HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, 
> HDFS-8546.006.patch
>
>
> We have some old-style try/finally to close files in DataStorage and Storage, 
> let's update them.
> Also a few small cleanups:
> * Actually check that tryLock returns a FileLock in isPreUpgradableLayout
> * Remove unused parameter from writeProperties
> * Add braces for one-line if statements per coding style



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602950#comment-14602950
 ] 

Hudson commented on HDFS-8462:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #238 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/238/])
HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. 
Contributed by Jagadesh Kiran N. (aajisaka: rev 
bc433908d35758ff0a7225cd6f5662959ef4d294)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md


> Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
> 
>
> Key: HDFS-8462
> URL: https://issues.apache.org/jira/browse/HDFS-8462
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Akira AJISAKA
>Assignee: Jagadesh Kiran N
> Fix For: 2.8.0
>
> Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, 
> HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, 
> HDFS-8462-05.patch, HDFS-8462-06.patch
>
>
> In Hadoop 2.7.0, WebImageViewer supports the following operations:
> * {{GETFILESTATUS}}
> * {{LISTSTATUS}}
> * {{GETACLSTATUS}}
> I'm thinking it would be better for administrators if {{GETXATTRS}} and 
> {{LISTXATTRS}} are supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602951#comment-14602951
 ] 

Hudson commented on HDFS-8640:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #238 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/238/])
HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka 
kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Make reserved RBW space visible through JMX
> ---
>
> Key: HDFS-8640
> URL: https://issues.apache.org/jira/browse/HDFS-8640
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: kanaka kumar avvaru
>Assignee: kanaka kumar avvaru
> Fix For: 2.8.0
>
> Attachments: HDFS-8640-00.patch
>
>
> At present there is no way to trace the reserved space for RBW  in DataNode 
> to identify any leaks like HDFS-8626
> Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX 
> bean can be referred to know the present reserved space



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602962#comment-14602962
 ] 

Hudson commented on HDFS-8665:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk #2186 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2186/])
HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev 
ff0e5e572f5dcf7b49381cbe901360f6e171d423)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Fix replication check in DFSTestUtils#waitForReplication
> 
>
> Key: HDFS-8665
> URL: https://issues.apache.org/jira/browse/HDFS-8665
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Trivial
> Fix For: 2.8.0
>
> Attachments: hdfs-8665.001.patch
>
>
> The check looks at the repl factor set on the file rather than reported # of 
> replica locations. Let's do the latter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602957#comment-14602957
 ] 

Hudson commented on HDFS-8462:
--

SUCCESS: Integrated in Hadoop-Hdfs-trunk #2168 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2168/])
HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. 
Contributed by Jagadesh Kiran N. (aajisaka: rev 
bc433908d35758ff0a7225cd6f5662959ef4d294)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md


> Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
> 
>
> Key: HDFS-8462
> URL: https://issues.apache.org/jira/browse/HDFS-8462
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Akira AJISAKA
>Assignee: Jagadesh Kiran N
> Fix For: 2.8.0
>
> Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, 
> HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, 
> HDFS-8462-05.patch, HDFS-8462-06.patch
>
>
> In Hadoop 2.7.0, WebImageViewer supports the following operations:
> * {{GETFILESTATUS}}
> * {{LISTSTATUS}}
> * {{GETACLSTATUS}}
> I'm thinking it would be better for administrators if {{GETXATTRS}} and 
> {{LISTXATTRS}} are supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602949#comment-14602949
 ] 

Hudson commented on HDFS-8665:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #238 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/238/])
HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev 
ff0e5e572f5dcf7b49381cbe901360f6e171d423)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java


> Fix replication check in DFSTestUtils#waitForReplication
> 
>
> Key: HDFS-8665
> URL: https://issues.apache.org/jira/browse/HDFS-8665
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Trivial
> Fix For: 2.8.0
>
> Attachments: hdfs-8665.001.patch
>
>
> The check looks at the repl factor set on the file rather than reported # of 
> replica locations. Let's do the latter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602959#comment-14602959
 ] 

Hudson commented on HDFS-8546:
--

SUCCESS: Integrated in Hadoop-Hdfs-trunk #2168 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2168/])
HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 
1403b84b122fb76ef2b085a728b5402c32499c1f)
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Use try with resources in DataStorage and Storage
> -
>
> Key: HDFS-8546
> URL: https://issues.apache.org/jira/browse/HDFS-8546
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.7.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, 
> HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, 
> HDFS-8546.006.patch
>
>
> We have some old-style try/finally to close files in DataStorage and Storage, 
> let's update them.
> Also a few small cleanups:
> * Actually check that tryLock returns a FileLock in isPreUpgradableLayout
> * Remove unused parameter from writeProperties
> * Add braces for one-line if statements per coding style



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602958#comment-14602958
 ] 

Hudson commented on HDFS-8640:
--

SUCCESS: Integrated in Hadoop-Hdfs-trunk #2168 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2168/])
HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka 
kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java


> Make reserved RBW space visible through JMX
> ---
>
> Key: HDFS-8640
> URL: https://issues.apache.org/jira/browse/HDFS-8640
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: kanaka kumar avvaru
>Assignee: kanaka kumar avvaru
> Fix For: 2.8.0
>
> Attachments: HDFS-8640-00.patch
>
>
> At present there is no way to trace the reserved space for RBW  in DataNode 
> to identify any leaks like HDFS-8626
> Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX 
> bean can be referred to know the present reserved space



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8640) Make reserved RBW space visible through JMX

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602964#comment-14602964
 ] 

Hudson commented on HDFS-8640:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk #2186 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2186/])
HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka 
kumar avvaru) (arp: rev 67a62da5c5f592b07d083440ced3666c7709b20d)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java


> Make reserved RBW space visible through JMX
> ---
>
> Key: HDFS-8640
> URL: https://issues.apache.org/jira/browse/HDFS-8640
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: kanaka kumar avvaru
>Assignee: kanaka kumar avvaru
> Fix For: 2.8.0
>
> Attachments: HDFS-8640-00.patch
>
>
> At present there is no way to trace the reserved space for RBW  in DataNode 
> to identify any leaks like HDFS-8626
> Idea is to add the value to {{VolumeInfo}} so that the {{DataNodeInfo}} JMX 
> bean can be referred to know the present reserved space



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602956#comment-14602956
 ] 

Hudson commented on HDFS-8665:
--

SUCCESS: Integrated in Hadoop-Hdfs-trunk #2168 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2168/])
HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. (wang: rev 
ff0e5e572f5dcf7b49381cbe901360f6e171d423)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java


> Fix replication check in DFSTestUtils#waitForReplication
> 
>
> Key: HDFS-8665
> URL: https://issues.apache.org/jira/browse/HDFS-8665
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Trivial
> Fix For: 2.8.0
>
> Attachments: hdfs-8665.001.patch
>
>
> The check looks at the repl factor set on the file rather than reported # of 
> replica locations. Let's do the latter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8665) Fix replication check in DFSTestUtils#waitForReplication

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8546) Use try with resources in DataStorage and Storage

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602965#comment-14602965
 ] 

Hudson commented on HDFS-8546:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk #2186 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2186/])
HDFS-8546. Use try with resources in DataStorage and Storage. (wang: rev 
1403b84b122fb76ef2b085a728b5402c32499c1f)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java


> Use try with resources in DataStorage and Storage
> -
>
> Key: HDFS-8546
> URL: https://issues.apache.org/jira/browse/HDFS-8546
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 2.7.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HDFS-8546.001.patch, HDFS-8546.002.patch, 
> HDFS-8546.003.patch, HDFS-8546.004.patch, HDFS-8546.005.patch, 
> HDFS-8546.006.patch
>
>
> We have some old-style try/finally to close files in DataStorage and Storage, 
> let's update them.
> Also a few small cleanups:
> * Actually check that tryLock returns a FileLock in isPreUpgradableLayout
> * Remove unused parameter from writeProperties
> * Add braces for one-line if statements per coding style



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8462) Implement GETXATTRS and LISTXATTRS operations for WebImageViewer

2015-06-26 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602963#comment-14602963
 ] 

Hudson commented on HDFS-8462:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk #2186 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2186/])
HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. 
Contributed by Jagadesh Kiran N. (aajisaka: rev 
bc433908d35758ff0a7225cd6f5662959ef4d294)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerForXAttr.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageHandler.java
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Implement GETXATTRS and LISTXATTRS operations for WebImageViewer
> 
>
> Key: HDFS-8462
> URL: https://issues.apache.org/jira/browse/HDFS-8462
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Akira AJISAKA
>Assignee: Jagadesh Kiran N
> Fix For: 2.8.0
>
> Attachments: HDFS-8462-00.patch, HDFS-8462-01.patch, 
> HDFS-8462-02.patch, HDFS-8462-03.patch, HDFS-8462-04.patch, 
> HDFS-8462-05.patch, HDFS-8462-06.patch
>
>
> In Hadoop 2.7.0, WebImageViewer supports the following operations:
> * {{GETFILESTATUS}}
> * {{LISTSTATUS}}
> * {{GETACLSTATUS}}
> I'm thinking it would be better for administrators if {{GETXATTRS}} and 
> {{LISTXATTRS}} are supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX

2015-06-26 Thread J.Andreina (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.Andreina updated HDFS-8670:
-
Status: Patch Available  (was: Open)

> Better to exclude decommissioned nodes for namenode NodeUsage JMX
> -
>
> Key: HDFS-8670
> URL: https://issues.apache.org/jira/browse/HDFS-8670
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: Ming Ma
>Assignee: J.Andreina
> Attachments: HDFS-8670.1.patch
>
>
> The namenode NodeUsage JMX has Max, Median, Min and Standard Deviation of 
> DataNodes usage, it currently includes decommissioned nodes for the 
> calculation. However, given balancer doesn't work on decommissioned nodes and 
> sometimes we could have nodes stay in decommissioned states for a long time; 
> it might be better to exclude decommissioned nodes for the metrics 
> calculation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX

2015-06-26 Thread J.Andreina (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.Andreina updated HDFS-8670:
-
Attachment: HDFS-8670.1.patch

Attached code patch along with a testcase.
Please review.

> Better to exclude decommissioned nodes for namenode NodeUsage JMX
> -
>
> Key: HDFS-8670
> URL: https://issues.apache.org/jira/browse/HDFS-8670
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: Ming Ma
>Assignee: J.Andreina
> Attachments: HDFS-8670.1.patch
>
>
> The namenode NodeUsage JMX has Max, Median, Min and Standard Deviation of 
> DataNodes usage, it currently includes decommissioned nodes for the 
> calculation. However, given balancer doesn't work on decommissioned nodes and 
> sometimes we could have nodes stay in decommissioned states for a long time; 
> it might be better to exclude decommissioned nodes for the metrics 
> calculation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Updated] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Updated] (HDFS-8670) Better to exclude decommissioned nodes for namenode NodeUsage JMX

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8579) Update HDFS usage with missing options

2015-06-26 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603126#comment-14603126
 ] 

Hadoop QA commented on HDFS-8579:
-

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  15m  3s | Pre-patch trunk compilation is 
healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any 
@author tags. |
| {color:red}-1{color} | tests included |   0m  0s | The patch doesn't appear 
to include any new or modified tests.  Please justify why no new tests are 
needed for this patch. Also please list what manual steps were performed to 
verify this patch. |
| {color:green}+1{color} | javac |   7m 40s | There were no new javac warning 
messages. |
| {color:green}+1{color} | javadoc |   9m 46s | There were no new javadoc 
warning messages. |
| {color:green}+1{color} | release audit |   0m 22s | The applied patch does 
not increase the total number of release audit warnings. |
| {color:green}+1{color} | shellcheck |   0m  6s | There were no new shellcheck 
(v0.3.3) issues. |
| {color:green}+1{color} | whitespace |   0m  0s | The patch has no lines that 
end in whitespace. |
| {color:green}+1{color} | install |   1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 32s | The patch built with 
eclipse:eclipse. |
| {color:green}+1{color} | native |   3m 17s | Pre-build of native portion |
| {color:green}+1{color} | hdfs tests | 159m 26s | Tests passed in hadoop-hdfs. 
|
| | | 197m 47s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12742113/HDFS-8579-trunk-1.patch
 |
| Optional Tests | javadoc javac unit shellcheck |
| git revision | trunk / 8ef07f7 |
| hadoop-hdfs test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11498/artifact/patchprocess/testrun_hadoop-hdfs.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11498/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11498/console |


This message was automatically generated.

> Update HDFS usage with missing options
> --
>
> Key: HDFS-8579
> URL: https://issues.apache.org/jira/browse/HDFS-8579
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: J.Andreina
>Assignee: J.Andreina
>Priority: Minor
> Attachments: HDFS-8579-branch-2.7-1.patch, HDFS-8579-trunk-1.patch
>
>
> Update hdfs usage with missing options (fetchdt and debug)
> {noformat}
>  1
> ./hdfs fetchdt
> fetchdt  
> Options:
>   --webservice   Url to contact NN on
>   --renewer Name of the delegation token renewer
>   --cancelCancel the delegation token
>   --renew Renew the delegation token.  Delegation token must have 
> been fetched using the --renewer  option.
>   --print Print the delegation token
> 2
>  ./hdfs debug
> Usage: hdfs debug  [arguments]
> verify [-meta ] [-block ]
> recoverLease [-path ] [-retries ]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8579) Update HDFS usage with missing options

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Created] (HDFS-8674) Improve performance of postponed block scans

2015-06-26 Thread Daryn Sharp (JIRA)
Daryn Sharp created HDFS-8674:
-

 Summary: Improve performance of postponed block scans
 Key: HDFS-8674
 URL: https://issues.apache.org/jira/browse/HDFS-8674
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: HDFS
Affects Versions: 2.6.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical


When a standby goes active, it marks all nodes as "stale" which will cause 
block invalidations for over-replicated blocks to be queued until full block 
reports are received from the nodes with the block.  The replication monitor 
scans the queue with O(N) runtime.  It picks a random offset and iterates 
through the set to randomize blocks scanned.

The result is devastating when a cluster loses multiple nodes during a rolling 
upgrade. Re-replication occurs, the nodes come back, the excess block 
invalidations are postponed. Rescanning just 2k blocks out of millions of 
postponed blocks may take multiple seconds. During the scan, the write lock is 
held which stalls all other processing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Created] (HDFS-8674) Improve performance of postponed block scans

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8656) Preserve compatibility of ClientProtocol#rollingUpgrade after finalization

2015-06-26 Thread Ming Ma (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603138#comment-14603138
 ] 

Ming Ma commented on HDFS-8656:
---

+1. Thanks Andrew. The unit test failures are unrelated.

> Preserve compatibility of ClientProtocol#rollingUpgrade after finalization
> --
>
> Key: HDFS-8656
> URL: https://issues.apache.org/jira/browse/HDFS-8656
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: rolling upgrades
>Affects Versions: 2.8.0
>Reporter: Andrew Wang
>Assignee: Andrew Wang
>Priority: Critical
> Attachments: hdfs-8656.001.patch, hdfs-8656.002.patch, 
> hdfs-8656.003.patch, hdfs-8656.004.patch
>
>
> HDFS-7645 changed rollingUpgradeInfo to still return an RUInfo after 
> finalization, so the DNs can differentiate between rollback and a 
> finalization. However, this breaks compatibility for the user facing APIs, 
> which always expect a null after finalization. Let's fix this and edify it in 
> unit tests.
> As an additional improvement, isFinalized and isStarted are part of the Java 
> API, but not in the JMX output of RollingUpgradeInfo. It'd be nice to expose 
> these booleans so JMX users don't need to do the != 0 check that possibly 
> exposes our implementation details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8656) Preserve compatibility of ClientProtocol#rollingUpgrade after finalization

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Commented] (HDFS-8627) NPE thrown if unable to fetch token from Namenode

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8627) NPE thrown if unable to fetch token from Namenode

2015-06-26 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603146#comment-14603146
 ] 

Hadoop QA commented on HDFS-8627:
-

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  17m 32s | Pre-patch trunk compilation is 
healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any 
@author tags. |
| {color:green}+1{color} | tests included |   0m  0s | The patch appears to 
include 1 new or modified test files. |
| {color:green}+1{color} | javac |   7m 26s | There were no new javac warning 
messages. |
| {color:green}+1{color} | javadoc |   9m 36s | There were no new javadoc 
warning messages. |
| {color:green}+1{color} | release audit |   0m 22s | The applied patch does 
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle |   2m 17s | There were no new checkstyle 
issues. |
| {color:red}-1{color} | whitespace |   0m  0s | The patch has 1  line(s) that 
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install |   1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 32s | The patch built with 
eclipse:eclipse. |
| {color:green}+1{color} | findbugs |   3m 15s | The patch does not introduce 
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native |   3m 14s | Pre-build of native portion |
| {color:green}+1{color} | hdfs tests | 161m 42s | Tests passed in hadoop-hdfs. 
|
| | | 207m 34s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12742110/HDFS-8627.1.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 8ef07f7 |
| whitespace | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11499/artifact/patchprocess/whitespace.txt
 |
| hadoop-hdfs test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11499/artifact/patchprocess/testrun_hadoop-hdfs.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11499/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11499/console |


This message was automatically generated.

> NPE thrown if unable to fetch token from Namenode
> -
>
> Key: HDFS-8627
> URL: https://issues.apache.org/jira/browse/HDFS-8627
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: J.Andreina
>Assignee: J.Andreina
> Attachments: HDFS-8627.1.patch
>
>
> DelegationTokenFetcher#saveDelegationToken
> Missed to check if token is null.
> {code}
> Token token = fs.getDelegationToken(renewer);
> Credentials cred = new Credentials();
> cred.addToken(token.getKind(), token);
> {code}
> {noformat}
> XX:~/hadoop/namenode/bin> ./hdfs fetchdt --renewer Rex 
> /home/REX/file1
> Exception in thread "main" java.lang.NullPointerException
> at 
> org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.saveDelegationToken(DelegationTokenFetcher.java:181)
> at 
> org.apache.hadoop.hdfs.tools.DelegationTokenFetcher$1.run(DelegationTokenFetcher.java:126)
> at 
> java.security.AccessController.doPrivileged(AccessController.java:314)
> at javax.security.auth.Subject.doAs(Subject.java:572)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1666)
> at 
> org.apache.hadoop.hdfs.tools.DelegationTokenFetcher.main(DelegationTokenFetcher.java:114)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HDFS-8675) IBRs from dead DNs go into infinite loop

2015-06-26 Thread Daryn Sharp (JIRA)
Daryn Sharp created HDFS-8675:
-

 Summary: IBRs from dead DNs go into infinite loop
 Key: HDFS-8675
 URL: https://issues.apache.org/jira/browse/HDFS-8675
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: datanode
Affects Versions: 2.6.0
Reporter: Daryn Sharp


If the DN sends an IBR after the NN declares it dead, the NN returns an IOE of 
unregistered or dead.  The DN catches the IOE, ignores it, and infinitely loops 
spamming the NN with retries.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Created] (HDFS-8675) IBRs from dead DNs go into infinite loop

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Created] (HDFS-8676) Delayed rolling upgrade finalization can cause heartbeat expiration

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Created] (HDFS-8676) Delayed rolling upgrade finalization can cause heartbeat expiration

2015-06-26 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-8676:


 Summary: Delayed rolling upgrade finalization can cause heartbeat 
expiration
 Key: HDFS-8676
 URL: https://issues.apache.org/jira/browse/HDFS-8676
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Kihwal Lee
Priority: Critical


In big busy clusters where the deletion rate is also high, a lot of blocks can 
pile up in the datanode trash directories until an upgrade is finalized.  When 
it is finally finalized, the deletion of trash is done in the service actor 
thread's context synchronously.  This blocks the heartbeat and can cause 
heartbeat expiration.  

We have seen a namenode losing hundreds of nodes after a delayed upgrade 
finalization.  The deletion of trash directories should be made asynchronous.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8675) IBRs from dead DNs go into infinite loop

2015-06-26 Thread Daryn Sharp (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603154#comment-14603154
 ] 

Daryn Sharp commented on HDFS-8675:
---

The solution isn't as simple as replacing the IOE with 
UnregisteredNodeException.  The DN will commit suicide.  Handling the exception 
will require buffering the IBR (or any other calls this may affect), 
registering, resending the rejected message.  Issue might be related to network 
issues or defects that stall BPOfferService like the synchronous clearing of 
the trash after RU finalize.

> IBRs from dead DNs go into infinite loop
> 
>
> Key: HDFS-8675
> URL: https://issues.apache.org/jira/browse/HDFS-8675
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode
>Affects Versions: 2.6.0
>Reporter: Daryn Sharp
>
> If the DN sends an IBR after the NN declares it dead, the NN returns an IOE 
> of unregistered or dead.  The DN catches the IOE, ignores it, and infinitely 
> loops spamming the NN with retries.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8675) IBRs from dead DNs go into infinite loop

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

Auto-Re: [jira] [Updated] (HDFS-8673) HDFS reports file already exists if there is a file/dir name end with ._COPYING_

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Updated] (HDFS-8673) HDFS reports file already exists if there is a file/dir name end with ._COPYING_

2015-06-26 Thread Chen He (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chen He updated HDFS-8673:
--
Assignee: (was: Chen He)

> HDFS reports file already exists if there is a file/dir name end with 
> ._COPYING_
> 
>
> Key: HDFS-8673
> URL: https://issues.apache.org/jira/browse/HDFS-8673
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: fs
>Affects Versions: 2.7.0
>Reporter: Chen He
>
> Because CLI is using CommandWithDestination.java which add "._COPYING_" to 
> the tail of file name when it does the copy. It will cause problem if there 
> is a file/dir already called *._COPYING_ on HDFS.
> For file:
> -bash-4.1$ hadoop fs -put 5M /user/occ/
> -bash-4.1$ hadoop fs -mv /user/occ/5M /user/occ/5M._COPYING_
> -bash-4.1$ hadoop fs -ls /user/occ/
> Found 1 items
> -rw-r--r--   1 occ supergroup5242880 2015-06-26 05:16 
> /user/occ/5M._COPYING_
> -bash-4.1$ hadoop fs -put 128K /user/occ/5M
> -bash-4.1$ hadoop fs -ls /user/occ/
> Found 1 items
> -rw-r--r--   1 occ supergroup 131072 2015-06-26 05:19 /user/occ/5M
> For dir:
> -bash-4.1$ hadoop fs -mkdir /user/occ/5M._COPYING_
> -bash-4.1$ hadoop fs -ls /user/occ/
> Found 1 items
> drwxr-xr-x   - occ supergroup  0 2015-06-26 05:24 
> /user/occ/5M._COPYING_
> -bash-4.1$ hadoop fs -put 128K /user/occ/5M
> put: /user/occ/5M._COPYING_ already exists as a directory
> -bash-4.1$ hadoop fs -ls /user/occ/
> (/user/occ/5M._COPYING_ is gone)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HDFS-8661) DataNode should filter the set of NameSpaceInfos passed to Datasets

2015-06-26 Thread Arpit Agarwal (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arpit Agarwal updated HDFS-8661:

Attachment: HDFS-8661-HDFS-7240.03.patch

v3 patch fixes checkstyle issues.

> DataNode should filter the set of NameSpaceInfos passed to Datasets
> ---
>
> Key: HDFS-8661
> URL: https://issues.apache.org/jira/browse/HDFS-8661
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: datanode
>Affects Versions: HDFS-7240
>Reporter: Arpit Agarwal
>Assignee: Arpit Agarwal
> Attachments: HDFS-8661-HDFS-7240.01.patch, 
> HDFS-8661-HDFS-7240.02.patch, HDFS-8661-HDFS-7240.03.patch
>
>
> {{DataNode#refreshVolumes}} passes the list of NamespaceInfos to each dataset 
> when adding new volumes.
> This list should be filtered by the correct NodeType(s) for each dataset. 
> e.g. in a shared HDFS+Ozone cluster, FsDatasets would be notified of NN block 
> pools and Ozone datasets would be notified of Ozone block pool(s).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Updated] (HDFS-8661) DataNode should filter the set of NameSpaceInfos passed to Datasets

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8664) Allow wildcards in dfs.datanode.data.dir

2015-06-26 Thread Patrick White (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603236#comment-14603236
 ] 

Patrick White commented on HDFS-8664:
-

interesting, lemme go over those tests.

> Allow wildcards in dfs.datanode.data.dir
> 
>
> Key: HDFS-8664
> URL: https://issues.apache.org/jira/browse/HDFS-8664
> Project: Hadoop HDFS
>  Issue Type: New Feature
>  Components: datanode, HDFS
>Affects Versions: 3.0.0
>Reporter: Patrick White
>Assignee: Patrick White
> Attachments: HDFS-8664.001.patch
>
>
> We have many disks per machine (12+) that don't always have the same 
> numbering when they come back from provisioning, but they're always in the 
> same tree following the same pattern.
> It would greatly reduce our config complexity to be able to specify a 
> wildcard for all the data directories.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8664) Allow wildcards in dfs.datanode.data.dir

2015-06-26 Thread wsb
您的邮件已收到!谢谢!

[jira] [Updated] (HDFS-8651) Make hadoop-hdfs-project Native code -Wall-clean

2015-06-26 Thread Colin Patrick McCabe (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-8651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin Patrick McCabe updated HDFS-8651:
---
   Resolution: Fixed
Fix Version/s: 2.8.0
   Status: Resolved  (was: Patch Available)

Committed to 2.8.  Thanks, Alan.

> Make hadoop-hdfs-project Native code -Wall-clean
> 
>
> Key: HDFS-8651
> URL: https://issues.apache.org/jira/browse/HDFS-8651
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: native
>Affects Versions: 2.7.0
>Reporter: Alan Burlison
>Assignee: Alan Burlison
> Fix For: 2.8.0
>
> Attachments: HDFS-8651.001.patch
>
>
> As we specify -Wall as a default compilation flag, it would be helpful if the 
> Native code was -Wall-clean



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


  1   2   3   >