[jira] [Updated] (HDFS-8823) Move replication factor into individual blocks

2015-08-11 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8823: - Attachment: HDFS-8823.002.patch Move replication factor into individual blocks

[jira] [Commented] (HDFS-8826) Balancer may not move blocks efficiently in some cases

2015-08-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14691417#comment-14691417 ] Tsz Wo Nicholas Sze commented on HDFS-8826: --- I suggest to add an option to

[jira] [Updated] (HDFS-8824) Do not use small blocks for balancing the cluster

2015-08-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8824: -- Attachment: h8824_20150811b.patch h8824_20150811b.patch: reverts the NN change. Will do

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

2015-08-11 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692302#comment-14692302 ] Zhe Zhang commented on HDFS-8833: - bq. Looks like Zhe planned to implement the XAttr-based

[jira] [Commented] (HDFS-6244) Make Trash Interval configurable for each of the namespaces

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682498#comment-14682498 ] Hadoop QA commented on HDFS-6244: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Updated] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-11 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated HDFS-8828: --- Attachment: HDFS-8828.006.patch Add exclude list while recursively traverse the created directories in

[jira] [Commented] (HDFS-8875) Optimize the wait time in Balancer for federation scenario

2015-08-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692284#comment-14692284 ] Tsz Wo Nicholas Sze commented on HDFS-8875: --- Balancer will exit if one of the NNs

[jira] [Updated] (HDFS-8888) Support volumes in HDFS

2015-08-11 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-: - Description: There are multiple types of zones (e.g., snapshottable directories, encryption

[jira] [Updated] (HDFS-7916) 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop

2015-08-11 Thread Vinod Kumar Vavilapalli (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated HDFS-7916: -- Labels: (was: 2.6.1-candidate) I earlier removed HDFS-7916 from the

[jira] [Updated] (HDFS-8859) Improve DataNode (ReplicaMap) memory footprint to save about 45%

2015-08-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8859: - Attachment: HDFS-8859.003.patch Improve DataNode (ReplicaMap) memory footprint to save about 45%

[jira] [Commented] (HDFS-8863) The remiaing space check in BlockPlacementPolicyDefault is flawed

2015-08-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692870#comment-14692870 ] Yi Liu commented on HDFS-8863: -- {quote} What if we let it check against storage type level sum

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692442#comment-14692442 ] Hadoop QA commented on HDFS-8828: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Updated] (HDFS-8871) Decommissioning of a node with a failed volume may not start

2015-08-11 Thread Gaurav Gupta (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Gupta updated HDFS-8871: --- Assignee: Daryn Sharp (was: Gaurav Gupta) Decommissioning of a node with a failed volume may not

[jira] [Assigned] (HDFS-8871) Decommissioning of a node with a failed volume may not start

2015-08-11 Thread Gaurav Gupta (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Gupta reassigned HDFS-8871: -- Assignee: Gaurav Gupta (was: Daryn Sharp) Decommissioning of a node with a failed volume may

[jira] [Updated] (HDFS-8884) Fail-fast check in BlockPlacementPolicyDefault#chooseTarget

2015-08-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8884: - Summary: Fail-fast check in BlockPlacementPolicyDefault#chooseTarget (was: Improve

[jira] [Updated] (HDFS-8884) Fail-fast check in BlockPlacementPolicyDefault#chooseTarget

2015-08-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8884: - Status: Patch Available (was: Open) Fail-fast check in BlockPlacementPolicyDefault#chooseTarget

[jira] [Commented] (HDFS-8815) DFS getStoragePolicy implementation using single RPC call

2015-08-11 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681292#comment-14681292 ] Surendra Singh Lilhore commented on HDFS-8815: -- I think it's good to add REST

[jira] [Commented] (HDFS-8078) HDFS client gets errors trying to to connect to IPv6 DataNode

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681371#comment-14681371 ] Hadoop QA commented on HDFS-8078: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Updated] (HDFS-8287) DFSStripedOutputStream.writeChunk should not wait for writing parity

2015-08-11 Thread Kai Sasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Sasaki updated HDFS-8287: - Attachment: HDFS-8287-HDFS-7285.01.patch DFSStripedOutputStream.writeChunk should not wait for writing

[jira] [Commented] (HDFS-8287) DFSStripedOutputStream.writeChunk should not wait for writing parity

2015-08-11 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681550#comment-14681550 ] Rakesh R commented on HDFS-8287: Thanks [~kaisasak], nice work. I've few comments, please

[jira] [Created] (HDFS-8885) ByteRangeInputStream used in webhdfs does not override available()

2015-08-11 Thread Shradha Revankar (JIRA)
Shradha Revankar created HDFS-8885: -- Summary: ByteRangeInputStream used in webhdfs does not override available() Key: HDFS-8885 URL: https://issues.apache.org/jira/browse/HDFS-8885 Project: Hadoop

[jira] [Commented] (HDFS-7285) Erasure Coding Support inside HDFS

2015-08-11 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681307#comment-14681307 ] Zhe Zhang commented on HDFS-7285: - [~wheat9] [~jingzhao] [~walter.k.su] Thanks for the

[jira] [Created] (HDFS-8884) Improve BlockPlacementPolicyDefault#chooseTarget to check candidate datanode first

2015-08-11 Thread Yi Liu (JIRA)
Yi Liu created HDFS-8884: Summary: Improve BlockPlacementPolicyDefault#chooseTarget to check candidate datanode first Key: HDFS-8884 URL: https://issues.apache.org/jira/browse/HDFS-8884 Project: Hadoop HDFS

[jira] [Updated] (HDFS-8884) Improve BlockPlacementPolicyDefault#chooseTarget to check candidate datanode first

2015-08-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8884: - Attachment: HDFS-8884.001.patch Improve BlockPlacementPolicyDefault#chooseTarget to check candidate datanode

[jira] [Commented] (HDFS-8287) DFSStripedOutputStream.writeChunk should not wait for writing parity

2015-08-11 Thread Kai Sasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681321#comment-14681321 ] Kai Sasaki commented on HDFS-8287: -- [~walter.k.su] Thank you for reviewing.

[jira] [Commented] (HDFS-7285) Erasure Coding Support inside HDFS

2015-08-11 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681311#comment-14681311 ] Zhe Zhang commented on HDFS-7285: - As explained below, I think each subtask should *at

[jira] [Commented] (HDFS-8884) Improve BlockPlacementPolicyDefault#chooseTarget to check candidate datanode first

2015-08-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681389#comment-14681389 ] Yi Liu commented on HDFS-8884: -- This JIRA will also remove the duplicate code in

[jira] [Commented] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

2015-08-11 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681614#comment-14681614 ] Brahma Reddy Battula commented on HDFS-8805: Test case failures are unrelated

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2015-08-11 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681648#comment-14681648 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Yarn-trunk #1014 (See

[jira] [Updated] (HDFS-8884) Fail-fast check in BlockPlacementPolicyDefault#chooseTarget

2015-08-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8884: - Description: In current BlockPlacementPolicyDefault, when choosing datanode storage to place block, we have

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2015-08-11 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681643#comment-14681643 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #284 (See

[jira] [Commented] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681595#comment-14681595 ] Hadoop QA commented on HDFS-8805: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-8287) DFSStripedOutputStream.writeChunk should not wait for writing parity

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681661#comment-14681661 ] Hadoop QA commented on HDFS-8287: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Comment Edited] (HDFS-8863) The remiaing space check in BlockPlacementPolicyDefault is flawed

2015-08-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681679#comment-14681679 ] Yi Liu edited comment on HDFS-8863 at 8/11/15 12:00 PM: I found an

[jira] [Commented] (HDFS-8863) The remiaing space check in BlockPlacementPolicyDefault is flawed

2015-08-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681766#comment-14681766 ] Yi Liu commented on HDFS-8863: -- {code} final long requiredSize = blockSize *

[jira] [Commented] (HDFS-8863) The remiaing space check in BlockPlacementPolicyDefault is flawed

2015-08-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681679#comment-14681679 ] Yi Liu commented on HDFS-8863: -- I found an issue here, I think {{getRemaining}} should take

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2015-08-11 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681918#comment-14681918 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #281 (See

[jira] [Updated] (HDFS-8863) The remiaing space check in BlockPlacementPolicyDefault is flawed

2015-08-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8863: - Attachment: HDFS-8863.v2.patch Attaching new patch. The remiaing space check in

[jira] [Commented] (HDFS-8884) Fail-fast check in BlockPlacementPolicyDefault#chooseTarget

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681818#comment-14681818 ] Hadoop QA commented on HDFS-8884: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2015-08-11 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681915#comment-14681915 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2230 (See

[jira] [Commented] (HDFS-8863) The remiaing space check in BlockPlacementPolicyDefault is flawed

2015-08-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681832#comment-14681832 ] Kihwal Lee commented on HDFS-8863: -- bq. it should just return current storage remaining

[jira] [Updated] (HDFS-8865) Improve quota initialization performance

2015-08-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8865: - Attachment: HDFS-8865.v2.checkstyle.patch Missed the one checkstyle warning. Improve quota

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2015-08-11 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681962#comment-14681962 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2211 (See

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2015-08-11 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681996#comment-14681996 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #273 (See

[jira] [Updated] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-08-11 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8808: Affects Version/s: 2.7.1 Target Version/s: 2.7.2 Status: Patch Available (was: Open)

[jira] [Reopened] (HDFS-8052) Move WebHdfsFileSystem into hadoop-hdfs-client

2015-08-11 Thread Gour Saha (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gour Saha reopened HDFS-8052: - Reopening because this is an incompatible change and breaks SLIDER-923 Move WebHdfsFileSystem into

[jira] [Updated] (HDFS-5274) Add Tracing to HDFS

2015-08-11 Thread Elliott Clark (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HDFS-5274: Resolution: Fixed Status: Resolved (was: Patch Available) Add Tracing to HDFS

[jira] [Updated] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-08-11 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8808: Attachment: HDFS-8808-00.patch dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

[jira] [Commented] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

2015-08-11 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682126#comment-14682126 ] Jing Zhao commented on HDFS-8805: - Thanks for updating the patch, [~brahmareddy]. +1 for

[jira] [Resolved] (HDFS-8052) Move WebHdfsFileSystem into hadoop-hdfs-client

2015-08-11 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai resolved HDFS-8052. -- Resolution: Fixed Closing this jira. {{RetryUtils}} is not annotated as a public API thus it might

[jira] [Updated] (HDFS-6244) Make Trash Interval configurable for each of the namespaces

2015-08-11 Thread Siqi Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li updated HDFS-6244: -- Attachment: HDFS-6244.v4.patch Make Trash Interval configurable for each of the namespaces

[jira] [Commented] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

2015-08-11 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682209#comment-14682209 ] Hudson commented on HDFS-8805: -- FAILURE: Integrated in Hadoop-trunk-Commit #8284 (See

[jira] [Updated] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

2015-08-11 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8805: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 2.6.0) 2.8.0

[jira] [Commented] (HDFS-8880) NameNode metrics logging

2015-08-11 Thread Jitendra Nath Pandey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682180#comment-14682180 ] Jitendra Nath Pandey commented on HDFS-8880: +1 NameNode metrics logging

[jira] [Commented] (HDFS-8859) Improve DataNode (ReplicaMap) memory footprint to save about 45%

2015-08-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682093#comment-14682093 ] Tsz Wo Nicholas Sze commented on HDFS-8859: --- - Is the only difference between

[jira] [Commented] (HDFS-8863) The remiaing space check in BlockPlacementPolicyDefault is flawed

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682328#comment-14682328 ] Hadoop QA commented on HDFS-8863: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-8052) Move WebHdfsFileSystem into hadoop-hdfs-client

2015-08-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682247#comment-14682247 ] Tsz Wo Nicholas Sze commented on HDFS-8052: --- Agree with Haohui that RetryUtils is

[jira] [Commented] (HDFS-8865) Improve quota initialization performance

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682301#comment-14682301 ] Hadoop QA commented on HDFS-8865: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Updated] (HDFS-6407) new namenode UI, lost ability to sort columns in datanode tab

2015-08-11 Thread Chang Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated HDFS-6407: --- Attachment: HDFS-6407.011.patch added non dfs usage back to column on .11 patch. [~wheat9], [~nroberts]

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

2015-08-11 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682419#comment-14682419 ] Jing Zhao commented on HDFS-8833: - bq. I agree that a pure XAttr-based solution is simpler,

[jira] [Commented] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682433#comment-14682433 ] Hadoop QA commented on HDFS-8808: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Created] (HDFS-8886) Not able to build with 'mvn compile -Pnative'

2015-08-11 Thread Puneeth P (JIRA)
Puneeth P created HDFS-8886: --- Summary: Not able to build with 'mvn compile -Pnative' Key: HDFS-8886 URL: https://issues.apache.org/jira/browse/HDFS-8886 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-8886) Not able to build with 'mvn compile -Pnative'

2015-08-11 Thread Puneeth P (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Puneeth P updated HDFS-8886: Description: I am running into a problem where i am not able to compile the native parts of hadoop-hdfs

[jira] [Commented] (HDFS-8277) Safemode enter fails when Standby NameNode is down

2015-08-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682399#comment-14682399 ] Arpit Agarwal commented on HDFS-8277: - Hi [~surendrasingh], the setting must be

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

2015-08-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682412#comment-14682412 ] Andrew Wang commented on HDFS-8833: --- [~jingzhao] any additional comment on file header

[jira] [Updated] (HDFS-8880) NameNode metrics logging

2015-08-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8880: Status: Open (was: Patch Available) NameNode metrics logging

[jira] [Updated] (HDFS-8880) NameNode metrics logging

2015-08-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8880: Status: Patch Available (was: Open) NameNode metrics logging

[jira] [Commented] (HDFS-8886) Not able to build with 'mvn compile -Pnative'

2015-08-11 Thread Puneeth P (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682469#comment-14682469 ] Puneeth P commented on HDFS-8886: - When i try to build it, it fails with {noformat} run

[jira] [Commented] (HDFS-8865) Improve quota initialization performance

2015-08-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682371#comment-14682371 ] Kihwal Lee commented on HDFS-8865: -- The checkstyle error is for the new config key, which

[jira] [Commented] (HDFS-6407) new namenode UI, lost ability to sort columns in datanode tab

2015-08-11 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682398#comment-14682398 ] Haohui Mai commented on HDFS-6407: -- The discussion on dfs usage and the sorting of the

[jira] [Commented] (HDFS-6407) new namenode UI, lost ability to sort columns in datanode tab

2015-08-11 Thread Chang Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682404#comment-14682404 ] Chang Li commented on HDFS-6407: ok. +1(non binding) new namenode UI, lost ability to

[jira] [Updated] (HDFS-8886) Not able to build with 'mvn compile -Pnative'

2015-08-11 Thread Puneeth P (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Puneeth P updated HDFS-8886: Description: I am running into a problem where i am not able to compile the native parts of hadoop-hdfs

[jira] [Updated] (HDFS-8886) Not able to build with 'mvn compile -Pnative'

2015-08-11 Thread Puneeth P (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Puneeth P updated HDFS-8886: Description: I am running into a problem where i am not able to compile the native parts of hadoop-hdfs

[jira] [Commented] (HDFS-8277) Safemode enter fails when Standby NameNode is down

2015-08-11 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681287#comment-14681287 ] Surendra Singh Lilhore commented on HDFS-8277: -- Hi, [~arpitagarwal] and

[jira] [Updated] (HDFS-8826) Balancer may not move blocks efficiently in some cases

2015-08-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8826: -- Status: Patch Available (was: Open) Balancer may not move blocks efficiently in some

[jira] [Updated] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-08-11 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8808: Attachment: HDFS-8808-01.patch Updating the patch with a unit test and the configuration option for

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

2015-08-11 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692549#comment-14692549 ] Jing Zhao commented on HDFS-8833: - Thanks for the summary, Zhe. The new proposal looks

[jira] [Commented] (HDFS-8888) Support the volume concepts in HDFS

2015-08-11 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692587#comment-14692587 ] Haohui Mai commented on HDFS-: -- From an implementation standpoint, the concepts of

[jira] [Commented] (HDFS-8855) Webhdfs client leaks active NameNode connections

2015-08-11 Thread Bob Hansen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692805#comment-14692805 ] Bob Hansen commented on HDFS-8855: -- Does 2200 DN-NN connections seem a bit... excessive...

[jira] [Commented] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-08-11 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692803#comment-14692803 ] Ajith S commented on HDFS-8808: --- +1 dfs.image.transfer.bandwidthPerSec should not apply to

[jira] [Commented] (HDFS-8879) Quota by storage type usage incorrectly initialized upon namenode restart

2015-08-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692863#comment-14692863 ] Arpit Agarwal commented on HDFS-8879: - +1 The test failures look unrelated. Quota by

[jira] [Updated] (HDFS-8118) Delay in checkpointing Trash can leave trash for 2 intervals before deleting

2015-08-11 Thread Casey Brotherton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Casey Brotherton updated HDFS-8118: --- Attachment: HDFS-8118.001.patch This is a simplified patch addressing only the defect, and not

[jira] [Created] (HDFS-8889) Erasure Coding: cover more test situations of datanode failure during client writing

2015-08-11 Thread Li Bo (JIRA)
Li Bo created HDFS-8889: --- Summary: Erasure Coding: cover more test situations of datanode failure during client writing Key: HDFS-8889 URL: https://issues.apache.org/jira/browse/HDFS-8889 Project: Hadoop HDFS

[jira] [Commented] (HDFS-8870) Lease is leaked on write failure

2015-08-11 Thread Sangjin Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692510#comment-14692510 ] Sangjin Lee commented on HDFS-8870: --- Should this be targeted to 2.6.2? We're trying to

[jira] [Commented] (HDFS-8888) Support the volume concepts in HDFS

2015-08-11 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692590#comment-14692590 ] Jing Zhao commented on HDFS-: - +1 to have volume in HDFS. Also congrats for the jira

[jira] [Updated] (HDFS-8870) Lease is leaked on write failure

2015-08-11 Thread Sangjin Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sangjin Lee updated HDFS-8870: -- Unless the patch is ready to go and the JIRA is a critical fix, we'll defer it to 2.6.2. Let me know if you

[jira] [Updated] (HDFS-8879) Quota by storage type usage incorrectly initialized upon namenode restart

2015-08-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-8879: - Target Version/s: 2.7.2 Quota by storage type usage incorrectly initialized upon namenode restart

[jira] [Created] (HDFS-8887) Expose storage type and storage ID in BlockLocation

2015-08-11 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-8887: - Summary: Expose storage type and storage ID in BlockLocation Key: HDFS-8887 URL: https://issues.apache.org/jira/browse/HDFS-8887 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-8887) Expose storage type and storage ID in BlockLocation

2015-08-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8887: -- Attachment: HDFS-8887.001.patch Patch attached. If this goes in, I'll file another follow-on JIRA to

[jira] [Updated] (HDFS-8887) Expose storage type and storage ID in BlockLocation

2015-08-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8887: -- Status: Patch Available (was: Open) Expose storage type and storage ID in BlockLocation

[jira] [Updated] (HDFS-8880) NameNode metrics logging

2015-08-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8880: Attachment: HDFS-8880.02.patch .02 patch: * Added test cases. * Replaced {{Timer}} with

[jira] [Created] (HDFS-8888) Support the volume concepts in HDFS

2015-08-11 Thread Haohui Mai (JIRA)
Haohui Mai created HDFS-: Summary: Support the volume concepts in HDFS Key: HDFS- URL: https://issues.apache.org/jira/browse/HDFS- Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-8887) Expose storage type and storage ID in BlockLocation

2015-08-11 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692594#comment-14692594 ] Lei (Eddy) Xu commented on HDFS-8887: - LGTM, Thanks [~andrew.wang]. +1, pending

[jira] [Updated] (HDFS-8888) Support volumes in HDFS

2015-08-11 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-: - Summary: Support volumes in HDFS (was: Support the volume concepts in HDFS) Support volumes in HDFS

[jira] [Commented] (HDFS-8824) Do not use small blocks for balancing the cluster

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692764#comment-14692764 ] Hadoop QA commented on HDFS-8824: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-8826) Balancer may not move blocks efficiently in some cases

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692888#comment-14692888 ] Hadoop QA commented on HDFS-8826: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-8879) Quota by storage type usage incorrectly initialized upon namenode restart

2015-08-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692908#comment-14692908 ] Xiaoyu Yao commented on HDFS-8879: -- Thanks [~arpit99] for the review. I will hold off

[jira] [Updated] (HDFS-8826) Balancer may not move blocks efficiently in some cases

2015-08-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8826: -- Attachment: h8826_20150811.patch h8826_20150811.patch: adds a new -source option. Will

[jira] [Updated] (HDFS-8078) HDFS client gets errors trying to to connect to IPv6 DataNode

2015-08-11 Thread Nate Edel (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nate Edel updated HDFS-8078: Status: Open (was: Patch Available) HDFS client gets errors trying to to connect to IPv6 DataNode

[jira] [Commented] (HDFS-6407) new namenode UI, lost ability to sort columns in datanode tab

2015-08-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692658#comment-14692658 ] Hadoop QA commented on HDFS-6407: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

  1   2   >