[jira] [Assigned] (HDFS-7285) Erasure Coding Support inside HDFS

2015-08-18 Thread qi jun wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qi jun wang reassigned HDFS-7285: - Assignee: qi jun wang (was: Zhe Zhang) > Erasure Coding Support inside HDFS > ---

[jira] [Updated] (HDFS-8803) Move DfsClientConf to hdfs-client

2015-08-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-8803: Attachment: HDFS-8803.003.patch This patch rebases from the branch trunk commit 71aedfa. > Move DfsC

[jira] [Commented] (HDFS-8863) The remaining space check in BlockPlacementPolicyDefault is flawed

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702562#comment-14702562 ] Hadoop QA commented on HDFS-8863: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702551#comment-14702551 ] Hadoop QA commented on HDFS-8828: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8777) Erasure Coding: add tests for taking snapshots on EC files

2015-08-18 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-8777: --- Labels: test (was: ) > Erasure Coding: add tests for taking snapshots on EC files > --

[jira] [Commented] (HDFS-8884) Fail-fast check in BlockPlacementPolicyDefault#chooseTarget

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702419#comment-14702419 ] Hadoop QA commented on HDFS-8884: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Comment Edited] (HDFS-8888) Support volumes in HDFS

2015-08-18 Thread Sanjay Radia (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702393#comment-14702393 ] Sanjay Radia edited comment on HDFS- at 8/19/15 3:58 AM: - T

[jira] [Updated] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-18 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated HDFS-8828: --- Attachment: HDFS-8828.010.patch > Utilize Snapshot diff report to build copy list in distcp > -

[jira] [Commented] (HDFS-8888) Support volumes in HDFS

2015-08-18 Thread Sanjay Radia (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702393#comment-14702393 ] Sanjay Radia commented on HDFS-: There are several motivations for introducing Volu

[jira] [Commented] (HDFS-7964) Add support for async edit logging

2015-08-18 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702369#comment-14702369 ] Yi Liu commented on HDFS-7964: -- {quote} All 2.6 clusters have been running with the feature en

[jira] [Commented] (HDFS-8829) DataNode sets SO_RCVBUF explicitly is disabling tcp auto-tuning

2015-08-18 Thread He Tianyi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702365#comment-14702365 ] He Tianyi commented on HDFS-8829: - I think auto-tuning does the magic when network latency

[jira] [Commented] (HDFS-8826) Balancer may not move blocks efficiently in some cases

2015-08-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702339#comment-14702339 ] Hudson commented on HDFS-8826: -- FAILURE: Integrated in Hadoop-trunk-Commit #8319 (See [https:

[jira] [Commented] (HDFS-8867) Enable optimized block reports

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702342#comment-14702342 ] Hadoop QA commented on HDFS-8867: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8435) Support CreateFlag in WebHdfs

2015-08-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702340#comment-14702340 ] Hudson commented on HDFS-8435: -- FAILURE: Integrated in Hadoop-trunk-Commit #8319 (See [https:

[jira] [Commented] (HDFS-8826) Balancer may not move blocks efficiently in some cases

2015-08-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702341#comment-14702341 ] Tsz Wo Nicholas Sze commented on HDFS-8826: --- > Also it may be a good idea to add

[jira] [Created] (HDFS-8921) Add an option to Balancer so tha it only uses the k-most over-utilized DNs or all over-utilized DNs as sources.

2015-08-18 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-8921: - Summary: Add an option to Balancer so tha it only uses the k-most over-utilized DNs or all over-utilized DNs as sources. Key: HDFS-8921 URL: https://issues.apache.org/ji

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702333#comment-14702333 ] Hadoop QA commented on HDFS-8828: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8884) Fail-fast check in BlockPlacementPolicyDefault#chooseTarget

2015-08-18 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8884: - Attachment: HDFS-8884.002.patch > Fail-fast check in BlockPlacementPolicyDefault#chooseTarget > ---

[jira] [Updated] (HDFS-8826) Balancer may not move blocks efficiently in some cases

2015-08-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8826: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Statu

[jira] [Commented] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702329#comment-14702329 ] Hadoop QA commented on HDFS-8808: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8884) Fail-fast check in BlockPlacementPolicyDefault#chooseTarget

2015-08-18 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8884: - Attachment: (was: HDFS-8884.002.patch) > Fail-fast check in BlockPlacementPolicyDefault#chooseTarget >

[jira] [Updated] (HDFS-8884) Fail-fast check in BlockPlacementPolicyDefault#chooseTarget

2015-08-18 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8884: - Attachment: HDFS-8884.002.patch Thanks [~vinayrpet] for the review! Update the patch to address your comment. I c

[jira] [Updated] (HDFS-8826) Balancer may not move blocks efficiently in some cases

2015-08-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8826: -- Attachment: h8826_20150818.patch h8826_20150818.patch: adds periods at the end of the sen

[jira] [Commented] (HDFS-8826) Balancer may not move blocks efficiently in some cases

2015-08-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702323#comment-14702323 ] Tsz Wo Nicholas Sze commented on HDFS-8826: --- The new patch only has minor changes

[jira] [Updated] (HDFS-8920) Erasure Coding: when recovering lost blocks, logs can be too verbose and hurt performance

2015-08-18 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-8920: Assignee: Rui Li > Erasure Coding: when recovering lost blocks, logs can be too verbose and hurt > performan

[jira] [Commented] (HDFS-8920) Erasure Coding: when recovering lost blocks, logs can be too verbose and hurt performance

2015-08-18 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702305#comment-14702305 ] Rui Li commented on HDFS-8920: -- cc [~drankye] > Erasure Coding: when recovering lost blocks,

[jira] [Created] (HDFS-8920) Erasure Coding: when recovering lost blocks, logs can be too verbose and hurt performance

2015-08-18 Thread Rui Li (JIRA)
Rui Li created HDFS-8920: Summary: Erasure Coding: when recovering lost blocks, logs can be too verbose and hurt performance Key: HDFS-8920 URL: https://issues.apache.org/jira/browse/HDFS-8920 Project: Hadoop

[jira] [Commented] (HDFS-8859) Improve DataNode ReplicaMap memory footprint to save about 45%

2015-08-18 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702289#comment-14702289 ] Yi Liu commented on HDFS-8859: -- Hi [~szetszwo], do you have time to help review latest patch?

[jira] [Commented] (HDFS-8863) The remaining space check in BlockPlacementPolicyDefault is flawed

2015-08-18 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702281#comment-14702281 ] Yi Liu commented on HDFS-8863: -- Thanks [~kihwal] for updating, +1 pending Jenkins. > The rema

[jira] [Commented] (HDFS-8809) HDFS fsck reports HBase WALs files (under construction) as "CORRUPT" (missing blocks) when HBase is running

2015-08-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702257#comment-14702257 ] Jing Zhao commented on HDFS-8809: - Looks like this is an issue caused by HDFS-8215. Before,

[jira] [Assigned] (HDFS-8809) HDFS fsck reports HBase WALs files (under construction) as "CORRUPT" (missing blocks) when HBase is running

2015-08-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reassigned HDFS-8809: --- Assignee: Jing Zhao > HDFS fsck reports HBase WALs files (under construction) as "CORRUPT" (missing >

[jira] [Commented] (HDFS-8867) Enable optimized block reports

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702249#comment-14702249 ] Hadoop QA commented on HDFS-8867: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8690) LeaseRenewer should not abort DFSClient when renew fails

2015-08-18 Thread Chang Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated HDFS-8690: --- Resolution: Duplicate Status: Resolved (was: Patch Available) > LeaseRenewer should not abort DFSClien

[jira] [Updated] (HDFS-8435) Support CreateFlag in WebHdfs

2015-08-18 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HDFS-8435: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Updated] (HDFS-8435) Support CreateFlag in WebHdfs

2015-08-18 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HDFS-8435: Summary: Support CreateFlag in WebHdfs (was: createNonRecursive support needed in WebHdfsFileSystem

[jira] [Commented] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702229#comment-14702229 ] Hadoop QA commented on HDFS-8808: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8287) DFSStripedOutputStream.writeChunk should not wait for writing parity

2015-08-18 Thread Kai Sasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Sasaki updated HDFS-8287: - Attachment: HDFS-8287-HDFS-7285.04.patch > DFSStripedOutputStream.writeChunk should not wait for writing pa

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-18 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702194#comment-14702194 ] Yufei Gu commented on HDFS-8828: Hi [~jingzhao], Thank you very much for great comments. I

[jira] [Updated] (HDFS-8890) Allow admin to specify which blockpools the balancer should run on

2015-08-18 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8890: --- Status: Patch Available (was: In Progress) > Allow admin to specify which blockpools the balancer shou

[jira] [Updated] (HDFS-8890) Allow admin to specify which blockpools the balancer should run on

2015-08-18 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8890: --- Attachment: HDFS-8890-trunk-v1.patch [~mingma] Attached is v1 patch for trunk. Balancer tests pass suc

[jira] [Updated] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-18 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated HDFS-8828: --- Attachment: HDFS-8828.009.patch > Utilize Snapshot diff report to build copy list in distcp > -

[jira] [Commented] (HDFS-8855) Webhdfs client leaks active NameNode connections

2015-08-18 Thread Bob Hansen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702188#comment-14702188 ] Bob Hansen commented on HDFS-8855: -- Jitendra and [~daryn] both hypothesized that there is

[jira] [Updated] (HDFS-3745) fsck prints that it's using KSSL even when it's in fact using SPNEGO for authentication

2015-08-18 Thread Asif A Bashar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif A Bashar updated HDFS-3745: Attachment: HDFS-3745.patch Renamed SSL to SPENGO > fsck prints that it's using KSSL even when it's

[jira] [Commented] (HDFS-8855) Webhdfs client leaks active NameNode connections

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702169#comment-14702169 ] Hadoop QA commented on HDFS-8855: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-3745) fsck prints that it's using KSSL even when it's in fact using SPNEGO for authentication

2015-08-18 Thread Asif A Bashar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif A Bashar updated HDFS-3745: Release Note: Renamed SSL to SPNEGO Target Version/s: 2.7.2 Status: Patch Availa

[jira] [Commented] (HDFS-8917) Cleanup BlockInfoUnderConstruction from comments and tests

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702158#comment-14702158 ] Hadoop QA commented on HDFS-8917: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8765) Implement local block reader in libhdfspp

2015-08-18 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702143#comment-14702143 ] James Clampffer commented on HDFS-8765: --- Hi Li, The short circuit interface will be z

[jira] [Created] (HDFS-8919) Use SLF4j in ProtobufRpcEngine class

2015-08-18 Thread Xiaoyu Yao (JIRA)
Xiaoyu Yao created HDFS-8919: Summary: Use SLF4j in ProtobufRpcEngine class Key: HDFS-8919 URL: https://issues.apache.org/jira/browse/HDFS-8919 Project: Hadoop HDFS Issue Type: Improvement Af

[jira] [Updated] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-08-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8808: Attachment: HDFS-8808.04.patch Thanks [~atm] for the review! Updating the patch to address the comments (not

[jira] [Updated] (HDFS-3745) fsck prints that it's using KSSL even when it's in fact using SPNEGO for authentication

2015-08-18 Thread Asif A Bashar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif A Bashar updated HDFS-3745: Status: Open (was: Patch Available) > fsck prints that it's using KSSL even when it's in fact using

[jira] [Updated] (HDFS-3745) fsck prints that it's using KSSL even when it's in fact using SPNEGO for authentication

2015-08-18 Thread Asif A Bashar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif A Bashar updated HDFS-3745: Fix Version/s: 2.7.2 Status: Patch Available (was: Open) > fsck prints that it's using KS

[jira] [Updated] (HDFS-3745) fsck prints that it's using KSSL even when it's in fact using SPNEGO for authentication

2015-08-18 Thread Asif A Bashar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif A Bashar updated HDFS-3745: Assignee: (was: Asif A Bashar) > fsck prints that it's using KSSL even when it's in fact using SP

[jira] [Assigned] (HDFS-3745) fsck prints that it's using KSSL even when it's in fact using SPNEGO for authentication

2015-08-18 Thread Asif A Bashar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif A Bashar reassigned HDFS-3745: --- Assignee: Asif A Bashar > fsck prints that it's using KSSL even when it's in fact using SPNEGO

[jira] [Updated] (HDFS-8486) DN startup may cause severe data loss

2015-08-18 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8486: Attachment: HDFS-8486-branch-2.6.addendum.patch Addendum patch to fix an issue introduced by conflict

[jira] [Commented] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-08-18 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702088#comment-14702088 ] Aaron T. Myers commented on HDFS-8808: -- Hey Zhe, latest patch looks pretty good to me.

[jira] [Commented] (HDFS-8765) Implement local block reader in libhdfspp

2015-08-18 Thread li zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702087#comment-14702087 ] li zhang commented on HDFS-8765: Hi [~James Clampffer] I'm working on a C++ project whi

[jira] [Updated] (HDFS-8863) The remaining space check in BlockPlacementPolicyDefault is flawed

2015-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8863: - Attachment: HDFS-8863.v3.patch > The remaining space check in BlockPlacementPolicyDefault is flawed > -

[jira] [Commented] (HDFS-8776) Decom manager should not be active on standby

2015-08-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702079#comment-14702079 ] Daryn Sharp commented on HDFS-8776: --- The change we used is just skipping the costly scan

[jira] [Updated] (HDFS-8435) createNonRecursive support needed in WebHdfsFileSystem to support HBase

2015-08-18 Thread Jakob Homan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-8435: -- Status: Patch Available (was: Open) > createNonRecursive support needed in WebHdfsFileSystem to support

[jira] [Updated] (HDFS-8435) createNonRecursive support needed in WebHdfsFileSystem to support HBase

2015-08-18 Thread Jakob Homan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-8435: -- Status: Open (was: Patch Available) > createNonRecursive support needed in WebHdfsFileSystem to support

[jira] [Commented] (HDFS-8823) Move replication factor into individual blocks

2015-08-18 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702035#comment-14702035 ] Haohui Mai commented on HDFS-8823: -- The tests look flaky. I ran the tests and all passed l

[jira] [Updated] (HDFS-8916) add nonDfsUsedSpace back to Name Node UI column

2015-08-18 Thread Chang Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated HDFS-8916: --- Status: Patch Available (was: Open) > add nonDfsUsedSpace back to Name Node UI column > --

[jira] [Commented] (HDFS-8855) Webhdfs client leaks active NameNode connections

2015-08-18 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702025#comment-14702025 ] Haohui Mai commented on HDFS-8855: -- I don't think this is correct as ugi / dt should be co

[jira] [Commented] (HDFS-8435) createNonRecursive support needed in WebHdfsFileSystem to support HBase

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702023#comment-14702023 ] Hadoop QA commented on HDFS-8435: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8855) Webhdfs client leaks active NameNode connections

2015-08-18 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-8855: Status: Patch Available (was: Open) > Webhdfs client leaks active NameNode connections > ---

[jira] [Updated] (HDFS-8855) Webhdfs client leaks active NameNode connections

2015-08-18 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-8855: Attachment: HDFS-8855.1.patch Made a patch for review. Idea's been to avoid creating brand new DFSCli

[jira] [Commented] (HDFS-7285) Erasure Coding Support inside HDFS

2015-08-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702000#comment-14702000 ] Zhe Zhang commented on HDFS-7285: - Given the feedbacks from the common-dev [discussion | h

[jira] [Commented] (HDFS-8867) Enable optimized block reports

2015-08-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701964#comment-14701964 ] Jing Zhao commented on HDFS-8867: - Thanks for the fix, Daryn. +1 pending Jenkins. > Enable

[jira] [Commented] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

2015-08-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701956#comment-14701956 ] Zhe Zhang commented on HDFS-8808: - The patch still applies against trunk. Triggering Jenkin

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701924#comment-14701924 ] Zhe Zhang commented on HDFS-8909: - Thanks Jing for the work! Most of the patch is clean ref

[jira] [Commented] (HDFS-8870) Lease is leaked on write failure

2015-08-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701862#comment-14701862 ] Daryn Sharp commented on HDFS-8870: --- A combination of errors occurred. Pipelines were fr

[jira] [Updated] (HDFS-8913) Documentation clarity regarding Secondary node, Checkpoint node & Backup node

2015-08-18 Thread Ravindra Babu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Babu updated HDFS-8913: Summary: Documentation clarity regarding Secondary node, Checkpoint node & Backup node (was: Documen

[jira] [Commented] (HDFS-7964) Add support for async edit logging

2015-08-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701823#comment-14701823 ] Daryn Sharp commented on HDFS-7964: --- [~hitliuyi], I'm sure I'll need to rebase. The batc

[jira] [Updated] (HDFS-8867) Enable optimized block reports

2015-08-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-8867: -- Status: Patch Available (was: Open) > Enable optimized block reports > -- >

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701789#comment-14701789 ] Hadoop QA commented on HDFS-8828: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8867) Enable optimized block reports

2015-08-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-8867: -- Attachment: HDFS-8867.patch Yes, the enum is lazy initialized so it wasn't advertised. Here's the patch

[jira] [Updated] (HDFS-8867) Enable optimized block reports

2015-08-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-8867: -- Target Version/s: 2.7.2 Fix Version/s: (was: 2.7.2) > Enable optimized block reports > ---

[jira] [Comment Edited] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701760#comment-14701760 ] Jing Zhao edited comment on HDFS-8828 at 8/18/15 6:54 PM: -- Thanks

[jira] [Comment Edited] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701760#comment-14701760 ] Jing Zhao edited comment on HDFS-8828 at 8/18/15 6:42 PM: -- Thanks

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701760#comment-14701760 ] Jing Zhao commented on HDFS-8828: - Thanks for updating the patch, Yufei! Some further comme

[jira] [Updated] (HDFS-8435) createNonRecursive support needed in WebHdfsFileSystem to support HBase

2015-08-18 Thread Jakob Homan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-8435: -- Attachment: HDFS-8435.005.patch One last patch that suppresses the warning rather than accepting it. Aga

[jira] [Updated] (HDFS-8435) createNonRecursive support needed in WebHdfsFileSystem to support HBase

2015-08-18 Thread Jakob Homan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-8435: -- Status: Patch Available (was: Open) > createNonRecursive support needed in WebHdfsFileSystem to support

[jira] [Updated] (HDFS-8435) createNonRecursive support needed in WebHdfsFileSystem to support HBase

2015-08-18 Thread Jakob Homan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-8435: -- Status: Open (was: Patch Available) > createNonRecursive support needed in WebHdfsFileSystem to support

[jira] [Updated] (HDFS-8913) Documentation correction regarding Secondary node, Checkpoint node & Backup node

2015-08-18 Thread Ravindra Babu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Babu updated HDFS-8913: Description: I checked with many people and almost all of them are confused on responsibilities of S

[jira] [Commented] (HDFS-8918) Convert BlockUnderConstructionFeature#replicas form list to array

2015-08-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701741#comment-14701741 ] Zhe Zhang commented on HDFS-8918: - [~jingzhao] Just saw that your HDFS-8909 patch made this

[jira] [Resolved] (HDFS-8835) Convert BlockInfoUnderConstruction as an interface

2015-08-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8835. - Resolution: Invalid HDFS-8801 has converted {{BlockInfoUC}} as a feature. > Convert BlockInfoUnderConstruc

[jira] [Commented] (HDFS-8863) The remaining space check in BlockPlacementPolicyDefault is flawed

2015-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701703#comment-14701703 ] Kihwal Lee commented on HDFS-8863: -- bq. I am thinking we can do as following: do storage t

[jira] [Commented] (HDFS-8801) Convert BlockInfoUnderConstruction as a feature

2015-08-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701715#comment-14701715 ] Zhe Zhang commented on HDFS-8801: - Thanks Jing again for the work. I just finished looking

[jira] [Created] (HDFS-8918) Convert BlockUnderConstructionFeature#replicas form list to array

2015-08-18 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8918: --- Summary: Convert BlockUnderConstructionFeature#replicas form list to array Key: HDFS-8918 URL: https://issues.apache.org/jira/browse/HDFS-8918 Project: Hadoop HDFS Is

[jira] [Updated] (HDFS-8917) Cleanup BlockInfoUnderConstruction from comments and tests

2015-08-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8917: Attachment: HDFS-8917.00.patch > Cleanup BlockInfoUnderConstruction from comments and tests > ---

[jira] [Updated] (HDFS-8917) Cleanup BlockInfoUnderConstruction from comments and tests

2015-08-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8917: Status: Patch Available (was: Open) > Cleanup BlockInfoUnderConstruction from comments and tests > -

[jira] [Created] (HDFS-8917) Cleanup BlockInfoUnderConstruction from comments and tests

2015-08-18 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8917: --- Summary: Cleanup BlockInfoUnderConstruction from comments and tests Key: HDFS-8917 URL: https://issues.apache.org/jira/browse/HDFS-8917 Project: Hadoop HDFS Issue Type

[jira] [Commented] (HDFS-8862) BlockManager#excessReplicateMap should use a HashMap

2015-08-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701660#comment-14701660 ] Hudson commented on HDFS-8862: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See [htt

[jira] [Commented] (HDFS-8895) Remove deprecated BlockStorageLocation APIs

2015-08-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701664#comment-14701664 ] Hudson commented on HDFS-8895: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See [htt

[jira] [Commented] (HDFS-8278) HDFS Balancer should consider remaining storage % when checking for under-utilized machines

2015-08-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701668#comment-14701668 ] Hudson commented on HDFS-8278: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See [htt

[jira] [Commented] (HDFS-8713) Convert DatanodeDescriptor to use SLF4J logging

2015-08-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701666#comment-14701666 ] Hudson commented on HDFS-8713: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See [htt

[jira] [Commented] (HDFS-8792) BlockManager#postponedMisreplicatedBlocks should use a LightWeightHashSet to save memory

2015-08-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701661#comment-14701661 ] Hudson commented on HDFS-8792: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See [htt

[jira] [Commented] (HDFS-8883) NameNode Metrics : Add FSNameSystem lock Queue Length

2015-08-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701667#comment-14701667 ] Hudson commented on HDFS-8883: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See [htt

[jira] [Commented] (HDFS-6407) Add sorting and pagination in the datanode tab of the NN Web UI

2015-08-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701659#comment-14701659 ] Hudson commented on HDFS-6407: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See [htt

[jira] [Commented] (HDFS-8880) NameNode metrics logging

2015-08-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701665#comment-14701665 ] Hudson commented on HDFS-8880: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See [htt

  1   2   >