[jira] [Commented] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037433#comment-15037433 ] Mingliang Liu commented on HDFS-9436: - The failing tests seem unrelated. Specially, {{

[jira] [Commented] (HDFS-9083) Replication violates block placement policy.

2015-12-02 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037436#comment-15037436 ] Xiaoyu Yao commented on HDFS-9083: -- Thanks [~brahmareddy] for the explanation. That helps

[jira] [Commented] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037435#comment-15037435 ] Mingliang Liu commented on HDFS-9484: - The failing tests seem unrelated. > NNThroughpu

[jira] [Commented] (HDFS-8705) BlockStoragePolicySuite uses equalsIgnoreCase for name lookup, won't work in all locales

2015-12-02 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037423#comment-15037423 ] Brahma Reddy Battula commented on HDFS-8705: As I mentioned in the earlier {{St

[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade

2015-12-02 Thread Phil Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HDFS-9500: Status: Patch Available (was: Open) > datanodesSoftwareVersions map may counting wrong when rolling upgrade

[jira] [Commented] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2015-12-02 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037400#comment-15037400 ] Brahma Reddy Battula commented on HDFS-9294: [~szetszwo] thanks a lot for revie

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037393#comment-15037393 ] Chris Trezzo commented on HDFS-8791: As a side note: I see that there are already multi

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037383#comment-15037383 ] Chris Trezzo commented on HDFS-8791: [~kihwal] For the sake of completeness, we upgrade

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Attachment: test-node-upgrade.txt > block ID-based DN storage layout can be very slow for datanode on e

[jira] [Commented] (HDFS-9083) Replication violates block placement policy.

2015-12-02 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037361#comment-15037361 ] Brahma Reddy Battula commented on HDFS-9083: [~xyao] thanks for pointing same..

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Attachment: test-node-upgrade.txt > block ID-based DN storage layout can be very slow for datanode on e

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8791: --- Attachment: (was: test-node-upgrade.txt) > block ID-based DN storage layout can be very slow for da

[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade

2015-12-02 Thread Phil Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HDFS-9500: Description: While rolling upgrading, namenode's website overview will report there are two versions datanod

[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade

2015-12-02 Thread Phil Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HDFS-9500: Description: While rolling upgrading, namenode's website overview will report there are two versions datanod

[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade

2015-12-02 Thread Phil Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HDFS-9500: Attachment: 9500-v1.patch > datanodesSoftwareVersions map may counting wrong when rolling upgrade > -

[jira] [Commented] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2015-12-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037339#comment-15037339 ] Hudson commented on HDFS-9294: -- ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #659 (See [htt

[jira] [Created] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade

2015-12-02 Thread Phil Yang (JIRA)
Phil Yang created HDFS-9500: --- Summary: datanodesSoftwareVersions map may counting wrong when rolling upgrade Key: HDFS-9500 URL: https://issues.apache.org/jira/browse/HDFS-9500 Project: Hadoop HDFS

[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade

2015-12-02 Thread Phil Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HDFS-9500: Description: While rolling upgrading, namenode's website overview will report there are two versions datanod

[jira] [Commented] (HDFS-9083) Replication violates block placement policy.

2015-12-02 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037293#comment-15037293 ] Xiaoyu Yao commented on HDFS-9083: -- The 2.7 patch caused failure of TestBalancer#testBalan

[jira] [Commented] (HDFS-9469) DiskBalancer : Add Planner

2015-12-02 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037290#comment-15037290 ] Anu Engineer commented on HDFS-9469: [~szetszwo], Thanks for the detailed review. Here

[jira] [Commented] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-12-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037287#comment-15037287 ] Hadoop QA commented on HDFS-9484: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Updated] (HDFS-9498) Move code that tracks orphan blocks to BlockManagerSafeMode

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9498: Attachment: HDFS-9498.000.patch > Move code that tracks orphan blocks to BlockManagerSafeMode > -

[jira] [Updated] (HDFS-9498) Move code that tracks orphan blocks to BlockManagerSafeMode

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9498: Status: Patch Available (was: Open) > Move code that tracks orphan blocks to BlockManagerSafeMode >

[jira] [Resolved] (HDFS-9496) Erasure coding: an erasure codec throughput benchmark tool

2015-12-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-9496. - Resolution: Duplicate Thanks Rui for clarifying this. > Erasure coding: an erasure codec throughput benchm

[jira] [Commented] (HDFS-9496) Erasure coding: an erasure codec throughput benchmark tool

2015-12-02 Thread Hui Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037240#comment-15037240 ] Hui Zheng commented on HDFS-9496: - HADOOP-11588 is good enough to me.Thanks [~lirui]. > Er

[jira] [Commented] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-12-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037220#comment-15037220 ] Hadoop QA commented on HDFS-9436: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HDFS-9448) Enable valgrind for libhdfspp unit tests

2015-12-02 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037181#comment-15037181 ] Allen Wittenauer commented on HDFS-9448: bq. Does that seem idiomatically in line w

[jira] [Updated] (HDFS-9498) Move code that tracks orphan blocks to BlockManagerSafeMode

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9498: Summary: Move code that tracks orphan blocks to BlockManagerSafeMode (was: Move BlockManager#numberO

[jira] [Commented] (HDFS-8705) BlockStoragePolicySuite uses equalsIgnoreCase for name lookup, won't work in all locales

2015-12-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037123#comment-15037123 ] Tsz Wo Nicholas Sze commented on HDFS-8705: --- I agree that equalsIgnoreCase is saf

[jira] [Updated] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9484: Attachment: HDFS-9484.001.patch Thanks for the comment, [~shv]. I totally agree with you that the co

[jira] [Updated] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2015-12-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-9294: -- Resolution: Fixed Fix Version/s: 2.7.2 Status: Resolved (was: Patch Ava

[jira] [Commented] (HDFS-8647) Abstract BlockManager's rack policy into BlockPlacementPolicy

2015-12-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037078#comment-15037078 ] Xiao Chen commented on HDFS-8647: - Locally applied both patches to corresponding branch and

[jira] [Commented] (HDFS-8647) Abstract BlockManager's rack policy into BlockPlacementPolicy

2015-12-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037076#comment-15037076 ] Xiao Chen commented on HDFS-8647: - Summary from branch-2.7 backport to branch-2.6 below: -

[jira] [Updated] (HDFS-8647) Abstract BlockManager's rack policy into BlockPlacementPolicy

2015-12-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-8647: Attachment: HDFS-8647-branch26.patch > Abstract BlockManager's rack policy into BlockPlacementPolicy > --

[jira] [Commented] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-12-02 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037072#comment-15037072 ] Konstantin Shvachko commented on HDFS-9436: --- Three is better. +1 > Make NNThroug

[jira] [Commented] (HDFS-8831) Trash Support for deletion in HDFS encryption zone

2015-12-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037068#comment-15037068 ] Hadoop QA commented on HDFS-8831: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-12-02 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037067#comment-15037067 ] Konstantin Shvachko commented on HDFS-9484: --- Yes indeed {{nrBlocks == blocks.size

[jira] [Commented] (HDFS-8705) BlockStoragePolicySuite uses equalsIgnoreCase for name lookup, won't work in all locales

2015-12-02 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037061#comment-15037061 ] Walter Su commented on HDFS-8705: - Hi [~ste...@apache.org]. I think {{String.equalsIgnoreCa

[jira] [Commented] (HDFS-9496) Erasure coding: an erasure codec throughput benchmark tool

2015-12-02 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037062#comment-15037062 ] Rui Li commented on HDFS-9496: -- [~huizane] and [~zhz] - We already have HADOOP-11588 to implem

[jira] [Commented] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2015-12-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037053#comment-15037053 ] Hudson commented on HDFS-9294: -- FAILURE: Integrated in Hadoop-trunk-Commit #8911 (See [https:

[jira] [Commented] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2015-12-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037035#comment-15037035 ] Tsz Wo Nicholas Sze commented on HDFS-9294: --- Thanks [~cnauroth]. This sounds not

[jira] [Commented] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2015-12-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037033#comment-15037033 ] Hadoop QA commented on HDFS-9294: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2015-12-02 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037004#comment-15037004 ] Chris Nauroth commented on HDFS-9294: - Hi [~szetszwo]. That relates to the recent spli

[jira] [Commented] (HDFS-8647) Abstract BlockManager's rack policy into BlockPlacementPolicy

2015-12-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037001#comment-15037001 ] Xiao Chen commented on HDFS-8647: - Sorry about the flooding comments here... Talked with Zh

[jira] [Commented] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2015-12-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036993#comment-15036993 ] Tsz Wo Nicholas Sze commented on HDFS-9294: --- [~aw], why there were only 49 tests

[jira] [Commented] (HDFS-9469) DiskBalancer : Add Planner

2015-12-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036990#comment-15036990 ] Tsz Wo Nicholas Sze commented on HDFS-9469: --- I think we need to change the data m

[jira] [Updated] (HDFS-8647) Abstract BlockManager's rack policy into BlockPlacementPolicy

2015-12-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-8647: Attachment: HDFS-8647-branch27.patch > Abstract BlockManager's rack policy into BlockPlacementPolicy > --

[jira] [Updated] (HDFS-8647) Abstract BlockManager's rack policy into BlockPlacementPolicy

2015-12-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-8647: Attachment: (was: HDFS-8647-branch26.patch) > Abstract BlockManager's rack policy into BlockPlacementPoli

[jira] [Updated] (HDFS-8647) Abstract BlockManager's rack policy into BlockPlacementPolicy

2015-12-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-8647: Attachment: (was: HDFS-8647-branch27.patch) > Abstract BlockManager's rack policy into BlockPlacementPoli

[jira] [Commented] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2015-12-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036953#comment-15036953 ] Hadoop QA commented on HDFS-9294: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Updated] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9436: Attachment: HDFS-9436.002.patch Thank you very much [~shv] for your review. Sending a couple of block

[jira] [Commented] (HDFS-9267) TestDiskError should get stored replicas through FsDatasetTestUtils.

2015-12-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036943#comment-15036943 ] Hadoop QA commented on HDFS-9267: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-12-02 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036932#comment-15036932 ] Konstantin Shvachko commented on HDFS-9436: --- {{numOpsRequired = 10}} and the {{nu

[jira] [Commented] (HDFS-9469) DiskBalancer : Add Planner

2015-12-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036918#comment-15036918 ] Hadoop QA commented on HDFS-9469: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Updated] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2015-12-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-9294: -- Hadoop Flags: Reviewed +1 the new patch looks good. > DFSClient deadlock when close fil

[jira] [Updated] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2015-12-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-9294: -- Attachment: HDFS-9294-002.patch Reposting the trunk patch to trigger Jenkins. > DFSClien

[jira] [Commented] (HDFS-9129) Move the safemode block count into BlockManager

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036894#comment-15036894 ] Mingliang Liu commented on HDFS-9129: - {noformat} Step 16 : RUN cd /usr/share/maven/lib

[jira] [Commented] (HDFS-9429) Tests in TestDFSAdminWithHA intermittently fail with EOFException

2015-12-02 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036881#comment-15036881 ] Colin Patrick McCabe commented on HDFS-9429: Thanks for the reminder, [~zhz].

[jira] [Commented] (HDFS-9144) Refactor libhdfs into stateful/ephemeral objects

2015-12-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036774#comment-15036774 ] Hadoop QA commented on HDFS-9144: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HDFS-9129) Move the safemode block count into BlockManager

2015-12-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036772#comment-15036772 ] Hadoop QA commented on HDFS-9129: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Updated] (HDFS-9499) Fix typos in DFSAdmin.java

2015-12-02 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HDFS-9499: --- Assignee: Nicole Pazmany > Fix typos in DFSAdmin.java > -- > >

[jira] [Commented] (HDFS-9417) Clean up the RAT warnings in the HDFS-8707 branch.

2015-12-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036758#comment-15036758 ] Hadoop QA commented on HDFS-9417: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HDFS-9490) MiniDFSCluster should change block generation stamp via FsDatasetTestUtils

2015-12-02 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036754#comment-15036754 ] Tony Wu commented on HDFS-9490: --- The failed tests are not related to this patch. Only {{Test

[jira] [Created] (HDFS-9499) Fix typos in DFSAdmin.java

2015-12-02 Thread Arpit Agarwal (JIRA)
Arpit Agarwal created HDFS-9499: --- Summary: Fix typos in DFSAdmin.java Key: HDFS-9499 URL: https://issues.apache.org/jira/browse/HDFS-9499 Project: Hadoop HDFS Issue Type: Bug Componen

[jira] [Comment Edited] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036721#comment-15036721 ] Kihwal Lee edited comment on HDFS-8791 at 12/2/15 10:06 PM: As

[jira] [Commented] (HDFS-9441) Do not construct path string when choosing block placement targets

2015-12-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036728#comment-15036728 ] Tsz Wo Nicholas Sze commented on HDFS-9441: --- In the past, the raid block placemen

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036721#comment-15036721 ] Kihwal Lee commented on HDFS-8791: -- As for making it run parallel, we could do it in {{Dat

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036716#comment-15036716 ] Kihwal Lee commented on HDFS-8791: -- bq. How long did it take to scan all the block pools

[jira] [Updated] (HDFS-8831) Trash Support for deletion in HDFS encryption zone

2015-12-02 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-8831: - Attachment: HDFS-8831.03.patch Update patch v03 based on [~arpit99]'s feedback. Please review, thanks! bq.

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036701#comment-15036701 ] Chris Trezzo commented on HDFS-8791: Thanks [~kihwal] for the testing info! I definitel

[jira] [Commented] (HDFS-9144) Refactor libhdfs into stateful/ephemeral objects

2015-12-02 Thread Bob Hansen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036692#comment-15036692 ] Bob Hansen commented on HDFS-9144: -- Thanks for rebasing those changes, [~James Clampffer].

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Vinod Kumar Vavilapalli (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1503#comment-1503 ] Vinod Kumar Vavilapalli commented on HDFS-8791: --- Agree with Kihwal that the r

[jira] [Created] (HDFS-9498) Move BlockManager#numberOfBytesInFutureBlocks to BlockManagerSafeMode

2015-12-02 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-9498: --- Summary: Move BlockManager#numberOfBytesInFutureBlocks to BlockManagerSafeMode Key: HDFS-9498 URL: https://issues.apache.org/jira/browse/HDFS-9498 Project: Hadoop HDFS

[jira] [Commented] (HDFS-9448) Enable valgrind for libhdfspp unit tests

2015-12-02 Thread Bob Hansen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036624#comment-15036624 ] Bob Hansen commented on HDFS-9448: -- I would very much like [~aw]'s feedback (at least at a

[jira] [Updated] (HDFS-9129) Move the safemode block count into BlockManager

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9129: Status: Patch Available (was: Reopened) > Move the safemode block count into BlockManager >

[jira] [Reopened] (HDFS-9129) Move the safemode block count into BlockManager

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reopened HDFS-9129: - > Move the safemode block count into BlockManager > --- > >

[jira] [Commented] (HDFS-8831) Trash Support for deletion in HDFS encryption zone

2015-12-02 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036603#comment-15036603 ] Xiaoyu Yao commented on HDFS-8831: -- Thanks [~arpit99] for the review and the detail feedba

[jira] [Commented] (HDFS-9448) Enable valgrind for libhdfspp unit tests

2015-12-02 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036601#comment-15036601 ] James Clampffer commented on HDFS-9448: --- This looks good to me, but I'd like to get a

[jira] [Commented] (HDFS-9441) Do not construct path string when choosing block placement targets

2015-12-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036566#comment-15036566 ] Kihwal Lee commented on HDFS-9441: -- bq. If the path argument is strictly for logging in bl

[jira] [Commented] (HDFS-9497) libhdfs++: move lib/proto/cpp_helpers to third-party since it won't have an ASF license

2015-12-02 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036561#comment-15036561 ] James Clampffer commented on HDFS-9497: --- Seems straightforward to me. +1. I plan on

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036547#comment-15036547 ] Kihwal Lee commented on HDFS-8791: -- This is not regarding the release planning, but the pa

[jira] [Commented] (HDFS-9144) Refactor libhdfs into stateful/ephemeral objects

2015-12-02 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036541#comment-15036541 ] James Clampffer commented on HDFS-9144: --- And by HDFS-9559 I meant HDFS-9359. > Refac

[jira] [Updated] (HDFS-9144) Refactor libhdfs into stateful/ephemeral objects

2015-12-02 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Clampffer updated HDFS-9144: -- Attachment: HDFS-9144.HDFS-8707.006.patch I took a stab at rebasing this to the head to get it se

[jira] [Assigned] (HDFS-9417) Clean up the RAT warnings in the HDFS-8707 branch.

2015-12-02 Thread Bob Hansen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bob Hansen reassigned HDFS-9417: Assignee: Bob Hansen (was: Xiaobing Zhou) > Clean up the RAT warnings in the HDFS-8707 branch. > --

[jira] [Updated] (HDFS-9417) Clean up the RAT warnings in the HDFS-8707 branch.

2015-12-02 Thread Bob Hansen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bob Hansen updated HDFS-9417: - Attachment: HDFS-9417.HDFS-8707.002.patch New patch: should take care of everything except cpp_helper.h, wh

[jira] [Commented] (HDFS-9417) Clean up the RAT warnings in the HDFS-8707 branch.

2015-12-02 Thread Bob Hansen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036508#comment-15036508 ] Bob Hansen commented on HDFS-9417: -- Of course. Thank you. Is there a format that the pat

[jira] [Updated] (HDFS-9417) Clean up the RAT warnings in the HDFS-8707 branch.

2015-12-02 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-9417: --- Status: Patch Available (was: Open) Clicking submit patch so that jenkins fires off. > Clean

[jira] [Commented] (HDFS-9430) waitforloadingfsimage() can be removed since checknnstartup() already ensures image loaded and namenode started.

2015-12-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036479#comment-15036479 ] Tsz Wo Nicholas Sze commented on HDFS-9430: --- > My main question was around the or

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036476#comment-15036476 ] Andrew Wang commented on HDFS-8791: --- [~wheat9] [~kihwal] are y'all okay with [~ctrezzo]'s

[jira] [Commented] (HDFS-9430) waitforloadingfsimage() can be removed since checknnstartup() already ensures image loaded and namenode started.

2015-12-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036457#comment-15036457 ] Tsz Wo Nicholas Sze commented on HDFS-9430: --- waitForLoadingFSImage() and checkNNS

[jira] [Commented] (HDFS-9490) MiniDFSCluster should change block generation stamp via FsDatasetTestUtils

2015-12-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036455#comment-15036455 ] Hadoop QA commented on HDFS-9490: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036442#comment-15036442 ] Haohui Mai commented on HDFS-8791: -- bq. I've also only seen one or two upgrade issues caus

[jira] [Updated] (HDFS-9469) DiskBalancer : Add Planner

2015-12-02 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-9469: --- Attachment: HDFS-9469-HDFS-1312.002.patch * Fix the white space issue. * Ignored the checkstyle issues

[jira] [Updated] (HDFS-1312) Re-balance disks within a Datanode

2015-12-02 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-1312: --- Attachment: HDFS-9469-HDFS-1312.002.patch * Fix the white space issue. * Ignored the checkstyle issues

[jira] [Updated] (HDFS-1312) Re-balance disks within a Datanode

2015-12-02 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-1312: --- Attachment: (was: HDFS-9469-HDFS-1312.002.patch) > Re-balance disks within a Datanode > ---

[jira] [Updated] (HDFS-9267) TestDiskError should get stored replicas through FsDatasetTestUtils.

2015-12-02 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-9267: Attachment: HDFS-9267.06.patch Hi, [~cmccabe] Thanks a lot for the reviews and the understandings.

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2015-12-02 Thread Joep Rottinghuis (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036421#comment-15036421 ] Joep Rottinghuis commented on HDFS-8791: Thanks [~ctrezzo], that seems like a reaso

[jira] [Commented] (HDFS-9441) Do not construct path string when choosing block placement targets

2015-12-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036419#comment-15036419 ] Tsz Wo Nicholas Sze commented on HDFS-9441: --- [~kihwal], you seems having some goo

[jira] [Commented] (HDFS-9430) waitforloadingfsimage() can be removed since checknnstartup() already ensures image loaded and namenode started.

2015-12-02 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036414#comment-15036414 ] Haohui Mai commented on HDFS-9430: -- It's okay to remove the {{waitForLoadingFsImage()}} fu

[jira] [Commented] (HDFS-9417) Clean up the RAT warnings in the HDFS-8707 branch.

2015-12-02 Thread Bob Hansen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036407#comment-15036407 ] Bob Hansen commented on HDFS-9417: -- [~aw] - any idea why this isn't kicking off a Jenkins

  1   2   >