[jira] [Updated] (HDFS-9867) Missing block exception should carry locatedBlocks information

2016-02-26 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9867: Component/s: erasure-coding > Missing block exception should carry locatedBlocks information > --

[jira] [Created] (HDFS-9867) Missing block exception should carry locatedBlocks information

2016-02-26 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-9867: --- Summary: Missing block exception should carry locatedBlocks information Key: HDFS-9867 URL: https://issues.apache.org/jira/browse/HDFS-9867 Project: Hadoop HDFS

[jira] [Updated] (HDFS-9867) Missing block exception should carry locatedBlocks information

2016-02-26 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9867: Attachment: HDFS-9867.000.patch > Missing block exception should carry locatedBlocks information > --

[jira] [Updated] (HDFS-9867) Missing block exception should carry locatedBlocks information

2016-02-26 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9867: Status: Patch Available (was: Open) > Missing block exception should carry locatedBlocks information

[jira] [Commented] (HDFS-9579) Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level

2016-02-26 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170428#comment-15170428 ] Mingliang Liu commented on HDFS-9579: - The latest patch looks pretty good to me overall

[jira] [Commented] (HDFS-9867) Missing block exception should carry locatedBlocks information

2016-02-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15171253#comment-15171253 ] Mingliang Liu commented on HDFS-9867: - Thank you [~jingzhao] for your reviewing and com

[jira] [Updated] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: HDFS-EC-file-flush-sync-design-version2.1.pdf Attached the v2.1 design doc, based on the

[jira] [Commented] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177055#comment-15177055 ] Mingliang Liu commented on HDFS-7661: - Thanks for the comments, [~sinall]! 1. Let's se

[jira] [Commented] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178387#comment-15178387 ] Mingliang Liu commented on HDFS-7661: - Thanks for the further review, [~demongaorui].

[jira] [Commented] (HDFS-9579) Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level

2016-03-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178651#comment-15178651 ] Mingliang Liu commented on HDFS-9579: - Thanks for updating the patch, [~mingma]! I like

[jira] [Commented] (HDFS-9579) Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level

2016-03-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178871#comment-15178871 ] Mingliang Liu commented on HDFS-9579: - +1 (non-binding) pending on Jenkins. > Provide

[jira] [Commented] (HDFS-9817) Use SLF4J in new classes

2016-03-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15181445#comment-15181445 ] Mingliang Liu commented on HDFS-9817: - Hi [~anu], I see you changed log level at severa

[jira] [Updated] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-08 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: (was: HDFS-EC-file-flush-sync-design-version2.0.pdf) > Erasure coding: support hflush

[jira] [Updated] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-08 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: (was: HDFS-EC-file-flush-sync-design-version2.1.pdf) > Erasure coding: support hflush

[jira] [Updated] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-08 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: HDFS-EC-file-flush-sync-design-version2.2.pdf Per offline discussion with [~jingzhao], [~

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189869#comment-15189869 ] Mingliang Liu commented on HDFS-9405: - Nits: 0. {{keyNames}} and {{kp}} seem final. 1.

[jira] [Commented] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190241#comment-15190241 ] Mingliang Liu commented on HDFS-9941: - This proposal makes sense to me. Thanks for repo

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190271#comment-15190271 ] Mingliang Liu commented on HDFS-9405: - Thanks for updating the patch, [~xiaochen]. I w

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Attachment: HDFS-10175.000.patch Thanks for creating this jira, [~venkateshrin]. The v0 patch is

[jira] [Work started] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-10175 started by Mingliang Liu. > add per-operation stats to FileSystem.Statistics >

[jira] [Commented] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200103#comment-15200103 ] Mingliang Liu commented on HDFS-7661: - According to the current design, we need to brea

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205086#comment-15205086 ] Mingliang Liu commented on HDFS-10175: -- Thanks for your comment, [~andrew.wang]. I wa

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Attachment: HDFS-10175.001.patch > add per-operation stats to FileSystem.Statistics >

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Status: Patch Available (was: In Progress) > add per-operation stats to FileSystem.Statistics > -

[jira] [Updated] (HDFS-10192) Namenode safemode not coming out during failover

2016-03-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10192: - Component/s: namenode > Namenode safemode not coming out during failover > ---

[jira] [Commented] (HDFS-10192) Namenode safemode not coming out during failover

2016-03-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207486#comment-15207486 ] Mingliang Liu commented on HDFS-10192: -- Thanks for reporting this, [~brahmareddy]. I

[jira] [Updated] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: HDFS-EC-file-flush-sync-design-v20160323.pdf > [umbrella] support hflush and hsync for er

[jira] [Updated] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: (was: HDFS-EC-file-flush-sync-design-version2.2.pdf) > [umbrella] support hflush and

[jira] [Created] (HDFS-10201) Implement undo log in parity datanode for hflush operations

2016-03-23 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-10201: Summary: Implement undo log in parity datanode for hflush operations Key: HDFS-10201 URL: https://issues.apache.org/jira/browse/HDFS-10201 Project: Hadoop HDFS

[jira] [Updated] (HDFS-10201) Implement undo log in parity datanode for hflush operations

2016-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10201: - Description: According to the current design doc for hflush support in erasure coding (see [HDFS-

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Attachment: HDFS-10175.002.patch Hi [~cmccabe], would you kindly review the patch? I'm not sure I

[jira] [Commented] (HDFS-8915) TestFSNamesystem.testFSLockGetWaiterCount fails intermittently in jenkins

2016-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209557#comment-15209557 ] Mingliang Liu commented on HDFS-8915: - Comparing with little sleep, I'm in favor of a {

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Attachment: HDFS-10175.003.patch Thanks [~cmccabe] for providing comments. I agree with you about

[jira] [Commented] (HDFS-10192) Namenode safemode not coming out during failover

2016-04-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15222540#comment-15222540 ] Mingliang Liu commented on HDFS-10192: -- I can verify that the test fails with trunk c

[jira] [Commented] (HDFS-10192) Namenode safemode not coming out during failover

2016-04-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15222641#comment-15222641 ] Mingliang Liu commented on HDFS-10192: -- I had a look at [HDFS-7046] and found that th

[jira] [Commented] (HDFS-10192) Namenode safemode not coming out during failover

2016-04-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15222642#comment-15222642 ] Mingliang Liu commented on HDFS-10192: -- s/avoid not/not/ > Namenode safemode not com

[jira] [Commented] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-04-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15227058#comment-15227058 ] Mingliang Liu commented on HDFS-7661: - Thanks for proposing solutions to this problem.

[jira] [Commented] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-04-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15227433#comment-15227433 ] Mingliang Liu commented on HDFS-7661: - Thanks for the detailed explanation, [~zhz]. I g

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15229596#comment-15229596 ] Mingliang Liu commented on HDFS-10175: -- Thanks for your test to quantify the overhead

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Attachment: HDFS-10175.004.patch > add per-operation stats to FileSystem.Statistics >

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15236382#comment-15236382 ] Mingliang Liu commented on HDFS-10175: -- Thank you very much [~mingma] for the comment

[jira] [Created] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-12 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-10281: Summary: o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently Key: HDFS-10281 URL: https://issues.apache.org/jira/browse/HDFS-10281 Project

[jira] [Commented] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237919#comment-15237919 ] Mingliang Liu commented on HDFS-10281: -- Basically, there is a fixed interval sleep in

[jira] [Updated] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10281: - Attachment: HDFS-10281.000.patch > o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fail

[jira] [Commented] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237949#comment-15237949 ] Mingliang Liu commented on HDFS-10281: -- [~jnp] Would you kindly review the patch? Tha

[jira] [Updated] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10281: - Status: Patch Available (was: Open) > o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages

[jira] [Created] (HDFS-10283) o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-10283: Summary: o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently Key: HDFS-10283 URL: https://issues.apache.org/jira/browse/HDFS-10

[jira] [Commented] (HDFS-10283) o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15240097#comment-15240097 ] Mingliang Liu commented on HDFS-10283: -- It happens in our internal daily UT Jenkins,

[jira] [Updated] (HDFS-10283) o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10283: - Status: Patch Available (was: Open) > o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSave

[jira] [Updated] (HDFS-10283) o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10283: - Attachment: HDFS-10283.000.patch A simple fix is to disable the replace-data-on-failure feature by

[jira] [Updated] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10281: - Status: Open (was: Patch Available) > o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages

[jira] [Updated] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10281: - Status: Patch Available (was: Open) > o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages

[jira] [Created] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-10284: Summary: o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently Key: HDFS-10284 URL: https://issues.apache.org/jira/browse/HDFS-10284

[jira] [Updated] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10284: - Priority: Minor (was: Major) > o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testChe

[jira] [Updated] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10284: - Description: *Stacktrace* {code} org.mockito.exceptions.misusing.UnfinishedStubbingException: Un

[jira] [Commented] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15240439#comment-15240439 ] Mingliang Liu commented on HDFS-10284: -- I think it's due to mocking {{fsn}} while bei

[jira] [Updated] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10284: - Attachment: HDFS-10284.000.patch For testing the safe mode state machine transition ({{testCheckSa

[jira] [Updated] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10284: - Status: Patch Available (was: Open) > o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.

[jira] [Commented] (HDFS-10282) The VolumeScanner should warn about replica files which are misplaced

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15240458#comment-15240458 ] Mingliang Liu commented on HDFS-10282: -- Failing test {{hadoop.hdfs.server.blockmanage

[jira] [Commented] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241567#comment-15241567 ] Mingliang Liu commented on HDFS-10284: -- Failing tests are not related. > o.a.h.hdfs.

[jira] [Updated] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10281: - Attachment: HDFS-10281.001.patch Thanks [~jnp] for reviewing. Re-upload the patch for triggering

[jira] [Commented] (HDFS-10287) MiniDFSCluster should implement AutoCloseable

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241634#comment-15241634 ] Mingliang Liu commented on HDFS-10287: -- This will be a good improvement and users of

[jira] [Created] (HDFS-10293) StripedFileTestUtil#readAll flaky

2016-04-14 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-10293: Summary: StripedFileTestUtil#readAll flaky Key: HDFS-10293 URL: https://issues.apache.org/jira/browse/HDFS-10293 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-10293) StripedFileTestUtil#readAll flaky

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10293: - Issue Type: Sub-task (was: Bug) Parent: HDFS-8031 > StripedFileTestUtil#readAll flaky > -

[jira] [Updated] (HDFS-10293) StripedFileTestUtil#readAll flaky

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10293: - Status: Patch Available (was: Open) > StripedFileTestUtil#readAll flaky > ---

[jira] [Updated] (HDFS-10293) StripedFileTestUtil#readAll flaky

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10293: - Attachment: HDFS-10293.000.patch The code is as following: {code} static int readAll(FSDataInp

[jira] [Commented] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241802#comment-15241802 ] Mingliang Liu commented on HDFS-10281: -- Thanks for studying the pre-commit build issu

[jira] [Commented] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241806#comment-15241806 ] Mingliang Liu commented on HDFS-10284: -- [~brahmareddy], would you review this for me?

[jira] [Commented] (HDFS-10292) Add block id when client got Unable to close file exception

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241815#comment-15241815 ] Mingliang Liu commented on HDFS-10292: -- A space between "block" and {{last}} will be

[jira] [Commented] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241859#comment-15241859 ] Mingliang Liu commented on HDFS-10281: -- Thanks for the review and commit, [~kihwal].

[jira] [Commented] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241897#comment-15241897 ] Mingliang Liu commented on HDFS-10284: -- I found the {{BlockManagerSafeMode$SafeModeMo

[jira] [Updated] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10284: - Attachment: HDFS-10284.001.patch > o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.test

[jira] [Commented] (HDFS-10283) o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241932#comment-15241932 ] Mingliang Liu commented on HDFS-10283: -- Failing tests are not related as the changes

[jira] [Commented] (HDFS-10293) StripedFileTestUtil#readAll flaky

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242189#comment-15242189 ] Mingliang Liu commented on HDFS-10293: -- I don't see related failing tests. Still, I'

[jira] [Commented] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242208#comment-15242208 ] Mingliang Liu commented on HDFS-10284: -- I see no related failing tests. > o.a.h.hdfs

[jira] [Commented] (HDFS-10283) o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently

2016-04-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15243319#comment-15243319 ] Mingliang Liu commented on HDFS-10283: -- Thank you [~jingzhao] for your review and com

[jira] [Commented] (HDFS-10293) StripedFileTestUtil#readAll flaky

2016-04-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15243326#comment-15243326 ] Mingliang Liu commented on HDFS-10293: -- Thanks for your review and commit, [~jing9].

[jira] [Commented] (HDFS-10291) TestShortCircuitLocalRead failing

2016-04-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15243553#comment-15243553 ] Mingliang Liu commented on HDFS-10291: -- Thanks for the detailed investigation, [~ste.

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Attachment: HDFS-10175.005.patch > add per-operation stats to FileSystem.Statistics >

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-15 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15243970#comment-15243970 ] Mingliang Liu commented on HDFS-10175: -- Thanks [~cmccabe] very much for the discussio

[jira] [Created] (HDFS-10306) SafeModeMonitor should not leave safe mode if name system is starting active service

2016-04-18 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-10306: Summary: SafeModeMonitor should not leave safe mode if name system is starting active service Key: HDFS-10306 URL: https://issues.apache.org/jira/browse/HDFS-10306 Pr

[jira] [Assigned] (HDFS-10306) SafeModeMonitor should not leave safe mode if name system is starting active service

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HDFS-10306: Assignee: Mingliang Liu (was: Brahma Reddy Battula) > SafeModeMonitor should not leave saf

[jira] [Updated] (HDFS-10306) SafeModeMonitor should not leave safe mode if name system is starting active service

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10306: - Hadoop Flags: (was: Reviewed) > SafeModeMonitor should not leave safe mode if name system is sta

[jira] [Updated] (HDFS-10306) SafeModeMonitor should not leave safe mode if name system is starting active service

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10306: - Fix Version/s: (was: 2.9.0) > SafeModeMonitor should not leave safe mode if name system is sta

[jira] [Updated] (HDFS-10306) SafeModeMonitor should not leave safe mode if name system is starting active service

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10306: - Description: This is a follow-up of [HDFS-10192]. The {{BlockManagerSafeMode$SafeModeMonitor#canL

[jira] [Updated] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10284: - Attachment: HDFS-10284.002.patch Thank you [~walter.k.su] and [~vinayrpet] for your kind review.

[jira] [Updated] (HDFS-10306) SafeModeMonitor should not leave safe mode if name system is starting active service

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10306: - Attachment: HDFS-10306.000.patch Thanks [~walter.k.su] for suggesting separating this code out of

[jira] [Updated] (HDFS-10306) SafeModeMonitor should not leave safe mode if name system is starting active service

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10306: - Status: Patch Available (was: Open) > SafeModeMonitor should not leave safe mode if name system i

[jira] [Updated] (HDFS-10306) SafeModeMonitor should not leave safe mode if name system is starting active service

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10306: - Description: This is a follow-up of [HDFS-10192]. The {{BlockManagerSafeMode$SafeModeMonitor#canL

[jira] [Commented] (HDFS-10305) Hdfs audit shouldn't log mkdir operaton if the directory already exists.

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246710#comment-15246710 ] Mingliang Liu commented on HDFS-10305: -- Agreed. > Hdfs audit shouldn't log mkdir ope

[jira] [Updated] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10284: - Priority: Major (was: Minor) > o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testChe

[jira] [Commented] (HDFS-10306) SafeModeMonitor should not leave safe mode if name system is starting active service

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246844#comment-15246844 ] Mingliang Liu commented on HDFS-10306: -- Test failures are not related. We don't add n

[jira] [Commented] (HDFS-10306) SafeModeMonitor should not leave safe mode if name system is starting active service

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246999#comment-15246999 ] Mingliang Liu commented on HDFS-10306: -- Thank you [~brahmareddy] for your review (and

[jira] [Updated] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10284: - Attachment: HDFS-10284.003.patch The v3 patch is to address [~brahmareddy]'s comment about code co

[jira] [Commented] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15247796#comment-15247796 ] Mingliang Liu commented on HDFS-10284: -- Thanks for the review [~brahmareddy]. Thanks

[jira] [Commented] (HDFS-10312) Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction.

2016-04-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15248713#comment-15248713 ] Mingliang Liu commented on HDFS-10312: -- +1 (non-binding) One nit is that, in the uni

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15248792#comment-15248792 ] Mingliang Liu commented on HDFS-10175: -- Hi [~cmccabe], would you kindly have a look a

[jira] [Commented] (HDFS-10312) Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction.

2016-04-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15248807#comment-15248807 ] Mingliang Liu commented on HDFS-10312: -- Shall we create a new jira for this? > Large

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15252797#comment-15252797 ] Mingliang Liu commented on HDFS-10175: -- Hi [~cmccabe], I think you proposed an innova

[jira] [Commented] (HDFS-10325) TestDFSShell#testMoveWithTargetPortEmpty fails

2016-04-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256686#comment-15256686 ] Mingliang Liu commented on HDFS-10325: -- Failing tests are not related. +1 (non-bindin

  1   2   3   4   5   6   7   8   9   10   >