[jira] [Updated] (HDFS-9677) Rename generationStampV1/generationStampV2 to legacyGenerationStamp/generationStamp

2016-01-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9677: Target Version/s: 3.0.0 Status: Patch Available (was: Open) > Rename

[jira] [Created] (HDFS-9689) Test o.a.h.hdfs.TestRenameWhileOpen fails intermittently

2016-01-22 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-9689: --- Summary: Test o.a.h.hdfs.TestRenameWhileOpen fails intermittently Key: HDFS-9689 URL: https://issues.apache.org/jira/browse/HDFS-9689 Project: Hadoop HDFS

[jira] [Updated] (HDFS-9689) Test o.a.h.hdfs.TestRenameWhileOpen fails intermittently

2016-01-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9689: Attachment: HDFS-9689.000.patch The tests bind the same name node (NN) port when building a new

[jira] [Updated] (HDFS-9689) Test o.a.h.hdfs.TestRenameWhileOpen fails intermittently

2016-01-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9689: Affects Version/s: 2.8.0 > Test o.a.h.hdfs.TestRenameWhileOpen fails intermittently >

[jira] [Updated] (HDFS-9689) Test o.a.h.hdfs.TestRenameWhileOpen fails intermittently

2016-01-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9689: Status: Patch Available (was: Open) > Test o.a.h.hdfs.TestRenameWhileOpen fails intermittently >

[jira] [Commented] (HDFS-9691) TestBlockManagerSafeMode#testCheckSafeMode fails intermittently

2016-02-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15146949#comment-15146949 ] Mingliang Liu commented on HDFS-9691: - Thanks for your review and commit, [~ajisakaa]! >

[jira] [Commented] (HDFS-9815) Move o.a.h.fs.Hdfs to hadoop-hdfs-client

2016-02-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149102#comment-15149102 ] Mingliang Liu commented on HDFS-9815: - The change is straightforward for me to +1 (non-binding). >

[jira] [Updated] (HDFS-7661) Erasure coding: support hflush and hsync

2016-02-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: HDFS-EC-file-flush-sync-design-version2.0.pdf Thanks for working on this [~demongaorui].

[jira] [Commented] (HDFS-7661) Erasure coding: support hflush and hsync

2016-02-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151729#comment-15151729 ] Mingliang Liu commented on HDFS-7661: - Thanks [~drankye] and [~sinall] for your prompt comments. As I

[jira] [Commented] (HDFS-9825) Balancer should not terminate if only one of the namenodes has error

2016-02-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153243#comment-15153243 ] Mingliang Liu commented on HDFS-9825: - The fix looks pretty good to me. +1 (non-binding) My minor

[jira] [Updated] (HDFS-9716) o.a.h.hdfs.TestRecoverStripedFile fails intermittently in trunk

2016-02-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9716: Component/s: erasure-coding > o.a.h.hdfs.TestRecoverStripedFile fails intermittently in trunk >

[jira] [Commented] (HDFS-9716) o.a.h.hdfs.TestRecoverStripedFile fails intermittently in trunk

2016-02-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154840#comment-15154840 ] Mingliang Liu commented on HDFS-9716: - Is it my imagination or the test is missing in trunk? If you're

[jira] [Updated] (HDFS-7661) Erasure coding: support hflush and hsync

2016-02-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: (was: HDFS-EC-file-flush-sync-design-V2.pdf) > Erasure coding: support hflush and

[jira] [Updated] (HDFS-7661) Erasure coding: support hflush and hsync

2016-02-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: HDFS-EC-file-flush-sync-design-V2.pdf > Erasure coding: support hflush and hsync >

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189869#comment-15189869 ] Mingliang Liu commented on HDFS-9405: - Nits: 0. {{keyNames}} and {{kp}} seem final. 1. I like the idea

[jira] [Commented] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190241#comment-15190241 ] Mingliang Liu commented on HDFS-9941: - This proposal makes sense to me. Thanks for reporting this,

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190271#comment-15190271 ] Mingliang Liu commented on HDFS-9405: - Thanks for updating the patch, [~xiaochen]. I was aware that we

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Attachment: HDFS-10175.000.patch Thanks for creating this jira, [~venkateshrin]. The v0 patch is

[jira] [Updated] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-08 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: (was: HDFS-EC-file-flush-sync-design-version2.0.pdf) > Erasure coding: support

[jira] [Updated] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-08 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: (was: HDFS-EC-file-flush-sync-design-version2.1.pdf) > Erasure coding: support

[jira] [Commented] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-04-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15227058#comment-15227058 ] Mingliang Liu commented on HDFS-7661: - Thanks for proposing solutions to this problem. I agree with

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229596#comment-15229596 ] Mingliang Liu commented on HDFS-10175: -- Thanks for your test to quantify the overhead of the per-op

[jira] [Commented] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237919#comment-15237919 ] Mingliang Liu commented on HDFS-10281: -- Basically, there is a fixed interval sleep in the test during

[jira] [Created] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-12 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-10281: Summary: o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently Key: HDFS-10281 URL: https://issues.apache.org/jira/browse/HDFS-10281

[jira] [Commented] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237949#comment-15237949 ] Mingliang Liu commented on HDFS-10281: -- [~jnp] Would you kindly review the patch? Thanks! >

[jira] [Updated] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10281: - Attachment: HDFS-10281.000.patch > o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages

[jira] [Updated] (HDFS-10281) o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently

2016-04-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10281: - Status: Patch Available (was: Open) > o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236382#comment-15236382 ] Mingliang Liu commented on HDFS-10175: -- Thank you very much [~mingma] for the comments. Yes NN also

[jira] [Commented] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-04-05 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15227433#comment-15227433 ] Mingliang Liu commented on HDFS-7661: - Thanks for the detailed explanation, [~zhz]. I got the idea of

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Attachment: HDFS-10175.004.patch > add per-operation stats to FileSystem.Statistics >

[jira] [Work started] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-10175 started by Mingliang Liu. > add per-operation stats to FileSystem.Statistics >

[jira] [Commented] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200103#comment-15200103 ] Mingliang Liu commented on HDFS-7661: - According to the current design, we need to break down this

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205086#comment-15205086 ] Mingliang Liu commented on HDFS-10175: -- Thanks for your comment, [~andrew.wang]. I was aware of the

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Status: Patch Available (was: In Progress) > add per-operation stats to FileSystem.Statistics >

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Attachment: HDFS-10175.001.patch > add per-operation stats to FileSystem.Statistics >

[jira] [Updated] (HDFS-10192) Namenode safemode not coming out during failover

2016-03-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10192: - Component/s: namenode > Namenode safemode not coming out during failover >

[jira] [Updated] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: (was: HDFS-EC-file-flush-sync-design-version2.2.pdf) > [umbrella] support hflush and

[jira] [Updated] (HDFS-10201) Implement undo log in parity datanode for hflush operations

2016-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10201: - Description: According to the current design doc for hflush support in erasure coding (see

[jira] [Commented] (HDFS-10192) Namenode safemode not coming out during failover

2016-03-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207486#comment-15207486 ] Mingliang Liu commented on HDFS-10192: -- Thanks for reporting this, [~brahmareddy]. I will have a look

[jira] [Commented] (HDFS-8915) TestFSNamesystem.testFSLockGetWaiterCount fails intermittently in jenkins

2016-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209557#comment-15209557 ] Mingliang Liu commented on HDFS-8915: - Comparing with little sleep, I'm in favor of a

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Attachment: HDFS-10175.002.patch Hi [~cmccabe], would you kindly review the patch? I'm not sure I

[jira] [Updated] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10175: - Attachment: HDFS-10175.003.patch Thanks [~cmccabe] for providing comments. I agree with you

[jira] [Updated] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: HDFS-EC-file-flush-sync-design-v20160323.pdf > [umbrella] support hflush and hsync for

[jira] [Created] (HDFS-10201) Implement undo log in parity datanode for hflush operations

2016-03-23 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-10201: Summary: Implement undo log in parity datanode for hflush operations Key: HDFS-10201 URL: https://issues.apache.org/jira/browse/HDFS-10201 Project: Hadoop HDFS

[jira] [Commented] (HDFS-9579) Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level

2016-03-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15178871#comment-15178871 ] Mingliang Liu commented on HDFS-9579: - +1 (non-binding) pending on Jenkins. > Provide

[jira] [Updated] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-7661: Attachment: HDFS-EC-file-flush-sync-design-version2.1.pdf Attached the v2.1 design doc, based on the

[jira] [Commented] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177055#comment-15177055 ] Mingliang Liu commented on HDFS-7661: - Thanks for the comments, [~sinall]! 1. Let's see if I get your

[jira] [Commented] (HDFS-7661) Erasure coding: support hflush and hsync

2016-03-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15178387#comment-15178387 ] Mingliang Liu commented on HDFS-7661: - Thanks for the further review, [~demongaorui]. 1. I should have

[jira] [Commented] (HDFS-9579) Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level

2016-03-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15178651#comment-15178651 ] Mingliang Liu commented on HDFS-9579: - Thanks for updating the patch, [~mingma]! I like the idea to

[jira] [Commented] (HDFS-9817) Use SLF4J in new classes

2016-03-04 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15181445#comment-15181445 ] Mingliang Liu commented on HDFS-9817: - Hi [~anu], I see you changed log level at several places. Is

[jira] [Commented] (HDFS-10192) Namenode safemode not coming out during failover

2016-04-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15222642#comment-15222642 ] Mingliang Liu commented on HDFS-10192: -- s/avoid not/not/ > Namenode safemode not coming out during

[jira] [Commented] (HDFS-10192) Namenode safemode not coming out during failover

2016-04-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15222641#comment-15222641 ] Mingliang Liu commented on HDFS-10192: -- I had a look at [HDFS-7046] and found that there was no easy

[jira] [Commented] (HDFS-10192) Namenode safemode not coming out during failover

2016-04-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15222540#comment-15222540 ] Mingliang Liu commented on HDFS-10192: -- I can verify that the test fails with trunk code. Thanks for

[jira] [Updated] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10284: - Description: *Stacktrace* {code} org.mockito.exceptions.misusing.UnfinishedStubbingException:

[jira] [Commented] (HDFS-10284) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently

2016-04-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240439#comment-15240439 ] Mingliang Liu commented on HDFS-10284: -- I think it's due to mocking {{fsn}} while being concurrently

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257099#comment-15257099 ] Mingliang Liu commented on HDFS-10175: -- Thanks for proposing the discussion. The time works for me.

[jira] [Commented] (HDFS-10325) TestDFSShell#testMoveWithTargetPortEmpty fails

2016-04-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256686#comment-15256686 ] Mingliang Liu commented on HDFS-10325: -- Failing tests are not related. +1 (non-binding). Thanks for

[jira] [Commented] (HDFS-10326) Disable setting tcp socket send/receive buffers for write pipelines

2016-04-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256716#comment-15256716 ] Mingliang Liu commented on HDFS-10326: -- Thanks for reporting this. FWIW, [HDFS-8829]and [HDFS-9259]

[jira] [Commented] (HDFS-10326) Disable setting tcp socket send/receive buffers for write pipelines

2016-04-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257008#comment-15257008 ] Mingliang Liu commented on HDFS-10326: -- {quote} If we strongly believe that auto-tuning should be the

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15252797#comment-15252797 ] Mingliang Liu commented on HDFS-10175: -- Hi [~cmccabe], I think you proposed an innovative idea for

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-26 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259234#comment-15259234 ] Mingliang Liu commented on HDFS-10175: -- Either tomorrow or next week will work for me. Thanks. > add

[jira] [Updated] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2016-04-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10335: - Status: Patch Available (was: In Progress) > Mover$Processor#chooseTarget() always chooses the

[jira] [Commented] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2016-04-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260943#comment-15260943 ] Mingliang Liu commented on HDFS-10335: -- Thanks for your discussion and review, [~szetszwo]. By the

[jira] [Commented] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2016-04-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260940#comment-15260940 ] Mingliang Liu commented on HDFS-10335: -- {code} Step 16 : RUN cabal update && cabal install shellcheck

[jira] [Commented] (HDFS-10338) DistCp masks potential CRC check failures

2016-04-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261127#comment-15261127 ] Mingliang Liu commented on HDFS-10338: -- I'm favor of propagating the IOException thrown by

[jira] [Updated] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2016-04-26 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10335: - Attachment: HDFS-10335.000.patch > Mover$Processor#chooseTarget() always chooses the first

[jira] [Updated] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2016-04-26 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10335: - Description: Currently the

[jira] [Created] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target datanode

2016-04-26 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-10335: Summary: Mover$Processor#chooseTarget() always chooses the first matching target datanode Key: HDFS-10335 URL: https://issues.apache.org/jira/browse/HDFS-10335

[jira] [Work started] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2016-04-26 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-10335 started by Mingliang Liu. > Mover$Processor#chooseTarget() always chooses the first matching target >

[jira] [Updated] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2016-04-26 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10335: - Summary: Mover$Processor#chooseTarget() always chooses the first matching target storage group

[jira] [Commented] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2016-04-26 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259401#comment-15259401 ] Mingliang Liu commented on HDFS-10335: -- The code is as following: {code} boolean

[jira] [Commented] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2016-04-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264494#comment-15264494 ] Mingliang Liu commented on HDFS-10335: -- Thanks for your review and commit, [~szetszwo]. >

[jira] [Updated] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2016-04-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10335: - Attachment: HDFS-10335.000.patch Re-uploading the same patch to trigger Jenkins. >

[jira] [Commented] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2016-04-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15263428#comment-15263428 ] Mingliang Liu commented on HDFS-10335: -- Failing tests are not related. Specially,

[jira] [Commented] (HDFS-10326) Disable setting tcp socket send/receive buffers for write pipelines

2016-04-26 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258605#comment-15258605 ] Mingliang Liu commented on HDFS-10326: -- Ping [~mingma] for more input. > Disable setting tcp socket

[jira] [Commented] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2

2016-05-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289540#comment-15289540 ] Mingliang Liu commented on HDFS-10415: -- Hi [~cmccabe], Thanks for the comment. Your proposal is a

[jira] [Commented] (HDFS-10383) Safely close resources in DFSTestUtil

2016-05-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289732#comment-15289732 ] Mingliang Liu commented on HDFS-10383: -- Thanks [~xyao] for your review and commit. Thanks

[jira] [Commented] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2

2016-05-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298714#comment-15298714 ] Mingliang Liu commented on HDFS-10415: -- If no objections about the 1st approach, does the v1 patch

[jira] [Commented] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2

2016-05-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292024#comment-15292024 ] Mingliang Liu commented on HDFS-10415: -- When [HADOOP-9418] was committed to {{trunk}} and

[jira] [Updated] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2016-05-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9442: Resolution: Won't Fix Status: Resolved (was: Patch Available) As stated in the promising

[jira] [Resolved] (HDFS-10386) DataTransferProtocol#writeBlock missing some javadocs

2016-05-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu resolved HDFS-10386. -- Resolution: Duplicate This is a duplicate of [HDFS-10387]. Re-open if I'm wrong. Thanks. >

[jira] [Commented] (HDFS-10387) DataTransferProtocol#writeBlock missing some javadocs

2016-05-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278755#comment-15278755 ] Mingliang Liu commented on HDFS-10387: -- Thanks for the patch. May I ask why {{allowLazyPersist}} is

[jira] [Updated] (HDFS-10387) DataTransferProtocol#writeBlock missing some javadocs

2016-05-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10387: - Priority: Minor (was: Major) > DataTransferProtocol#writeBlock missing some javadocs >

[jira] [Commented] (HDFS-10385) LocalFileSystem rename() function should return false when destination file exists

2016-05-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278785#comment-15278785 ] Mingliang Liu commented on HDFS-10385: -- For {{ChecksumFileSystem}}, the rename operation succeeds iff

[jira] [Commented] (HDFS-10387) DataTransferProtocol#writeBlock missing some javadocs

2016-05-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278975#comment-15278975 ] Mingliang Liu commented on HDFS-10387: -- That makes sense to me. Thanks for the explanation,

[jira] [Commented] (HDFS-10383) Safely close resources in DFSTestUtil

2016-05-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280945#comment-15280945 ] Mingliang Liu commented on HDFS-10383: -- Thanks [~xyao] for your comment. I'll address them in the

[jira] [Updated] (HDFS-10383) Safely close resources in DFSTestUtil

2016-05-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10383: - Priority: Major (was: Minor) The v1 patch revisited the {{DFSTestUtil}} class. #

[jira] [Updated] (HDFS-10383) Safely close resources in DFSTestUtil

2016-05-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10383: - Attachment: HDFS-10383.001.patch > Safely close resources in DFSTestUtil >

[jira] [Updated] (HDFS-10383) Safely close resources in DFSTestUtil

2016-05-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10383: - Issue Type: Improvement (was: Bug) > Safely close resources in DFSTestUtil >

[jira] [Commented] (HDFS-10383) Safely close resources in DFSTestUtil

2016-05-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280976#comment-15280976 ] Mingliang Liu commented on HDFS-10383: -- That makes sense. I'll post a new patch soon. > Safely close

[jira] [Updated] (HDFS-10383) Safely close resources in DFSTestUtil

2016-05-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10383: - Attachment: HDFS-10383.002.patch The v2 patch is to revert {{createStripedFile()}} and

[jira] [Created] (HDFS-10395) GlobalStorageStatistics should check null FileSystem scheme to avoid NPE

2016-05-12 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-10395: Summary: GlobalStorageStatistics should check null FileSystem scheme to avoid NPE Key: HDFS-10395 URL: https://issues.apache.org/jira/browse/HDFS-10395 Project:

[jira] [Updated] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting

2016-05-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10397: - Attachment: HDFS-10397.003.patch Uploading the same patch for pre-commit Jenkins run. > Distcp

[jira] [Assigned] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2

2016-05-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HDFS-10415: Assignee: Mingliang Liu > TestDistributedFileSystem#testDFSCloseOrdering() fails on

[jira] [Updated] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2

2016-05-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10415: - Attachment: HDFS-10415-branch-2.000.patch Thanks for reporting this [~sjlee0] and

[jira] [Updated] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2

2016-05-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10415: - Status: Patch Available (was: Open) If the patch looks good, I think we can apply it to

[jira] [Commented] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2

2016-05-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287671#comment-15287671 ] Mingliang Liu commented on HDFS-10415: -- # If we call the {{initialize()}} method in the test, it will

[jira] [Commented] (HDFS-10397) Distcp should ignore -delete option if -diff option is provided instead of exiting

2016-05-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287823#comment-15287823 ] Mingliang Liu commented on HDFS-10397: -- Thanks [~jingzhao] and [~yzhangal] for the review and

[jira] [Updated] (HDFS-10383) Safely close resources in DFSTestUtil

2016-05-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-10383: - Attachment: HDFS-10383-branch-2.000.patch Thanks [~xyao] for the review and commit. I attached

[jira] [Commented] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2

2016-05-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287977#comment-15287977 ] Mingliang Liu commented on HDFS-10415: -- [~sjlee0], interesting question. I see code difference in

<    4   5   6   7   8   9   10   11   12   13   >