[jira] [Commented] (HDFS-9407) TestFileTruncate fails with BindException

2015-11-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999154#comment-14999154 ] Mingliang Liu commented on HDFS-9407: - Thanks for reporting and working on this. > Tes

[jira] [Commented] (HDFS-9400) TestRollingUpgradeRollback fails on branch-2.

2015-11-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999177#comment-14999177 ] Mingliang Liu commented on HDFS-9400: - The test report seems good? > TestRollingUpgrad

[jira] [Commented] (HDFS-9349) Reconfigure NN protected directories on the fly

2015-11-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999523#comment-14999523 ] Mingliang Liu commented on HDFS-9349: - Thanks for working on this, [~xiaobingo]. Is it

[jira] [Commented] (HDFS-9349) Reconfigure NN protected directories on the fly

2015-11-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999641#comment-14999641 ] Mingliang Liu commented on HDFS-9349: - The v2 patch looks good overall to me. Further

[jira] [Commented] (HDFS-9245) Fix findbugs warnings in hdfs-nfs/WriteCtx

2015-11-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999695#comment-14999695 ] Mingliang Liu commented on HDFS-9245: - Thanks [~gtCarrera9] and [~brandonli] for your r

[jira] [Updated] (HDFS-9129) Move the safemode block count into BlockManager

2015-11-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9129: Attachment: HDFS-9129.022.patch Thanks [~jingzhao] for your detailed review and I appreciate it very

[jira] [Updated] (HDFS-9129) Move the safemode block count into BlockManager

2015-11-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9129: Attachment: HDFS-9129.023.patch The v23 patch is to address the findbugs warnings. > Move the safemo

[jira] [Commented] (HDFS-9387) Parse namenodeUri parameter only once in NNThroughputBenchmark$OperationStatsBase#verifyOpArgument()

2015-11-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003280#comment-15003280 ] Mingliang Liu commented on HDFS-9387: - Thanks for your comment. It's a good catch. I re

[jira] [Commented] (HDFS-9387) Parse namenodeUri parameter only once in NNThroughputBenchmark$OperationStatsBase#verifyOpArgument()

2015-11-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003294#comment-15003294 ] Mingliang Liu commented on HDFS-9387: - Yes, I agree with you. Fixing it provides obviou

[jira] [Assigned] (HDFS-9421) NNThroughputBenchmark replication test NPE with -namenode option

2015-11-12 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HDFS-9421: --- Assignee: Mingliang Liu > NNThroughputBenchmark replication test NPE with -namenode option > -

[jira] [Updated] (HDFS-9421) NNThroughputBenchmark replication test NPE with -namenode option

2015-11-13 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9421: Attachment: HDFS-9241.000.patch Per discussion in [HDFS-9387], the {{replication}} test is not suppor

[jira] [Updated] (HDFS-9421) NNThroughputBenchmark replication test NPE with -namenode option

2015-11-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9421: Status: Patch Available (was: Open) > NNThroughputBenchmark replication test NPE with -namenode opti

[jira] [Commented] (HDFS-9431) DistributedFileSystem#concat fails if the target path is relative.

2015-11-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005606#comment-15005606 ] Mingliang Liu commented on HDFS-9431: - Thanks for working on this, [~fjk]. It's helpfu

[jira] [Updated] (HDFS-9431) DistributedFileSystem#concat fails if the target path is relative.

2015-11-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9431: Assignee: Kazuho Fujii > DistributedFileSystem#concat fails if the target path is relative. > ---

[jira] [Updated] (HDFS-9431) DistributedFileSystem#concat fails if the target path is relative.

2015-11-14 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9431: Status: Patch Available (was: Open) > DistributedFileSystem#concat fails if the target path is relat

[jira] [Commented] (HDFS-9421) NNThroughputBenchmark replication test NPE with -namenode option

2015-11-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007714#comment-15007714 ] Mingliang Liu commented on HDFS-9421: - Thanks for your reporting this issue, reviewing

[jira] [Commented] (HDFS-9387) Fix namenodeUri parameter parsing in NNThroughputBenchmark

2015-11-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007717#comment-15007717 ] Mingliang Liu commented on HDFS-9387: - Thanks for your review and commit! > Fix nameno

[jira] [Commented] (HDFS-9434) Recommission a datanode with 500k blocks may pause NN for 30 seconds

2015-11-16 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007756#comment-15007756 ] Mingliang Liu commented on HDFS-9434: - Thanks for working on this . The patch looks goo

[jira] [Commented] (HDFS-9379) Make NNThroughputBenchmark$BlockReportStats support more than 10 datanodes

2015-11-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009393#comment-15009393 ] Mingliang Liu commented on HDFS-9379: - Thanks for your comment [~shv]. If the default v

[jira] [Updated] (HDFS-9379) Make NNThroughputBenchmark$BlockReportStats support more than 10 datanodes

2015-11-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9379: Component/s: (was: namenode) test > Make NNThroughputBenchmark$BlockReportStats

[jira] [Created] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-11-17 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-9436: --- Summary: Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default Key: HDFS-9436 URL: https://issues.apache.org/jira/browse/HDFS-9436 Project: Hadoo

[jira] [Updated] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-11-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9436: Description: This is a follow-up of [HDFS-9379]. Though for actual benchmarking the defaults are rar

[jira] [Updated] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-11-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9436: Hadoop Flags: (was: Reviewed) > Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes b

[jira] [Updated] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-11-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9436: Priority: Minor (was: Major) > Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by

[jira] [Commented] (HDFS-9379) Make NNThroughputBenchmark$BlockReportStats support more than 10 datanodes

2015-11-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009630#comment-15009630 ] Mingliang Liu commented on HDFS-9379: - Thanks for your confirmation. I filed [HDFS-9436

[jira] [Commented] (HDFS-9422) Unnecessary FsDatasetImpl locking in DirectoryScanner cause periodic datanode pauses

2015-11-17 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010058#comment-15010058 ] Mingliang Liu commented on HDFS-9422: - Agreed. The most heavy I/O operations scanning t

[jira] [Created] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-11-18 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-9442: --- Summary: Move block replication logic from BlockManager to a new class ReplicationManager Key: HDFS-9442 URL: https://issues.apache.org/jira/browse/HDFS-9442 Project: H

[jira] [Commented] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-11-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15012804#comment-15012804 ] Mingliang Liu commented on HDFS-9442: - I think we can work on {{trunk}} branch for this

[jira] [Commented] (HDFS-9407) TestFileTruncate fails with BindException

2015-11-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014210#comment-15014210 ] Mingliang Liu commented on HDFS-9407: - {quote} In TestFileTruncate we restart the whol

[jira] [Commented] (HDFS-9407) TestFileTruncate fails with BindException

2015-11-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014354#comment-15014354 ] Mingliang Liu commented on HDFS-9407: - Thanks for your clarification. Yes, the intrinsi

[jira] [Updated] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-11-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9442: Attachment: HDFS-9442.000.patch The v0 patch moves pending, over and mis-replicated blocks to {{Repl

[jira] [Updated] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-11-19 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9442: Status: Patch Available (was: Open) > Move block replication logic from BlockManager to a new class

[jira] [Updated] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-11-20 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9442: Attachment: HDFS-9442.001.patch The v1 patch moves the {{neededReplication}} from {{BlockManager}} to

[jira] [Updated] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-11-20 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9442: Attachment: HDFS-9442.001.patch > Move block replication logic from BlockManager to a new class > Re

[jira] [Updated] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-11-20 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9442: Attachment: (was: HDFS-9442.001.patch) > Move block replication logic from BlockManager to a new

[jira] [Updated] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-11-20 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9436: Attachment: HDFS-9436.000.patch The v0 patch makes number of datanodes in {{BlockReportStat}} 10 by d

[jira] [Updated] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-11-20 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9436: Status: Patch Available (was: Open) > Make NNThroughputBenchmark$BlockReportStats run with 10 datano

[jira] [Commented] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-11-20 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15020262#comment-15020262 ] Mingliang Liu commented on HDFS-9436: - Failing tests and ASF license warnings are not r

[jira] [Updated] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-11-21 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9442: Attachment: HDFS-9442.002.patch The v2 patch is to address the checkstyle, javac and findbugs warning

[jira] [Commented] (HDFS-9438) TestPipelinesFailover assumes Linux ifconfig

2015-11-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023447#comment-15023447 ] Mingliang Liu commented on HDFS-9438: - Agreed. Ignoring the test instead of failing is

[jira] [Updated] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-11-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9442: Attachment: HDFS-9442.003.patch > Move block replication logic from BlockManager to a new class > Re

[jira] [Commented] (HDFS-9457) Datanode Logging Improvement

2015-11-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025149#comment-15025149 ] Mingliang Liu commented on HDFS-9457: - Thanks for working on this. {{DataNode}} has ad

[jira] [Commented] (HDFS-7435) PB encoding of block reports is very inefficient

2015-11-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025772#comment-15025772 ] Mingliang Liu commented on HDFS-7435: - In {{NNThroughputBenchmark}} class, the change {

[jira] [Updated] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-11-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9436: Attachment: HDFS-9436.001.patch Thanks for your review and comment. No, the change was not intention

[jira] [Created] (HDFS-9467) Fix data race accessing writeLockHeldTimeStamp in FSNamesystem

2015-11-24 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-9467: --- Summary: Fix data race accessing writeLockHeldTimeStamp in FSNamesystem Key: HDFS-9467 URL: https://issues.apache.org/jira/browse/HDFS-9467 Project: Hadoop HDFS

[jira] [Commented] (HDFS-9145) Tracking methods that hold FSNamesytemLock for too long

2015-11-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026129#comment-15026129 ] Mingliang Liu commented on HDFS-9145: - Sure. Nice catch! I filed [HDFS-9467] to track

[jira] [Updated] (HDFS-9467) Fix data race accessing writeLockHeldTimeStamp in FSNamesystem

2015-11-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9467: Affects Version/s: 2.8.0 > Fix data race accessing writeLockHeldTimeStamp in FSNamesystem > -

[jira] [Updated] (HDFS-9467) Fix data race accessing writeLockHeldTimeStamp in FSNamesystem

2015-11-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9467: Status: Patch Available (was: Open) > Fix data race accessing writeLockHeldTimeStamp in FSNamesystem

[jira] [Updated] (HDFS-9467) Fix data race accessing writeLockHeldTimeStamp in FSNamesystem

2015-11-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9467: Attachment: HDFS-9467.000.patch > Fix data race accessing writeLockHeldTimeStamp in FSNamesystem > --

[jira] [Commented] (HDFS-9457) Datanode Logging Improvement

2015-11-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026158#comment-15026158 ] Mingliang Liu commented on HDFS-9457: - Thanks for your further effort. I meant to remo

[jira] [Updated] (HDFS-9457) Datanode Logging Improvement

2015-11-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9457: Assignee: BELUGA BEHR > Datanode Logging Improvement > > >

[jira] [Updated] (HDFS-9457) Datanode Logging Improvement

2015-11-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9457: Priority: Minor (was: Trivial) > Datanode Logging Improvement > > >

[jira] [Commented] (HDFS-9457) Datanode Logging Improvement

2015-11-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026368#comment-15026368 ] Mingliang Liu commented on HDFS-9457: - Option 1 for sure. I think LOG.isTraceEnabled()

[jira] [Updated] (HDFS-9467) Fix data race accessing writeLockHeldTimeStamp in FSNamesystem

2015-11-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9467: Attachment: HDFS-9467.001.patch The failing tests seem unrelated. Thanks [~jingzhao] for your review

[jira] [Commented] (HDFS-9467) Fix data race accessing writeLockHeldTimeStamp in FSNamesystem

2015-11-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027725#comment-15027725 ] Mingliang Liu commented on HDFS-9467: - Thank you for reviewing this patch again and com

[jira] [Commented] (HDFS-9457) Datanode Logging Improvement

2015-11-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027834#comment-15027834 ] Mingliang Liu commented on HDFS-9457: - Thanks for updating the patch. Please refer to

[jira] [Commented] (HDFS-9214) Reconfigure DN concurrent move on the fly

2015-11-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027919#comment-15027919 ] Mingliang Liu commented on HDFS-9214: - Overall looks good. Minor comments: # Any loggi

[jira] [Commented] (HDFS-6363) Improve concurrency while checking inclusion and exclusion of datanodes

2015-11-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027921#comment-15027921 ] Mingliang Liu commented on HDFS-6363: - +1 (non-binding) > Improve concurrency while ch

[jira] [Updated] (HDFS-9129) Move the safemode block count into BlockManager

2015-11-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9129: Attachment: HDFS-9129.024.patch Per offline discussion with [~jingzhao], the v24 patch removes the {

[jira] [Commented] (HDFS-9457) Datanode Logging Improvement

2015-11-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027972#comment-15027972 ] Mingliang Liu commented on HDFS-9457: - Instead of deleting the previous patch and uploa

[jira] [Commented] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-11-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027999#comment-15027999 ] Mingliang Liu commented on HDFS-9436: - Thank you very much for the detailed explanation

[jira] [Created] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-11-30 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-9484: --- Summary: NNThroughputBenchmark$BlockReportStats should not send empty block reports Key: HDFS-9484 URL: https://issues.apache.org/jira/browse/HDFS-9484 Project: Hadoop

[jira] [Commented] (HDFS-7435) PB encoding of block reports is very inefficient

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15032285#comment-15032285 ] Mingliang Liu commented on HDFS-7435: - Thanks for your comment, [~daryn] and [~shv]. I

[jira] [Updated] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9484: Description: In {{NNThroughputBenchmark$BlockReportStats#formBlockReport()}}, the {{blockReportList}

[jira] [Created] (HDFS-9485) Make BlockManager#removeFromExcessReplicateMap accept BlockInfo instead of Block

2015-11-30 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-9485: --- Summary: Make BlockManager#removeFromExcessReplicateMap accept BlockInfo instead of Block Key: HDFS-9485 URL: https://issues.apache.org/jira/browse/HDFS-9485 Project: H

[jira] [Updated] (HDFS-9485) Make BlockManager#removeFromExcessReplicateMap accept BlockInfo instead of Block

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9485: Status: Patch Available (was: Open) > Make BlockManager#removeFromExcessReplicateMap accept BlockInf

[jira] [Updated] (HDFS-9485) Make BlockManager#removeFromExcessReplicateMap accept BlockInfo instead of Block

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9485: Attachment: HDFS-9485.000.patch > Make BlockManager#removeFromExcessReplicateMap accept BlockInfo ins

[jira] [Updated] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9484: Description: In {{NNThroughputBenchmark$BlockReportStats#formBlockReport()}}, the {{blockReportList}

[jira] [Updated] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9484: Description: There are two potential bugs that make the {{NNThroughputBenchmark$BlockReportStats}} s

[jira] [Commented] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15032704#comment-15032704 ] Mingliang Liu commented on HDFS-9484: - Thanks [~cmccabe] for your confirmation. I updat

[jira] [Commented] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15032708#comment-15032708 ] Mingliang Liu commented on HDFS-9436: - I updated the JIRA [HDFS-9484] to track the effo

[jira] [Updated] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9442: Attachment: HDFS-9442.004.patch Per offline discussion with [~wheat9] and [~jingzhao], the v4 patch m

[jira] [Updated] (HDFS-9129) Move the safemode block count into BlockManager

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9129: Attachment: HDFS-9129.025.patch Thanks [~jingzhao] for the review. I revisited the TODO comment and I

[jira] [Commented] (HDFS-9485) Make BlockManager#removeFromExcessReplicateMap accept BlockInfo instead of Block

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15033048#comment-15033048 ] Mingliang Liu commented on HDFS-9485: - Thanks [~jingzhao] for reviewing this. Failing

[jira] [Commented] (HDFS-9129) Move the safemode block count into BlockManager

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15033170#comment-15033170 ] Mingliang Liu commented on HDFS-9129: - The _javac_ (both 1.7 and 1.8), _checkstyle_ and

[jira] [Updated] (HDFS-9129) Move the safemode block count into BlockManager

2015-11-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9129: Component/s: namenode > Move the safemode block count into BlockManager > ---

[jira] [Updated] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-12-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9484: Attachment: HDFS-9484.000.patch > NNThroughputBenchmark$BlockReportStats should not send empty block

[jira] [Updated] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-12-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9484: Status: Patch Available (was: Open) > NNThroughputBenchmark$BlockReportStats should not send empty b

[jira] [Commented] (HDFS-9485) Make BlockManager#removeFromExcessReplicateMap accept BlockInfo instead of Block

2015-12-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034593#comment-15034593 ] Mingliang Liu commented on HDFS-9485: - Thanks [~jingzhao] for your review and commit.

[jira] [Commented] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-12-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034803#comment-15034803 ] Mingliang Liu commented on HDFS-9484: - Failing tests are not related and can not re-pro

[jira] [Updated] (HDFS-9300) TestDirectoryScanner.testThrottle() is still a little flakey

2015-12-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9300: Summary: TestDirectoryScanner.testThrottle() is still a little flakey (was: DirectoryScanner.testThr

[jira] [Created] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2015-12-01 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-9493: --- Summary: Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk Key: HDFS-9493 URL: https://issues.apache.org/jira/browse/HDFS-9493 Project: Hadoop HDFS

[jira] [Commented] (HDFS-9445) Deadlock in datanode

2015-12-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15035016#comment-15035016 ] Mingliang Liu commented on HDFS-9445: - The _bposLock --> FSDatasetLock_ locking order l

[jira] [Updated] (HDFS-9129) Move the safemode block count into BlockManager

2015-12-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9129: Attachment: HDFS-9129-branch-2.025.patch Thank you very much [~jingzhao] for your review, discussion

[jira] [Updated] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-12-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9442: Attachment: HDFS-9442.005.patch The v5 patch rebases from {{trunk}} branch and fixes conflicts. > Mo

[jira] [Reopened] (HDFS-9129) Move the safemode block count into BlockManager

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reopened HDFS-9129: - > Move the safemode block count into BlockManager > --- > >

[jira] [Updated] (HDFS-9129) Move the safemode block count into BlockManager

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9129: Status: Patch Available (was: Reopened) > Move the safemode block count into BlockManager >

[jira] [Created] (HDFS-9498) Move BlockManager#numberOfBytesInFutureBlocks to BlockManagerSafeMode

2015-12-02 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-9498: --- Summary: Move BlockManager#numberOfBytesInFutureBlocks to BlockManagerSafeMode Key: HDFS-9498 URL: https://issues.apache.org/jira/browse/HDFS-9498 Project: Hadoop HDFS

[jira] [Commented] (HDFS-9129) Move the safemode block count into BlockManager

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036894#comment-15036894 ] Mingliang Liu commented on HDFS-9129: - {noformat} Step 16 : RUN cd /usr/share/maven/lib

[jira] [Updated] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9436: Attachment: HDFS-9436.002.patch Thank you very much [~shv] for your review. Sending a couple of block

[jira] [Updated] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9484: Attachment: HDFS-9484.001.patch Thanks for the comment, [~shv]. I totally agree with you that the co

[jira] [Updated] (HDFS-9498) Move code that tracks orphan blocks to BlockManagerSafeMode

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9498: Summary: Move code that tracks orphan blocks to BlockManagerSafeMode (was: Move BlockManager#numberO

[jira] [Updated] (HDFS-9498) Move code that tracks orphan blocks to BlockManagerSafeMode

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9498: Status: Patch Available (was: Open) > Move code that tracks orphan blocks to BlockManagerSafeMode >

[jira] [Updated] (HDFS-9498) Move code that tracks orphan blocks to BlockManagerSafeMode

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9498: Attachment: HDFS-9498.000.patch > Move code that tracks orphan blocks to BlockManagerSafeMode > -

[jira] [Commented] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037435#comment-15037435 ] Mingliang Liu commented on HDFS-9484: - The failing tests seem unrelated. > NNThroughpu

[jira] [Commented] (HDFS-9436) Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default

2015-12-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037433#comment-15037433 ] Mingliang Liu commented on HDFS-9436: - The failing tests seem unrelated. Specially, {{

[jira] [Commented] (HDFS-9484) NNThroughputBenchmark$BlockReportStats should not send empty block reports

2015-12-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15038353#comment-15038353 ] Mingliang Liu commented on HDFS-9484: - [~shv] Thank you very much for your review, disc

[jira] [Assigned] (HDFS-9503) Replace -namenode option with -fs for NNThroughputBenchmark

2015-12-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HDFS-9503: --- Assignee: Mingliang Liu > Replace -namenode option with -fs for NNThroughputBenchmark > --

[jira] [Commented] (HDFS-9503) Replace -namenode option with -fs for NNThroughputBenchmark

2015-12-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15038411#comment-15038411 ] Mingliang Liu commented on HDFS-9503: - Feel free to re-assign it to someone who's activ

[jira] [Updated] (HDFS-9442) Move block replication logic from BlockManager to a new class ReplicationManager

2015-12-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9442: Attachment: HDFS-9442.006.patch Thanks [~wheat9] for your review. The v5 patch is to address this. >

<    1   2   3   4   5   6   7   8   9   10   >