[jira] [Commented] (HBASE-15302) Reenable the other tests disabled by HBASE-14678

2016-02-22 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157901#comment-15157901 ] Duo Zhang commented on HBASE-15302: --- +1 if it pass pre commit again. > Reenable the other tests

[jira] [Created] (HBASE-15339) Add archive tiers for date based tiered compactions

2016-02-25 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-15339: - Summary: Add archive tiers for date based tiered compactions Key: HBASE-15339 URL: https://issues.apache.org/jira/browse/HBASE-15339 Project: HBase Issue Type:

[jira] [Updated] (HBASE-15339) Add archive tiers for date based tiered compaction

2016-02-25 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15339: -- Summary: Add archive tiers for date based tiered compaction (was: Add archive tiers for date based

[jira] [Commented] (HBASE-15339) Add archive tiers for date based tiered compaction

2016-02-27 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170916#comment-15170916 ] Duo Zhang commented on HBASE-15339: --- OK, I went through the patch. The window generating algorithm is

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-28 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15265: -- Attachment: HBASE-15265-v3.patch Addressing the comments on rb. > Implement an asynchronous FSHLog >

[jira] [Commented] (HBASE-15358) canEnforceTimeLimitFromScope should use timeScope instead of sizeScope

2016-02-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171592#comment-15171592 ] Duo Zhang commented on HBASE-15358: --- Yes this should be a typo. Can we add a testcase for it? >

[jira] [Commented] (HBASE-15358) canEnforceTimeLimitFromScope should use timeScope instead of sizeScope

2016-02-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171623#comment-15171623 ] Duo Zhang commented on HBASE-15358: --- Fine. Let's just commit this. > canEnforceTimeLimitFromScope

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Resolution: Fixed Hadoop Flags: Reviewed Release Note: Implement a fan-out asynchronous

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15265: -- Attachment: HBASE-15265.patch The patch is big so put it here for reviewing first. Will work on the

[jira] [Commented] (HBASE-15322) HBase 1.1.3 crashing

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15166641#comment-15166641 ] Duo Zhang commented on HBASE-15322: --- {{NoClassDefFoundError}} could happen when you have that class on

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15265: -- Status: Patch Available (was: Open) > Implement an asynchronous FSHLog >

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15265: -- Fix Version/s: 2.0.0 Component/s: wal > Implement an asynchronous FSHLog >

[jira] [Commented] (HBASE-15222) Use less contended classes for metrics

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15166829#comment-15166829 ] Duo Zhang commented on HBASE-15222: --- For {{TestRpcMetrics}}, first we need to add null check for two

[jira] [Comment Edited] (HBASE-15326) NPE in TestHRegion.testBatchPut_whileNoRowLocksHeld

2016-02-25 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15166950#comment-15166950 ] Duo Zhang edited comment on HBASE-15326 at 2/25/16 8:57 AM: Yes I have posted

[jira] [Commented] (HBASE-15326) NPE in TestHRegion.testBatchPut_whileNoRowLocksHeld

2016-02-25 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15166950#comment-15166950 ] Duo Zhang commented on HBASE-15326: --- Yes I have posted on that page.

[jira] [Issue Comment Deleted] (HBASE-15222) Use less contended classes for metrics

2016-02-25 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15222: -- Comment: was deleted (was: 你是中国人吗?不好意思,英语不好,想请教关于hbase的两个问题:

[jira] [Commented] (HBASE-15222) Use less contended classes for metrics

2016-02-24 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15166747#comment-15166747 ] Duo Zhang commented on HBASE-15222: --- Seems this causes several tests fail? See

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15265: -- Attachment: HBASE-15265-v4.patch Delete file if create on namenode succeeded but connect to datanode

[jira] [Commented] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171886#comment-15171886 ] Duo Zhang commented on HBASE-15265: --- The retry logic does not handle all the cases... Let me fix it...

[jira] [Updated] (HBASE-15358) canEnforceTimeLimitFromScope should use timeScope instead of sizeScope

2016-02-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15358: -- Affects Version/s: 1.4.0 1.3.0 2.0.0 Fix

[jira] [Resolved] (HBASE-15358) canEnforceTimeLimitFromScope should use timeScope instead of sizeScope

2016-02-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-15358. --- Resolution: Fixed Hadoop Flags: Reviewed Pushed to 1.1+. Thanks [~yangzhe1991] for the patch.

[jira] [Commented] (HBASE-15339) Add archive tiers for date based tiered compaction

2016-02-26 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170083#comment-15170083 ] Duo Zhang commented on HBASE-15339: --- Oh, I see the picture in the design doc, it is not the same with

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-27 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15265: -- Attachment: HBASE-15265-v2.patch Add some comments. TestAsyncLogRolling is passed locally, so add a

[jira] [Commented] (HBASE-15352) FST BlockEncoder

2016-02-26 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170424#comment-15170424 ] Duo Zhang commented on HBASE-15352: --- I used to think about this but there is a little difference

[jira] [Commented] (HBASE-15352) FST BlockEncoder

2016-02-26 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170433#comment-15170433 ] Duo Zhang commented on HBASE-15352: --- Oh, data block encoding, sorry, I have missed the keyword... Yes,

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-22 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158051#comment-15158051 ] Duo Zhang commented on HBASE-15264: --- [~stack] What do you think we should go next? On master or a new

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-26 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15265: -- Attachment: HBASE-15265-v1.patch Addressing the create retry and shutdown issue and also the comments

[jira] [Commented] (HBASE-15181) A simple implementation of date based tiered compaction

2016-02-25 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168302#comment-15168302 ] Duo Zhang commented on HBASE-15181: --- +1. This feature is very important for our MiCloud

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2016-01-21 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15111963#comment-15111963 ] Duo Zhang commented on HBASE-14790: ---

[jira] [Commented] (HBASE-14969) Add throughput controller for flush

2016-01-22 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15112438#comment-15112438 ] Duo Zhang commented on HBASE-14969: --- [~apurtell] In this patch we move and remove some classes marked

[jira] [Created] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-12 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-15265: - Summary: Implement an asynchronous FSHLog Key: HBASE-15265 URL: https://issues.apache.org/jira/browse/HBASE-15265 Project: HBase Issue Type: Sub-task

[jira] [Created] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-12 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-15264: - Summary: Implement a fan out HDFS OutputStream Key: HBASE-15264 URL: https://issues.apache.org/jira/browse/HBASE-15264 Project: HBase Issue Type: Sub-task

[jira] [Commented] (HBASE-14790) Implement a new DFSOutputStream for logging WAL only

2016-02-12 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145757#comment-15145757 ] Duo Zhang commented on HBASE-14790: --- {quote} E.g. gen stamp bumping, token renewal. {quote} Yes, this

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-13 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Attachment: HBASE-14949-v4.patch New patch addressing review comments from [~tedyu]. > Resolve name

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-15 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: HBASE-15264-v1.patch Addressing comments on rb and hadoop compatible issue. > Implement a

[jira] [Commented] (HBASE-13839) Fix AssgnmentManagerTmpl.jamon issues (coloring, content etc.)

2016-02-15 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-13839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147991#comment-15147991 ] Duo Zhang commented on HBASE-13839: --- [~stack] This causes the

[jira] [Commented] (HBASE-13839) Fix AssgnmentManagerTmpl.jamon issues (coloring, content etc.)

2016-02-15 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-13839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147997#comment-15147997 ] Duo Zhang commented on HBASE-13839: --- So you will do it [~stack]? > Fix AssgnmentManagerTmpl.jamon

[jira] [Commented] (HBASE-13839) Fix AssgnmentManagerTmpl.jamon issues (coloring, content etc.)

2016-02-15 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-13839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148001#comment-15148001 ] Duo Zhang commented on HBASE-13839: --- OK, I saw the new commit :) > Fix AssgnmentManagerTmpl.jamon

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-15 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: HBASE-15264-v2.patch Addressing the findbugs issue. > Implement a fan out HDFS

[jira] [Commented] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149675#comment-15149675 ] Duo Zhang commented on HBASE-14949: --- There are two places where we might have file name conflict, one

[jira] [Commented] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149677#comment-15149677 ] Duo Zhang commented on HBASE-14949: --- {quote} We still need the migration switch to not use new naming

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149715#comment-15149715 ] Duo Zhang commented on HBASE-15264: --- Basically, the stream works as follow 1. {{write}} will just copy

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149763#comment-15149763 ] Duo Zhang commented on HBASE-15264: --- There are several advantages 1. The fan out mechanism. This will

[jira] [Commented] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149921#comment-15149921 ] Duo Zhang commented on HBASE-14949: --- The problem here is we can not change the meaning of final file

[jira] [Commented] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149920#comment-15149920 ] Duo Zhang commented on HBASE-14949: --- The problem here is we can not change the meaning of final file

[jira] [Issue Comment Deleted] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Comment: was deleted (was: The problem here is we can not change the meaning of final file name

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149959#comment-15149959 ] Duo Zhang commented on HBASE-15264: --- [~stack] You can see this, it is almost done.

[jira] [Commented] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149982#comment-15149982 ] Duo Zhang commented on HBASE-14949: --- {quote} We use to have an isCreate flag. We don't have it anymore.

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-16 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150003#comment-15150003 ] Duo Zhang commented on HBASE-15264: --- {quote} That Payload looks like RingBufferTruck Is the RingBuffer

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-17 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: HBASE-15264-v3.patch Add comments. Remove unused methods and add methods for implementing

[jira] [Commented] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-17 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151568#comment-15151568 ] Duo Zhang commented on HBASE-14949: --- {quote} if my question on use of formatRecoveredEditsFileName from

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-17 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151576#comment-15151576 ] Duo Zhang commented on HBASE-15264: --- {quote} You want to commit this Duo Zhang to master branch ahead

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-17 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Attachment: HBASE-14949-v5.patch formatRecoveredEditsFileName only has external referenced from tests,

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-17 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Attachment: (was: HBASE-14949-v5.patch) > Resolve name conflict when splitting if there are

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-17 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Attachment: HBASE-14949-v5.patch Docker build failed... Retry... > Resolve name conflict when

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Fix Version/s: 1.3.0 2.0.0 > Resolve name conflict when splitting if there are

[jira] [Commented] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152161#comment-15152161 ] Duo Zhang commented on HBASE-14949: --- More than 7 hours... Let me check the failed testcases... >

[jira] [Commented] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152175#comment-15152175 ] Duo Zhang commented on HBASE-14949: --- Seems all failed tests are cause by error executing shell command,

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Component/s: wal > Resolve name conflict when splitting if there are duplicated WAL entries >

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available)

[jira] [Reopened] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-14949: --- branch-1 compilation error > Resolve name conflict when splitting if there are duplicated WAL entries >

[jira] [Updated] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-14949: -- Attachment: HBASE-14949-addendum-branch-1.patch On branch-1, we can not use getSequenceId instead of

[jira] [Resolved] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-14949. --- Resolution: Fixed Release Note: Now we can write duplicated WAL entries into different WAL

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: HBASE-15264-v4.patch Simple change for implementing HBASE-15265. > Implement a fan out

[jira] [Commented] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153676#comment-15153676 ] Duo Zhang commented on HBASE-15265: --- There are two problems here, so the comments of this test file.

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Fix Version/s: 2.0.0 Component/s: wal util Only in 2.0 first? [~stack] >

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-19 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153951#comment-15153951 ] Duo Zhang commented on HBASE-15264: --- Oh, I forgot an important thing, we can not close WAL writer

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-19 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153990#comment-15153990 ] Duo Zhang commented on HBASE-15264: --- Oh, master will move the RS's log directory to another place to

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-19 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154014#comment-15154014 ] Duo Zhang commented on HBASE-15264: --- So let's commit this and focus on HBASE-15265? [~stack] Thanks.

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-19 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155163#comment-15155163 ] Duo Zhang commented on HBASE-15264: --- Here we just want to split a big patch into several little ones,

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-19 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155254#comment-15155254 ] Duo Zhang commented on HBASE-15264: --- There is no problem if master renamed our WAL directory because we

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-19 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155257#comment-15155257 ] Duo Zhang commented on HBASE-15264: --- I placed them in regionserver.wal on the POC branch. And here, as

[jira] [Commented] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-19 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155351#comment-15155351 ] Duo Zhang commented on HBASE-15265: --- {quote} The way you describe it, current WAL implementation could

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-19 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: HBASE-15264-v5.patch Add createParent param for createOutput method. We need to set

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-20 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: HBASE-15264-v6.patch Forget to use the createParent param... Add a testcase to verify

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-21 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156412#comment-15156412 ] Duo Zhang commented on HBASE-15264: --- The javadoc and license issue can be fixed when committing. >

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-14 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Attachment: HBASE-15264.patch FanOutOneBlockAsyncDFSOutput implementation. > Implement a fan out HDFS

[jira] [Updated] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-14 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15264: -- Status: Patch Available (was: Open) > Implement a fan out HDFS OutputStream >

[jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream

2016-02-14 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15146518#comment-15146518 ] Duo Zhang commented on HBASE-15264: --- Let me try addressing the hadoop version issue... For checkstyle,

[jira] [Commented] (HBASE-14949) Resolve name conflict when splitting if there are duplicated WAL entries

2016-02-14 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15146452#comment-15146452 ] Duo Zhang commented on HBASE-14949: --- [~stack] This is needed by the {{AsyncFSHLog}}, PTAL. Thanks. >

[jira] [Created] (HBASE-15252) Data loss when replaying wal if HDFS timeout

2016-02-10 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-15252: - Summary: Data loss when replaying wal if HDFS timeout Key: HBASE-15252 URL: https://issues.apache.org/jira/browse/HBASE-15252 Project: HBase Issue Type: Bug

[jira] [Commented] (HBASE-15302) Reenable the other tests disabled by HBASE-14678

2016-02-21 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156550#comment-15156550 ] Duo Zhang commented on HBASE-15302: --- {{fileBeingSplit}} used to be {{isCreate}}, and after HBASE-14949

[jira] [Updated] (HBASE-15265) Implement an asynchronous FSHLog

2016-02-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15265: -- Attachment: HBASE-15265-v5.patch Forget to reset highestUnsyncedSequence... > Implement an

[jira] [Commented] (HBASE-15339) Add archive tiers for date based tiered compaction

2016-02-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173181#comment-15173181 ] Duo Zhang commented on HBASE-15339: --- For example, usually we consider hot data to be 'data that written

[jira] [Created] (HBASE-15365) Do not write to '/tmp' in TestHBaseConfiguration

2016-02-29 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-15365: - Summary: Do not write to '/tmp' in TestHBaseConfiguration Key: HBASE-15365 URL: https://issues.apache.org/jira/browse/HBASE-15365 Project: HBase Issue Type: Bug

[jira] [Updated] (HBASE-15365) Do not write to '/tmp' in TestHBaseConfiguration

2016-02-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15365: -- Status: Patch Available (was: Open) > Do not write to '/tmp' in TestHBaseConfiguration >

[jira] [Updated] (HBASE-15365) Do not write to '/tmp' in TestHBaseConfiguration

2016-02-29 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15365: -- Attachment: HBASE-15365.patch A trivial fix. Just write to a tmp directory and delete the directoty

[jira] [Updated] (HBASE-15365) Do not write to '/tmp' in TestHBaseConfiguration

2016-03-01 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15365: -- Attachment: HBASE-15365-v1.patch Use {{HBaseCommonTestingUtility}}. > Do not write to '/tmp' in

[jira] [Commented] (HBASE-15368) Add relative window support

2016-03-01 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174804#comment-15174804 ] Duo Zhang commented on HBASE-15368: --- And I could try both strategies in our cluster to see which one

[jira] [Commented] (HBASE-15368) Add relative window support

2016-03-01 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174868#comment-15174868 ] Duo Zhang commented on HBASE-15368: --- Yes, it will pull new data into older windows. So without writing

[jira] [Created] (HBASE-15374) Avoid using '/tmp' directory in our unit tests

2016-03-01 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-15374: - Summary: Avoid using '/tmp' directory in our unit tests Key: HBASE-15374 URL: https://issues.apache.org/jira/browse/HBASE-15374 Project: HBase Issue Type:

[jira] [Updated] (HBASE-15365) Do not write to '/tmp' in TestHBaseConfiguration

2016-03-01 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15365: -- Issue Type: Sub-task (was: Bug) Parent: HBASE-15374 > Do not write to '/tmp' in

[jira] [Commented] (HBASE-15368) Add relative window support

2016-03-01 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174792#comment-15174792 ] Duo Zhang commented on HBASE-15368: --- Oh, right. We should use minTimestamp instead of maxTimestamp

[jira] [Commented] (HBASE-15265) Implement an asynchronous FSHLog

2016-03-09 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188824#comment-15188824 ] Duo Zhang commented on HBASE-15265: --- [~stack] {code} LOG.info("Instantiating WALProvider of type " +

[jira] [Commented] (HBASE-15265) Implement an asynchronous FSHLog

2016-03-09 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188835#comment-15188835 ] Duo Zhang commented on HBASE-15265: --- {quote} Why is this option called asyncfs and not something like

[jira] [Commented] (HBASE-15265) Implement an asynchronous FSHLog

2016-03-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190311#comment-15190311 ] Duo Zhang commented on HBASE-15265: --- {quote} On the name, yeah, default is 'filesystem' but next one is

[jira] [Commented] (HBASE-15389) Write out multiple files when compaction

2016-03-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202381#comment-15202381 ] Duo Zhang commented on HBASE-15389: --- {quote} Why was TestStripeCompactor test removed? {quote} I just

[jira] [Commented] (HBASE-15481) Add pre/post roll to WALObserver

2016-03-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202547#comment-15202547 ] Duo Zhang commented on HBASE-15481: --- I agree with [~busbey] that we may have a WAL which does not rely

[jira] [Commented] (HBASE-15389) Write out multiple files when compaction

2016-03-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202553#comment-15202553 ] Duo Zhang commented on HBASE-15389: --- {quote} So HBASE-15400 sits on top of this, and without it, this

[jira] [Updated] (HBASE-15389) Write out multiple files when compaction

2016-03-18 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15389: -- Attachment: HBASE-15389-v13.patch Addressing the comments on RB. > Write out multiple files when

<    1   2   3   4   5   6   7   8   9   10   >