[jira] [Created] (HBASE-18130) Refactor ReplicationSource
Guanghao Zhang created HBASE-18130: -- Summary: Refactor ReplicationSource Key: HBASE-18130 URL: https://issues.apache.org/jira/browse/HBASE-18130 Project: HBase Issue Type: Improvement Reporter: Guanghao Zhang Assignee: Guanghao Zhang One basic idea is move the code about recovered queue to a new subclass RecoveredReplicationSource. Then ReplicationSource will don't need call isQueueRecovered many times. This will make the code more clearly. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18130) Refactor ReplicationSource
[ https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-18130: --- Attachment: HBASE-18130.master.001.patch > Refactor ReplicationSource > -- > > Key: HBASE-18130 > URL: https://issues.apache.org/jira/browse/HBASE-18130 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-18130.master.001.patch > > > One basic idea is move the code about recovered queue to a new subclass > RecoveredReplicationSource. Then ReplicationSource will don't need call > isQueueRecovered many times. This will make the code more clearly. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18130) Refactor ReplicationSource
[ https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-18130: --- Status: Patch Available (was: Open) > Refactor ReplicationSource > -- > > Key: HBASE-18130 > URL: https://issues.apache.org/jira/browse/HBASE-18130 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-18130.master.001.patch > > > One basic idea is move the code about recovered queue to a new subclass > RecoveredReplicationSource. Then ReplicationSource will don't need call > isQueueRecovered many times. This will make the code more clearly. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17887) Row-level consistency is broken for read
[ https://issues.apache.org/jira/browse/HBASE-17887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027786#comment-16027786 ] Mikhail Antonov commented on HBASE-17887: - I have been waiting for another issues that I though might make it to 1.3.2 (possibly RPC / cell format sanity checks), but seems like no point to wait. I'll spin up 1.3.2 tomorrow or Monday. > Row-level consistency is broken for read > > > Key: HBASE-17887 > URL: https://issues.apache.org/jira/browse/HBASE-17887 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 1.3.0 >Reporter: Umesh Agashe >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.3.2 > > Attachments: HBASE-17887.branch-1.v0.patch, > HBASE-17887.branch-1.v1.patch, HBASE-17887.branch-1.v1.patch, > HBASE-17887.branch-1.v2.patch, HBASE-17887.branch-1.v2.patch, > HBASE-17887.branch-1.v3.patch, HBASE-17887.branch-1.v4.patch, > HBASE-17887.branch-1.v4.patch, HBASE-17887.branch-1.v4.patch, > HBASE-17887.branch-1.v5.patch, HBASE-17887.branch-1.v6.patch, > HBASE-17887.ut.patch, HBASE-17887.v0.patch, HBASE-17887.v1.patch, > HBASE-17887.v2.patch, HBASE-17887.v3.patch, HBASE-17887.v4.patch, > HBASE-17887.v5.patch, HBASE-17887.v5.patch > > > The scanner of latest memstore may be lost if we make quick flushes. The > following step may help explain this issue. > # put data_A (seq id = 10, active store data_A and snapshots is empty) > # snapshot of 1st flush (active is empty and snapshot stores data_A) > # put data_B (seq id = 11, active store data_B and snapshot store data_A) > # create user scanner (read point = 11, so It should see the data_B) > # commit of 1st flush > #* clear snapshot ((hfile_A has data_A, active store data_B, and snapshot is > empty) > #* update the reader (the user scanner receives the hfile_A) > # snapshot of 2st flush (active is empty and snapshot store data_B) > # commit of 2st flush > #* clear snapshot (hfile_A has data_A, hfile_B has data_B, active is empty, > and snapshot is empty) – this is critical piece. > #* -update the reader- (haven't happen) > # user scanner update the kv scanners (it creates scanner of hfile_A but > nothing of memstore) > # user see the older data A – wrong result -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18130) Refactor ReplicationSource
[ https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027817#comment-16027817 ] Hadoop QA commented on HBASE-18130: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 33s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 2 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 22s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 21s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 29s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 24s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 7s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 29s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 21s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 21s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 27s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 25s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 56m 51s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 22s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 191m 20s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 26s {color} | {color:red} The patch generated 4 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 274m 17s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12870239/HBASE-18130.master.001.patch | | JIRA Issue | HBASE-18130 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2706d026b127 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / 6846b03 | | Default Java | 1.8.0_131 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/6987/artifact/patchprocess/patch-unit-hbase-server.txt | | unit test logs | https://builds.apache.org/job/PreCommit-HBASE-Build/6987/artifact/patchprocess/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/6987/testReport/ | | asflicense | https://builds.apache.org/job/PreCommit-HBASE-Build/6987/artifact/patchprocess/patch-asflicense-problems.txt | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/6987/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > Refac
[jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027874#comment-16027874 ] Jonathan Lawlor commented on HBASE-11544: - [~karanmehta93] Only the last result should be a partial. Repeated is likely a bug. Please file an issue > [Ergonomics] hbase.client.scanner.caching is dogged and will try to return > batch even if it means OOME > -- > > Key: HBASE-11544 > URL: https://issues.apache.org/jira/browse/HBASE-11544 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Jonathan Lawlor >Priority: Critical > Fix For: 2.0.0, 1.1.0 > > Attachments: Allocation_Hot_Spots.html, gc.j.png, > HBASE-11544-addendum-v1.patch, HBASE-11544-addendum-v2.patch, > HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, > HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, > HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, > HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v7.patch, > HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch, hits.j.png, h.png, > mean.png, m.png, net.j.png, q (2).png > > > Running some tests, I set hbase.client.scanner.caching=1000. Dataset has > large cells. I kept OOME'ing. > Serverside, we should measure how much we've accumulated and return to the > client whatever we've gathered once we pass out a certain size threshold > rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-13603) Write test asserting desired priority of RS->Master RPCs
[ https://issues.apache.org/jira/browse/HBASE-13603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13603: Fix Version/s: (was: 1.2.6) 1.2.7 > Write test asserting desired priority of RS->Master RPCs > > > Key: HBASE-13603 > URL: https://issues.apache.org/jira/browse/HBASE-13603 > Project: HBase > Issue Type: Test > Components: IPC/RPC, test >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Minor > Fix For: 2.0.0, 1.1.7, 1.3.2, 1.2.7 > > > From HBASE-13351: > {quote} > Any way we can write a FT test to assert that the RS->Master APIs are treated > with higher priority. I see your UT for asserting the annotation. > {quote} > Write a test that verifies expected RPCs are run on the correct pools in as > real of an environment possible. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17453) add Ping into HBase server for deprecated GetProtocolVersion
[ https://issues.apache.org/jira/browse/HBASE-17453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-17453: Fix Version/s: (was: 1.2.6) 1.4.0 > add Ping into HBase server for deprecated GetProtocolVersion > > > Key: HBASE-17453 > URL: https://issues.apache.org/jira/browse/HBASE-17453 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 1.2.2 >Reporter: Tianying Chang >Assignee: Tianying Chang >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17453-1.2.patch, > HBASE-17453-master-fixWhiteSpace.patch, HBASE-17453-master.patch, > HBASE-17453-master-v1.patch, HBASE-17453-master-v2.patch > > > Our HBase service is hosted in AWS. We saw cases where the connection between > the client (Asynchbase in our case) and server stop working but did not throw > any exception, therefore traffic stuck. So we added a "Ping" feature in > AsyncHBase 1.5 by utilizing the GetProtocolVersion() API provided at RS side, > if no traffic for given time, we send the "Ping", if no response back for > "Ping", we assume the connect is bad and reconnect. > Now we are upgrading cluster from 94 to 1.2. However, GetProtocolVersion() is > deprecated. To be able to support same detect/reconnect feature, we added > Ping() in our internal HBase 1.2 branch, and also patched accordingly in > Asynchbase 1.7. > We would like to open source this feature since it is useful for use case in > AWS environment. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17453) add Ping into HBase server for deprecated GetProtocolVersion
[ https://issues.apache.org/jira/browse/HBASE-17453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16028026#comment-16028026 ] Sean Busbey commented on HBASE-17453: - What's the word here? Is this switching to AdminProtos or staying in ClientProtos? > add Ping into HBase server for deprecated GetProtocolVersion > > > Key: HBASE-17453 > URL: https://issues.apache.org/jira/browse/HBASE-17453 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 1.2.2 >Reporter: Tianying Chang >Assignee: Tianying Chang >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17453-1.2.patch, > HBASE-17453-master-fixWhiteSpace.patch, HBASE-17453-master.patch, > HBASE-17453-master-v1.patch, HBASE-17453-master-v2.patch > > > Our HBase service is hosted in AWS. We saw cases where the connection between > the client (Asynchbase in our case) and server stop working but did not throw > any exception, therefore traffic stuck. So we added a "Ping" feature in > AsyncHBase 1.5 by utilizing the GetProtocolVersion() API provided at RS side, > if no traffic for given time, we send the "Ping", if no response back for > "Ping", we assume the connect is bad and reconnect. > Now we are upgrading cluster from 94 to 1.2. However, GetProtocolVersion() is > deprecated. To be able to support same detect/reconnect feature, we added > Ping() in our internal HBase 1.2 branch, and also patched accordingly in > Asynchbase 1.7. > We would like to open source this feature since it is useful for use case in > AWS environment. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-8758) Error in RegionCoprocessorHost class preScanner method documentation.
[ https://issues.apache.org/jira/browse/HBASE-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-8758: --- Fix Version/s: (was: 1.2.7) 1.2.6 > Error in RegionCoprocessorHost class preScanner method documentation. > - > > Key: HBASE-8758 > URL: https://issues.apache.org/jira/browse/HBASE-8758 > Project: HBase > Issue Type: Bug > Components: Coprocessors, documentation >Affects Versions: 0.98.0, 0.95.2, 0.94.9 > Environment: Any. Actually it is just wrong comment in code. >Reporter: Roman Nikitchenko >Assignee: Roman Nikitchenko >Priority: Minor > Labels: beginner, comments, coprocessors, documentation > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2 > > Attachments: 8758-r1545168.patch, HBASE-8758-r1545168.patch, > HBASE-8758-r1545168.patch > > > preScannerOpen() method of RegionCoprocessorHost class is documented to > return 'false' value in negative case (default operation should not be > bypassed). Actual implementation returns 'null' value. > Proposed solution is just to correct comment to match existing implementation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (HBASE-8758) Error in RegionCoprocessorHost class preScanner method documentation.
[ https://issues.apache.org/jira/browse/HBASE-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned HBASE-8758: -- Assignee: Roman Nikitchenko > Error in RegionCoprocessorHost class preScanner method documentation. > - > > Key: HBASE-8758 > URL: https://issues.apache.org/jira/browse/HBASE-8758 > Project: HBase > Issue Type: Bug > Components: Coprocessors, documentation >Affects Versions: 0.98.0, 0.95.2, 0.94.9 > Environment: Any. Actually it is just wrong comment in code. >Reporter: Roman Nikitchenko >Assignee: Roman Nikitchenko >Priority: Minor > Labels: beginner, comments, coprocessors, documentation > Fix For: 2.0.0, 1.4.0, 1.2.6, 1.3.2 > > Attachments: 8758-r1545168.patch, HBASE-8758-r1545168.patch, > HBASE-8758-r1545168.patch > > > preScannerOpen() method of RegionCoprocessorHost class is documented to > return 'false' value in negative case (default operation should not be > bypassed). Actual implementation returns 'null' value. > Proposed solution is just to correct comment to match existing implementation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17890) FuzzyRowFilter fail if unaligned support is false
[ https://issues.apache.org/jira/browse/HBASE-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-17890: Fix Version/s: (was: 1.2.6) 1.2.7 > FuzzyRowFilter fail if unaligned support is false > - > > Key: HBASE-17890 > URL: https://issues.apache.org/jira/browse/HBASE-17890 > Project: HBase > Issue Type: Sub-task > Components: util >Affects Versions: 2.0.0, 1.2.5 >Reporter: Jerry He >Assignee: Chia-Ping Tsai > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7 > > Attachments: HBASE-17890.v0.branch-1.patch, HBASE-17890.v0.patch, > HBASE-17890.v1.branch-1.patch, HBASE-17890.v1.patch, HBASE-17890.v2.patch, > HBASE-17890.v3.patch, HBASE-17890.v3.patch, HBASE-17890.v3.patch, > HBASE-17890.v3.patch, HBASE-17890.v3.patch > > > When unaligned support is false, FuzzyRow tests fail: > {noformat} > Failed tests: > TestFuzzyRowAndColumnRangeFilter.Test:134->runTest:157->runScanner:186 > expected:<10> but was:<0> > TestFuzzyRowFilter.testSatisfiesForward:81 expected: but was: > TestFuzzyRowFilter.testSatisfiesReverse:121 expected: but > was: > TestFuzzyRowFilterEndToEnd.testEndToEnd:247->runTest1:278->runScanner:343 > expected:<6250> but was:<0> > TestFuzzyRowFilterEndToEnd.testFilterList:385->runTest:417->runScanner:445 > expected:<5> but was:<0> > TestFuzzyRowFilterEndToEnd.testHBASE14782:204 expected:<6> but was:<0> > {noformat} > This can be reproduced in the case described in HBASE-17869. Or on a platform > really without unaligned support. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Work started] (HBASE-17876) Release 1.2.6
[ https://issues.apache.org/jira/browse/HBASE-17876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-17876 started by Sean Busbey. --- > Release 1.2.6 > - > > Key: HBASE-17876 > URL: https://issues.apache.org/jira/browse/HBASE-17876 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 1.2.6 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17885) Backport HBASE-15871 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-17885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-17885: Fix Version/s: (was: 1.2.6) 1.2.7 > Backport HBASE-15871 to branch-1 > > > Key: HBASE-17885 > URL: https://issues.apache.org/jira/browse/HBASE-17885 > Project: HBase > Issue Type: Bug > Components: Scanners >Affects Versions: 1.3.1, 1.2.5, 1.1.8 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 1.3.2, 1.1.11, 1.2.7 > > > Will try to rebase the branch-1 patch at the earliest. Hope the fix versions > are correct. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15691) Port HBASE-10205 (ConcurrentModificationException in BucketAllocator) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15691: Fix Version/s: (was: 1.2.6) 1.2.7 > Port HBASE-10205 (ConcurrentModificationException in BucketAllocator) to > branch-1 > - > > Key: HBASE-15691 > URL: https://issues.apache.org/jira/browse/HBASE-15691 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.0 >Reporter: Andrew Purtell >Assignee: Stephen Yuan Jiang > Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7 > > Attachments: HBASE-15691-branch-1.patch, HBASE-15691.v2-branch-1.patch > > > HBASE-10205 was committed to trunk and 0.98 branches only. To preserve > continuity we should commit it to branch-1. The change requires more than > nontrivial fixups so I will attach a backport of the change from trunk to > current branch-1 here. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15580) Tag coprocessor limitedprivate scope to StoreFile.Reader
[ https://issues.apache.org/jira/browse/HBASE-15580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15580: Fix Version/s: 1.3.2 1.4.0 > Tag coprocessor limitedprivate scope to StoreFile.Reader > > > Key: HBASE-15580 > URL: https://issues.apache.org/jira/browse/HBASE-15580 > Project: HBase > Issue Type: Improvement >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.1.11, 1.2.7 > > Attachments: HBASE-15580_branch-1.0.patch, HBASE-15580.patch > > > For phoenix local indexing we need to have custom storefile reader > constructor(IndexHalfStoreFileReader) to distinguish from other storefile > readers. So wanted to mark StoreFile.Reader scope as > InterfaceAudience.LimitedPrivate("Coprocessor") -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15580) Tag coprocessor limitedprivate scope to StoreFile.Reader
[ https://issues.apache.org/jira/browse/HBASE-15580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15580: Fix Version/s: (was: 1.2.6) (was: 1.1.7) (was: 1.0.4) 1.2.7 1.1.11 > Tag coprocessor limitedprivate scope to StoreFile.Reader > > > Key: HBASE-15580 > URL: https://issues.apache.org/jira/browse/HBASE-15580 > Project: HBase > Issue Type: Improvement >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.1.11, 1.2.7 > > Attachments: HBASE-15580_branch-1.0.patch, HBASE-15580.patch > > > For phoenix local indexing we need to have custom storefile reader > constructor(IndexHalfStoreFileReader) to distinguish from other storefile > readers. So wanted to mark StoreFile.Reader scope as > InterfaceAudience.LimitedPrivate("Coprocessor") -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15302) Reenable the other tests disabled by HBASE-14678
[ https://issues.apache.org/jira/browse/HBASE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16028048#comment-16028048 ] Sean Busbey commented on HBASE-15302: - we have commits for this issue on master and branch-1 from ages ago. {code} Busbey-MBA:hbase busbey$ git log --grep HBASE-15302 origin/master commit 30cec72f9ade972d7e9ce4bba527b0e6074cae60 Author: Phil Yang Date: Mon Feb 22 14:17:24 2016 +0800 HBASE-15302 Reenable the other tests disabled by HBASE-14678 Signed-off-by: stack Busbey-MBA:hbase busbey$ git log --grep HBASE-15302 origin/branch-1 commit 6977c5dee5edb71deba91c8884d18940f448fcf6 Author: tedyu Date: Fri Mar 18 20:56:03 2016 -0700 HBASE-15302 Revert TestWALSplitCompressed which extends TestWALSplit commit 27d59d3ebb7466a601ae66db32ed9ff34dfd213a Author: tedyu Date: Fri Mar 18 20:53:29 2016 -0700 HBASE-15302 Revert two tests which hung in build #31 commit d942d4e5a1fe5644a51b5775f3a3e27dd219ad1d Author: tedyu Date: Fri Mar 18 09:06:34 2016 -0700 HBASE-15302 Reenable the other tests disabled by HBASE-14678 (Phil Yang) Busbey-MBA:hbase busbey$ git log --grep HBASE-15302 origin/branch-1.3 Busbey-MBA:hbase busbey$ git log --grep HBASE-15302 origin/branch-1.2 Busbey-MBA:hbase busbey$ git log --grep HBASE-15302 origin/branch-1.1 Busbey-MBA:hbase busbey$ {code} Can we resolve this issue with just those commits and track any additional work with a different jira? > Reenable the other tests disabled by HBASE-14678 > > > Key: HBASE-15302 > URL: https://issues.apache.org/jira/browse/HBASE-15302 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7 > > Attachments: HBASE-15302-branch-1.3-append-v1.patch, > HBASE-15302-branch-1.3-append-v1.patch, HBASE-15302-branch-1-append-v1.patch, > HBASE-15302-branch-1-v1.patch, HBASE-15302-v1.txt, HBASE-15302-v1.txt > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-15302) Reenable the other tests disabled by HBASE-14678
[ https://issues.apache.org/jira/browse/HBASE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15302: Fix Version/s: (was: 1.2.6) 1.2.7 > Reenable the other tests disabled by HBASE-14678 > > > Key: HBASE-15302 > URL: https://issues.apache.org/jira/browse/HBASE-15302 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.2.7 > > Attachments: HBASE-15302-branch-1.3-append-v1.patch, > HBASE-15302-branch-1.3-append-v1.patch, HBASE-15302-branch-1-append-v1.patch, > HBASE-15302-branch-1-v1.patch, HBASE-15302-v1.txt, HBASE-15302-v1.txt > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14610) IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client
[ https://issues.apache.org/jira/browse/HBASE-14610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14610: Fix Version/s: (was: 1.2.6) (was: 1.1.7) (was: 1.0.4) 1.2.7 1.1.11 1.4.0 > IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client > -- > > Key: HBASE-14610 > URL: https://issues.apache.org/jira/browse/HBASE-14610 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: Enis Soztutar > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.1.11, 1.2.7 > > Attachments: output > > > HBASE-14535 introduces an IT to simulate a running cluster with RPC servers > and RPC clients doing requests against the servers. > It passes with the sync client, but fails with async client. Probably we need > to take a look. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14223) Meta WALs are not cleared if meta region was closed and RS aborts
[ https://issues.apache.org/jira/browse/HBASE-14223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14223: Fix Version/s: (was: 1.2.6) (was: 1.1.7) (was: 1.0.4) 1.2.7 1.1.11 1.4.0 > Meta WALs are not cleared if meta region was closed and RS aborts > - > > Key: HBASE-14223 > URL: https://issues.apache.org/jira/browse/HBASE-14223 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.1.11, 1.2.7 > > Attachments: HBASE-14223logs, hbase-14223_v0.patch, > hbase-14223_v1-branch-1.patch, hbase-14223_v2-branch-1.patch, > hbase-14223_v3-branch-1.patch, hbase-14223_v3-branch-1.patch, > hbase-14223_v3-master.patch > > > When an RS opens meta, and later closes it, the WAL(FSHlog) is not closed. > The last WAL file just sits there in the RS WAL directory. If RS stops > gracefully, the WAL file for meta is deleted. Otherwise if RS aborts, WAL for > meta is not cleaned. It is also not split (which is correct) since master > determines that the RS no longer hosts meta at the time of RS abort. > From a cluster after running ITBLL with CM, I see a lot of {{-splitting}} > directories left uncleaned: > {code} > [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls > /apps/hbase/data/WALs > Found 31 items > drwxr-xr-x - hbase hadoop 0 2015-06-05 01:14 > /apps/hbase/data/WALs/hregion-58203265 > drwxr-xr-x - hbase hadoop 0 2015-06-05 07:54 > /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433489308745-splitting > drwxr-xr-x - hbase hadoop 0 2015-06-05 09:28 > /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433494382959-splitting > drwxr-xr-x - hbase hadoop 0 2015-06-05 10:01 > /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433498252205-splitting > ... > {code} > The directories contain WALs from meta: > {code} > [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls > /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting > Found 2 items > -rw-r--r-- 3 hbase hadoop 201608 2015-06-05 03:15 > /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta > -rw-r--r-- 3 hbase hadoop 44420 2015-06-05 04:36 > /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta > {code} > The RS hosted the meta region for some time: > {code} > 2015-06-05 03:14:28,692 INFO [PostOpenDeployTasks:1588230740] > zookeeper.MetaTableLocator: Setting hbase:meta region location in ZooKeeper > as os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285 > ... > 2015-06-05 03:15:17,302 INFO > [RS_CLOSE_META-os-enis-dal-test-jun-4-5:16020-0] regionserver.HRegion: Closed > hbase:meta,,1.1588230740 > {code} > In between, a WAL is created: > {code} > 2015-06-05 03:15:11,707 INFO > [RS_OPEN_META-os-enis-dal-test-jun-4-5:16020-0-MetaLogRoller] wal.FSHLog: > Rolled WAL > /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta > with entries=385, filesize=196.88 KB; new WAL > /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta > {code} > When CM killed the region server later master did not see these WAL files: > {code} > ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:46,075 > INFO [MASTER_SERVER_OPERATIONS-os-enis-dal-test-jun-4-3:16000-0] > master.SplitLogManager: started splitting 2 logs in > [hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting] > for [os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285] > ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:47,300 > INFO [main-EventThread] wal.WALSplitter: Archived processed log > hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.1433475074436 > to > hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/oldWALs/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.14334750
[jira] [Updated] (HBASE-14391) Empty regionserver WAL will never be deleted although the coresponding regionserver has been stale
[ https://issues.apache.org/jira/browse/HBASE-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14391: Fix Version/s: 1.4.0 > Empty regionserver WAL will never be deleted although the coresponding > regionserver has been stale > -- > > Key: HBASE-14391 > URL: https://issues.apache.org/jira/browse/HBASE-14391 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 1.0.2 >Reporter: Qianxi Zhang >Assignee: Qianxi Zhang > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.1.11, 1.2.7 > > Attachments: HBASE-14391-master-v3.patch, > HBASE_14391_master_v4.patch, HBASE_14391_trunk_v1.patch, > HBASE_14391_trunk_v2.patch, WALs-leftover-dir.txt > > > When I restarted the hbase cluster in which there was few data, I found there > are two directories for one host with different timestamp which indicates > that the old regionserver wal directory is not deleted. > FHLog#989 > {code} > @Override > public void close() throws IOException { > shutdown(); > final FileStatus[] files = getFiles(); > if (null != files && 0 != files.length) { > for (FileStatus file : files) { > Path p = getWALArchivePath(this.fullPathArchiveDir, file.getPath()); > // Tell our listeners that a log is going to be archived. > if (!this.listeners.isEmpty()) { > for (WALActionsListener i : this.listeners) { > i.preLogArchive(file.getPath(), p); > } > } > if (!FSUtils.renameAndSetModifyTime(fs, file.getPath(), p)) { > throw new IOException("Unable to rename " + file.getPath() + " to " > + p); > } > // Tell our listeners that a log was archived. > if (!this.listeners.isEmpty()) { > for (WALActionsListener i : this.listeners) { > i.postLogArchive(file.getPath(), p); > } > } > } > LOG.debug("Moved " + files.length + " WAL file(s) to " + > FSUtils.getPath(this.fullPathArchiveDir)); > } > LOG.info("Closed WAL: " + toString()); > } > {code} > When regionserver is stopped, the hlog will be archived, so wal/regionserver > is empty in hdfs. > MasterFileSystem#252 > {code} > if (curLogFiles == null || curLogFiles.length == 0) { > // Empty log folder. No recovery needed > continue; > } > {code} > The regionserver directory will be not splitted, it makes sense. But it will > be not deleted. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14391) Empty regionserver WAL will never be deleted although the coresponding regionserver has been stale
[ https://issues.apache.org/jira/browse/HBASE-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14391: Fix Version/s: (was: 1.2.6) (was: 1.1.7) 1.2.7 1.1.11 > Empty regionserver WAL will never be deleted although the coresponding > regionserver has been stale > -- > > Key: HBASE-14391 > URL: https://issues.apache.org/jira/browse/HBASE-14391 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 1.0.2 >Reporter: Qianxi Zhang >Assignee: Qianxi Zhang > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.1.11, 1.2.7 > > Attachments: HBASE-14391-master-v3.patch, > HBASE_14391_master_v4.patch, HBASE_14391_trunk_v1.patch, > HBASE_14391_trunk_v2.patch, WALs-leftover-dir.txt > > > When I restarted the hbase cluster in which there was few data, I found there > are two directories for one host with different timestamp which indicates > that the old regionserver wal directory is not deleted. > FHLog#989 > {code} > @Override > public void close() throws IOException { > shutdown(); > final FileStatus[] files = getFiles(); > if (null != files && 0 != files.length) { > for (FileStatus file : files) { > Path p = getWALArchivePath(this.fullPathArchiveDir, file.getPath()); > // Tell our listeners that a log is going to be archived. > if (!this.listeners.isEmpty()) { > for (WALActionsListener i : this.listeners) { > i.preLogArchive(file.getPath(), p); > } > } > if (!FSUtils.renameAndSetModifyTime(fs, file.getPath(), p)) { > throw new IOException("Unable to rename " + file.getPath() + " to " > + p); > } > // Tell our listeners that a log was archived. > if (!this.listeners.isEmpty()) { > for (WALActionsListener i : this.listeners) { > i.postLogArchive(file.getPath(), p); > } > } > } > LOG.debug("Moved " + files.length + " WAL file(s) to " + > FSUtils.getPath(this.fullPathArchiveDir)); > } > LOG.info("Closed WAL: " + toString()); > } > {code} > When regionserver is stopped, the hlog will be archived, so wal/regionserver > is empty in hdfs. > MasterFileSystem#252 > {code} > if (curLogFiles == null || curLogFiles.length == 0) { > // Empty log folder. No recovery needed > continue; > } > {code} > The regionserver directory will be not splitted, it makes sense. But it will > be not deleted. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-13587) TestSnapshotCloneIndependence.testOnlineSnapshotDeleteIndependent is flakey
[ https://issues.apache.org/jira/browse/HBASE-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13587: Fix Version/s: (was: 1.2.6) (was: 1.1.7) 1.2.7 1.1.11 1.4.0 > TestSnapshotCloneIndependence.testOnlineSnapshotDeleteIndependent is flakey > --- > > Key: HBASE-13587 > URL: https://issues.apache.org/jira/browse/HBASE-13587 > Project: HBase > Issue Type: Test >Reporter: Nick Dimiduk > Fix For: 2.0.0, 1.4.0, 1.3.2, 1.1.11, 1.2.7 > > > Looking at our [build > history|https://builds.apache.org/job/HBase-1.1/buildTimeTrend], it seems > this test is flakey. See builds 428, 431, 432, 433. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17716) Formalize Scan Metric names
[ https://issues.apache.org/jira/browse/HBASE-17716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-17716: Fix Version/s: (was: 1.2.5) 1.2.6 > Formalize Scan Metric names > --- > > Key: HBASE-17716 > URL: https://issues.apache.org/jira/browse/HBASE-17716 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Karan Mehta >Assignee: Karan Mehta >Priority: Minor > Fix For: 2.0.0, 1.3.1, 1.2.6 > > Attachments: HBASE-17716.patch, HBASE-17716_v2.patch, > HBASE-17716_v2.patch > > > HBase provides various metrics through the API's exposed by ScanMetrics > class. > The JIRA PHOENIX-3248 requires them to be surfaced through the Phoenix > Metrics API. Currently these metrics are referred via hard-coded strings, > which are not formal and can break the Phoenix API. Hence we need to refactor > the code to assign enums for these metrics. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-17722) Metrics subsystem stop/start messages add a lot of useless bulk to operational logging
[ https://issues.apache.org/jira/browse/HBASE-17722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-17722: Fix Version/s: (was: 1.2.5) 1.2.6 > Metrics subsystem stop/start messages add a lot of useless bulk to > operational logging > -- > > Key: HBASE-17722 > URL: https://issues.apache.org/jira/browse/HBASE-17722 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 1.3.0, 1.2.4 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.1.10, 1.2.6 > > Attachments: HBASE-17722.patch, HBASE-17722.patch > > > Metrics subsystem stop/start messages add a lot of useless bulk to > operational logging. Say you are collecting logs from a fleet of thousands of > servers and want to have them around for ~month or longer. It adds up. > I think these should at least be at DEBUG level and ideally at TRACE. They > don't offer much utility. Unfortunately they are Hadoop classes so we can > tweak log4j.properties defaults instead. We do this in test resources but not > in what we ship in conf/ . > {noformat} > INFO [] impl.MetricsSystemImpl: HBase metrics system started > INFO [] impl.MetricsSystemImpl: Stopping HBase metrics > system... > INFO [] impl.MetricsSystemImpl: HBase metrics system stopped. > INFO [] impl.MetricsConfig: loaded properties from > hadoop-metrics2-hbase.properties > INFO [] impl.MetricsSystemImpl: Scheduled snapshot period at > 10 second(s). > {noformat} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17876) Release 1.2.6
[ https://issues.apache.org/jira/browse/HBASE-17876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16028076#comment-16028076 ] Hudson commented on HBASE-17876: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #876 (See [https://builds.apache.org/job/HBase-1.2-IT/876/]) HBASE-17876 update version to 1.2.6 for RC0. (busbey: rev f6ecbf972e15da81718e44ba44a0c2a780a45c1e) * (edit) hbase-prefix-tree/pom.xml * (edit) hbase-external-blockcache/pom.xml * (edit) hbase-it/pom.xml * (edit) hbase-server/pom.xml * (edit) hbase-common/pom.xml * (edit) hbase-shaded/hbase-shaded-server/pom.xml * (edit) hbase-thrift/pom.xml * (edit) hbase-resource-bundle/pom.xml * (edit) hbase-annotations/pom.xml * (edit) hbase-assembly/pom.xml * (edit) hbase-examples/pom.xml * (edit) hbase-rest/pom.xml * (edit) hbase-shaded/hbase-shaded-client/pom.xml * (edit) pom.xml * (edit) hbase-checkstyle/pom.xml * (edit) hbase-procedure/pom.xml * (edit) hbase-hadoop2-compat/pom.xml * (edit) hbase-client/pom.xml * (edit) hbase-testing-util/pom.xml * (edit) hbase-shaded/pom.xml * (edit) hbase-shell/pom.xml * (edit) hbase-hadoop-compat/pom.xml * (edit) hbase-protocol/pom.xml HBASE-17876 update CHANGES.txt for 1.2.6 RC0. (busbey: rev 2f9b9e17d0522e36063bf52ecc58576243d20b3f) * (edit) CHANGES.txt > Release 1.2.6 > - > > Key: HBASE-17876 > URL: https://issues.apache.org/jira/browse/HBASE-17876 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 1.2.6 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)