[jira] [Updated] (HBASE-14422) Fix TestFastFailWithoutTestUtil
[ https://issues.apache.org/jira/browse/HBASE-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Ryakhovskiy updated HBASE-14422: --- Status: Patch Available (was: Open) > Fix TestFastFailWithoutTestUtil > --- > > Key: HBASE-14422 > URL: https://issues.apache.org/jira/browse/HBASE-14422 > Project: HBase > Issue Type: Task > Components: test >Reporter: stack >Assignee: Konstantin Ryakhovskiy >Priority: Minor > Labels: beginner > Attachments: HBASE-14422.master.001.patch, > HBASE-14422.master.002.patch, HBASE-14422.master.003.patch, > HBASE-14422.master.004.patch, log.txt, trace.log > > > TestFastFailWithoutTestUtil has a unit test that does > testInterceptorIntercept50Times Usually it passes but on occasion, the > latching between thread 1 and thread 2 goes awry and the test hangs and the > test hangs out. Depends on the hardware but it seems to happen about one in > four runs here on an internal rig. > HBASE-14421 changed the wait-on-latch to timeout and do a thread dump and > just let the test keep going. > This issue is about digging in on figuring why the hang up on latches and > then fixing it so the test doesn't have to have the latch timeout. Hopefully > the threaddump helps. > This one could be hard to fix since it not easy to reproduce. Marking it > beginner anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14422) Fix TestFastFailWithoutTestUtil
[ https://issues.apache.org/jira/browse/HBASE-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Ryakhovskiy updated HBASE-14422: --- Attachment: HBASE-14422.master.004.patch re-submitting, fix whitespaces > Fix TestFastFailWithoutTestUtil > --- > > Key: HBASE-14422 > URL: https://issues.apache.org/jira/browse/HBASE-14422 > Project: HBase > Issue Type: Task > Components: test >Reporter: stack >Assignee: Konstantin Ryakhovskiy >Priority: Minor > Labels: beginner > Attachments: HBASE-14422.master.001.patch, > HBASE-14422.master.002.patch, HBASE-14422.master.003.patch, > HBASE-14422.master.004.patch, log.txt, trace.log > > > TestFastFailWithoutTestUtil has a unit test that does > testInterceptorIntercept50Times Usually it passes but on occasion, the > latching between thread 1 and thread 2 goes awry and the test hangs and the > test hangs out. Depends on the hardware but it seems to happen about one in > four runs here on an internal rig. > HBASE-14421 changed the wait-on-latch to timeout and do a thread dump and > just let the test keep going. > This issue is about digging in on figuring why the hang up on latches and > then fixing it so the test doesn't have to have the latch timeout. Hopefully > the threaddump helps. > This one could be hard to fix since it not easy to reproduce. Marking it > beginner anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14422) Fix TestFastFailWithoutTestUtil
[ https://issues.apache.org/jira/browse/HBASE-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Ryakhovskiy updated HBASE-14422: --- Status: Open (was: Patch Available) > Fix TestFastFailWithoutTestUtil > --- > > Key: HBASE-14422 > URL: https://issues.apache.org/jira/browse/HBASE-14422 > Project: HBase > Issue Type: Task > Components: test >Reporter: stack >Assignee: Konstantin Ryakhovskiy >Priority: Minor > Labels: beginner > Attachments: HBASE-14422.master.001.patch, > HBASE-14422.master.002.patch, HBASE-14422.master.003.patch, log.txt, trace.log > > > TestFastFailWithoutTestUtil has a unit test that does > testInterceptorIntercept50Times Usually it passes but on occasion, the > latching between thread 1 and thread 2 goes awry and the test hangs and the > test hangs out. Depends on the hardware but it seems to happen about one in > four runs here on an internal rig. > HBASE-14421 changed the wait-on-latch to timeout and do a thread dump and > just let the test keep going. > This issue is about digging in on figuring why the hang up on latches and > then fixing it so the test doesn't have to have the latch timeout. Hopefully > the threaddump helps. > This one could be hard to fix since it not easy to reproduce. Marking it > beginner anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16156) In write heavy scenario creating in memory flushes leads to contention
[ https://issues.apache.org/jira/browse/HBASE-16156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358503#comment-15358503 ] ramkrishna.s.vasudevan commented on HBASE-16156: Yes. In my testing I had already removed these logs that were coming a lot of times. > In write heavy scenario creating in memory flushes leads to contention > -- > > Key: HBASE-16156 > URL: https://issues.apache.org/jira/browse/HBASE-16156 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > > In write heavy cases, the inmemory flushes blocks the write because it takes > a update lock. Dumps show that it is causing heavy contention and leads to > memstore filling up and so intern blocks the flush from happening sooner. > This JIRA is to discuss optimal settings and then make suitable changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16144) Replication queue's lock will live forever if RS acquiring the lock has died prematurely
[ https://issues.apache.org/jira/browse/HBASE-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358501#comment-15358501 ] Phil Yang commented on HBASE-16144: --- Failed tests are unrelated and can pass locally > Replication queue's lock will live forever if RS acquiring the lock has died > prematurely > > > Key: HBASE-16144 > URL: https://issues.apache.org/jira/browse/HBASE-16144 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.1, 1.1.5, 0.98.20 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-16144-v1.patch, HBASE-16144-v2.patch > > > In default, we will use multi operation when we claimQueues from ZK. But if > we set hbase.zookeeper.useMulti=false, we will add a lock first, then copy > nodes, finally clean old queue and the lock. > However, if the RS acquiring the lock crash before claimQueues done, the lock > will always be there and other RS can never claim the queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16156) In write heavy scenario creating in memory flushes leads to contention
[
https://issues.apache.org/jira/browse/HBASE-16156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358498#comment-15358498
]
Anoop Sam John commented on HBASE-16156:
{code}
getRegionServices().blockUpdates();
try {
MutableSegment active = getActive();
LOG.info("IN-MEMORY FLUSH: Pushing active segment into compaction
pipeline, "
+ "and initiating compaction.");
pushActiveToPipeline(active);
} finally {
getRegionServices().unblockUpdates();
}
{code}
It was this way. So under the lock we do log also Now I changed level
to debug so that u will see better. Even we need change that also. After the
lock is released we can debug log that the active is pushed.
> In write heavy scenario creating in memory flushes leads to contention
> --
>
> Key: HBASE-16156
> URL: https://issues.apache.org/jira/browse/HBASE-16156
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
>
> In write heavy cases, the inmemory flushes blocks the write because it takes
> a update lock. Dumps show that it is causing heavy contention and leads to
> memstore filling up and so intern blocks the flush from happening sooner.
> This JIRA is to discuss optimal settings and then make suitable changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16157) The incorrect block cache count and size are caused by removing duplicate block key in the HBase LruBlockCache
[
https://issues.apache.org/jira/browse/HBASE-16157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358492#comment-15358492
]
Hadoop QA commented on HBASE-16157:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 29s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 85m 35s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 127m 28s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
| | org.apache.hadoop.hbase.master.TestSplitLogManager |
| | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer |
| | org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster |
| | org.apache.hadoop.hbase.master.TestRestartCluster |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12815679/HBASE-16157-v2.patch |
| JIRA Issue | HBASE-16157 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slav
[jira] [Commented] (HBASE-16144) Replication queue's lock will live forever if RS acquiring the lock has died prematurely
[
https://issues.apache.org/jira/browse/HBASE-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358488#comment-15358488
]
Hadoop QA commented on HBASE-16144:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
1s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 18s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 14s
{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 87m 8s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 137m 14s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | org.apache.hadoop.hbase.master.cleaner.ReplicationZKLockCleanerChore.TTL
isn't final but should be At ReplicationZKLockCleanerChore.java:be At
ReplicationZKLockCleanerChore.java:[line 57] |
| Failed junit tests | hadoop.hbase.snapshot.TestMobFlushSnapshotFromClient |
| | hadoop.hbase.client.TestHCM |
| Timed out junit tests |
[jira] [Commented] (HBASE-16161) Remove a few unnecessary (uncontended) synchronizes
[ https://issues.apache.org/jira/browse/HBASE-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358484#comment-15358484 ] Heng Chen commented on HBASE-16161: --- Only branch-1? It seems master has this issue too. > Remove a few unnecessary (uncontended) synchronizes > --- > > Key: HBASE-16161 > URL: https://issues.apache.org/jira/browse/HBASE-16161 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Minor > Attachments: HBASE-16161.branch-1.001.patch > > > This is followon from HBASE-15716. We have a few odd looking synchronizes. A > few are probably elided after analysis concludes single thread consumer but > lets remove to be clear. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14743) Add metrics around HeapMemoryManager
[
https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358477#comment-15358477
]
Hadoop QA commented on HBASE-14743:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 14s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 55s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 15s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 19s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 96m 7s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 144m 54s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestAcidGuarantees |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12815672/HBASE-14743.010.v2.patch
|
| JIRA Issue | HBASE-14743 |
| Optional Tests | asflicens
[jira] [Commented] (HBASE-16124) Make check_compatibility.sh less verbose when building HBase
[
https://issues.apache.org/jira/browse/HBASE-16124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358470#comment-15358470
]
Hudson commented on HBASE-16124:
FAILURE: Integrated in HBase-1.3 #764 (See
[https://builds.apache.org/job/HBase-1.3/764/])
HBASE-16124 Make check_compatibility.sh less verbose when building HBase
(busbey: rev 959eb2d22740d078ef1722b4c8832b6e4c61d13a)
* dev-support/check_compatibility.sh
> Make check_compatibility.sh less verbose when building HBase
>
>
> Key: HBASE-16124
> URL: https://issues.apache.org/jira/browse/HBASE-16124
> Project: HBase
> Issue Type: Improvement
> Components: build, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 0.98.21
>
> Attachments: HBASE-16124_v1.patch
>
>
> {{[check_compatibility.sh|https://github.com/apache/hbase/blob/master/dev-support/check_compatibility.sh]}}
> is a bit verbose when building HBase JARs, which makes it kind of a
> nightmare when used in a Jenkins job. Let's run those steps in Maven's batch
> mode, which means less unnecessary output and no expectation of user
> interaction.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[ https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358467#comment-15358467 ] Hudson commented on HBASE-16073: FAILURE: Integrated in HBase-1.3 #764 (See [https://builds.apache.org/job/HBase-1.3/764/]) HBASE-16073 update compatibility_checker for jacc dropping comma sep (busbey: rev 32f53b37c40968b6cc26cc036e465564d820c4d1) * dev-support/check_compatibility.sh > update compatibility_checker for jacc dropping comma sep args > - > > Key: HBASE-16073 > URL: https://issues.apache.org/jira/browse/HBASE-16073 > Project: HBase > Issue Type: Task > Components: build, documentation >Reporter: Sean Busbey >Assignee: Dima Spivak >Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-16073_v1.patch, HBASE-16073_v2.patch > > > the japi-compliance-checker has a change in place (post the 1.7 release) that > removes the ability to give a comma separated list of jars on the cli. > we should switch to generating descriptor xml docs since that will still be > supported, or update to use the expanded tooling suggested in the issue: > https://github.com/lvc/japi-compliance-checker/issues/27 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16129) check_compatibility.sh is broken when using Java API Compliance Checker v1.7
[ https://issues.apache.org/jira/browse/HBASE-16129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358468#comment-15358468 ] Hudson commented on HBASE-16129: FAILURE: Integrated in HBase-1.3 #764 (See [https://builds.apache.org/job/HBase-1.3/764/]) HBASE-16129 check_compatibility.sh is broken when using Java API (busbey: rev 7335a1f987eca8bd09a2d4f14e533e15751ed877) * dev-support/check_compatibility.sh > check_compatibility.sh is broken when using Java API Compliance Checker v1.7 > > > Key: HBASE-16129 > URL: https://issues.apache.org/jira/browse/HBASE-16129 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Dima Spivak >Assignee: Dima Spivak > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-16129_v1.patch, HBASE-16129_v2.patch, > HBASE-16129_v3.patch > > > As part of HBASE-16073, we hardcoded check_compatiblity.sh to check out the > v1.7 tag of Java ACC. Unfortunately, just running it between two branches > that I know have incompatibilities, I get 0 incompatibilities (and 0 classes > read). Looks like this version doesn't properly traverse through JARs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15729) Remove old JDiff wrapper scripts in dev-support
[ https://issues.apache.org/jira/browse/HBASE-15729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358466#comment-15358466 ] Hudson commented on HBASE-15729: FAILURE: Integrated in HBase-1.3 #764 (See [https://builds.apache.org/job/HBase-1.3/764/]) HBASE-15729 Remove old JDiff wrapper scripts in dev-support (busbey: rev 7697c114d9fbe6c49159fa9c73bbb7c774c8b6da) * dev-support/jdiffHBasePublicAPI.sh * dev-support/hbase_jdiff_afterSingularityTemplate.xml * dev-support/jdiffHBasePublicAPI_common.sh * dev-support/hbase_jdiff_acrossSingularityTemplate.xml * dev-support/hbase_jdiff_template.xml > Remove old JDiff wrapper scripts in dev-support > --- > > Key: HBASE-15729 > URL: https://issues.apache.org/jira/browse/HBASE-15729 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Dima Spivak >Assignee: Dima Spivak >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-15729.patch > > > Since HBASE-12808, we've been using the Java API Compliance Checker instead > of JDiff to look at API compatibility. Probably makes sense to remove the old > wrapper scripts that aren't being used anymore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15119) Include git SHA in check_compatibility reports
[ https://issues.apache.org/jira/browse/HBASE-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358469#comment-15358469 ] Hudson commented on HBASE-15119: FAILURE: Integrated in HBase-1.3 #764 (See [https://builds.apache.org/job/HBase-1.3/764/]) HBASE-15119 Include git SHA in check_compatibility reports (busbey: rev 2c18bb49b0dc044f7145cdf824785bd83e11146f) * dev-support/check_compatibility.sh > Include git SHA in check_compatibility reports > -- > > Key: HBASE-15119 > URL: https://issues.apache.org/jira/browse/HBASE-15119 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-15119.v00.patch > > > Since some refs change over time (ie, branches), it would be nice to include > git shas in the version info included in check compatibility reports. It'll > also help interested parties to be sure of what they're looking at. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16161) Remove a few unnecessary (uncontended) synchronizes
[
https://issues.apache.org/jira/browse/HBASE-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358465#comment-15358465
]
Hadoop QA commented on HBASE-16161:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
22s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 51s
{color} | {color:red} hbase-server in branch-1 has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 23s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 21s
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0. {color}
|
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 21s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 22s
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_80.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 22s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_80. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m 47s
{color} | {color:red} Patch causes 29 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 33s
{color} | {color:red} Patch causes 29 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 20s
{color} | {color:red} Patch causes 29 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 5s
{color} | {color:red} Patch causes 29 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 48s
{color} | {color:red} Patch causes 29 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 35s
{color} | {color:red} Patch causes 29 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 22s
{color} | {color:red} Patch causes 29 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 11s
{color} | {color:red} Patch causes 29 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 4s
{color} | {color:red} Patch causes 29 errors with Hadoop v2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 22s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:
[jira] [Commented] (HBASE-16096) Replication keeps accumulating znodes
[
https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358449#comment-15358449
]
Hadoop QA commented on HBASE-16096:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 17s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
0s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 27s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 56s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 104m 20s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 154m 1s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12815668/HBASE-16096.patch |
| JIRA Issue | HBASE-16096 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
[jira] [Commented] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358444#comment-15358444
]
Hadoop QA commented on HBASE-15844:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 5s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 10s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 104m 42s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 147m 49s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
| | hadoop.hbase.snapshot.TestFlushSnapshotFromClient |
| Timed out junit tests |
org.apache.hadoop.hbase.replication.multiwal.TestReplicationKillMasterRSCompressedWithMultipleAsyncWAL
|
| |
org.apache.hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL
|
| |
org.apache.hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleAsyncWAL
|
| |
org.apache.hadoop.hbase.replication.multiwal.TestReplicationKillMasterRSCompressedWithMultipleWAL
|
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12815670/HBASE-15844.patch |
|
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358441#comment-15358441 ] stack commented on HBASE-15716: --- Issue that picks up some of the changes made in here but not committed as part of this issue. > HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random > read > -- > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack > Attachments: > 15716.implementation.using.ScannerReadPoints.branch-1.patch, > 15716.prune.synchronizations.patch, 15716.prune.synchronizations.v3.patch, > 15716.prune.synchronizations.v4.patch, 15716.prune.synchronizations.v4.patch, > 15716.wip.more_to_be_done.patch, HBASE-15716.branch-1.001.patch, > HBASE-15716.branch-1.002.patch, HBASE-15716.branch-1.003.patch, > HBASE-15716.branch-1.004.patch, HBASE-15716.branch-1.005.patch, > ScannerReadPoints.java, Screen Shot 2016-04-26 at 2.05.45 PM.png, Screen Shot > 2016-04-26 at 2.06.14 PM.png, Screen Shot 2016-04-26 at 2.07.06 PM.png, > Screen Shot 2016-04-26 at 2.25.26 PM.png, Screen Shot 2016-04-26 at 6.02.29 > PM.png, Screen Shot 2016-04-27 at 9.49.35 AM.png, Screen Shot 2016-06-30 at > 9.52.52 PM.png, Screen Shot 2016-06-30 at 9.54.08 PM.png, > TestScannerReadPoints.java, before_after.png, > current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, > remove.locks.patch, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16161) Remove a few unnecessary (uncontended) synchronizes
[ https://issues.apache.org/jira/browse/HBASE-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358434#comment-15358434 ] stack commented on HBASE-16161: --- Here is the patch. Let me run some tests and post some compares. > Remove a few unnecessary (uncontended) synchronizes > --- > > Key: HBASE-16161 > URL: https://issues.apache.org/jira/browse/HBASE-16161 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-16161.branch-1.001.patch > > > This is followon from HBASE-15716. We have a few odd looking synchronizes. A > few are probably elided after analysis concludes single thread consumer but > lets remove to be clear. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16161) Remove a few unnecessary (uncontended) synchronizes
[ https://issues.apache.org/jira/browse/HBASE-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16161: -- Priority: Minor (was: Major) > Remove a few unnecessary (uncontended) synchronizes > --- > > Key: HBASE-16161 > URL: https://issues.apache.org/jira/browse/HBASE-16161 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Minor > Attachments: HBASE-16161.branch-1.001.patch > > > This is followon from HBASE-15716. We have a few odd looking synchronizes. A > few are probably elided after analysis concludes single thread consumer but > lets remove to be clear. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted
[ https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-16135: -- Attachment: HBASE-16135-v3.patch Haven't found the root cause yet, but peer id '2' is only introduced in the new test. So I changed the server hostname to hostname2 in the new test to make them independent. > PeerClusterZnode under rs of removed peer may never be deleted > -- > > Key: HBASE-16135 > URL: https://issues.apache.org/jira/browse/HBASE-16135 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2, 0.98.20 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3 > > Attachments: HBASE-16135-0.98.patch, HBASE-16135-branch-1.1.patch, > HBASE-16135-branch-1.2.patch, HBASE-16135-branch-1.patch, > HBASE-16135-v1.patch, HBASE-16135-v2.patch, HBASE-16135-v3.patch, > HBASE-16135.patch > > > One of our cluster run out of space recently, and we found that the .oldlogs > directory had almost the same size as the data directory. > Finally we found the problem is that, we removed a peer abort 3 months ago, > but there are still some replication queue znode under some rs nodes. This > prevents the deletion of .oldlogs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16161) Remove a few unnecessary (uncontended) synchronizes
[ https://issues.apache.org/jira/browse/HBASE-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16161: -- Assignee: stack Status: Patch Available (was: Open) > Remove a few unnecessary (uncontended) synchronizes > --- > > Key: HBASE-16161 > URL: https://issues.apache.org/jira/browse/HBASE-16161 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-16161.branch-1.001.patch > > > This is followon from HBASE-15716. We have a few odd looking synchronizes. A > few are probably elided after analysis concludes single thread consumer but > lets remove to be clear. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16161) Remove a few unnecessary (uncontended) synchronizes
[ https://issues.apache.org/jira/browse/HBASE-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358433#comment-15358433 ] stack commented on HBASE-16161: --- Remove some unnecessary synchronizes. M hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java Remove synchronize on sendResponseIfReady, and Reader#doRunLoop. The former is safe for many threads to call and doRunLoop has one thread only except reading the 'adding' field. M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Cache hashcode. Add hash code on region scanner (and equals) Remove synchronize on the scanner close and next call. Remove some dup'd code between handleException and close. > Remove a few unnecessary (uncontended) synchronizes > --- > > Key: HBASE-16161 > URL: https://issues.apache.org/jira/browse/HBASE-16161 > Project: HBase > Issue Type: Bug >Reporter: stack > Attachments: HBASE-16161.branch-1.001.patch > > > This is followon from HBASE-15716. We have a few odd looking synchronizes. A > few are probably elided after analysis concludes single thread consumer but > lets remove to be clear. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16161) Remove a few unnecessary (uncontended) synchronizes
[ https://issues.apache.org/jira/browse/HBASE-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16161: -- Attachment: HBASE-16161.branch-1.001.patch > Remove a few unnecessary (uncontended) synchronizes > --- > > Key: HBASE-16161 > URL: https://issues.apache.org/jira/browse/HBASE-16161 > Project: HBase > Issue Type: Bug >Reporter: stack > Attachments: HBASE-16161.branch-1.001.patch > > > This is followon from HBASE-15716. We have a few odd looking synchronizes. A > few are probably elided after analysis concludes single thread consumer but > lets remove to be clear. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
[
https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358422#comment-15358422
]
Hadoop QA commented on HBASE-16160:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 12s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 88m 2s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 131m 7s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster |
| Timed out junit tests |
org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestExportSnapshot |
| | org.apache.hadoop.hbase.snapshot.TestMobRestoreFlushSnapshotFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12815669/HBASE-16160.001.patch
|
| JIRA Issue | HBASE-16160 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64
[jira] [Updated] (HBASE-16068) Procedure v2 - use consts for conf properties in tests
[
https://issues.apache.org/jira/browse/HBASE-16068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-16068:
Fix Version/s: (was: 1.2.3)
1.2.2
> Procedure v2 - use consts for conf properties in tests
> ---
>
> Key: HBASE-16068
> URL: https://issues.apache.org/jira/browse/HBASE-16068
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, test
>Affects Versions: 2.0.0, 1.3.0, 1.2.1
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 2.0.0, 1.2.2, 1.3.1
>
> Attachments: HBASE-16068-v0.patch
>
>
> replace the hardcoded properties string conf.set("foo.key", v) in the tests
> with the use of the configuration property constants that we already have
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted
[
https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358416#comment-15358416
]
Duo Zhang commented on HBASE-16135:
---
I can not reproduce it locally. The strange log output is
{noformat}
2016-07-01 04:09:57,901 DEBUG [main] replication.ReplicationQueueInfo(112):
Found dead servers:[hostname1.example.org,1234,1]
2016-07-01 04:09:57,910 INFO [main]
replication.TableBasedReplicationQueuesImpl(250): hostname.example.org,1234,1
has deleted abandoned queue 2-hostname1.example.org,1234,1 from
hostname1.example.org,1234,1
{noformat}
It should be
{noformat}
2016-07-01 13:19:01,981 DEBUG [main] replication.ReplicationQueueInfo(112):
Found dead servers:[hostname1.example.org,1234,1]
2016-07-01 13:19:01,983 INFO [main]
replication.TableBasedReplicationQueuesImpl(246):
dummyserver1.example.org,1234,1 has claimed queue
1-hostname1.example.org,1234,1 from hostname1.example.org,1234,1
{noformat}
Let me dig more.
> PeerClusterZnode under rs of removed peer may never be deleted
> --
>
> Key: HBASE-16135
> URL: https://issues.apache.org/jira/browse/HBASE-16135
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2, 0.98.20
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3
>
> Attachments: HBASE-16135-0.98.patch, HBASE-16135-branch-1.1.patch,
> HBASE-16135-branch-1.2.patch, HBASE-16135-branch-1.patch,
> HBASE-16135-v1.patch, HBASE-16135-v2.patch, HBASE-16135.patch
>
>
> One of our cluster run out of space recently, and we found that the .oldlogs
> directory had almost the same size as the data directory.
> Finally we found the problem is that, we removed a peer abort 3 months ago,
> but there are still some replication queue znode under some rs nodes. This
> prevents the deletion of .oldlogs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16096) Replication keeps accumulating znodes
[
https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358414#comment-15358414
]
Hadoop QA commented on HBASE-16096:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 1s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 58s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 102m 27s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 153m 35s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
| | hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster |
| Timed out junit tests |
org.apache.hadoop.hbase.util.TestMiniClusterLoadEncoded |
| | org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential |
| | org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| | org.apache.hadoop.hbase.util.Tes
[jira] [Updated] (HBASE-16117) Fix Connection leak in mapred.TableOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-16117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16117: Fix Version/s: (was: 1.2.2) 1.2.3 > Fix Connection leak in mapred.TableOutputFormat > > > Key: HBASE-16117 > URL: https://issues.apache.org/jira/browse/HBASE-16117 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0, 1.3.0, 1.2.2, 1.1.6 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0, 1.3.0, 1.1.6, 1.2.3 > > Attachments: hbase-16117.branch-1.patch, hbase-16117.patch, > hbase-16117.v2.branch-1.patch, hbase-16117.v2.patch, > hbase-16117.v3.branch-1.patch, hbase-16117.v3.patch, hbase-16117.v4.patch > > > Spark seems to instantiate multiple instances of output formats within a > single process. When mapred.TableOutputFormat (not > mapreduce.TableOutputFormat) is used, this may cause connection leaks that > slowly exhaust the cluster's zk connections. > This patch fixes that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16117) Fix Connection leak in mapred.TableOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-16117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358409#comment-15358409 ] Sean Busbey commented on HBASE-16117: - In HBASE-15645 we noted that an added API couldn't be relied on by annotating it IA.Private in the earlier branch. I'd like to do that with the new constructor from HBASE-16017. > Fix Connection leak in mapred.TableOutputFormat > > > Key: HBASE-16117 > URL: https://issues.apache.org/jira/browse/HBASE-16117 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0, 1.3.0, 1.2.2, 1.1.6 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6 > > Attachments: hbase-16117.branch-1.patch, hbase-16117.patch, > hbase-16117.v2.branch-1.patch, hbase-16117.v2.patch, > hbase-16117.v3.branch-1.patch, hbase-16117.v3.patch, hbase-16117.v4.patch > > > Spark seems to instantiate multiple instances of output formats within a > single process. When mapred.TableOutputFormat (not > mapreduce.TableOutputFormat) is used, this may cause connection leaks that > slowly exhaust the cluster's zk connections. > This patch fixes that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16161) Remove a few unnecessary (uncontended) synchronizes
stack created HBASE-16161: - Summary: Remove a few unnecessary (uncontended) synchronizes Key: HBASE-16161 URL: https://issues.apache.org/jira/browse/HBASE-16161 Project: HBase Issue Type: Bug Reporter: stack This is followon from HBASE-15716. We have a few odd looking synchronizes. A few are probably elided after analysis concludes single thread consumer but lets remove to be clear. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16093) Splits failed before creating daughter regions leave meta inconsistent
[ https://issues.apache.org/jira/browse/HBASE-16093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved HBASE-16093. - Resolution: Fixed > Splits failed before creating daughter regions leave meta inconsistent > -- > > Key: HBASE-16093 > URL: https://issues.apache.org/jira/browse/HBASE-16093 > Project: HBase > Issue Type: Bug > Components: master, Region Assignment >Affects Versions: 1.3.0, 1.2.1 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 1.3.0, 1.4.0, 1.2.2, 1.1.6 > > Attachments: HBASE-16093.branch-1.patch > > > This is on branch-1 based code only. > Here's the sequence of events. > # A regionserver opens a new region. That regions looks like it should split. > # So the regionserver starts a split transaction. > # Split transaction starts execute > # Split transaction encounters an error in stepsBeforePONR > # Split transaction starts rollback > # Split transaction notifies master that it's rolling back using > HMasterRpcServices#reportRegionStateTransition > # AssignmentManager#onRegionTransition is called with SPLIT_REVERTED > # AssignmentManager#onRegionSplitReverted is called. > # That onlines the parent region and offlines the daughter regions. > However the daughter regions were never created in meta so all that gets done > is that state for those rows gets OFFLINE. Now all clients trying to get the > parent instead get the offline daughter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15844:
--
Fix Version/s: (was: 0.98.21)
(was: 1.1.6)
(was: 1.2.2)
(was: 1.0.4)
(was: 1.3.0)
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15844.patch, HBASE-15844_v1.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358405#comment-15358405
]
Heng Chen commented on HBASE-15844:
---
OK. Let's commit it to branch-1 and master
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15844.patch, HBASE-15844_v1.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358399#comment-15358399 ] stack commented on HBASE-15716: --- Ok. Changed my mind. The 005 patch has ONLY the [~ikeda] changes. I will make a new issue for removing unnecessary synchronizes and addition of hash calculations. This patch adds in the suggested test class. Good by you [~ikeda]? [~lhofhansl], want to take a look at this? > HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random > read > -- > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack > Attachments: > 15716.implementation.using.ScannerReadPoints.branch-1.patch, > 15716.prune.synchronizations.patch, 15716.prune.synchronizations.v3.patch, > 15716.prune.synchronizations.v4.patch, 15716.prune.synchronizations.v4.patch, > 15716.wip.more_to_be_done.patch, HBASE-15716.branch-1.001.patch, > HBASE-15716.branch-1.002.patch, HBASE-15716.branch-1.003.patch, > HBASE-15716.branch-1.004.patch, HBASE-15716.branch-1.005.patch, > ScannerReadPoints.java, Screen Shot 2016-04-26 at 2.05.45 PM.png, Screen Shot > 2016-04-26 at 2.06.14 PM.png, Screen Shot 2016-04-26 at 2.07.06 PM.png, > Screen Shot 2016-04-26 at 2.25.26 PM.png, Screen Shot 2016-04-26 at 6.02.29 > PM.png, Screen Shot 2016-04-27 at 9.49.35 AM.png, Screen Shot 2016-06-30 at > 9.52.52 PM.png, Screen Shot 2016-06-30 at 9.54.08 PM.png, > TestScannerReadPoints.java, before_after.png, > current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, > remove.locks.patch, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15716: -- Attachment: HBASE-15716.branch-1.005.patch > HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random > read > -- > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack > Attachments: > 15716.implementation.using.ScannerReadPoints.branch-1.patch, > 15716.prune.synchronizations.patch, 15716.prune.synchronizations.v3.patch, > 15716.prune.synchronizations.v4.patch, 15716.prune.synchronizations.v4.patch, > 15716.wip.more_to_be_done.patch, HBASE-15716.branch-1.001.patch, > HBASE-15716.branch-1.002.patch, HBASE-15716.branch-1.003.patch, > HBASE-15716.branch-1.004.patch, HBASE-15716.branch-1.005.patch, > ScannerReadPoints.java, Screen Shot 2016-04-26 at 2.05.45 PM.png, Screen Shot > 2016-04-26 at 2.06.14 PM.png, Screen Shot 2016-04-26 at 2.07.06 PM.png, > Screen Shot 2016-04-26 at 2.25.26 PM.png, Screen Shot 2016-04-26 at 6.02.29 > PM.png, Screen Shot 2016-04-27 at 9.49.35 AM.png, Screen Shot 2016-06-30 at > 9.52.52 PM.png, Screen Shot 2016-06-30 at 9.54.08 PM.png, > TestScannerReadPoints.java, before_after.png, > current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, > remove.locks.patch, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted
[ https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358391#comment-15358391 ] Duo Zhang commented on HBASE-16135: --- Let me check the failed UT. > PeerClusterZnode under rs of removed peer may never be deleted > -- > > Key: HBASE-16135 > URL: https://issues.apache.org/jira/browse/HBASE-16135 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2, 0.98.20 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3 > > Attachments: HBASE-16135-0.98.patch, HBASE-16135-branch-1.1.patch, > HBASE-16135-branch-1.2.patch, HBASE-16135-branch-1.patch, > HBASE-16135-v1.patch, HBASE-16135-v2.patch, HBASE-16135.patch > > > One of our cluster run out of space recently, and we found that the .oldlogs > directory had almost the same size as the data directory. > Finally we found the problem is that, we removed a peer abort 3 months ago, > but there are still some replication queue znode under some rs nodes. This > prevents the deletion of .oldlogs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15716: -- Attachment: Screen Shot 2016-06-30 at 9.54.08 PM.png Screen Shot 2016-06-30 at 9.52.52 PM.png Before and after humps from a regionserver doing heavy workloadc (the trough is me doing the JFR pull). Also include picture of our JFR in JMC where it shows the thousands of contention points for registration of region scanner as now gone. Here are the lines from honest profiler flat view... before: 62 (t 19.4,s 0.2) org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl:: and after: 21 (t 18.9,s 0.8) org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl:: > HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random > read > -- > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack > Attachments: > 15716.implementation.using.ScannerReadPoints.branch-1.patch, > 15716.prune.synchronizations.patch, 15716.prune.synchronizations.v3.patch, > 15716.prune.synchronizations.v4.patch, 15716.prune.synchronizations.v4.patch, > 15716.wip.more_to_be_done.patch, HBASE-15716.branch-1.001.patch, > HBASE-15716.branch-1.002.patch, HBASE-15716.branch-1.003.patch, > HBASE-15716.branch-1.004.patch, ScannerReadPoints.java, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at > 9.49.35 AM.png, Screen Shot 2016-06-30 at 9.52.52 PM.png, Screen Shot > 2016-06-30 at 9.54.08 PM.png, TestScannerReadPoints.java, before_after.png, > current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, > remove.locks.patch, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358380#comment-15358380 ] stack commented on HBASE-15716: --- I think it is fine. I went to hack on your class because I'd thought you'd left off consideration of isolationlevel...but I think the way you do it is clean. The other method is for ourside consumers and should probably go away anyways (it seems to be used by Scanners only...). Let me make up a patch that has your nice class [~ikeda] and the other changes too... I spent a bit of a while testing this patch. Looks good. JFR no longer reports this as a point of contention. If I thread dump, safe point no longer has the region scanner init showing. Comparing perf runs, i see we are doing the same throughput, perhaps a very slight bit more. Looking before and after using honest profiler https://github.com/RichardWarburton/honest-profiler, it says that org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl appears now MORE often in stack traces when samping but that as an endpoint in a stack trace, there is less incidence; percentages are small -- less than 1% -- which would seem to indicate that focus on this as a point of contention is probably overblown because it appears in safepoint thread dumps (i.e. my manual thread dumping -- see "Why (Most) Sampling Java Profilers Are Fucking Terrible" http://psy-lob-saw.blogspot.co.za/2016/02/why-most-sampling-java-profilers-are.html -- though this post says JFR does not rely on safe point). > HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random > read > -- > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack > Attachments: > 15716.implementation.using.ScannerReadPoints.branch-1.patch, > 15716.prune.synchronizations.patch, 15716.prune.synchronizations.v3.patch, > 15716.prune.synchronizations.v4.patch, 15716.prune.synchronizations.v4.patch, > 15716.wip.more_to_be_done.patch, HBASE-15716.branch-1.001.patch, > HBASE-15716.branch-1.002.patch, HBASE-15716.branch-1.003.patch, > HBASE-15716.branch-1.004.patch, ScannerReadPoints.java, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at > 9.49.35 AM.png, TestScannerReadPoints.java, before_after.png, > current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, > remove.locks.patch, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16155) Compacting Memstore : Few log improvements
[
https://issues.apache.org/jira/browse/HBASE-16155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-16155:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the review Stack.
> Compacting Memstore : Few log improvements
> --
>
> Key: HBASE-16155
> URL: https://issues.apache.org/jira/browse/HBASE-16155
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16155.patch
>
>
> When testing this I can see large log files for RS. Many log lines in INFO
> level which can be lower level
> LOG.info("FLUSHING TO DISK: region "+ getRegionServices().getRegionInfo()
> .getRegionNameAsString() + "store: "+ getFamilyName());
> To be DEBUG
> LOG.info("IN-MEMORY FLUSH: Pushing active segment into compaction pipeline, "
> +
> "and initiating compaction.");
> To be DEBUG
> LOG.info("Swapping pipeline suffix with compacted item. "
> +"Just before the swap the number of segments in pipeline is:"
> +versionedList.getStoreSegments().size()
> +", and the number of cells in new segment
> is:"+segment.getCellsCount());
> To be DEBUG
> LOG.info("Suffix size: "+ suffixSize+" compacted item size: "+newSize+
> " globalMemstoreSize: "+globalMemstoreSize);
> To be DEBUG
> LOG.info("Starting the MemStore in-memory compaction for store " +
> compactingMemStore.getStore().getColumnFamilyName());
> To be DEBUG
> And one now in DEBUG can be better at INFO level
> LOG.debug("Setting in-memory flush size threshold to " + inmemoryFlushSize);
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16124) Make check_compatibility.sh less verbose when building HBase
[
https://issues.apache.org/jira/browse/HBASE-16124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-16124:
Fix Version/s: 0.98.21
1.1.6
1.2.2
1.4.0
1.3.0
> Make check_compatibility.sh less verbose when building HBase
>
>
> Key: HBASE-16124
> URL: https://issues.apache.org/jira/browse/HBASE-16124
> Project: HBase
> Issue Type: Improvement
> Components: build, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 0.98.21
>
> Attachments: HBASE-16124_v1.patch
>
>
> {{[check_compatibility.sh|https://github.com/apache/hbase/blob/master/dev-support/check_compatibility.sh]}}
> is a bit verbose when building HBase JARs, which makes it kind of a
> nightmare when used in a Jenkins job. Let's run those steps in Maven's batch
> mode, which means less unnecessary output and no expectation of user
> interaction.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[ https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16073: Fix Version/s: 0.98.21 1.1.6 1.2.2 1.4.0 1.3.0 > update compatibility_checker for jacc dropping comma sep args > - > > Key: HBASE-16073 > URL: https://issues.apache.org/jira/browse/HBASE-16073 > Project: HBase > Issue Type: Task > Components: build, documentation >Reporter: Sean Busbey >Assignee: Dima Spivak >Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-16073_v1.patch, HBASE-16073_v2.patch > > > the japi-compliance-checker has a change in place (post the 1.7 release) that > removes the ability to give a comma separated list of jars on the cli. > we should switch to generating descriptor xml docs since that will still be > supported, or update to use the expanded tooling suggested in the issue: > https://github.com/lvc/japi-compliance-checker/issues/27 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15729) Remove old JDiff wrapper scripts in dev-support
[ https://issues.apache.org/jira/browse/HBASE-15729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15729: Fix Version/s: 0.98.21 1.1.6 1.2.2 1.4.0 1.3.0 > Remove old JDiff wrapper scripts in dev-support > --- > > Key: HBASE-15729 > URL: https://issues.apache.org/jira/browse/HBASE-15729 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Dima Spivak >Assignee: Dima Spivak >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-15729.patch > > > Since HBASE-12808, we've been using the Java API Compliance Checker instead > of JDiff to look at API compatibility. Probably makes sense to remove the old > wrapper scripts that aren't being used anymore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358367#comment-15358367
]
Anoop Sam John commented on HBASE-15844:
We can commit to trunk and branch-1 alone? Dont think it is so serious to be
committed to patch branches. WDYT?
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.2.2, 1.1.6, 0.98.21
>
> Attachments: HBASE-15844.patch, HBASE-15844_v1.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16129) check_compatibility.sh is broken when using Java API Compliance Checker v1.7
[ https://issues.apache.org/jira/browse/HBASE-16129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16129: Fix Version/s: 0.98.21 1.1.6 1.2.2 1.4.0 1.3.0 > check_compatibility.sh is broken when using Java API Compliance Checker v1.7 > > > Key: HBASE-16129 > URL: https://issues.apache.org/jira/browse/HBASE-16129 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Dima Spivak >Assignee: Dima Spivak > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-16129_v1.patch, HBASE-16129_v2.patch, > HBASE-16129_v3.patch > > > As part of HBASE-16073, we hardcoded check_compatiblity.sh to check out the > v1.7 tag of Java ACC. Unfortunately, just running it between two branches > that I know have incompatibilities, I get 0 incompatibilities (and 0 classes > read). Looks like this version doesn't properly traverse through JARs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15119) Include git SHA in check_compatibility reports
[ https://issues.apache.org/jira/browse/HBASE-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-15119: Fix Version/s: 0.98.21 1.1.6 1.2.2 1.4.0 1.3.0 > Include git SHA in check_compatibility reports > -- > > Key: HBASE-15119 > URL: https://issues.apache.org/jira/browse/HBASE-15119 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-15119.v00.patch > > > Since some refs change over time (ie, branches), it would be nice to include > git shas in the version info included in check compatibility reports. It'll > also help interested parties to be sure of what they're looking at. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16154) bring non-master branches up to date wrt check_compatibility script
[ https://issues.apache.org/jira/browse/HBASE-16154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-16154: Resolution: Fixed Status: Resolved (was: Patch Available) > bring non-master branches up to date wrt check_compatibility script > --- > > Key: HBASE-16154 > URL: https://issues.apache.org/jira/browse/HBASE-16154 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 1.2.2, 1.1.6, 0.98.21 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 1.3.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-16154-branch-1.v1.patch, > HBASE-16154-branch-1.v2.patch > > > There have been some fixes to check_compatibility.sh that impact correctness, > we should bring everything back to the earlier branches in case RMs are > running from their branch rather than master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15844:
--
Fix Version/s: 0.98.21
1.1.6
1.2.2
1.4.0
1.0.4
1.3.0
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.2.2, 1.1.6, 0.98.21
>
> Attachments: HBASE-15844.patch, HBASE-15844_v1.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16157) The incorrect block cache count and size are caused by removing duplicate block key in the HBase LruBlockCache
[
https://issues.apache.org/jira/browse/HBASE-16157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-16157:
--
Status: Patch Available (was: Open)
> The incorrect block cache count and size are caused by removing duplicate
> block key in the HBase LruBlockCache
> --
>
> Key: HBASE-16157
> URL: https://issues.apache.org/jira/browse/HBASE-16157
> Project: HBase
> Issue Type: Bug
>Reporter: ChiaPing Tsai
>Priority: Trivial
> Attachments: HBASE-16157-v1.patch, HBASE-16157-v2.patch
>
>
> {code:title=LruBlockCache.java|borderStyle=solid}
> // Check return value from the Map#remove before updating the metrics
> protected long evictBlock(LruCachedBlock block, boolean
> evictedByEvictionProcess) {
> map.remove(block.getCacheKey());
> updateSizeMetrics(block, true);
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16157) The incorrect block cache count and size are caused by removing duplicate block key in the HBase LruBlockCache
[
https://issues.apache.org/jira/browse/HBASE-16157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-16157:
--
Attachment: HBASE-16157-v2.patch
The following test is passed in my local.
1) TestHCM#testConnectionNotAllowsInterrupt
2) TestTableBasedReplicationSourceManagerImpl#testCleanupFailoverQueues test
> The incorrect block cache count and size are caused by removing duplicate
> block key in the HBase LruBlockCache
> --
>
> Key: HBASE-16157
> URL: https://issues.apache.org/jira/browse/HBASE-16157
> Project: HBase
> Issue Type: Bug
>Reporter: ChiaPing Tsai
>Priority: Trivial
> Attachments: HBASE-16157-v1.patch, HBASE-16157-v2.patch
>
>
> {code:title=LruBlockCache.java|borderStyle=solid}
> // Check return value from the Map#remove before updating the metrics
> protected long evictBlock(LruCachedBlock block, boolean
> evictedByEvictionProcess) {
> map.remove(block.getCacheKey());
> updateSizeMetrics(block, true);
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16114) Get regionLocation of required regions only for MR jobs
[ https://issues.apache.org/jira/browse/HBASE-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16114: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Get regionLocation of required regions only for MR jobs > --- > > Key: HBASE-16114 > URL: https://issues.apache.org/jira/browse/HBASE-16114 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 1.2.1 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16114.master.001.patch, > HBASE-16114.master.001.patch, HBASE-16114.master.002.patch > > > We should only get the location of regions required during the MR job. This > will help for jobs with large regions but the job itself scans only a small > portion of it. Similar changes can be seen in MultiInputFormatBase.java. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15844:
--
Attachment: HBASE-15844_v1.patch
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-15844.patch, HBASE-15844_v1.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15844:
--
Attachment: (was: HBASE-15844_v1.patch)
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-15844.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16157) The incorrect block cache count and size are caused by removing duplicate block key in the HBase LruBlockCache
[
https://issues.apache.org/jira/browse/HBASE-16157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaPing Tsai updated HBASE-16157:
--
Status: Open (was: Patch Available)
> The incorrect block cache count and size are caused by removing duplicate
> block key in the HBase LruBlockCache
> --
>
> Key: HBASE-16157
> URL: https://issues.apache.org/jira/browse/HBASE-16157
> Project: HBase
> Issue Type: Bug
>Reporter: ChiaPing Tsai
>Priority: Trivial
> Attachments: HBASE-16157-v1.patch
>
>
> {code:title=LruBlockCache.java|borderStyle=solid}
> // Check return value from the Map#remove before updating the metrics
> protected long evictBlock(LruCachedBlock block, boolean
> evictedByEvictionProcess) {
> map.remove(block.getCacheKey());
> updateSizeMetrics(block, true);
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15844:
--
Attachment: HBASE-15844_v1.patch
upload patch v1, we do cache root index block when write
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-15844.patch, HBASE-15844_v1.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted
[
https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358353#comment-15358353
]
Hadoop QA commented on HBASE-16135:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 16s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 17s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m 5s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 146m 22s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.regionserver.TestTableBasedReplicationSourceManagerImpl
|
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12815661/HBASE-16135-v2.patch |
| JIRA Issue | HBASE-16135 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf901.gq1.yg
[jira] [Commented] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358345#comment-15358345
]
Anoop Sam John commented on HBASE-15844:
That make sense. Seems a miss while coding.. Ya when cache on write is true
and we cache leaf and intermediate level blocks, there is no point not caching
Root. Actually Root should get more priority. Better correct here only?
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-15844.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16144) Replication queue's lock will live forever if RS acquiring the lock has died prematurely
[ https://issues.apache.org/jira/browse/HBASE-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-16144: -- Attachment: HBASE-16144-v2.patch > Replication queue's lock will live forever if RS acquiring the lock has died > prematurely > > > Key: HBASE-16144 > URL: https://issues.apache.org/jira/browse/HBASE-16144 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.1, 1.1.5, 0.98.20 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-16144-v1.patch, HBASE-16144-v2.patch > > > In default, we will use multi operation when we claimQueues from ZK. But if > we set hbase.zookeeper.useMulti=false, we will add a lock first, then copy > nodes, finally clean old queue and the lock. > However, if the RS acquiring the lock crash before claimQueues done, the lock > will always be there and other RS can never claim the queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16149) Log the underlying RPC exception in RpcRetryingCallerImpl
[
https://issues.apache.org/jira/browse/HBASE-16149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerry He updated HBASE-16149:
-
Attachment: HBASE-16149-branch-1.patch
Thanks, [~stack] Attached patch for branch-1.
> Log the underlying RPC exception in RpcRetryingCallerImpl
> --
>
> Key: HBASE-16149
> URL: https://issues.apache.org/jira/browse/HBASE-16149
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.2.0
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16149-branch-1.patch, HBASE-16149.patch
>
>
> In RpcRetryingCallerImpl:
> {code}
> public T callWithRetries(RetryingCallable callable, int callTimeout)
> throws IOException, RuntimeException {
> ...
> for (int tries = 0;; tries++) {
> try {
> ...
> return callable.call(getTimeout(callTimeout));
> ...
> } catch (Throwable t) {
> ExceptionUtil.rethrowIfInterrupt(t);
> if (tries > startLogErrorsCnt) {
> LOG.info("Call exception, tries=" + tries + ", maxAttempts=" +
> maxAttempts + ", started="
> + (EnvironmentEdgeManager.currentTime() -
> tracker.getStartTime()) + " ms ago, "
> + "cancelled=" + cancelled.get() + ", msg="
> + callable.getExceptionMessageAdditionalDetail());
> }
> ...
> {code}
> We log the callable.getExceptionMessageAdditionalDetail() msg. But
> callable.getExceptionMessageAdditionalDetail() may not provide the underlying
> cause..
> For example, in AbstractRegionServerCallable,
> {code}
> public String getExceptionMessageAdditionalDetail() {
> return "row '" + Bytes.toString(row) + "' on table '" + tableName + "' at
> " + location;
> }
> {code}
> Let's add the underlying exception cause to the message as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16144) Replication queue's lock will live forever if RS acquiring the lock has died prematurely
[ https://issues.apache.org/jira/browse/HBASE-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358346#comment-15358346 ] Phil Yang commented on HBASE-16144: --- Got it, thanks > Replication queue's lock will live forever if RS acquiring the lock has died > prematurely > > > Key: HBASE-16144 > URL: https://issues.apache.org/jira/browse/HBASE-16144 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.1, 1.1.5, 0.98.20 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-16144-v1.patch > > > In default, we will use multi operation when we claimQueues from ZK. But if > we set hbase.zookeeper.useMulti=false, we will add a lock first, then copy > nodes, finally clean old queue and the lock. > However, if the RS acquiring the lock crash before claimQueues done, the lock > will always be there and other RS can never claim the queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted
[ https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358344#comment-15358344 ] Duo Zhang commented on HBASE-16135: --- You can file another issue to do refactoring. > PeerClusterZnode under rs of removed peer may never be deleted > -- > > Key: HBASE-16135 > URL: https://issues.apache.org/jira/browse/HBASE-16135 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2, 0.98.20 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3 > > Attachments: HBASE-16135-0.98.patch, HBASE-16135-branch-1.1.patch, > HBASE-16135-branch-1.2.patch, HBASE-16135-branch-1.patch, > HBASE-16135-v1.patch, HBASE-16135-v2.patch, HBASE-16135.patch > > > One of our cluster run out of space recently, and we found that the .oldlogs > directory had almost the same size as the data directory. > Finally we found the problem is that, we removed a peer abort 3 months ago, > but there are still some replication queue znode under some rs nodes. This > prevents the deletion of .oldlogs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16144) Replication queue's lock will live forever if RS acquiring the lock has died prematurely
[ https://issues.apache.org/jira/browse/HBASE-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358341#comment-15358341 ] Ted Yu commented on HBASE-16144: bq. if we use EnvironmentEdge, it may be wrong when it is not System.currentTimeMillis()? Using EnvironmentEdgeManager is common practice. It is used in classes such as HRegion. I would suggest using EnvironmentEdgeManager in the above mentioned place. > Replication queue's lock will live forever if RS acquiring the lock has died > prematurely > > > Key: HBASE-16144 > URL: https://issues.apache.org/jira/browse/HBASE-16144 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.1, 1.1.5, 0.98.20 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-16144-v1.patch > > > In default, we will use multi operation when we claimQueues from ZK. But if > we set hbase.zookeeper.useMulti=false, we will add a lock first, then copy > nodes, finally clean old queue and the lock. > However, if the RS acquiring the lock crash before claimQueues done, the lock > will always be there and other RS can never claim the queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16144) Replication queue's lock will live forever if RS acquiring the lock has died prematurely
[ https://issues.apache.org/jira/browse/HBASE-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358336#comment-15358336 ] Phil Yang commented on HBASE-16144: --- Here array.length will always positive, so i think it is OK The timestamp of mtime is set by zk, which is always System.currentTimeMillis(), if we use EnvironmentEdge, it may be wrong when it is not System.currentTimeMillis()? Will fix other issues soon > Replication queue's lock will live forever if RS acquiring the lock has died > prematurely > > > Key: HBASE-16144 > URL: https://issues.apache.org/jira/browse/HBASE-16144 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.1, 1.1.5, 0.98.20 >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-16144-v1.patch > > > In default, we will use multi operation when we claimQueues from ZK. But if > we set hbase.zookeeper.useMulti=false, we will add a lock first, then copy > nodes, finally clean old queue and the lock. > However, if the RS acquiring the lock crash before claimQueues done, the lock > will always be there and other RS can never claim the queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358335#comment-15358335
]
Heng Chen commented on HBASE-15844:
---
Thanks [~anoop.hbase] for your suggestions. What do you think about my
proposal for root index block, open another issue for it?
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-15844.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16157) The incorrect block cache count and size are caused by removing duplicate block key in the HBase LruBlockCache
[
https://issues.apache.org/jira/browse/HBASE-16157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358327#comment-15358327
]
ChiaPing Tsai commented on HBASE-16157:
---
copy that.
> The incorrect block cache count and size are caused by removing duplicate
> block key in the HBase LruBlockCache
> --
>
> Key: HBASE-16157
> URL: https://issues.apache.org/jira/browse/HBASE-16157
> Project: HBase
> Issue Type: Bug
>Reporter: ChiaPing Tsai
>Priority: Trivial
> Attachments: HBASE-16157-v1.patch
>
>
> {code:title=LruBlockCache.java|borderStyle=solid}
> // Check return value from the Map#remove before updating the metrics
> protected long evictBlock(LruCachedBlock block, boolean
> evictedByEvictionProcess) {
> map.remove(block.getCacheKey());
> updateSizeMetrics(block, true);
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted
[ https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358326#comment-15358326 ] Ted Yu commented on HBASE-16135: To remove element from CopyOnWriteArrayList, we don't need to use iterator. We can iterate oldsources using index (in reverse) and call closeRecoveredQueue() passing the element at the index. Just for your reference. > PeerClusterZnode under rs of removed peer may never be deleted > -- > > Key: HBASE-16135 > URL: https://issues.apache.org/jira/browse/HBASE-16135 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2, 0.98.20 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3 > > Attachments: HBASE-16135-0.98.patch, HBASE-16135-branch-1.1.patch, > HBASE-16135-branch-1.2.patch, HBASE-16135-branch-1.patch, > HBASE-16135-v1.patch, HBASE-16135-v2.patch, HBASE-16135.patch > > > One of our cluster run out of space recently, and we found that the .oldlogs > directory had almost the same size as the data directory. > Finally we found the problem is that, we removed a peer abort 3 months ago, > but there are still some replication queue znode under some rs nodes. This > prevents the deletion of .oldlogs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14743) Add metrics around HeapMemoryManager
[ https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-14743: -- Attachment: HBASE-14743.010.v2.patch can't pass the tests in the TestHeapMemoryManager because of the default value of the min/max range. i remove them in the patch 011, and just keep them as origin. the metrics by default will be off and show zero in the MBeans, improve it in the feature as suggestion. sorry for my stubborn. > Add metrics around HeapMemoryManager > > > Key: HBASE-14743 > URL: https://issues.apache.org/jira/browse/HBASE-14743 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Reid Chan >Priority: Minor > Attachments: HBASE-14743.001.patch, HBASE-14743.002.patch, > HBASE-14743.003.patch, HBASE-14743.004.patch, HBASE-14743.005.patch, > HBASE-14743.006.patch, HBASE-14743.007.patch, HBASE-14743.008.patch, > HBASE-14743.009.patch, HBASE-14743.009.rw3.patch, HBASE-14743.009.v2.patch, > HBASE-14743.010.patch, HBASE-14743.010.v2.patch, HBASE-14743.011.patch, > Metrics snapshot 2016-6-30.png, Screen Shot 2016-06-16 at 5.39.13 PM.png, > test2_1.png, test2_2.png, test2_3.png, test2_4.png > > > it would be good to know how many invocations there have been. > How many decided to expand memstore. > How many decided to expand block cache. > How many decided to do nothing. > etc. > When that's done use those metrics to clean up the tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[
https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358319#comment-15358319
]
Hiroshi Ikeda commented on HBASE-15716:
---
{code}
+ } // Otherwise head.references is asynchronously　changed from 0; Check
again.
{code}
Sorry, I wrote a double-byte space between "asynchronously" and "changed".
I a little worry about duplicate logic about getting a read point with an
isolation level between HRegion.getReadpoint and ScannerReadPoints.getEntry. It
might be (or not) better to create a new method in ScannerReadPoints and
delegate, in order to collect the duplicate logic just inside the same class
(but that might become ScannerReadPoints messy).
> HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random
> read
> --
>
> Key: HBASE-15716
> URL: https://issues.apache.org/jira/browse/HBASE-15716
> Project: HBase
> Issue Type: Bug
> Components: Performance
>Reporter: stack
>Assignee: stack
> Attachments:
> 15716.implementation.using.ScannerReadPoints.branch-1.patch,
> 15716.prune.synchronizations.patch, 15716.prune.synchronizations.v3.patch,
> 15716.prune.synchronizations.v4.patch, 15716.prune.synchronizations.v4.patch,
> 15716.wip.more_to_be_done.patch, HBASE-15716.branch-1.001.patch,
> HBASE-15716.branch-1.002.patch, HBASE-15716.branch-1.003.patch,
> HBASE-15716.branch-1.004.patch, ScannerReadPoints.java, Screen Shot
> 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png,
> Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26
> PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at
> 9.49.35 AM.png, TestScannerReadPoints.java, before_after.png,
> current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png,
> remove.locks.patch, remove_cslm.patch
>
>
> Here is a [~lhofhansl] special.
> When we construct the region scanner, we get our read point and then store it
> with the scanner instance in a Region scoped CSLM. This is done under a
> synchronize on the CSLM.
> This synchronize on a region-scoped Map creating region scanners is the
> outstanding point of lock contention according to flight recorder (My work
> load is workload c, random reads).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358320#comment-15358320
]
Anoop Sam John commented on HBASE-15844:
+1
Minor suggestion can fix on commit. We have a private method getCacheOnWrite()
doing this new if check. Just use that.
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-15844.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16044) Fix 'hbase shell' output parsing in graceful_stop.sh
[
https://issues.apache.org/jira/browse/HBASE-16044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358318#comment-15358318
]
Appy commented on HBASE-16044:
--
[~asamir], does this look good to you?
> Fix 'hbase shell' output parsing in graceful_stop.sh
>
>
> Key: HBASE-16044
> URL: https://issues.apache.org/jira/browse/HBASE-16044
> Project: HBase
> Issue Type: Bug
> Components: scripts, shell
>Affects Versions: 2.0.0
>Reporter: Samir Ahmic
>Assignee: Samir Ahmic
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-16044.master.001.patch
>
>
> In some of our bash scripts we are piping command in hbase shell and then
> parsing response to define variables. Since 'hbase shell' output format is
> changed we are picking wrong values from output Here is example form
> gracful_stop.sh:
> {code}
> HBASE_BALANCER_STATE=$(echo 'balance_switch false' | "$bin"/hbase --config
> "${HBASE_CONF_DIR}" shell | tail -3 | head -1)
> {code}
> this will return "balance_switch true" instead of previous balancer state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[
https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358316#comment-15358316
]
Hiroshi Ikeda commented on HBASE-15716:
---
I meant, in Listener.doAccept, change
{code}
Reader reader = getReader();
try {
reader.startAdd();
SelectionKey readKey = reader.registerChannel(channel);
// calling channel.register(readSelector, SelectionKey.OP_READ),
// possibly blocked until the reader returns from the
readSelector.select,
// depending on VM implementations
c = getConnection(channel, System.currentTimeMillis());
readKey.attach(c);
...
} finally {
reader.finishAdd();
// The reader will be blocked until calling this.
}
{code}
with
{code}
Reader reader = getReader();
c = getConnection(channel, System.currentTimeMillis());
...
reader.registrationQueue.offer(c);
// The channel is still just-accepted, and the reader thread will register
later.
reader.readSelector.wakeup();
{code}
In Reader.doRunLoop, change
{code}
readSelector.select();
while (adding) {
this.wait(1000);
}
...
{code}
with
{code}
readSelector.select();
Connection c;
while ((c = registrationQueue.poll()) != null) {
// Register anything in the queue.
try {
c.channel.register(readSelector, SelectionKey.OP_READ, c)
} catch (ClosedChannelException e) {
// The channel is already closed somewhere.
// That's legal, and you might want to just log something.
}
}
...
{code}
That is a little part of the logic of the patch I created in HBASE-14479.
Selector is subtle against multi-threads.
> HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random
> read
> --
>
> Key: HBASE-15716
> URL: https://issues.apache.org/jira/browse/HBASE-15716
> Project: HBase
> Issue Type: Bug
> Components: Performance
>Reporter: stack
>Assignee: stack
> Attachments:
> 15716.implementation.using.ScannerReadPoints.branch-1.patch,
> 15716.prune.synchronizations.patch, 15716.prune.synchronizations.v3.patch,
> 15716.prune.synchronizations.v4.patch, 15716.prune.synchronizations.v4.patch,
> 15716.wip.more_to_be_done.patch, HBASE-15716.branch-1.001.patch,
> HBASE-15716.branch-1.002.patch, HBASE-15716.branch-1.003.patch,
> HBASE-15716.branch-1.004.patch, ScannerReadPoints.java, Screen Shot
> 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png,
> Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26
> PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at
> 9.49.35 AM.png, TestScannerReadPoints.java, before_after.png,
> current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png,
> remove.locks.patch, remove_cslm.patch
>
>
> Here is a [~lhofhansl] special.
> When we construct the region scanner, we get our read point and then store it
> with the scanner instance in a Region scoped CSLM. This is done under a
> synchronize on the CSLM.
> This synchronize on a region-scoped Map creating region scanners is the
> outstanding point of lock contention according to flight recorder (My work
> load is workload c, random reads).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358315#comment-15358315
]
Heng Chen commented on HBASE-15844:
---
And more thoughts, i think we should cache root index block when write by
default. Currently it seems not do this work
{code: title=BlockIndexWriter#writeIndexBlocks}
// write the root level
long rootLevelIndexPos = out.getPos();
{
DataOutput blockStream =
blockWriter.startWriting(BlockType.ROOT_INDEX);
rootChunk.writeRoot(blockStream);
if (midKeyMetadata != null)
blockStream.write(midKeyMetadata);
blockWriter.writeHeaderAndData(out);
}
{code}
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-15844.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14743) Add metrics around HeapMemoryManager
[
https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358311#comment-15358311
]
Hadoop QA commented on HBASE-14743:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 2s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 43s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 16s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 0s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 38s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 31s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 19s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 25s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 51s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 91m 41s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHeapMemoryManager |
| | hadoop.hbase.regionserver.TestMetricsHeapMemoryManager |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12815664/HBASE-1
[jira] [Updated] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15844:
--
Fix Version/s: 2.0.0
Status: Patch Available (was: Open)
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-15844.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
[
https://issues.apache.org/jira/browse/HBASE-15844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-15844:
--
Attachment: HBASE-15844.patch
upload a simple patch
> We should respect hfile.block.index.cacheonwrite when write intermediate
> index Block
>
>
> Key: HBASE-15844
> URL: https://issues.apache.org/jira/browse/HBASE-15844
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
> Attachments: HBASE-15844.patch
>
>
> {code: title=BlockIndexWriter#writeIntermediateBlock}
> if (cacheConf != null) {
> HFileBlock blockForCaching =
> blockWriter.getBlockForCaching(cacheConf);
> cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
> beginOffset, true, blockForCaching.getBlockType()),
> blockForCaching);
> }
> {code}
> The if condition should be ?
> {code}
> if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes
[ https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16096: --- Status: Patch Available (was: Open) > Replication keeps accumulating znodes > - > > Key: HBASE-16096 > URL: https://issues.apache.org/jira/browse/HBASE-16096 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.2.0, 2.0.0, 1.3.0 >Reporter: Ashu Pachauri >Assignee: Joseph > Attachments: HBASE-16096.patch > > > If there is an error while creating the replication source on adding the > peer, the source if not added to the in memory list of sources but the > replication peer is. > However, in such a scenario, when you remove the peer, it is deleted from > zookeeper successfully but for removing the in memory list of peers, we wait > for the corresponding sources to get deleted (which as we said don't exist > because of error creating the source). > The problem here is the ordering of operations for adding/removing source and > peer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes
[ https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16096: --- Status: Open (was: Patch Available) Overriding ReplicationPeerZkImpl.nodeCreated() without calling the super method caused the watcher to fail as ReplicationPeerZkImpl.nodeDataChanged() eventually called nodeCreated(). This also led to an infinite recursive loop. Fixed this in the newest patch and ran the failed test cases. > Replication keeps accumulating znodes > - > > Key: HBASE-16096 > URL: https://issues.apache.org/jira/browse/HBASE-16096 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.2.0, 2.0.0, 1.3.0 >Reporter: Ashu Pachauri >Assignee: Joseph > Attachments: HBASE-16096.patch > > > If there is an error while creating the replication source on adding the > peer, the source if not added to the in memory list of sources but the > replication peer is. > However, in such a scenario, when you remove the peer, it is deleted from > zookeeper successfully but for removing the in memory list of peers, we wait > for the corresponding sources to get deleted (which as we said don't exist > because of error creating the source). > The problem here is the ordering of operations for adding/removing source and > peer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
[ https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma updated HBASE-16160: - Attachment: HBASE-16160.001.patch > Get the UnsupportedOperationException when using delegation token with > encryption > - > > Key: HBASE-16160 > URL: https://issues.apache.org/jira/browse/HBASE-16160 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma > Attachments: HBASE-16160.001.patch > > > Using delegation token with encryption, when do the Put operation, will get > the following exception: > [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] > ipc.CallRunner(161): > RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: > java.lang.UnsupportedOperationException > at java.nio.ByteBuffer.array(ByteBuffer.java:959) > at > org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
[ https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma updated HBASE-16160: - Status: Patch Available (was: Open) > Get the UnsupportedOperationException when using delegation token with > encryption > - > > Key: HBASE-16160 > URL: https://issues.apache.org/jira/browse/HBASE-16160 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma > Attachments: HBASE-16160.001.patch > > > Using delegation token with encryption, when do the Put operation, will get > the following exception: > [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] > ipc.CallRunner(161): > RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: > java.lang.UnsupportedOperationException > at java.nio.ByteBuffer.array(ByteBuffer.java:959) > at > org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes
[ https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16096: --- Attachment: (was: HBASE-16096.patch) > Replication keeps accumulating znodes > - > > Key: HBASE-16096 > URL: https://issues.apache.org/jira/browse/HBASE-16096 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Ashu Pachauri >Assignee: Joseph > Attachments: HBASE-16096.patch > > > If there is an error while creating the replication source on adding the > peer, the source if not added to the in memory list of sources but the > replication peer is. > However, in such a scenario, when you remove the peer, it is deleted from > zookeeper successfully but for removing the in memory list of peers, we wait > for the corresponding sources to get deleted (which as we said don't exist > because of error creating the source). > The problem here is the ordering of operations for adding/removing source and > peer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes
[ https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16096: --- Attachment: (was: HBASE-16096-v2.patch) > Replication keeps accumulating znodes > - > > Key: HBASE-16096 > URL: https://issues.apache.org/jira/browse/HBASE-16096 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Ashu Pachauri >Assignee: Joseph > Attachments: HBASE-16096.patch > > > If there is an error while creating the replication source on adding the > peer, the source if not added to the in memory list of sources but the > replication peer is. > However, in such a scenario, when you remove the peer, it is deleted from > zookeeper successfully but for removing the in memory list of peers, we wait > for the corresponding sources to get deleted (which as we said don't exist > because of error creating the source). > The problem here is the ordering of operations for adding/removing source and > peer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes
[ https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16096: --- Attachment: HBASE-16096.patch > Replication keeps accumulating znodes > - > > Key: HBASE-16096 > URL: https://issues.apache.org/jira/browse/HBASE-16096 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Ashu Pachauri >Assignee: Joseph > Attachments: HBASE-16096.patch > > > If there is an error while creating the replication source on adding the > peer, the source if not added to the in memory list of sources but the > replication peer is. > However, in such a scenario, when you remove the peer, it is deleted from > zookeeper successfully but for removing the in memory list of peers, we wait > for the corresponding sources to get deleted (which as we said don't exist > because of error creating the source). > The problem here is the ordering of operations for adding/removing source and > peer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted
[ https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358299#comment-15358299 ] Ted Yu commented on HBASE-16135: Since removing from CopyOnWriteArrayList involves shifting, looks like we should traverse oldSourcesToDelete in reverse order. Can be done in another JIRA. > PeerClusterZnode under rs of removed peer may never be deleted > -- > > Key: HBASE-16135 > URL: https://issues.apache.org/jira/browse/HBASE-16135 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2, 0.98.20 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3 > > Attachments: HBASE-16135-0.98.patch, HBASE-16135-branch-1.1.patch, > HBASE-16135-branch-1.2.patch, HBASE-16135-branch-1.patch, > HBASE-16135-v1.patch, HBASE-16135-v2.patch, HBASE-16135.patch > > > One of our cluster run out of space recently, and we found that the .oldlogs > directory had almost the same size as the data directory. > Finally we found the problem is that, we removed a peer abort 3 months ago, > but there are still some replication queue znode under some rs nodes. This > prevents the deletion of .oldlogs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
Colin Ma created HBASE-16160: Summary: Get the UnsupportedOperationException when using delegation token with encryption Key: HBASE-16160 URL: https://issues.apache.org/jira/browse/HBASE-16160 Project: HBase Issue Type: Bug Reporter: Colin Ma Assignee: Colin Ma Using delegation token with encryption, when do the Put operation, will get the following exception: [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] ipc.CallRunner(161): RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: java.lang.UnsupportedOperationException at java.nio.ByteBuffer.array(ByteBuffer.java:959) at org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66) at org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547) at org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16150) Remove ConcurrentIndex
[ https://issues.apache.org/jira/browse/HBASE-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358278#comment-15358278 ] Hudson commented on HBASE-16150: SUCCESS: Integrated in HBase-1.3 #763 (See [https://builds.apache.org/job/HBase-1.3/763/]) HBASE-16150 Remove ConcurrentIndex (Hiroshi Ikeda) (stack: rev b8c6233ba3a63b542b2b85a62f4b16cadcf0ed45) * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ConcurrentIndex.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java > Remove ConcurrentIndex > -- > > Key: HBASE-16150 > URL: https://issues.apache.org/jira/browse/HBASE-16150 > Project: HBase > Issue Type: Bug >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda >Priority: Minor > Attachments: HBASE-16150-branch-1.patch, HBASE-16150-master.patch, > remove_index.png > > > ConcurrentIndex has race condition and it is enough to use > ConcurrentSkipListSet instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15729) Remove old JDiff wrapper scripts in dev-support
[ https://issues.apache.org/jira/browse/HBASE-15729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358262#comment-15358262 ] Hudson commented on HBASE-15729: FAILURE: Integrated in HBase-1.4 #261 (See [https://builds.apache.org/job/HBase-1.4/261/]) HBASE-15729 Remove old JDiff wrapper scripts in dev-support (busbey: rev f280fa21fb0b20871f27aa0bff45b379b4d9b0e5) * dev-support/hbase_jdiff_afterSingularityTemplate.xml * dev-support/jdiffHBasePublicAPI.sh * dev-support/hbase_jdiff_template.xml * dev-support/hbase_jdiff_acrossSingularityTemplate.xml * dev-support/jdiffHBasePublicAPI_common.sh > Remove old JDiff wrapper scripts in dev-support > --- > > Key: HBASE-15729 > URL: https://issues.apache.org/jira/browse/HBASE-15729 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Dima Spivak >Assignee: Dima Spivak >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15729.patch > > > Since HBASE-12808, we've been using the Java API Compliance Checker instead > of JDiff to look at API compatibility. Probably makes sense to remove the old > wrapper scripts that aren't being used anymore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes
[ https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16096: --- Attachment: HBASE-16096-v2.patch > Replication keeps accumulating znodes > - > > Key: HBASE-16096 > URL: https://issues.apache.org/jira/browse/HBASE-16096 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Ashu Pachauri >Assignee: Joseph > Attachments: HBASE-16096-v2.patch, HBASE-16096.patch > > > If there is an error while creating the replication source on adding the > peer, the source if not added to the in memory list of sources but the > replication peer is. > However, in such a scenario, when you remove the peer, it is deleted from > zookeeper successfully but for removing the in memory list of peers, we wait > for the corresponding sources to get deleted (which as we said don't exist > because of error creating the source). > The problem here is the ordering of operations for adding/removing source and > peer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16150) Remove ConcurrentIndex
[ https://issues.apache.org/jira/browse/HBASE-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358266#comment-15358266 ] Hudson commented on HBASE-16150: FAILURE: Integrated in HBase-1.4 #261 (See [https://builds.apache.org/job/HBase-1.4/261/]) HBASE-16150 Remove ConcurrentIndex (Hiroshi Ikeda) (stack: rev f4d7cd5f58ad79fd2b4d0f3f8d7547c5196a19c5) * hbase-common/src/main/java/org/apache/hadoop/hbase/util/ConcurrentIndex.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java > Remove ConcurrentIndex > -- > > Key: HBASE-16150 > URL: https://issues.apache.org/jira/browse/HBASE-16150 > Project: HBase > Issue Type: Bug >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda >Priority: Minor > Attachments: HBASE-16150-branch-1.patch, HBASE-16150-master.patch, > remove_index.png > > > ConcurrentIndex has race condition and it is enough to use > ConcurrentSkipListSet instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16124) Make check_compatibility.sh less verbose when building HBase
[
https://issues.apache.org/jira/browse/HBASE-16124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358268#comment-15358268
]
Hudson commented on HBASE-16124:
FAILURE: Integrated in HBase-1.4 #261 (See
[https://builds.apache.org/job/HBase-1.4/261/])
HBASE-16124 Make check_compatibility.sh less verbose when building HBase
(busbey: rev 7f56fb2de79d54c82b4e55dec75087b2306d505c)
* dev-support/check_compatibility.sh
> Make check_compatibility.sh less verbose when building HBase
>
>
> Key: HBASE-16124
> URL: https://issues.apache.org/jira/browse/HBASE-16124
> Project: HBase
> Issue Type: Improvement
> Components: build, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16124_v1.patch
>
>
> {{[check_compatibility.sh|https://github.com/apache/hbase/blob/master/dev-support/check_compatibility.sh]}}
> is a bit verbose when building HBase JARs, which makes it kind of a
> nightmare when used in a Jenkins job. Let's run those steps in Maven's batch
> mode, which means less unnecessary output and no expectation of user
> interaction.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16129) check_compatibility.sh is broken when using Java API Compliance Checker v1.7
[ https://issues.apache.org/jira/browse/HBASE-16129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358265#comment-15358265 ] Hudson commented on HBASE-16129: FAILURE: Integrated in HBase-1.4 #261 (See [https://builds.apache.org/job/HBase-1.4/261/]) HBASE-16129 check_compatibility.sh is broken when using Java API (busbey: rev aa361a20ced90d016dc45d6dd533a075e5f12b19) * dev-support/check_compatibility.sh > check_compatibility.sh is broken when using Java API Compliance Checker v1.7 > > > Key: HBASE-16129 > URL: https://issues.apache.org/jira/browse/HBASE-16129 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Dima Spivak >Assignee: Dima Spivak > Fix For: 2.0.0 > > Attachments: HBASE-16129_v1.patch, HBASE-16129_v2.patch, > HBASE-16129_v3.patch > > > As part of HBASE-16073, we hardcoded check_compatiblity.sh to check out the > v1.7 tag of Java ACC. Unfortunately, just running it between two branches > that I know have incompatibilities, I get 0 incompatibilities (and 0 classes > read). Looks like this version doesn't properly traverse through JARs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15119) Include git SHA in check_compatibility reports
[ https://issues.apache.org/jira/browse/HBASE-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358267#comment-15358267 ] Hudson commented on HBASE-15119: FAILURE: Integrated in HBase-1.4 #261 (See [https://builds.apache.org/job/HBase-1.4/261/]) HBASE-15119 Include git SHA in check_compatibility reports (busbey: rev 13535f4d43b1699f19d2177daf04278288c63da1) * dev-support/check_compatibility.sh > Include git SHA in check_compatibility reports > -- > > Key: HBASE-15119 > URL: https://issues.apache.org/jira/browse/HBASE-15119 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15119.v00.patch > > > Since some refs change over time (ie, branches), it would be nice to include > git shas in the version info included in check compatibility reports. It'll > also help interested parties to be sure of what they're looking at. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[ https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358263#comment-15358263 ] Hudson commented on HBASE-16073: FAILURE: Integrated in HBase-1.4 #261 (See [https://builds.apache.org/job/HBase-1.4/261/]) HBASE-16073 update compatibility_checker for jacc dropping comma sep (busbey: rev 854e796ea93242f3bb91d900e8fe37dc7a3fb4ef) * dev-support/check_compatibility.sh > update compatibility_checker for jacc dropping comma sep args > - > > Key: HBASE-16073 > URL: https://issues.apache.org/jira/browse/HBASE-16073 > Project: HBase > Issue Type: Task > Components: build, documentation >Reporter: Sean Busbey >Assignee: Dima Spivak >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-16073_v1.patch, HBASE-16073_v2.patch > > > the japi-compliance-checker has a change in place (post the 1.7 release) that > removes the ability to give a comma separated list of jars on the cli. > we should switch to generating descriptor xml docs since that will still be > supported, or update to use the expanded tooling suggested in the issue: > https://github.com/lvc/japi-compliance-checker/issues/27 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16147) Add ruby wrapper for getting compaction state
[ https://issues.apache.org/jira/browse/HBASE-16147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358264#comment-15358264 ] Hudson commented on HBASE-16147: FAILURE: Integrated in HBase-1.4 #261 (See [https://builds.apache.org/job/HBase-1.4/261/]) HBASE-16147 Addendum fixes syntax in admin_test (tedyu: rev cbc99f6d5ae004c78e5b2572c2ac9dec7d3e3ffc) * hbase-shell/src/test/ruby/hbase/admin_test.rb HBASE-16147 Addendum fixes syntax in admin_test (tedyu: rev b2f9f131b2e0d19439bcad701d34169995bc34b3) * hbase-shell/src/test/ruby/hbase/admin_test.rb > Add ruby wrapper for getting compaction state > - > > Key: HBASE-16147 > URL: https://issues.apache.org/jira/browse/HBASE-16147 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 2.0.0, 1.4.0 > > Attachments: 16147.v1.txt, 16147.v2.txt > > > [~romil.choksi] was asking for command that can poll compaction status from > hbase shell. > This issue is to add ruby wrapper for getting compaction state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted
[
https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358260#comment-15358260
]
Duo Zhang commented on HBASE-16135:
---
oldsources is a {{CopyOnWriteArrayList}}, its iterator does not support remove
which means we can not remove element from it on fly.
> PeerClusterZnode under rs of removed peer may never be deleted
> --
>
> Key: HBASE-16135
> URL: https://issues.apache.org/jira/browse/HBASE-16135
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2, 0.98.20
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3
>
> Attachments: HBASE-16135-0.98.patch, HBASE-16135-branch-1.1.patch,
> HBASE-16135-branch-1.2.patch, HBASE-16135-branch-1.patch,
> HBASE-16135-v1.patch, HBASE-16135-v2.patch, HBASE-16135.patch
>
>
> One of our cluster run out of space recently, and we found that the .oldlogs
> directory had almost the same size as the data directory.
> Finally we found the problem is that, we removed a peer abort 3 months ago,
> but there are still some replication queue znode under some rs nodes. This
> prevents the deletion of .oldlogs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16159) OutOfMemory exception when using AsyncRpcClient with encryption to read rpc response
[
https://issues.apache.org/jira/browse/HBASE-16159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358269#comment-15358269
]
Hadoop QA commented on HBASE-16159:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 28s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
40m 1s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 20s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 6s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12815660/HBASE-16159.001.patch
|
| JIRA Issue | HBASE-16159 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf910.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d1d8cc7 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/home/jenkins/jenkins-slave/tools/hudson.mod
[jira] [Commented] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted
[
https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358241#comment-15358241
]
Ted Yu commented on HBASE-16135:
If you want put the following inside the synchronized (oldsources) block:
{code}
+ for (ReplicationSourceInterface src : oldSourcesToDelete) {
+src.terminate(terminateMessage);
+closeRecoveredQueue(src);
{code}
oldSourcesToDelete becomes unnecessary.
> PeerClusterZnode under rs of removed peer may never be deleted
> --
>
> Key: HBASE-16135
> URL: https://issues.apache.org/jira/browse/HBASE-16135
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2, 0.98.20
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 0.98.21, 1.2.3
>
> Attachments: HBASE-16135-0.98.patch, HBASE-16135-branch-1.1.patch,
> HBASE-16135-branch-1.2.patch, HBASE-16135-branch-1.patch,
> HBASE-16135-v1.patch, HBASE-16135-v2.patch, HBASE-16135.patch
>
>
> One of our cluster run out of space recently, and we found that the .oldlogs
> directory had almost the same size as the data directory.
> Finally we found the problem is that, we removed a peer abort 3 months ago,
> but there are still some replication queue znode under some rs nodes. This
> prevents the deletion of .oldlogs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15965) Shell test changes. Use @shell.command instead directly calling functions in admin.rb and other libraries.
[
https://issues.apache.org/jira/browse/HBASE-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358223#comment-15358223
]
Junegunn Choi commented on HBASE-15965:
---
[~Appy]
I was reminded of the same xkcd strip when I first ran into this :)
bq. You should be able to get the return value using -n (non-interactive) flag.
Yes, but it's quite useful to be able to *incrementally explore* the data
interactively on the REPL. Internally, we've been using a customized version of
the shell with a few extra commands that allow us to explore the status of the
cluster without having to write verbose Java code with Admin API. One such
command is {{regions}}, and this is an example of what we do when things happen.
{code}
# See the number of regions
regions.count
# See the number of regions whose localities are below 50%
regions.select { |r| r[:locality] < 50 }
# Pretty-print those regions grouped by their tables
require 'pp'
pp regions.select { |r| r[:locality] < 50 }.group_by { |r| r[:table] }
# Flush and major compact regions that match the criteria
regions.select { |r| r[:locality] < 50 && r[:files] > 2 }.each { |r| flush
r[:name]; major_compact r[:name] }
{code}
If I had to use {{-n}} flag, I'll have to re-run hbase shell multiple times.
There's nothing wrong with harnessing the power of REPL.
bq. Changing return value to contain more information will break someone else's
workflow
Yes, the concern led me to the idea of introducing new commands instead of
changing the return values of the existing ones.
bq. That approach requires duplicating classes and we over 100 commands
I think we have a misunderstanding here. I was not suggesting that we should
replicate every one of the commands, but to add just a handful of new ones
(tables, snapshots, regions, servers, etc.) under a new command group named
something like QUERY or INSPECTION whose members are supposed to return values
with clear semantics. They aren't for side-effects, they are not responsible
for formatting and printing the output, they should just return values that can
be used by the user programmatically even in "interactive" mode. I don't
disagree with suppressing the return values of the commands in other groups for
cleaner output, but the commands in this group are solely for their return
values, so we make an exception. What do you think?
> Shell test changes. Use @shell.command instead directly calling functions in
> admin.rb and other libraries.
> --
>
> Key: HBASE-15965
> URL: https://issues.apache.org/jira/browse/HBASE-15965
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Fix For: 2.0.0
>
> Attachments: HBASE-15965.master.001.patch,
> HBASE-15965.master.002.patch, HBASE-15965.master.003.patch
>
>
> Testing by executing a command will cover the exact path users will trigger,
> so its better then directly calling library functions in tests. Changing the
> tests to use @shell.command(:, args) to execute them like it's a
> command coming from shell.
> Norm change:
> Commands should print the output user would like to see, but in the end,
> should also return the relevant value. This way:
> - Tests can use returned value to check that functionality works
> - Tests can capture stdout to assert particular kind of output user should
> see.
> - We do not print the return value in interactive mode and keep the output
> clean. See Shell.command() function.
> Bugs found due to this change:
> - Uncovered bug in major_compact.rb with this approach. It was calling
> admin.majorCompact() which doesn't exist but our tests didn't catch it since
> they directly tested admin.major_compact()
> - Enabled TestReplicationShell. If it's bad, flaky infra will take care of it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16087) Replication shouldn't start on a master if if only hosts system tables
[ https://issues.apache.org/jira/browse/HBASE-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358218#comment-15358218 ] Ashish Singhi commented on HBASE-16087: --- Hi Gary, As I already mentioned in the below comment we have a use case where we have to replicate these tables. To be clear I am not objecting this patch to get pushed in, only thing I am asking is to make it configurable. > Replication shouldn't start on a master if if only hosts system tables > -- > > Key: HBASE-16087 > URL: https://issues.apache.org/jira/browse/HBASE-16087 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-16087.patch, HBASE-16087.v1.patch > > > System tables aren't replicated so we shouldn't start up a replication master > if there are no user tables on the master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14743) Add metrics around HeapMemoryManager
[ https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-14743: -- Attachment: HBASE-14743.011.patch HadoopQA didn't test last 010.patch...? This patch change the initial value of: "globalMemStorePercentMinRange = conf.getFloat(MEMSTORE_SIZE_MIN_RANGE_KEY, globalMemStorePercent / 3);" and "blockCachePercentMinRange = conf.getFloat(BLOCK_CACHE_SIZE_MIN_RANGE_KEY, blockCachePercent / 4);" Thanks > Add metrics around HeapMemoryManager > > > Key: HBASE-14743 > URL: https://issues.apache.org/jira/browse/HBASE-14743 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Reid Chan >Priority: Minor > Attachments: HBASE-14743.001.patch, HBASE-14743.002.patch, > HBASE-14743.003.patch, HBASE-14743.004.patch, HBASE-14743.005.patch, > HBASE-14743.006.patch, HBASE-14743.007.patch, HBASE-14743.008.patch, > HBASE-14743.009.patch, HBASE-14743.009.rw3.patch, HBASE-14743.009.v2.patch, > HBASE-14743.010.patch, HBASE-14743.011.patch, Metrics snapshot 2016-6-30.png, > Screen Shot 2016-06-16 at 5.39.13 PM.png, test2_1.png, test2_2.png, > test2_3.png, test2_4.png > > > it would be good to know how many invocations there have been. > How many decided to expand memstore. > How many decided to expand block cache. > How many decided to do nothing. > etc. > When that's done use those metrics to clean up the tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
