[jira] [Commented] (HBASE-15698) Increment TimeRange not serialized to server
[ https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318001#comment-15318001 ] Hudson commented on HBASE-15698: SUCCESS: Integrated in HBase-1.3-IT #693 (See [https://builds.apache.org/job/HBase-1.3-IT/693/]) HBASE-15698 Increment TimeRange not serialized to server (Ted Yu) (apurtell: rev 05e1013b08f7bd3699a6417014833efe7340c91b) * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestIncrementTimeRange.java > Increment TimeRange not serialized to server > > > Key: HBASE-15698 > URL: https://issues.apache.org/jira/browse/HBASE-15698 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0 >Reporter: James Taylor >Assignee: Ted Yu >Priority: Blocker > Labels: phoenix > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt, > 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, 15698.v6.txt, > 15698.v7.txt, HBASE-15698.1.patch > > > Before HBase-1.2, the Increment TimeRange set on the client was serialized > over to the server. As of HBase 1.2, this appears to no longer be true, as my > preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value > of increment.getTimeRange().getMax() regardless of what the client has > specified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[
https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317973#comment-15317973
]
Hadoop QA commented on HBASE-15948:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 47s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 16s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 88m 36s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 132m 27s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12808549/HBASE-15948.master.004.patch
|
| JIRA Issue | HBASE-15948 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/L
[jira] [Updated] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-14644: - Attachment: HBASE-14644-v002.patch Patch v2 cleans up some comments. Manually run of the failured test class passed locally (not caused by this change) > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Attachments: HBASE-14644-v001.patch, HBASE-14644-v002.patch, > branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15976) RegionServerMetricsWrapperRunnable will be failure when disable blockcache.
[ https://issues.apache.org/jira/browse/HBASE-15976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-15976: - Attachment: (was: HBASE-15976.patch) > RegionServerMetricsWrapperRunnable will be failure when disable blockcache. > > > Key: HBASE-15976 > URL: https://issues.apache.org/jira/browse/HBASE-15976 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.15 >Reporter: Liu Junhong >Assignee: Jingcheng Du > Attachments: HBASE-15976-0.98.patch > > > When i disable blockcache, the code "cacheStats = blockCache.getStats();" > will occur NPE in > org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.RegionServerMetricsWrapperRunnable. > It lead to many regionserver's metrics' value always equal 0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15976) RegionServerMetricsWrapperRunnable will be failure when disable blockcache.
[ https://issues.apache.org/jira/browse/HBASE-15976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-15976: - Attachment: HBASE-15976-0.98.patch Re-upload the patch for 0.98. This issue is also in other branches, I will upload patches to fix them. > RegionServerMetricsWrapperRunnable will be failure when disable blockcache. > > > Key: HBASE-15976 > URL: https://issues.apache.org/jira/browse/HBASE-15976 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.15 >Reporter: Liu Junhong >Assignee: Jingcheng Du > Attachments: HBASE-15976-0.98.patch > > > When i disable blockcache, the code "cacheStats = blockCache.getStats();" > will occur NPE in > org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.RegionServerMetricsWrapperRunnable. > It lead to many regionserver's metrics' value always equal 0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15976) RegionServerMetricsWrapperRunnable will be failure when disable blockcache.
[ https://issues.apache.org/jira/browse/HBASE-15976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-15976: - Attachment: HBASE-15976.patch Upload a patch to fix this. > RegionServerMetricsWrapperRunnable will be failure when disable blockcache. > > > Key: HBASE-15976 > URL: https://issues.apache.org/jira/browse/HBASE-15976 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.15 >Reporter: Liu Junhong >Assignee: Jingcheng Du > Attachments: HBASE-15976.patch > > > When i disable blockcache, the code "cacheStats = blockCache.getStats();" > will occur NPE in > org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.RegionServerMetricsWrapperRunnable. > It lead to many regionserver's metrics' value always equal 0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317948#comment-15317948 ] huaxiang sun commented on HBASE-14644: -- The unittest failures seem unrelated. I will resubmit a updated patch with comments cleaned up. > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Attachments: HBASE-14644-v001.patch, branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15967) Metric for active ipc Readers and make default fraction of cpu count
[ https://issues.apache.org/jira/browse/HBASE-15967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317950#comment-15317950 ] Hiroshi Ikeda commented on HBASE-15967: --- I think it is enough to queue and pay attention to requests' priority just when the requests are too many so that most reader threads are going to handle requests. One or some of reader threads should still remain to accept and enqueue requests until the queue is full. The priority or schedule becomes not so useful once the queue becomes full. Newly coming requests will be left on the native socket's buffer regardless of their priority. Another port should have been used for the priority purpose. I'll check HBASE-15971 when I have time but it seems difficult for me so don't expect much. > Metric for active ipc Readers and make default fraction of cpu count > > > Key: HBASE-15967 > URL: https://issues.apache.org/jira/browse/HBASE-15967 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Attachments: HBASE-15967.master.001.patch > > > Our ipc Readers are hard coded at 10 regardless since . Running w/ less > Readers, we go faster..(e.g. 12 Readers has us doing 135k with workloadc and > 6 readers has us doing 145k).. .but hard to tell what count of Readers are > needed since no metric. > This issue changes Readers to be 1/4 the installed CPUs or 8, whichever is > the minimum, and then adds a new hbase.regionserver.ipc.runningReaders metric > so you have a chance seeing whats needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-15976) RegionServerMetricsWrapperRunnable will be failure when disable blockcache.
[ https://issues.apache.org/jira/browse/HBASE-15976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du reassigned HBASE-15976: Assignee: Jingcheng Du > RegionServerMetricsWrapperRunnable will be failure when disable blockcache. > > > Key: HBASE-15976 > URL: https://issues.apache.org/jira/browse/HBASE-15976 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.15 >Reporter: Liu Junhong >Assignee: Jingcheng Du > > When i disable blockcache, the code "cacheStats = blockCache.getStats();" > will occur NPE in > org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.RegionServerMetricsWrapperRunnable. > It lead to many regionserver's metrics' value always equal 0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15698) Increment TimeRange not serialized to server
[ https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15698: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) Pushed up to all branches > Increment TimeRange not serialized to server > > > Key: HBASE-15698 > URL: https://issues.apache.org/jira/browse/HBASE-15698 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0 >Reporter: James Taylor >Assignee: Ted Yu >Priority: Blocker > Labels: phoenix > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt, > 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, 15698.v6.txt, > 15698.v7.txt, HBASE-15698.1.patch > > > Before HBase-1.2, the Increment TimeRange set on the client was serialized > over to the server. As of HBase 1.2, this appears to no longer be true, as my > preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value > of increment.getTimeRange().getMax() regardless of what the client has > specified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15975) logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
[
https://issues.apache.org/jira/browse/HBASE-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317930#comment-15317930
]
huaxiang sun commented on HBASE-15975:
--
Thanks [~mbertozzi], I will make changes as suggested, will upload a new patch
tomorrow.
> logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
>
>
> Key: HBASE-15975
> URL: https://issues.apache.org/jira/browse/HBASE-15975
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: master
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Trivial
> Attachments: HBASE-15975-v001.patch
>
>
> While working on an unitest case for HBASE-14644, crossed over
> testAddCoprocessorWithSpecStr().
> {code}
>HTableDescriptor htd = new HTableDescriptor(TableName.META_TABLE_NAME);
> String cpName = "a.b.c.d";
> boolean expected = false;
> try {
> htd.addCoprocessorWithSpec(cpName);
> } catch (IllegalArgumentException iae) {
> expected = true;
> }
> if (!expected) fail();
> // Try minimal spec.
> try {
> htd.addCoprocessorWithSpec("file:///some/path" + "|" + cpName);
> } catch (IllegalArgumentException iae) {
> expected = false;
> }
> if (expected) fail();
> // Try more spec.
> String spec =
> "hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2";
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IllegalArgumentException iae) {
> expected = false; It should be true as it is expected to succeed.
> }
> if (expected) fail();
> // Try double add of same coprocessor
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IOException ioe) {
> expected = true;
> }
> if (!expected) fail();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15970) Move Replication Peers into an HBase table too
[ https://issues.apache.org/jira/browse/HBASE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317916#comment-15317916 ] Ted Yu commented on HBASE-15970: I think we should target using one system table (hbase:replication) for tracking both queues and replication peers. > Move Replication Peers into an HBase table too > -- > > Key: HBASE-15970 > URL: https://issues.apache.org/jira/browse/HBASE-15970 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > > Currently ReplicationQueuesHBaseTableImpl relies on ReplicationStateZkImpl to > track information about the available replication peers (used during > claimQueues). We can also move this into an HBase table instead of relying on > ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15965) Shell test changes. Use @shell.command instead directly calling functions in admin.rb and other libraries.
[ https://issues.apache.org/jira/browse/HBASE-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317902#comment-15317902 ] Hudson commented on HBASE-15965: SUCCESS: Integrated in HBase-1.4 #198 (See [https://builds.apache.org/job/HBase-1.4/198/]) HBASE-15965 - Testing by executing a command will cover the exact path (appy: rev c2b4c6f6375346b5cf76a53efdcb36ea6239a30f) * hbase-shell/src/test/ruby/test_helper.rb * hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestReplicationShell.java * hbase-shell/src/main/ruby/shell/commands/show_peer_tableCFs.rb * hbase-shell/src/main/ruby/hbase/admin.rb * hbase-shell/src/main/ruby/shell/commands/create.rb * hbase-shell/src/main/ruby/shell/commands/list_peer_configs.rb * hbase-shell/src/main/ruby/shell/commands/get_peer_config.rb * hbase-shell/src/main/ruby/shell/commands/truncate.rb * hbase-shell/src/test/ruby/hbase/replication_admin_test.rb * hbase-shell/src/main/ruby/shell/commands.rb * hbase-shell/src/main/ruby/shell.rb * hbase-shell/src/main/ruby/shell/commands/list_peers.rb * hbase-shell/src/main/ruby/shell/commands/list_namespace_tables.rb * hbase-shell/src/main/ruby/hbase/table.rb * hbase-shell/src/main/ruby/shell/commands/exists.rb * hbase-shell/src/main/ruby/shell/commands/is_enabled.rb * hbase-shell/src/test/ruby/hbase/visibility_labels_admin_test.rb * hbase-shell/src/main/ruby/shell/commands/locate_region.rb * hbase-shell/src/main/ruby/shell/commands/get_auths.rb * hbase-shell/src/test/ruby/hbase/admin_test.rb * hbase-shell/src/main/ruby/shell/commands/truncate_preserve.rb > Shell test changes. Use @shell.command instead directly calling functions in > admin.rb and other libraries. > -- > > Key: HBASE-15965 > URL: https://issues.apache.org/jira/browse/HBASE-15965 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15965.master.001.patch, > HBASE-15965.master.002.patch, HBASE-15965.master.003.patch > > > Testing by executing a command will cover the exact path users will trigger, > so its better then directly calling library functions in tests. Changing the > tests to use @shell.command(:, args) to execute them like it's a > command coming from shell. > Norm change: > Commands should print the output user would like to see, but in the end, > should also return the relevant value. This way: > - Tests can use returned value to check that functionality works > - Tests can capture stdout to assert particular kind of output user should > see. > - We do not print the return value in interactive mode and keep the output > clean. See Shell.command() function. > Bugs found due to this change: > - Uncovered bug in major_compact.rb with this approach. It was calling > admin.majorCompact() which doesn't exist but our tests didn't catch it since > they directly tested admin.major_compact() > - Enabled TestReplicationShell. If it's bad, flaky infra will take care of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15952) Bulk load data replication is not working when RS user does not have permission on hfile-refs node
[ https://issues.apache.org/jira/browse/HBASE-15952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-15952: -- Attachment: HBASE-15952.v1.patch > Bulk load data replication is not working when RS user does not have > permission on hfile-refs node > -- > > Key: HBASE-15952 > URL: https://issues.apache.org/jira/browse/HBASE-15952 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.3.0 >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15952.patch, HBASE-15952.v1.patch > > > In our recent testing in secure cluster we found that when a RS user does not > have permission on hfile-refs znode then RS was failing to replicate the bulk > loaded data as the hfile-refs znode is created by hbase client and RS user > may not have permission to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15971) Regression: Random Read/WorkloadC slower in 1.x than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317882#comment-15317882 ] stack commented on HBASE-15971: --- I just checked. It is there in branch-1.0. Yes, it is for all blocks in cache. > Regression: Random Read/WorkloadC slower in 1.x than 0.98 > - > > Key: HBASE-15971 > URL: https://issues.apache.org/jira/browse/HBASE-15971 > Project: HBase > Issue Type: Sub-task > Components: rpc >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 098.hits.png, 098.png, branch-1.hits.png, branch-1.png > > > branch-1 is slower than 0.98 doing YCSB random read/workloadC. It seems to be > doing about 1/2 the throughput of 0.98. > In branch-1, we have low handler occupancy compared to 0.98. Hacking in > reader thread occupancy metric, is about the same in both. In parent issue, > hacking out the scheduler, I am able to get branch-1 to go 3x faster so will > dig in here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15965) Shell test changes. Use @shell.command instead directly calling functions in admin.rb and other libraries.
[ https://issues.apache.org/jira/browse/HBASE-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15965: - Resolution: Fixed Fix Version/s: 1.4.0 2.0.0 Status: Resolved (was: Patch Available) > Shell test changes. Use @shell.command instead directly calling functions in > admin.rb and other libraries. > -- > > Key: HBASE-15965 > URL: https://issues.apache.org/jira/browse/HBASE-15965 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-15965.master.001.patch, > HBASE-15965.master.002.patch, HBASE-15965.master.003.patch > > > Testing by executing a command will cover the exact path users will trigger, > so its better then directly calling library functions in tests. Changing the > tests to use @shell.command(:, args) to execute them like it's a > command coming from shell. > Norm change: > Commands should print the output user would like to see, but in the end, > should also return the relevant value. This way: > - Tests can use returned value to check that functionality works > - Tests can capture stdout to assert particular kind of output user should > see. > - We do not print the return value in interactive mode and keep the output > clean. See Shell.command() function. > Bugs found due to this change: > - Uncovered bug in major_compact.rb with this approach. It was calling > admin.majorCompact() which doesn't exist but our tests didn't catch it since > they directly tested admin.major_compact() > - Enabled TestReplicationShell. If it's bad, flaky infra will take care of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15943) Add page displaying JVM process metrics
[ https://issues.apache.org/jira/browse/HBASE-15943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samir Ahmic updated HBASE-15943: Status: Patch Available (was: Open) > Add page displaying JVM process metrics > --- > > Key: HBASE-15943 > URL: https://issues.apache.org/jira/browse/HBASE-15943 > Project: HBase > Issue Type: New Feature > Components: UI >Affects Versions: 2.0.0 >Reporter: Samir Ahmic >Assignee: Samir Ahmic >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15943_v0.patch, HBASE-15943_v1.patch, > processInfo.png > > > It would be useful to have page displaying some JVM metrics like PID, process > owner. threads info, GC info, etc. This ticked will create two jsp pages (for > master and rs) displaying stats listed above. > - -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15975) logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
[
https://issues.apache.org/jira/browse/HBASE-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317855#comment-15317855
]
Matteo Bertozzi commented on HBASE-15975:
-
I had an hard time to read this test with the use of "expected" without having
it reset every step.
can you change the test to avoid the use of "expected" and do something like a
bit more readable like:
{code}
// Try double add of same coprocessor
try {
htd.addCoprocessorWithSpec(spec);
fatal(); // the coproc already exist so we should fail with IOException
} catch (IOException ioe) {
// We expect Coprocessor com.foo.FooRegionObserver already exists.
}
{code}
> logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
>
>
> Key: HBASE-15975
> URL: https://issues.apache.org/jira/browse/HBASE-15975
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: master
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Trivial
> Attachments: HBASE-15975-v001.patch
>
>
> While working on an unitest case for HBASE-14644, crossed over
> testAddCoprocessorWithSpecStr().
> {code}
>HTableDescriptor htd = new HTableDescriptor(TableName.META_TABLE_NAME);
> String cpName = "a.b.c.d";
> boolean expected = false;
> try {
> htd.addCoprocessorWithSpec(cpName);
> } catch (IllegalArgumentException iae) {
> expected = true;
> }
> if (!expected) fail();
> // Try minimal spec.
> try {
> htd.addCoprocessorWithSpec("file:///some/path" + "|" + cpName);
> } catch (IllegalArgumentException iae) {
> expected = false;
> }
> if (expected) fail();
> // Try more spec.
> String spec =
> "hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2";
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IllegalArgumentException iae) {
> expected = false; It should be true as it is expected to succeed.
> }
> if (expected) fail();
> // Try double add of same coprocessor
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IOException ioe) {
> expected = true;
> }
> if (!expected) fail();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15970) Move Replication Peers into an HBase table too
[ https://issues.apache.org/jira/browse/HBASE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317834#comment-15317834 ] Joseph commented on HBASE-15970: I'm not super sure yet, but I feel like we would probably use another table. This one would only be tracking the replication peers, so we would only be storing cluster-id's as rowkeys? hbase:replication currently tracks queues. > Move Replication Peers into an HBase table too > -- > > Key: HBASE-15970 > URL: https://issues.apache.org/jira/browse/HBASE-15970 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > > Currently ReplicationQueuesHBaseTableImpl relies on ReplicationStateZkImpl to > track information about the available replication peers (used during > claimQueues). We can also move this into an HBase table instead of relying on > ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15948) Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers"
[ https://issues.apache.org/jira/browse/HBASE-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15948: -- Attachment: HBASE-15948.master.004.patch Ok. 005 doesn't work. We are skipping handling of a Connection. Will go w/ 004 which makes us same as hadoop rpc and smoothes a blast of new Connections by doing a bit of batch processing as they come in. > Port "HADOOP-9956 RPC listener inefficiently assigns connections to readers" > - > > Key: HBASE-15948 > URL: https://issues.apache.org/jira/browse/HBASE-15948 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: HBASE-15948.master.001.patch, > HBASE-15948.master.002.patch, HBASE-15948.master.003.patch, > HBASE-15948.master.004.patch, HBASE-15948.master.004.patch, > HBASE-15948.master.005.patch, Screen Shot 2016-06-02 at 6.16.39 PM.png > > > Esteban noticed we were missing this upstream issue. Seems to make no > difference in profiling but here is the patch anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15965) Shell test changes. Use @shell.command instead directly calling functions in admin.rb and other libraries.
[ https://issues.apache.org/jira/browse/HBASE-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317799#comment-15317799 ] Hudson commented on HBASE-15965: FAILURE: Integrated in HBase-Trunk_matrix #1001 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1001/]) HBASE-15965 - Testing by executing a command will cover the exact path (appy: rev 15c03fd1c97c271aca6dc30feab35ec0c9f8edbe) * hbase-shell/src/main/ruby/shell/commands/show_peer_tableCFs.rb * hbase-shell/src/main/ruby/shell/commands/major_compact.rb * hbase-shell/src/main/ruby/shell/commands/list_namespace_tables.rb * hbase-shell/src/test/ruby/hbase/visibility_labels_admin_test.rb * hbase-shell/src/main/ruby/shell/commands/get_peer_config.rb * hbase-shell/src/main/ruby/hbase/table.rb * hbase-shell/src/main/ruby/shell/commands/truncate.rb * hbase-shell/src/main/ruby/shell/commands/balance_rsgroup.rb * hbase-shell/src/main/ruby/shell.rb * hbase-shell/src/main/ruby/shell/commands/locate_region.rb * hbase-shell/src/main/ruby/shell/commands/create.rb * hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestReplicationShell.java * hbase-shell/src/main/ruby/shell/commands/list_peers.rb * hbase-shell/src/main/ruby/shell/commands/list_peer_configs.rb * hbase-shell/src/main/ruby/shell/commands/is_enabled.rb * hbase-shell/src/test/ruby/test_helper.rb * hbase-shell/src/main/ruby/shell/commands.rb * hbase-shell/src/test/ruby/hbase/replication_admin_test.rb * hbase-shell/src/main/ruby/shell/commands/truncate_preserve.rb * hbase-shell/src/main/ruby/shell/commands/exists.rb * hbase-shell/src/main/ruby/shell/commands/get_auths.rb * hbase-shell/src/main/ruby/hbase/admin.rb * hbase-shell/src/test/ruby/hbase/admin_test.rb * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerConfig.java > Shell test changes. Use @shell.command instead directly calling functions in > admin.rb and other libraries. > -- > > Key: HBASE-15965 > URL: https://issues.apache.org/jira/browse/HBASE-15965 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-15965.master.001.patch, > HBASE-15965.master.002.patch, HBASE-15965.master.003.patch > > > Testing by executing a command will cover the exact path users will trigger, > so its better then directly calling library functions in tests. Changing the > tests to use @shell.command(:, args) to execute them like it's a > command coming from shell. > Norm change: > Commands should print the output user would like to see, but in the end, > should also return the relevant value. This way: > - Tests can use returned value to check that functionality works > - Tests can capture stdout to assert particular kind of output user should > see. > - We do not print the return value in interactive mode and keep the output > clean. See Shell.command() function. > Bugs found due to this change: > - Uncovered bug in major_compact.rb with this approach. It was calling > admin.majorCompact() which doesn't exist but our tests didn't catch it since > they directly tested admin.major_compact() > - Enabled TestReplicationShell. If it's bad, flaky infra will take care of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15954) REST server should log requests with TRACE instead of DEBUG
[ https://issues.apache.org/jira/browse/HBASE-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317798#comment-15317798 ] Hudson commented on HBASE-15954: FAILURE: Integrated in HBase-Trunk_matrix #1001 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1001/]) HBASE-15954 REST server should log requests with TRACE instead of DEBUG (enis: rev 3d7840a173aab97fb72409fa8c0f161fd7ad0e8f) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ProtobufStreamingUtil.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/VersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterVersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/ConnectionCache.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RegionsResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RootResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServlet.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/filter/RestCsrfPreventionFilter.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/filter/AuthFilter.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/provider/consumer/ProtobufMessageBodyConsumer.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/Client.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerInstanceResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesInstanceResource.java > REST server should log requests with TRACE instead of DEBUG > --- > > Key: HBASE-15954 > URL: https://issues.apache.org/jira/browse/HBASE-15954 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: hbase-15954_v1.patch, hbase-15954_v2.patch > > > One of the surprising findings of one of the HBaseCon presentations from > [~nitinverma], [~pravinmittal], [~maxluk] was that REST server is logging > every request in DEBUG which is enabled by default. This obviously causes > out-of-the-box experience to be pretty bad in terms of throughput unless > DEBUG logging is turned off. > We should bring REST server to be on par with the RS level log conventions. > Individual requests to be only logged at the TRACE level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14644) Region in transition metric is broken
[
https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317783#comment-15317783
]
Hadoop QA commented on HBASE-14644:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 27s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 84m 56s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 125m 46s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster |
| Timed out junit tests |
org.apache.hadoop.hbase.namespace.TestNamespaceAuditor |
| | org.apache.hadoop.hbase.TestMultiVersions |
| | org.apache.hadoop.hbase.mapred.TestTableMapReduce |
| | org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12808528/HBASE-14644-v001.patch
|
| JIRA Issue | HBASE-14644 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/worksp
[jira] [Commented] (HBASE-15967) Metric for active ipc Readers and make default fraction of cpu count
[ https://issues.apache.org/jira/browse/HBASE-15967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1531#comment-1531 ] stack commented on HBASE-15967: --- Which one you want [~bbeaudreault]? We don't seem to have the meter in our armory (see DynamicMetricsRegistry) to do the percentage that you point to in the 0.8 abve. We have a Rate as in 0.9 pointer above. What did you fellows do? I like the idea of percentage occupancy so you can tell if you need to up Readers/Handlers. Thanks. > Metric for active ipc Readers and make default fraction of cpu count > > > Key: HBASE-15967 > URL: https://issues.apache.org/jira/browse/HBASE-15967 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Attachments: HBASE-15967.master.001.patch > > > Our ipc Readers are hard coded at 10 regardless since . Running w/ less > Readers, we go faster..(e.g. 12 Readers has us doing 135k with workloadc and > 6 readers has us doing 145k).. .but hard to tell what count of Readers are > needed since no metric. > This issue changes Readers to be 1/4 the installed CPUs or 8, whichever is > the minimum, and then adds a new hbase.regionserver.ipc.runningReaders metric > so you have a chance seeing whats needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15954) REST server should log requests with TRACE instead of DEBUG
[ https://issues.apache.org/jira/browse/HBASE-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317747#comment-15317747 ] Hudson commented on HBASE-15954: SUCCESS: Integrated in HBase-1.2 #642 (See [https://builds.apache.org/job/HBase-1.2/642/]) HBASE-15954 REST server should log requests with TRACE instead of DEBUG (enis: rev 70593efa2760a4c0f5df353047200e5ed14c1035) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RootResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/Client.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerInstanceResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RowResource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/ConnectionCache.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/filter/AuthFilter.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/provider/consumer/ProtobufMessageBodyConsumer.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServlet.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/VersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesInstanceResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ProtobufStreamingUtil.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterVersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RegionsResource.java > REST server should log requests with TRACE instead of DEBUG > --- > > Key: HBASE-15954 > URL: https://issues.apache.org/jira/browse/HBASE-15954 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: hbase-15954_v1.patch, hbase-15954_v2.patch > > > One of the surprising findings of one of the HBaseCon presentations from > [~nitinverma], [~pravinmittal], [~maxluk] was that REST server is logging > every request in DEBUG which is enabled by default. This obviously causes > out-of-the-box experience to be pretty bad in terms of throughput unless > DEBUG logging is turned off. > We should bring REST server to be on par with the RS level log conventions. > Individual requests to be only logged at the TRACE level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15967) Metric for active ipc Readers and make default fraction of cpu count
[ https://issues.apache.org/jira/browse/HBASE-15967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317742#comment-15317742 ] stack commented on HBASE-15967: --- Say more [~ikeda] ... I think we need the workers behind the queue (the 'handler' threads) because we want to do some scheduling... If priority rpc, it should happen above 'normal' priority ipc. If we only have a pool of readers, and they do the parse of the request and then do the handling instead of handing it off, then we don't have opportunity to 'schedule' (we will run much faster though!) See related HBASE-15971. Thanks. > Metric for active ipc Readers and make default fraction of cpu count > > > Key: HBASE-15967 > URL: https://issues.apache.org/jira/browse/HBASE-15967 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Attachments: HBASE-15967.master.001.patch > > > Our ipc Readers are hard coded at 10 regardless since . Running w/ less > Readers, we go faster..(e.g. 12 Readers has us doing 135k with workloadc and > 6 readers has us doing 145k).. .but hard to tell what count of Readers are > needed since no metric. > This issue changes Readers to be 1/4 the installed CPUs or 8, whichever is > the minimum, and then adds a new hbase.regionserver.ipc.runningReaders metric > so you have a chance seeing whats needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15976) RegionServerMetricsWrapperRunnable will be failure when disable blockcache.
Liu Junhong created HBASE-15976: --- Summary: RegionServerMetricsWrapperRunnable will be failure when disable blockcache. Key: HBASE-15976 URL: https://issues.apache.org/jira/browse/HBASE-15976 Project: HBase Issue Type: Bug Affects Versions: 0.98.15 Reporter: Liu Junhong When i disable blockcache, the code "cacheStats = blockCache.getStats();" will occur NPE in org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.RegionServerMetricsWrapperRunnable. It lead to many regionserver's metrics' value always equal 0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-5291) Add Kerberos HTTP SPNEGO authentication support to HBase web consoles
[
https://issues.apache.org/jira/browse/HBASE-5291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-5291:
--
Attachment: HBASE-5291.002.patch
.002 Serious patch for master. Includes feature (using existing hadoop-auth
wiring in HttpServer.Builder), unit tests, and documentation updates.
> Add Kerberos HTTP SPNEGO authentication support to HBase web consoles
> -
>
> Key: HBASE-5291
> URL: https://issues.apache.org/jira/browse/HBASE-5291
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver, security
>Reporter: Andrew Purtell
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-5291.001.patch, HBASE-5291.002.patch
>
>
> Like HADOOP-7119, the same motivations:
> {quote}
> Hadoop RPC already supports Kerberos authentication.
> {quote}
> As does the HBase secure RPC engine.
> {quote}
> Kerberos enables single sign-on.
> Popular browsers (Firefox and Internet Explorer) have support for Kerberos
> HTTP SPNEGO.
> Adding support for Kerberos HTTP SPNEGO to [HBase] web consoles would provide
> a unified authentication mechanism and single sign-on for web UI and RPC.
> {quote}
> Also like HADOOP-7119, the same solution:
> A servlet filter is configured in front of all Hadoop web consoles for
> authentication.
> This filter verifies if the incoming request is already authenticated by the
> presence of a signed HTTP cookie. If the cookie is present, its signature is
> valid and its value didn't expire; then the request continues its way to the
> page invoked by the request. If the cookie is not present, it is invalid or
> it expired; then the request is delegated to an authenticator handler. The
> authenticator handler then is responsible for requesting/validating the
> user-agent for the user credentials. This may require one or more additional
> interactions between the authenticator handler and the user-agent (which will
> be multiple HTTP requests). Once the authenticator handler verifies the
> credentials and generates an authentication token, a signed cookie is
> returned to the user-agent for all subsequent invocations.
> The authenticator handler is pluggable and 2 implementations are provided out
> of the box: pseudo/simple and kerberos.
> 1. The pseudo/simple authenticator handler is equivalent to the Hadoop
> pseudo/simple authentication. It trusts the value of the user.name query
> string parameter. The pseudo/simple authenticator handler supports an
> anonymous mode which accepts any request without requiring the user.name
> query string parameter to create the token. This is the default behavior,
> preserving the behavior of the HBase web consoles before this patch.
> 2. The kerberos authenticator handler implements the Kerberos HTTP SPNEGO
> implementation. This authenticator handler will generate a token only if a
> successful Kerberos HTTP SPNEGO interaction is performed between the
> user-agent and the authenticator. Browsers like Firefox and Internet Explorer
> support Kerberos HTTP SPNEGO.
> We can build on the support added to Hadoop via HADOOP-7119. Should just be a
> matter of wiring up the filter to our infoservers in a similar manner.
> And from
> https://issues.apache.org/jira/browse/HBASE-5050?focusedCommentId=13171086&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13171086
> {quote}
> Hadoop 0.23 onwards has a hadoop-auth artifact that provides SPNEGO/Kerberos
> authentication for webapps via a filter. You should consider using it. You
> don't have to move Hbase to 0.23 for that, just consume the hadoop-auth
> artifact, which has no dependencies on the rest of Hadoop 0.23 artifacts.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15954) REST server should log requests with TRACE instead of DEBUG
[ https://issues.apache.org/jira/browse/HBASE-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317706#comment-15317706 ] Hudson commented on HBASE-15954: FAILURE: Integrated in HBase-1.3 #725 (See [https://builds.apache.org/job/HBase-1.3/725/]) HBASE-15954 REST server should log requests with TRACE instead of DEBUG (enis: rev 466eb31648a4783c79d9b044fdd84d0db25c3d12) * hbase-server/src/main/java/org/apache/hadoop/hbase/util/ConnectionCache.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/provider/consumer/ProtobufMessageBodyConsumer.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterVersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/filter/AuthFilter.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesInstanceResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RootResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServlet.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/VersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ProtobufStreamingUtil.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerInstanceResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/filter/RestCsrfPreventionFilter.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/Client.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RegionsResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java > REST server should log requests with TRACE instead of DEBUG > --- > > Key: HBASE-15954 > URL: https://issues.apache.org/jira/browse/HBASE-15954 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: hbase-15954_v1.patch, hbase-15954_v2.patch > > > One of the surprising findings of one of the HBaseCon presentations from > [~nitinverma], [~pravinmittal], [~maxluk] was that REST server is logging > every request in DEBUG which is enabled by default. This obviously causes > out-of-the-box experience to be pretty bad in terms of throughput unless > DEBUG logging is turned off. > We should bring REST server to be on par with the RS level log conventions. > Individual requests to be only logged at the TRACE level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15968) Strong semantics of versions
[
https://issues.apache.org/jira/browse/HBASE-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phil Yang updated HBASE-15968:
--
Description:
In HBase book, we have a section in Versions called "Current Limitations" see
http://hbase.apache.org/book.html#_current_limitations
{quote}
28.3. Current Limitations
28.3.1. Deletes mask Puts
Deletes mask puts, even puts that happened after the delete was entered. See
HBASE-2256. Remember that a delete writes a tombstone, which only disappears
after then next major compaction has run. Suppose you do a delete of everything
⇐ T. After this you do a new put with a timestamp ⇐ T. This put, even if it
happened after the delete, will be masked by the delete tombstone. Performing
the put will not fail, but when you do a get you will notice the put did have
no effect. It will start working again after the major compaction has run.
These issues should not be a problem if you use always-increasing versions for
new puts to a row. But they can occur even if you do not care about time: just
do delete and put immediately after each other, and there is some chance they
happen within the same millisecond.
28.3.2. Major compactions change query results
…create three cell versions at t1, t2 and t3, with a maximum-versions setting
of 2. So when getting all versions, only the values at t2 and t3 will be
returned. But if you delete the version at t2 or t3, the one at t1 will appear
again. Obviously, once a major compaction has run, such behavior will not be
the case anymore… (See Garbage Collection in Bending time in HBase.)
{quote}
These limitations result from the current implementation on multi-versions: we
only consider timestamp, no matter when it comes; we will not remove old
version immediately if there are enough number of new versions.
So we can get a stronger semantics of versions by two guarantees:
1, Delete will not mask Put that comes after it.
2, If a version is masked by enough number of higher versions (VERSIONS in cf's
conf), it will never be seen any more.
Some examples for understanding:
(delete t<=3 means use Delete.addColumns to delete all versions whose ts is not
greater than 3, and delete t3 means use Delete.addColumn to delete the version
whose ts=3)
case 1: put t2 -> put t3 -> delete t<=3 -> put t1, and we will get t1 because
the put is after delete.
case 2: maxversion=2, put t1 -> put t2 -> put t3 -> delete t3, and we will
always get t2 no matter if there is a major compaction, because t1 is masked
when we put t3 so t1 will never be seen.
case 3: maxversion=2, put t1 -> put t2 -> put t3 -> delete t2 -> delete t3, and
we will get nothing.
case 4: maxversion=3, put t1 -> put t2 -> put t3 -> delete t2 -> delete t3, and
we will get t1 because it is not masked.
case 5: maxversion=2, put t1 -> put t2 -> put t3 -> delete t3 -> put t1, and we
can get t3+t1 because when we put t1 at second time it is the 2nd latest
version and it can be read.
case 6:maxversion=2, put t3->put t2->put t1, and we will get t3+t2 just like
what we can get now, ts is still the key of versions.
Different VERSIONS may result in different results even the size of result is
smaller than VERSIONS(see case 3 and 4). So Get/Scan.setMaxVersions will be
handled at end after we read correct data according to CF's VERSIONS setting.
The semantics is different from the current HBase, and we may need more logic
to support the new semantic, so it is configurable and default is disabled.
was:
In HBase book, we have a section in Versions called "Current Limitations" see
http://hbase.apache.org/book.html#_current_limitations
{quote}
28.3. Current Limitations
28.3.1. Deletes mask Puts
Deletes mask puts, even puts that happened after the delete was entered. See
HBASE-2256. Remember that a delete writes a tombstone, which only disappears
after then next major compaction has run. Suppose you do a delete of everything
⇐ T. After this you do a new put with a timestamp ⇐ T. This put, even if it
happened after the delete, will be masked by the delete tombstone. Performing
the put will not fail, but when you do a get you will notice the put did have
no effect. It will start working again after the major compaction has run.
These issues should not be a problem if you use always-increasing versions for
new puts to a row. But they can occur even if you do not care about time: just
do delete and put immediately after each other, and there is some chance they
happen within the same millisecond.
28.3.2. Major compactions change query results
…create three cell versions at t1, t2 and t3, with a maximum-versions setting
of 2. So when getting all versions, only the values at t2 and t3 will be
returned. But if you delete the version at t2 or t3, the one at t1 will appear
again. Obviously, once a major compaction has run, such behavior will not be
the case anymore… (See Garbage Collection in Ben
[jira] [Commented] (HBASE-15967) Metric for active ipc Readers and make default fraction of cpu count
[ https://issues.apache.org/jira/browse/HBASE-15967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317689#comment-15317689 ] Hiroshi Ikeda commented on HBASE-15967: --- If we apply the leader/followers pattern, remove the internal worker threads behind the queue, and make the reader threads work instead, then we collect the most of active threads into one pool and that might make it more clear to configure. > Metric for active ipc Readers and make default fraction of cpu count > > > Key: HBASE-15967 > URL: https://issues.apache.org/jira/browse/HBASE-15967 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Attachments: HBASE-15967.master.001.patch > > > Our ipc Readers are hard coded at 10 regardless since . Running w/ less > Readers, we go faster..(e.g. 12 Readers has us doing 135k with workloadc and > 6 readers has us doing 145k).. .but hard to tell what count of Readers are > needed since no metric. > This issue changes Readers to be 1/4 the installed CPUs or 8, whichever is > the minimum, and then adds a new hbase.regionserver.ipc.runningReaders metric > so you have a chance seeing whats needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15954) REST server should log requests with TRACE instead of DEBUG
[ https://issues.apache.org/jira/browse/HBASE-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317666#comment-15317666 ] Hudson commented on HBASE-15954: FAILURE: Integrated in HBase-1.4 #197 (See [https://builds.apache.org/job/HBase-1.4/197/]) HBASE-15954 REST server should log requests with TRACE instead of DEBUG (enis: rev 4a0a9a20dd5bbfdafe2ec95196b449d2e1a45a13) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/filter/AuthFilter.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServlet.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerInstanceResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/filter/RestCsrfPreventionFilter.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/VersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterVersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/ConnectionCache.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesInstanceResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ProtobufStreamingUtil.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RegionsResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/provider/consumer/ProtobufMessageBodyConsumer.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/Client.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RootResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RowResource.java > REST server should log requests with TRACE instead of DEBUG > --- > > Key: HBASE-15954 > URL: https://issues.apache.org/jira/browse/HBASE-15954 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: hbase-15954_v1.patch, hbase-15954_v2.patch > > > One of the surprising findings of one of the HBaseCon presentations from > [~nitinverma], [~pravinmittal], [~maxluk] was that REST server is logging > every request in DEBUG which is enabled by default. This obviously causes > out-of-the-box experience to be pretty bad in terms of throughput unless > DEBUG logging is turned off. > We should bring REST server to be on par with the RS level log conventions. > Individual requests to be only logged at the TRACE level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15958) Implement ClaimQueues on top of HBase
[
https://issues.apache.org/jira/browse/HBASE-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317660#comment-15317660
]
Hadoop QA commented on HBASE-15958:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m 50s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 10s
{color} | {color:green} hbase-client-jdk1.8.0 with JDK v1.8.0 generated 0 new +
4 unchanged - 2 fixed = 4 total (was 6) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s
{color} | {color:green} hbase-client in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} hbase-server in the patch passed with JDK v1.8.0.
{color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 0s
{color} | {color:green} hbase-client-jdk1.7.0_79 with JDK v1.7.0_79 generated 0
new + 4 unchanged - 2 fixed = 4 total (was 6) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} hbase-client in the patch passed with JDK v1.7.0_79.
{color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} hbase-server in the patch passed with JDK v1.7.0_79.
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 9s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 16s
{color} | {color:red} hbase-client generated 3 new + 0 unchanged - 0 fixed = 3
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 56s
{color} | {color:green} hbase-client in the patch pass
[jira] [Commented] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317659#comment-15317659 ] huaxiang sun commented on HBASE-14644: -- One comment in test case is wrong, it should be 3 seconds instead of 6 seconds, will correct later. > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Attachments: HBASE-14644-v001.patch, branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-14644: - Status: Patch Available (was: Open) > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Attachments: HBASE-14644-v001.patch, branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-14644: - Attachment: HBASE-14644-v001.patch Patch with unitest on master branch, for branch-1, the unittest will be different. > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Attachments: HBASE-14644-v001.patch, branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15975) logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
[
https://issues.apache.org/jira/browse/HBASE-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317633#comment-15317633
]
Hadoop QA commented on HBASE-15975:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 25s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 29s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 0s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 39s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12808517/HBASE-15975-v001.patch
|
| JIRA Issue | HBASE-15975 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 15c03fd |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/2128/testReport/ |
| modules
[jira] [Updated] (HBASE-14415) Full backup based on Snapshot v2
[ https://issues.apache.org/jira/browse/HBASE-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14415: -- Description: Full backup is based on table snapshot. Current implementation of table snapshots is not robust at scale and snapshot verification stage may be very slow for tables with large number of regions. If region gets split during snapshot - verification and snapshot will fail. > Full backup based on Snapshot v2 > > > Key: HBASE-14415 > URL: https://issues.apache.org/jira/browse/HBASE-14415 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > > Full backup is based on table snapshot. Current implementation of table > snapshots is not robust at scale and snapshot verification stage may be very > slow for tables with large number of regions. If region gets split during > snapshot - verification and snapshot will fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14142) HBase Backup/Restore Phase 3: Edits deduplication during backup
[ https://issues.apache.org/jira/browse/HBASE-14142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14142: -- Summary: HBase Backup/Restore Phase 3: Edits deduplication during backup (was: HBase Backup/Restore Phase 3: Cells deduplication during backup) > HBase Backup/Restore Phase 3: Edits deduplication during backup > --- > > Key: HBASE-14142 > URL: https://issues.apache.org/jira/browse/HBASE-14142 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > > As since we do not record last backed up sequence ids (MVCC) and do not > restore up to that sequence id - that is kind of tricky, there will be some > duplicates of KVs in store files after first incremental restore after full > backup. These duplicates are result of how we do full backup and first > incremental backup after full one. During full backup we perform distributed > log roll and record, for every RS, last WAL timestamp, then we do snapshot. > The next WAL after recorded one will make it into a next incremental backup > set, but it will contains some edits (puts, deletes) which have been recorded > by a previous snapshot. During restore, we, first, restore snapshot, then we > will re-play WALs and this operation can create some duplicates of KVs in > different store files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14142) HBase Backup/Restore Phase 3: Cells deduplication during backup
[ https://issues.apache.org/jira/browse/HBASE-14142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317621#comment-15317621 ] Vladimir Rodionov commented on HBASE-14142: --- Dangerous for operations which are not idempotent (Increment, Append) > HBase Backup/Restore Phase 3: Cells deduplication during backup > --- > > Key: HBASE-14142 > URL: https://issues.apache.org/jira/browse/HBASE-14142 > Project: HBase > Issue Type: New Feature >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > > As since we do not record last backed up sequence ids (MVCC) and do not > restore up to that sequence id - that is kind of tricky, there will be some > duplicates of KVs in store files after first incremental restore after full > backup. These duplicates are result of how we do full backup and first > incremental backup after full one. During full backup we perform distributed > log roll and record, for every RS, last WAL timestamp, then we do snapshot. > The next WAL after recorded one will make it into a next incremental backup > set, but it will contains some edits (puts, deletes) which have been recorded > by a previous snapshot. During restore, we, first, restore snapshot, then we > will re-play WALs and this operation can create some duplicates of KVs in > different store files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14413) Procedure V2 - Snapshot V2
[ https://issues.apache.org/jira/browse/HBASE-14413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-14413: --- Summary: Procedure V2 - Snapshot V2 (was: HBase snapshots v2) > Procedure V2 - Snapshot V2 > -- > > Key: HBASE-14413 > URL: https://issues.apache.org/jira/browse/HBASE-14413 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > > We need new implementation of snapshot feature that is more robust and > performant. Ideally, it will work with multiple tables as well. The possible > areas of improvements: > # Must be flushless. Coordinated memstore flushes across a cluster are bad. > # Verification phase must be distributed, done in parallel and not on Master. > In theory, the only info we need to record snapshot of a table: list of WAL > files, list of HFiles and max sequence id of an edit which has been flushed > per Region. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15957) RpcClientImpl.close never ends in some circumstances
[
https://issues.apache.org/jira/browse/HBASE-15957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317604#comment-15317604
]
Enis Soztutar commented on HBASE-15957:
---
Thanks [~sergey.soldatov]. The patch makes sense since not in all cases after
calling markClosed(), we call close() from that thread. So, if a thread calls
markClosed() without subsequent call to close(), then the write thread does
{{if (markClosed())}} check the thread will just hang.
> RpcClientImpl.close never ends in some circumstances
>
>
> Key: HBASE-15957
> URL: https://issues.apache.org/jira/browse/HBASE-15957
> Project: HBase
> Issue Type: Bug
> Components: Client, rpc
>Affects Versions: 1.1.2
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Attachments: HBASE-15957-2.patch, HBASE-15957.patch
>
>
> This bug is related to HBASE-14241 and HBASE-13851.
> Fix for HBASE-13851 introduced the check for non alive connections and if
> connection is not alive, it close it:
> {noformat}
> if (!conn.isAlive()) {
> if (connsToClose == null) {
> connsToClose = new HashSet();
> }
> connsToClose.add(conn);
> }
>
> if (connsToClose != null) {
> for (Connection conn : connsToClose) {
> if (conn.markClosed(new InterruptedIOException("RpcClient is
> closing"))) {
> conn.close();
> }
> }
> }
> {noformat}
> That worked fine until fix for HBASE-14241 introduced handling for interrupt
> in writer thread:
> {noformat}
> try {
> cts = callsToWrite.take();
> } catch (InterruptedException e) {
> markClosed(new InterruptedIOException());
> }
> {noformat}
> So, if writer thread is running, but connection thread is not started yet,
> interrupt will cause calling of markClosed which will set
> shouldCloseConnection flag for the parent connection. And the next time
> during the handling of non alive connections markClosed will return false and
> close will not be called. As the result connection will not be removed from
> the connections pool and RpcClientImpl.close never finish.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-15330) HBase Backup/Restore Phase 3: support delete/truncate table
[ https://issues.apache.org/jira/browse/HBASE-15330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov resolved HBASE-15330. --- Resolution: Duplicate Release Note: HBASE-15449 > HBase Backup/Restore Phase 3: support delete/truncate table > --- > > Key: HBASE-15330 > URL: https://issues.apache.org/jira/browse/HBASE-15330 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov > > Currently, we do not track delete/recreate or truncate table events -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15803) ZooKeeperWatcher's constructor can leak a ZooKeeper instance with throwing ZooKeeperConnectionException when canCreateBaseZNode is true
[
https://issues.apache.org/jira/browse/HBASE-15803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317595#comment-15317595
]
Hiroshi Ikeda commented on HBASE-15803:
---
+1
Anyway the patch should make it better.
> ZooKeeperWatcher's constructor can leak a ZooKeeper instance with throwing
> ZooKeeperConnectionException when canCreateBaseZNode is true
> ---
>
> Key: HBASE-15803
> URL: https://issues.apache.org/jira/browse/HBASE-15803
> Project: HBase
> Issue Type: Bug
>Reporter: Hiroshi Ikeda
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 15803.v1.txt, 15803.v2.txt
>
>
> {code}
> public ZooKeeperWatcher(Configuration conf, String identifier,
> Abortable abortable, boolean canCreateBaseZNode)
> throws IOException, ZooKeeperConnectionException {
> ...skip...
> this.recoverableZooKeeper = ZKUtil.connect(...
> ...skip...
> if (canCreateBaseZNode) {
> createBaseZNodes();
> }
> }
> private void createBaseZNodes() throws ZooKeeperConnectionException {
> {code}
> The registered watcher doesn't seem to close the Zookeeper instance by watch
> events, and the instance keeps alive when createBaseZNodes is failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15226) HBase Backup/Restore Phase 3: Add FSM to Backup state
[ https://issues.apache.org/jira/browse/HBASE-15226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-15226: -- Description: Finate State Machine. (was: Final State Machine. ) > HBase Backup/Restore Phase 3: Add FSM to Backup state > - > > Key: HBASE-15226 > URL: https://issues.apache.org/jira/browse/HBASE-15226 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > > Finate State Machine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15975) logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
[
https://issues.apache.org/jira/browse/HBASE-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huaxiang sun updated HBASE-15975:
-
Attachment: HBASE-15975-v001.patch
Minor fix.
> logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
>
>
> Key: HBASE-15975
> URL: https://issues.apache.org/jira/browse/HBASE-15975
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: master
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Trivial
> Attachments: HBASE-15975-v001.patch
>
>
> While working on an unitest case for HBASE-14644, crossed over
> testAddCoprocessorWithSpecStr().
> {code}
>HTableDescriptor htd = new HTableDescriptor(TableName.META_TABLE_NAME);
> String cpName = "a.b.c.d";
> boolean expected = false;
> try {
> htd.addCoprocessorWithSpec(cpName);
> } catch (IllegalArgumentException iae) {
> expected = true;
> }
> if (!expected) fail();
> // Try minimal spec.
> try {
> htd.addCoprocessorWithSpec("file:///some/path" + "|" + cpName);
> } catch (IllegalArgumentException iae) {
> expected = false;
> }
> if (expected) fail();
> // Try more spec.
> String spec =
> "hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2";
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IllegalArgumentException iae) {
> expected = false; It should be true as it is expected to succeed.
> }
> if (expected) fail();
> // Try double add of same coprocessor
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IOException ioe) {
> expected = true;
> }
> if (!expected) fail();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15975) logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
[
https://issues.apache.org/jira/browse/HBASE-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huaxiang sun updated HBASE-15975:
-
Status: Patch Available (was: Open)
> logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
>
>
> Key: HBASE-15975
> URL: https://issues.apache.org/jira/browse/HBASE-15975
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: master
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Trivial
> Attachments: HBASE-15975-v001.patch
>
>
> While working on an unitest case for HBASE-14644, crossed over
> testAddCoprocessorWithSpecStr().
> {code}
>HTableDescriptor htd = new HTableDescriptor(TableName.META_TABLE_NAME);
> String cpName = "a.b.c.d";
> boolean expected = false;
> try {
> htd.addCoprocessorWithSpec(cpName);
> } catch (IllegalArgumentException iae) {
> expected = true;
> }
> if (!expected) fail();
> // Try minimal spec.
> try {
> htd.addCoprocessorWithSpec("file:///some/path" + "|" + cpName);
> } catch (IllegalArgumentException iae) {
> expected = false;
> }
> if (expected) fail();
> // Try more spec.
> String spec =
> "hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2";
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IllegalArgumentException iae) {
> expected = false; It should be true as it is expected to succeed.
> }
> if (expected) fail();
> // Try double add of same coprocessor
> try {
> htd.addCoprocessorWithSpec(spec);
> } catch (IOException ioe) {
> expected = true;
> }
> if (!expected) fail();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15975) logic in TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
huaxiang sun created HBASE-15975:
Summary: logic in
TestHTableDescriptor#testAddCoprocessorWithSpecStr is wrong
Key: HBASE-15975
URL: https://issues.apache.org/jira/browse/HBASE-15975
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: master
Reporter: huaxiang sun
Assignee: huaxiang sun
Priority: Trivial
While working on an unitest case for HBASE-14644, crossed over
testAddCoprocessorWithSpecStr().
{code}
HTableDescriptor htd = new HTableDescriptor(TableName.META_TABLE_NAME);
String cpName = "a.b.c.d";
boolean expected = false;
try {
htd.addCoprocessorWithSpec(cpName);
} catch (IllegalArgumentException iae) {
expected = true;
}
if (!expected) fail();
// Try minimal spec.
try {
htd.addCoprocessorWithSpec("file:///some/path" + "|" + cpName);
} catch (IllegalArgumentException iae) {
expected = false;
}
if (expected) fail();
// Try more spec.
String spec =
"hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2";
try {
htd.addCoprocessorWithSpec(spec);
} catch (IllegalArgumentException iae) {
expected = false; It should be true as it is expected to succeed.
}
if (expected) fail();
// Try double add of same coprocessor
try {
htd.addCoprocessorWithSpec(spec);
} catch (IOException ioe) {
expected = true;
}
if (!expected) fail();
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15935) Have a separate Walker task running concurrently with Generator
[
https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317550#comment-15317550
]
Hadoop QA commented on HBASE-15935:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 0s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
39m 16s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 24s
{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 12s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12808500/HBASE-15935.patch |
| JIRA Issue | HBASE-15935 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux pomona.apache.org 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT
Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3d7840a |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/2127/artifact/patchprocess/white
[jira] [Commented] (HBASE-15965) Shell test changes. Use @shell.command instead directly calling functions in admin.rb and other libraries.
[
https://issues.apache.org/jira/browse/HBASE-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317543#comment-15317543
]
Hadoop QA commented on HBASE-15965:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 1s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 1s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 18s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 59s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m 37s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 12s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12808504/HBASE-15965.master.003.patch
|
| JIRA Issue
[jira] [Created] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
Joseph created HBASE-15974: -- Summary: Create a ReplicationQueuesClientHBaseImpl Key: HBASE-15974 URL: https://issues.apache.org/jira/browse/HBASE-15974 Project: HBase Issue Type: Sub-task Reporter: Joseph Currently ReplicationQueuesClient utilizes a ZooKeeper implementation ReplicationQueuesClientZkImpl that attempts to read from the ZNode where ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase implementation for ReplicationQueuesClient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-15974) Create a ReplicationQueuesClientHBaseImpl
[ https://issues.apache.org/jira/browse/HBASE-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph reassigned HBASE-15974: -- Assignee: Joseph > Create a ReplicationQueuesClientHBaseImpl > - > > Key: HBASE-15974 > URL: https://issues.apache.org/jira/browse/HBASE-15974 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > > Currently ReplicationQueuesClient utilizes a ZooKeeper implementation > ReplicationQueuesClientZkImpl that attempts to read from the ZNode where > ReplicationQueuesZkImpl tracked WAL's. So we need to create a HBase > implementation for ReplicationQueuesClient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15698) Increment TimeRange not serialized to server
[ https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317518#comment-15317518 ] Andrew Purtell commented on HBASE-15698: Great. Thanks [~tedyu]. I will try committing the v7 patch now back to 0.98 with fixups. Back soon. > Increment TimeRange not serialized to server > > > Key: HBASE-15698 > URL: https://issues.apache.org/jira/browse/HBASE-15698 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0 >Reporter: James Taylor >Assignee: Ted Yu >Priority: Blocker > Labels: phoenix > Fix For: 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt, > 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, 15698.v6.txt, > 15698.v7.txt, HBASE-15698.1.patch > > > Before HBase-1.2, the Increment TimeRange set on the client was serialized > over to the server. As of HBase 1.2, this appears to no longer be true, as my > preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value > of increment.getTimeRange().getMax() regardless of what the client has > specified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15954) REST server should log requests with TRACE instead of DEBUG
[ https://issues.apache.org/jira/browse/HBASE-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317506#comment-15317506 ] Hudson commented on HBASE-15954: SUCCESS: Integrated in HBase-1.2-IT #524 (See [https://builds.apache.org/job/HBase-1.2-IT/524/]) HBASE-15954 REST server should log requests with TRACE instead of DEBUG (enis: rev 70593efa2760a4c0f5df353047200e5ed14c1035) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/provider/consumer/ProtobufMessageBodyConsumer.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesInstanceResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterVersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerInstanceResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServlet.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/filter/AuthFilter.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RegionsResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RootResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/Client.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/ConnectionCache.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ProtobufStreamingUtil.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/VersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java > REST server should log requests with TRACE instead of DEBUG > --- > > Key: HBASE-15954 > URL: https://issues.apache.org/jira/browse/HBASE-15954 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: hbase-15954_v1.patch, hbase-15954_v2.patch > > > One of the surprising findings of one of the HBaseCon presentations from > [~nitinverma], [~pravinmittal], [~maxluk] was that REST server is logging > every request in DEBUG which is enabled by default. This obviously causes > out-of-the-box experience to be pretty bad in terms of throughput unless > DEBUG logging is turned off. > We should bring REST server to be on par with the RS level log conventions. > Individual requests to be only logged at the TRACE level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15972) hbase backup set command should not accept non-existing table
[ https://issues.apache.org/jira/browse/HBASE-15972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317505#comment-15317505 ] Vladimir Rodionov commented on HBASE-15972: --- OK. > hbase backup set command should not accept non-existing table > - > > Key: HBASE-15972 > URL: https://issues.apache.org/jira/browse/HBASE-15972 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Labels: backup > Attachments: 15972.v1.txt > > > [~romil.choksi] discovered that 'backup set add' command doesn't check > whether the table exists before adding the table to the named set. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15969) add precommit check for commit message formating
[
https://issues.apache.org/jira/browse/HBASE-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317493#comment-15317493
]
Andrew Purtell commented on HBASE-15969:
The check must admit a second valid form:
{noformat}
HBASE- Single line summary (Contributor attribution)
Optional extended message
{noformat}
No signed-off line. This is what gets committed when a plain patch, not
something produced by 'git format-patch' is submitted.
> add precommit check for commit message formating
>
>
> Key: HBASE-15969
> URL: https://issues.apache.org/jira/browse/HBASE-15969
> Project: HBase
> Issue Type: Improvement
> Components: build
>Reporter: Sean Busbey
>
> add a precommit check that sees if commit messages are of the form:
> {code}
> HBASE-X Single line summary of issues, pref under 76 col
> Optional extended description that might contain pre-existing signed-off by
> entries if e.g. it's a version posted by a reviewer as an update.
> Signed-off-by: Jane Reviewer
> {code}
> Open to other things we can easily check for via regex (please add in
> comments).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15973) PeriodicMemstoreFlusher is causing excessive flushes when back-in-time inserts are happening
[ https://issues.apache.org/jira/browse/HBASE-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317489#comment-15317489 ] Enis Soztutar commented on HBASE-15973: --- Instead of looking at time of oldest edit, we should track time of first insertion to the memstore after initialization, or flush start. > PeriodicMemstoreFlusher is causing excessive flushes when back-in-time > inserts are happening > > > Key: HBASE-15973 > URL: https://issues.apache.org/jira/browse/HBASE-15973 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0 > > > In a production cluster, we have noticed a case where flushes were happening > for 200-400KB in sizes. Turns out the periodic memstore flusher is force > flushing because cells with older timestamps (in this case days old) were > being inserted. > We have periodic memstore flusher with 1 hour defaulted, so in a case where > replication is lagging, or phoenix secondary index rebuild or the user doing > back-in-time inserts with cell timestamps older than 1 hour, we will flush > extremely frequently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15973) PeriodicMemstoreFlusher is causing excessive flushes when back-in-time inserts are happening
Enis Soztutar created HBASE-15973: - Summary: PeriodicMemstoreFlusher is causing excessive flushes when back-in-time inserts are happening Key: HBASE-15973 URL: https://issues.apache.org/jira/browse/HBASE-15973 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Fix For: 2.0.0, 1.3.0, 1.4.0 In a production cluster, we have noticed a case where flushes were happening for 200-400KB in sizes. Turns out the periodic memstore flusher is force flushing because cells with older timestamps (in this case days old) were being inserted. We have periodic memstore flusher with 1 hour defaulted, so in a case where replication is lagging, or phoenix secondary index rebuild or the user doing back-in-time inserts with cell timestamps older than 1 hour, we will flush extremely frequently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15965) Shell test changes. Use @shell.command instead directly calling functions in admin.rb and other libraries.
[ https://issues.apache.org/jira/browse/HBASE-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15965: - Description: Testing by executing a command will cover the exact path users will trigger, so its better then directly calling library functions in tests. Changing the tests to use @shell.command(:, args) to execute them like it's a command coming from shell. Norm change: Commands should print the output user would like to see, but in the end, should also return the relevant value. This way: - Tests can use returned value to check that functionality works - Tests can capture stdout to assert particular kind of output user should see. - We do not print the return value in interactive mode and keep the output clean. See Shell.command() function. Bugs found due to this change: - Uncovered bug in major_compact.rb with this approach. It was calling admin.majorCompact() which doesn't exist but our tests didn't catch it since they directly tested admin.major_compact() - Enabled TestReplicationShell. If it's bad, flaky infra will take care of it. was: - Changes tests to use @shell.command(:, args) instead of directly using functions in admin.rb, etc. - Testing by executing a command will cover the exact path users will trigger, so its better testing compared to just testing library functions. - Uncovered bug in major_compact.rb with this approach. It was calling admin.majorCompact() which doesn't exist but our tests didn't catch it since they directly tested admin.major_compact() > Shell test changes. Use @shell.command instead directly calling functions in > admin.rb and other libraries. > -- > > Key: HBASE-15965 > URL: https://issues.apache.org/jira/browse/HBASE-15965 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-15965.master.001.patch, > HBASE-15965.master.002.patch, HBASE-15965.master.003.patch > > > Testing by executing a command will cover the exact path users will trigger, > so its better then directly calling library functions in tests. Changing the > tests to use @shell.command(:, args) to execute them like it's a > command coming from shell. > Norm change: > Commands should print the output user would like to see, but in the end, > should also return the relevant value. This way: > - Tests can use returned value to check that functionality works > - Tests can capture stdout to assert particular kind of output user should > see. > - We do not print the return value in interactive mode and keep the output > clean. See Shell.command() function. > Bugs found due to this change: > - Uncovered bug in major_compact.rb with this approach. It was calling > admin.majorCompact() which doesn't exist but our tests didn't catch it since > they directly tested admin.major_compact() > - Enabled TestReplicationShell. If it's bad, flaky infra will take care of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-15946:
Affects Version/s: 1.3.0
1.2.1
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Mikhail Antonov
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-v1.patch, HBASE-15946-v2.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-15946:
Fix Version/s: 1.2.2
1.3.0
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Sean Mackrory
>Assignee: Mikhail Antonov
> Fix For: 1.3.0, 1.2.2
>
> Attachments: HBASE-15946-v1.patch, HBASE-15946-v2.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15954) REST server should log requests with TRACE instead of DEBUG
[ https://issues.apache.org/jira/browse/HBASE-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-15954: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to 0.98+. > REST server should log requests with TRACE instead of DEBUG > --- > > Key: HBASE-15954 > URL: https://issues.apache.org/jira/browse/HBASE-15954 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: hbase-15954_v1.patch, hbase-15954_v2.patch > > > One of the surprising findings of one of the HBaseCon presentations from > [~nitinverma], [~pravinmittal], [~maxluk] was that REST server is logging > every request in DEBUG which is enabled by default. This obviously causes > out-of-the-box experience to be pretty bad in terms of throughput unless > DEBUG logging is turned off. > We should bring REST server to be on par with the RS level log conventions. > Individual requests to be only logged at the TRACE level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15965) Shell test changes. Use @shell.command instead directly calling functions in admin.rb and other libraries.
[ https://issues.apache.org/jira/browse/HBASE-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15965: - Attachment: HBASE-15965.master.003.patch > Shell test changes. Use @shell.command instead directly calling functions in > admin.rb and other libraries. > -- > > Key: HBASE-15965 > URL: https://issues.apache.org/jira/browse/HBASE-15965 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-15965.master.001.patch, > HBASE-15965.master.002.patch, HBASE-15965.master.003.patch > > > - Changes tests to use @shell.command(:, args) instead of > directly using functions in admin.rb, etc. > - Testing by executing a command will cover the exact path users will > trigger, so its better testing compared to just testing library functions. > - Uncovered bug in major_compact.rb with this approach. It was calling > admin.majorCompact() which doesn't exist but our tests didn't catch it since > they directly tested admin.major_compact() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15972) hbase backup set command should not accept non-existing table
[ https://issues.apache.org/jira/browse/HBASE-15972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15972: --- Attachment: 15972.v1.txt Patch v1 raises IOException when non-existent table is detected. Comment is welcome. > hbase backup set command should not accept non-existing table > - > > Key: HBASE-15972 > URL: https://issues.apache.org/jira/browse/HBASE-15972 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Labels: backup > Attachments: 15972.v1.txt > > > [~romil.choksi] discovered that 'backup set add' command doesn't check > whether the table exists before adding the table to the named set. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15972) hbase backup set command should not accept non-existing table
Ted Yu created HBASE-15972: -- Summary: hbase backup set command should not accept non-existing table Key: HBASE-15972 URL: https://issues.apache.org/jira/browse/HBASE-15972 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu [~romil.choksi] discovered that 'backup set add' command doesn't check whether the table exists before adding the table to the named set. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15935) Have a separate Walker task running concurrently with Generator
[ https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15935: --- Status: Patch Available (was: Open) > Have a separate Walker task running concurrently with Generator > -- > > Key: HBASE-15935 > URL: https://issues.apache.org/jira/browse/HBASE-15935 > Project: HBase > Issue Type: Sub-task > Components: integration tests >Reporter: Joseph >Assignee: Joseph >Priority: Minor > Attachments: HBASE-15935.patch > > > Keep track of which linked lists have been flushed in an HBase table, so that > we can concurrently Walk these lists during the Generation phase. This will > allow us to test: > 1. HBase under concurrent read/writes > 2. Availability of data immediately after flushes (as opposed to waiting till > the Verification phase) > The review diff can be found at: > https://reviews.apache.org/r/48294/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15958) Implement ClaimQueues on top of HBase
[ https://issues.apache.org/jira/browse/HBASE-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15958: --- Attachment: HBASE-15958.patch > Implement ClaimQueues on top of HBase > - > > Key: HBASE-15958 > URL: https://issues.apache.org/jira/browse/HBASE-15958 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15958.patch > > > Building on HBase-15883. Now implementing the claim queues procedure within > an HBase table. Peer tracking will still be performed by ZooKeeper though. > Revision at https://reviews.apache.org/r/48232/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15935) Have a separate Walker task running concurrently with Generator
[ https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15935: --- Attachment: HBASE-15935.patch > Have a separate Walker task running concurrently with Generator > -- > > Key: HBASE-15935 > URL: https://issues.apache.org/jira/browse/HBASE-15935 > Project: HBase > Issue Type: Sub-task > Components: integration tests >Reporter: Joseph >Assignee: Joseph >Priority: Minor > Attachments: HBASE-15935.patch > > > Keep track of which linked lists have been flushed in an HBase table, so that > we can concurrently Walk these lists during the Generation phase. This will > allow us to test: > 1. HBase under concurrent read/writes > 2. Availability of data immediately after flushes (as opposed to waiting till > the Verification phase) > The review diff can be found at: > https://reviews.apache.org/r/48294/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15958) Implement ClaimQueues on top of HBase
[ https://issues.apache.org/jira/browse/HBASE-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15958: --- Status: Patch Available (was: Open) > Implement ClaimQueues on top of HBase > - > > Key: HBASE-15958 > URL: https://issues.apache.org/jira/browse/HBASE-15958 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15958.patch > > > Building on HBase-15883. Now implementing the claim queues procedure within > an HBase table. Peer tracking will still be performed by ZooKeeper though. > Revision at https://reviews.apache.org/r/48232/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15972) hbase backup set command should not accept non-existing table
[
https://issues.apache.org/jira/browse/HBASE-15972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317451#comment-15317451
]
Ted Yu commented on HBASE-15972:
We can check whether the table exists before adding the table to the backup set.
Question is:
how do we convey that certain table(s) is not added to the set.
{code}
public void addToBackupSet(String name, TableName[] tables) throws
IOException;
{code}
Maybe change the return type for the above method in BackupAdmin ?
> hbase backup set command should not accept non-existing table
> -
>
> Key: HBASE-15972
> URL: https://issues.apache.org/jira/browse/HBASE-15972
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Labels: backup
>
> [~romil.choksi] discovered that 'backup set add' command doesn't check
> whether the table exists before adding the table to the named set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15972) hbase backup set command should not accept non-existing table
[ https://issues.apache.org/jira/browse/HBASE-15972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15972: --- Labels: backup (was: ) > hbase backup set command should not accept non-existing table > - > > Key: HBASE-15972 > URL: https://issues.apache.org/jira/browse/HBASE-15972 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Labels: backup > > [~romil.choksi] discovered that 'backup set add' command doesn't check > whether the table exists before adding the table to the named set. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15935) Have a separate Walker task running concurrently with Generator
[ https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15935: --- Description: Keep track of which linked lists have been flushed in an HBase table, so that we can concurrently Walk these lists during the Generation phase. This will allow us to test: 1. HBase under concurrent read/writes 2. Availability of data immediately after flushes (as opposed to waiting till the Verification phase) The review diff can be found at: https://reviews.apache.org/r/48294/ was: Keep track of which linked lists have been flushed in an HBase table, so that we can concurrently Walk these lists during the Generation phase. This will allow us to test: 1. HBase under concurrent read/writes 2. Availability of data immediately after flushes (as opposed to waiting till the Verification phase) > Have a separate Walker task running concurrently with Generator > -- > > Key: HBASE-15935 > URL: https://issues.apache.org/jira/browse/HBASE-15935 > Project: HBase > Issue Type: Sub-task > Components: integration tests >Reporter: Joseph >Assignee: Joseph >Priority: Minor > > Keep track of which linked lists have been flushed in an HBase table, so that > we can concurrently Walk these lists during the Generation phase. This will > allow us to test: > 1. HBase under concurrent read/writes > 2. Availability of data immediately after flushes (as opposed to waiting till > the Verification phase) > The review diff can be found at: > https://reviews.apache.org/r/48294/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15946) Eliminate possible security concerns in RS web UI's store file metrics
[
https://issues.apache.org/jira/browse/HBASE-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317447#comment-15317447
]
Mikhail Antonov commented on HBASE-15946:
-
Since we know for fact that in case of this specific JSP the output is going to
be small (only file metadata), could we use ByteArrayOutputStream to back
PrintWriter?
> Eliminate possible security concerns in RS web UI's store file metrics
> --
>
> Key: HBASE-15946
> URL: https://issues.apache.org/jira/browse/HBASE-15946
> Project: HBase
> Issue Type: Bug
>Reporter: Sean Mackrory
>Assignee: Mikhail Antonov
> Attachments: HBASE-15946-v1.patch, HBASE-15946-v2.patch
>
>
> More from static code analysis: it warns about the invoking of a separate
> command ("hbase hfile -s -f ...") as a possible security issue in
> hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp.
> It looks to me like one cannot inject arbitrary shell script or even
> arbitrary arguments: ProcessBuilder makes that fairly safe and only allows
> the user to specify the argument that comes after -f. However that does
> potentially allow them to have the daemon's user access files they shouldn't
> be able to touch, albeit only for reading.
> To more explicitly eliminate any threats here, we should add some validation
> that the file is at least within HBase's root directory and use the Java API
> directly instead of invoking a separate executable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15954) REST server should log requests with TRACE instead of DEBUG
[ https://issues.apache.org/jira/browse/HBASE-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317436#comment-15317436 ] Hudson commented on HBASE-15954: SUCCESS: Integrated in HBase-1.3-IT #692 (See [https://builds.apache.org/job/HBase-1.3-IT/692/]) HBASE-15954 REST server should log requests with TRACE instead of DEBUG (enis: rev 466eb31648a4783c79d9b044fdd84d0db25c3d12) * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/TableResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/provider/consumer/ProtobufMessageBodyConsumer.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RootResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/VersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterVersionResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerInstanceResource.java * hbase-server/src/main/java/org/apache/hadoop/hbase/util/ConnectionCache.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RowResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/NamespacesInstanceResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/filter/AuthFilter.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/SchemaResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ProtobufStreamingUtil.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/Client.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/ScannerResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/filter/RestCsrfPreventionFilter.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServlet.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RegionsResource.java * hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/MultiRowResource.java > REST server should log requests with TRACE instead of DEBUG > --- > > Key: HBASE-15954 > URL: https://issues.apache.org/jira/browse/HBASE-15954 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: hbase-15954_v1.patch, hbase-15954_v2.patch > > > One of the surprising findings of one of the HBaseCon presentations from > [~nitinverma], [~pravinmittal], [~maxluk] was that REST server is logging > every request in DEBUG which is enabled by default. This obviously causes > out-of-the-box experience to be pretty bad in terms of throughput unless > DEBUG logging is turned off. > We should bring REST server to be on par with the RS level log conventions. > Individual requests to be only logged at the TRACE level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15889) String case conversions are locale-sensitive, used without locale
[
https://issues.apache.org/jira/browse/HBASE-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317376#comment-15317376
]
Hudson commented on HBASE-15889:
SUCCESS: Integrated in HBase-1.4 #196 (See
[https://builds.apache.org/job/HBase-1.4/196/])
HBASE-15889. String case conversions are locale-sensitive, used without
(busbey: rev 878b1ea7212a461bb9abb0eff2bf41a6bb77)
*
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceFactoryImpl.java
*
hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/KeyStoreKeyProvider.java
*
hbase-annotations/src/main/java/org/apache/hadoop/hbase/classification/tools/StabilityOptions.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/TimedOutTestsListener.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerCommandLine.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/util/PoolMap.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SubstringComparator.java
*
hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/test/MetricsAssertHelperImpl.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcServer.java
* hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftUtilities.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableInputFormat.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/security/SaslUtil.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableInputFormat.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanBase.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AsyncRpcChannel.java
* hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/filter/GzipFilter.java
* hbase-it/src/test/java/org/apache/hadoop/hbase/HBaseClusterManager.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java
*
hbase-it/src/test/java/org/apache/hadoop/hbase/StripeCompactionsPerformanceEvaluation.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/BaseRowProcessor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/DirectMemoryUtils.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
*
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/HThreadedSelectorServerArgs.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/MetaTableLocator.java
* hbase-it/src/test/java/org/apache/hadoop/hbase/RESTApiClusterManager.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/MultiTableInputFormatTestBase.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/TestIPv6NIOServerSocketChannel.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/CreateSnapshot.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
> String case conversions are locale-sensitive, used without locale
> -
>
> Key: HBASE-15889
> URL: https://issues.apache.org/jira/browse/HBASE-15889
> Project: HBase
> Issue Type: Bug
> Components: localization
>Affects Versions: 1.2.0
>Reporter: Sean Mackrory
>Assignee: Sean Mackrory
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15889-branch-1.v2.patch, HBASE-15889-v1.patch,
> HBASE-15891-v2.patch
>
>
> Static code analysis is flagging cases of String.toLowerCase and
> String.toUpperCase being used without Locale. From the API reference:
> {quote}
> Note: This method is locale sensitive, and may produce unexpected results if
> used for strings that are intended to be interpreted locale independently.
> Examples are programming language identifiers, protocol keys, and HTML tags.
> For instance, "TITLE".toLowerCase() in a Turkish locale returns "t\u0131tle",
> where '\u0131' is the LATIN SMALL LETTER DOTLESS I character. To obtain
> correct results for locale insensitive strings, use toLowerCase(Locale.ROOT).
> {quote}
> Many uses o
[jira] [Commented] (HBASE-15971) Regression: Random Read/WorkloadC slower in 1.x than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317369#comment-15317369 ] Mikhail Antonov commented on HBASE-15971: - Is that for all blocks in the cache, I assume? > Regression: Random Read/WorkloadC slower in 1.x than 0.98 > - > > Key: HBASE-15971 > URL: https://issues.apache.org/jira/browse/HBASE-15971 > Project: HBase > Issue Type: Sub-task > Components: rpc >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 098.hits.png, 098.png, branch-1.hits.png, branch-1.png > > > branch-1 is slower than 0.98 doing YCSB random read/workloadC. It seems to be > doing about 1/2 the throughput of 0.98. > In branch-1, we have low handler occupancy compared to 0.98. Hacking in > reader thread occupancy metric, is about the same in both. In parent issue, > hacking out the scheduler, I am able to get branch-1 to go 3x faster so will > dig in here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15971) Regression: Random Read/WorkloadC slower in 1.x than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317366#comment-15317366 ] Mikhail Antonov commented on HBASE-15971: - Ouch. Do you have any feeling at what point this regression creeped in? Likely a blocker for 1.3 (and other branches). Also from the posted charts I'm not sure I immediately see lower ipc readers, but numCallsInGeneralQueue is different drastically.. > Regression: Random Read/WorkloadC slower in 1.x than 0.98 > - > > Key: HBASE-15971 > URL: https://issues.apache.org/jira/browse/HBASE-15971 > Project: HBase > Issue Type: Sub-task > Components: rpc >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 098.hits.png, 098.png, branch-1.hits.png, branch-1.png > > > branch-1 is slower than 0.98 doing YCSB random read/workloadC. It seems to be > doing about 1/2 the throughput of 0.98. > In branch-1, we have low handler occupancy compared to 0.98. Hacking in > reader thread occupancy metric, is about the same in both. In parent issue, > hacking out the scheduler, I am able to get branch-1 to go 3x faster so will > dig in here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[
https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317364#comment-15317364
]
Vladimir Rodionov commented on HBASE-15862:
---
>From MySQL:
{quote}
MySQL:
Full Versus Point-in-Time (Incremental) Recovery
A full recovery restores all data from a full backup. This restores the server
instance to the state that it had when the backup was made. If that state is
not sufficiently current, a full recovery can be followed by recovery of
incremental backups made since the full backup, to bring the server to a more
up-to-date state.
Incremental recovery is recovery of changes made during a given time span. This
is also called point-in-time recovery because it makes a server's state current
up to a given time. Point-in-time recovery is based on the binary log and
typically follows a full recovery from the backup files that restores the
server to its state when the backup was made. Then the data changes written in
the binary log files are applied as incremental recovery to redo data
modifications and bring the server up to the desired point in time.
{quote}
All restores are PIT restores. Others do not make sense. Let us not
overcomplicate stuff and default overwrite mode.
> Backup - Delete- Restore does not restore deleted data
> --
>
> Key: HBASE-15862
> URL: https://issues.apache.org/jira/browse/HBASE-15862
> Project: HBase
> Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-15862-v1.patch
>
>
> This was discovered during testing. If we delete row after full backup and
> perform immediately restore, the deleted row still remains deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15967) Metric for active ipc Readers and make default fraction of cpu count
[ https://issues.apache.org/jira/browse/HBASE-15967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317346#comment-15317346 ] stack commented on HBASE-15967: --- Thanks for the useful pointers [~bbeaudreault] Let me see if I can do similar. Our current metrics suffer another issue, one you do not mention, and that is that they are really expensive (in perf, w/ this random read loading, the most CPU is spent counting). > Metric for active ipc Readers and make default fraction of cpu count > > > Key: HBASE-15967 > URL: https://issues.apache.org/jira/browse/HBASE-15967 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Attachments: HBASE-15967.master.001.patch > > > Our ipc Readers are hard coded at 10 regardless since . Running w/ less > Readers, we go faster..(e.g. 12 Readers has us doing 135k with workloadc and > 6 readers has us doing 145k).. .but hard to tell what count of Readers are > needed since no metric. > This issue changes Readers to be 1/4 the installed CPUs or 8, whichever is > the minimum, and then adds a new hbase.regionserver.ipc.runningReaders metric > so you have a chance seeing whats needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15971) Regression: Random Read/WorkloadC slower in 1.x than 0.98
[ https://issues.apache.org/jira/browse/HBASE-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15971: -- Attachment: 098.png branch-1.png branch-1.hits.png 098.hits.png Here are graphs of me running ycsb -- both asynchbase and hbase10 -- on 8 servers pounding a single node carrying all regions. See how we do about 125k in branch-1 and 300k in 0.98. See how the handler occupancy is less in branch-1. > Regression: Random Read/WorkloadC slower in 1.x than 0.98 > - > > Key: HBASE-15971 > URL: https://issues.apache.org/jira/browse/HBASE-15971 > Project: HBase > Issue Type: Sub-task > Components: rpc >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 098.hits.png, 098.png, branch-1.hits.png, branch-1.png > > > branch-1 is slower than 0.98 doing YCSB random read/workloadC. It seems to be > doing about 1/2 the throughput of 0.98. > In branch-1, we have low handler occupancy compared to 0.98. Hacking in > reader thread occupancy metric, is about the same in both. In parent issue, > hacking out the scheduler, I am able to get branch-1 to go 3x faster so will > dig in here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15971) Regression: Random Read/WorkloadC slower in 1.x than 0.98
stack created HBASE-15971: - Summary: Regression: Random Read/WorkloadC slower in 1.x than 0.98 Key: HBASE-15971 URL: https://issues.apache.org/jira/browse/HBASE-15971 Project: HBase Issue Type: Sub-task Components: rpc Reporter: stack Assignee: stack Priority: Critical branch-1 is slower than 0.98 doing YCSB random read/workloadC. It seems to be doing about 1/2 the throughput of 0.98. In branch-1, we have low handler occupancy compared to 0.98. Hacking in reader thread occupancy metric, is about the same in both. In parent issue, hacking out the scheduler, I am able to get branch-1 to go 3x faster so will dig in here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317302#comment-15317302 ] huaxiang sun commented on HBASE-14644: -- I removed doMetrics() from HRegionServer as currently it does nothing there. Basically, I made it master-only. If this is not right, I can keep it as the old way. > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Attachments: branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14644) Region in transition metric is broken
[ https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-14644: - Attachment: branch-1.diff A quick patch just want to get some earlier feedbacks. As I mentioned earlier, master branch by default works, so I posted a branch-1 diff first. I have verified that ritCount/totalRITsOverThreshold works as expected in my local testing. I am working on an unitest case and polish the code as well. > Region in transition metric is broken > - > > Key: HBASE-14644 > URL: https://issues.apache.org/jira/browse/HBASE-14644 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: huaxiang sun > Attachments: branch-1.diff > > > ritCount stays 0 no matter what -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15698) Increment TimeRange not serialized to server
[
https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317288#comment-15317288
]
Hadoop QA commented on HBASE-15698:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 2s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 40s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 85m 30s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 133m 42s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12808449/15698.v7.txt |
| JIRA Issue | HBASE-15698 |
|
[jira] [Commented] (HBASE-15970) Move Replication Peers into an HBase table too
[ https://issues.apache.org/jira/browse/HBASE-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317266#comment-15317266 ] Ted Yu commented on HBASE-15970: Would hbase:replication be used for this tracking ? > Move Replication Peers into an HBase table too > -- > > Key: HBASE-15970 > URL: https://issues.apache.org/jira/browse/HBASE-15970 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > > Currently ReplicationQueuesHBaseTableImpl relies on ReplicationStateZkImpl to > track information about the available replication peers (used during > claimQueues). We can also move this into an HBase table instead of relying on > ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15594) [YCSB] Improvements
[
https://issues.apache.org/jira/browse/HBASE-15594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317250#comment-15317250
]
stack commented on HBASE-15594:
---
Yeah. 300k for raw 0.98. Handlers are more occupied, running at about 42 out of
48 occupied. Let me add in my reader metric and see...
> [YCSB] Improvements
> ---
>
> Key: HBASE-15594
> URL: https://issues.apache.org/jira/browse/HBASE-15594
> Project: HBase
> Issue Type: Umbrella
>Reporter: stack
>Priority: Critical
> Attachments: fast.patch
>
>
> Running YCSB and getting good results is an arcane art. For example, in my
> testing, a few handlers (100) with as many readers as I had CPUs (48), and
> upping connections on clients to same as #cpus made for 2-3x the throughput.
> The above config changes came of lore; which configurations need tweaking is
> not obvious going by their names, there were no indications from the app on
> where/why we were blocked or on which metrics are important to consider. Nor
> was any of this stuff written down in docs.
> Even still, I am stuck trying to make use of all of the machine. I am unable
> to overrun a server though 8 client nodes trying to beat up a single node
> (workloadc, all random-read, with no data returned -p readallfields=false).
> There is also a strange phenomenon where if I add a few machines, rather than
> 3x the YCSB throughput when 3 nodes in cluster, each machine instead is doing
> about 1/3rd.
> This umbrella issue is to host items that improve our defaults and noting how
> to get good numbers running YCSB. In particular, I want to be able to
> saturate a machine.
> Here are the configs I'm currently working with. I've not done the work to
> figure client-side if they are optimal (weird is how big a difference
> client-side changes can make -- need to fix this). On my 48 cpu machine, I
> can do about 370k random reads a second from data totally cached in
> bucketcache. If I short-circuit the user gets so they don't do any work but
> return immediately, I can do 600k ops a second but the CPUs are at 60-70%
> only. I cannot get them to go above this. Working on it.
> {code}
>
>
> hbase.ipc.server.read.threadpool.size
>
> 48
>
>
>
> hbase.regionserver.handler.count
>
> 100
>
>
>
> hbase.client.ipc.pool.size
>
> 100
>
>
>
> hbase.htable.threads.max
>
> 48
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15594) [YCSB] Improvements
[
https://issues.apache.org/jira/browse/HBASE-15594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317219#comment-15317219
]
stack commented on HBASE-15594:
---
0.98 is faster... 2x? 300k vs 125/140k. Reading from totally cached data
(workloadc).
> [YCSB] Improvements
> ---
>
> Key: HBASE-15594
> URL: https://issues.apache.org/jira/browse/HBASE-15594
> Project: HBase
> Issue Type: Umbrella
>Reporter: stack
>Priority: Critical
> Attachments: fast.patch
>
>
> Running YCSB and getting good results is an arcane art. For example, in my
> testing, a few handlers (100) with as many readers as I had CPUs (48), and
> upping connections on clients to same as #cpus made for 2-3x the throughput.
> The above config changes came of lore; which configurations need tweaking is
> not obvious going by their names, there were no indications from the app on
> where/why we were blocked or on which metrics are important to consider. Nor
> was any of this stuff written down in docs.
> Even still, I am stuck trying to make use of all of the machine. I am unable
> to overrun a server though 8 client nodes trying to beat up a single node
> (workloadc, all random-read, with no data returned -p readallfields=false).
> There is also a strange phenomenon where if I add a few machines, rather than
> 3x the YCSB throughput when 3 nodes in cluster, each machine instead is doing
> about 1/3rd.
> This umbrella issue is to host items that improve our defaults and noting how
> to get good numbers running YCSB. In particular, I want to be able to
> saturate a machine.
> Here are the configs I'm currently working with. I've not done the work to
> figure client-side if they are optimal (weird is how big a difference
> client-side changes can make -- need to fix this). On my 48 cpu machine, I
> can do about 370k random reads a second from data totally cached in
> bucketcache. If I short-circuit the user gets so they don't do any work but
> return immediately, I can do 600k ops a second but the CPUs are at 60-70%
> only. I cannot get them to go above this. Working on it.
> {code}
>
>
> hbase.ipc.server.read.threadpool.size
>
> 48
>
>
>
> hbase.regionserver.handler.count
>
> 100
>
>
>
> hbase.client.ipc.pool.size
>
> 100
>
>
>
> hbase.htable.threads.max
>
> 48
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[
https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317167#comment-15317167
]
Vladimir Rodionov commented on HBASE-15862:
---
{quote}
You can decide if you want to keep the current 'overwrite', which actually
means 'Dump whatever datafiles in the backup onto the live table, no offline.
But be aware of the multi-version and delete issue.'
It can still be useful in certain use cases.
{quote}
I though this is exactly no-overwrite semantics?
> Backup - Delete- Restore does not restore deleted data
> --
>
> Key: HBASE-15862
> URL: https://issues.apache.org/jira/browse/HBASE-15862
> Project: HBase
> Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-15862-v1.patch
>
>
> This was discovered during testing. If we delete row after full backup and
> perform immediately restore, the deleted row still remains deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317160#comment-15317160 ] Jerry He commented on HBASE-15862: -- bq. Then the user can do the needful and either delete or not, take snapshot, etc. This is a good option. You can decide if you want to keep the current 'overwrite', which actually means 'Dump whatever datafiles in the backup onto the live table, no offline. But be aware of the multi-version and delete issue.' It can still be useful in certain use cases. > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-15862-v1.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317094#comment-15317094 ] Devaraj Das commented on HBASE-15862: - I'd say that we punt on it, and let the full-restore fail if the table already exists? Then the user can do the needful and either delete or not, take snapshot, etc. > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-15862-v1.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317092#comment-15317092 ] Jerry He commented on HBASE-15862: -- Hi, [~vrodionov] Yes, that is PIT restore. But you can define your own 'overwrite'. We can leave to the user to clean and then restore. Or we can clean as part of the restore. In the database world, there are different support and choices from different vendors. For example, this is for DB2. https://www.ibm.com/support/knowledgecenter/SSEPGG_10.5.0/com.ibm.db2.luw.admin.ha.doc/doc/c0006250.html The compact before restore won't work for PIT, because HBase can have multiple versions for the cell. So if the restore puts back the older version, the newer version will still be effective. > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-15862-v1.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15698) Increment TimeRange not serialized to server
[ https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317026#comment-15317026 ] Ted Yu commented on HBASE-15698: Turned out that I missed one operation which needs manually incrementing the clock. Patch v7 passes locally. Let's see what QA bot says. > Increment TimeRange not serialized to server > > > Key: HBASE-15698 > URL: https://issues.apache.org/jira/browse/HBASE-15698 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0 >Reporter: James Taylor >Assignee: Ted Yu >Priority: Blocker > Labels: phoenix > Fix For: 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt, > 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, 15698.v6.txt, > 15698.v7.txt, HBASE-15698.1.patch > > > Before HBase-1.2, the Increment TimeRange set on the client was serialized > over to the server. As of HBase 1.2, this appears to no longer be true, as my > preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value > of increment.getTimeRange().getMax() regardless of what the client has > specified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15698) Increment TimeRange not serialized to server
[ https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15698: --- Attachment: 15698.v7.txt > Increment TimeRange not serialized to server > > > Key: HBASE-15698 > URL: https://issues.apache.org/jira/browse/HBASE-15698 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0 >Reporter: James Taylor >Assignee: Ted Yu >Priority: Blocker > Labels: phoenix > Fix For: 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt, > 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, 15698.v6.txt, > 15698.v7.txt, HBASE-15698.1.patch > > > Before HBase-1.2, the Increment TimeRange set on the client was serialized > over to the server. As of HBase 1.2, this appears to no longer be true, as my > preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value > of increment.getTimeRange().getMax() regardless of what the client has > specified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15698) Increment TimeRange not serialized to server
[ https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15698: --- Status: Patch Available (was: Open) > Increment TimeRange not serialized to server > > > Key: HBASE-15698 > URL: https://issues.apache.org/jira/browse/HBASE-15698 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0 >Reporter: James Taylor >Assignee: Ted Yu >Priority: Blocker > Labels: phoenix > Fix For: 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt, > 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, 15698.v6.txt, > 15698.v7.txt, HBASE-15698.1.patch > > > Before HBase-1.2, the Increment TimeRange set on the client was serialized > over to the server. As of HBase 1.2, this appears to no longer be true, as my > preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value > of increment.getTimeRange().getMax() regardless of what the client has > specified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15698) Increment TimeRange not serialized to server
[ https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317016#comment-15317016 ] Andrew Purtell commented on HBASE-15698: bq. See if I used ManualEnvironmentEdge correctly. Haven't looked at the patch. Are you manually incrementing the clock as needed? I'm pretty busy. Don't wait on me. > Increment TimeRange not serialized to server > > > Key: HBASE-15698 > URL: https://issues.apache.org/jira/browse/HBASE-15698 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0 >Reporter: James Taylor >Assignee: Ted Yu >Priority: Blocker > Labels: phoenix > Fix For: 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt, > 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, 15698.v6.txt, > 15698.v7.txt, HBASE-15698.1.patch > > > Before HBase-1.2, the Increment TimeRange set on the client was serialized > over to the server. As of HBase 1.2, this appears to no longer be true, as my > preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value > of increment.getTimeRange().getMax() regardless of what the client has > specified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15698) Increment TimeRange not serialized to server
[
https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316957#comment-15316957
]
Ted Yu commented on HBASE-15698:
Haven't been able to get v6 to pass.
{code}
testHTableInterfaceMethods(org.apache.hadoop.hbase.coprocessor.TestIncrementTimeRange)
Time elapsed: 1.773 sec <<< ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.hbase.coprocessor.TestIncrementTimeRange.checkHTableInterfaceMethods(TestIncrementTimeRange.java:178)
at
org.apache.hadoop.hbase.coprocessor.TestIncrementTimeRange.testHTableInterfaceMethods(TestIncrementTimeRange.java:151)
{code}
The above meant that observer wasn't activated for the two Increment's of 2L.
See if I used ManualEnvironmentEdge correctly.
> Increment TimeRange not serialized to server
>
>
> Key: HBASE-15698
> URL: https://issues.apache.org/jira/browse/HBASE-15698
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0
>Reporter: James Taylor
>Assignee: Ted Yu
>Priority: Blocker
> Labels: phoenix
> Fix For: 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6
>
> Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt,
> 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, 15698.v6.txt,
> HBASE-15698.1.patch
>
>
> Before HBase-1.2, the Increment TimeRange set on the client was serialized
> over to the server. As of HBase 1.2, this appears to no longer be true, as my
> preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value
> of increment.getTimeRange().getMax() regardless of what the client has
> specified.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15698) Increment TimeRange not serialized to server
[ https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15698: --- Attachment: 15698.v6.txt > Increment TimeRange not serialized to server > > > Key: HBASE-15698 > URL: https://issues.apache.org/jira/browse/HBASE-15698 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0 >Reporter: James Taylor >Assignee: Ted Yu >Priority: Blocker > Labels: phoenix > Fix For: 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt, > 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, 15698.v6.txt, > HBASE-15698.1.patch > > > Before HBase-1.2, the Increment TimeRange set on the client was serialized > over to the server. As of HBase 1.2, this appears to no longer be true, as my > preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value > of increment.getTimeRange().getMax() regardless of what the client has > specified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15698) Increment TimeRange not serialized to server
[ https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15698: --- Attachment: (was: 15698.v6.txt) > Increment TimeRange not serialized to server > > > Key: HBASE-15698 > URL: https://issues.apache.org/jira/browse/HBASE-15698 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0 >Reporter: James Taylor >Assignee: Ted Yu >Priority: Blocker > Labels: phoenix > Fix For: 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt, > 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, HBASE-15698.1.patch > > > Before HBase-1.2, the Increment TimeRange set on the client was serialized > over to the server. As of HBase 1.2, this appears to no longer be true, as my > preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value > of increment.getTimeRange().getMax() regardless of what the client has > specified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15698) Increment TimeRange not serialized to server
[ https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15698: --- Attachment: 15698.v6.txt See if patch v6 is closer to what you're looking for. Both Increment's in the batch carry the same TimeRange. > Increment TimeRange not serialized to server > > > Key: HBASE-15698 > URL: https://issues.apache.org/jira/browse/HBASE-15698 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0 >Reporter: James Taylor >Assignee: Ted Yu >Priority: Blocker > Labels: phoenix > Fix For: 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6 > > Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt, > 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, 15698.v6.txt, > HBASE-15698.1.patch > > > Before HBase-1.2, the Increment TimeRange set on the client was serialized > over to the server. As of HBase 1.2, this appears to no longer be true, as my > preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value > of increment.getTimeRange().getMax() regardless of what the client has > specified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15698) Increment TimeRange not serialized to server
[
https://issues.apache.org/jira/browse/HBASE-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316942#comment-15316942
]
Sean Busbey commented on HBASE-15698:
-
{quote}
bq. Sean Busbey is on vacation this week
Bah. I owe you a favor Sean Busbey as apology so please feel free to call it in
whenever.
{quote}
No worries. Sorry I needed to disappear for a bit without any notice. FWIW, my
plan on my return was just to commit whatever Ted had come up with as long as
tests were passing.
> Increment TimeRange not serialized to server
>
>
> Key: HBASE-15698
> URL: https://issues.apache.org/jira/browse/HBASE-15698
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.0, 1.3.0
>Reporter: James Taylor
>Assignee: Ted Yu
>Priority: Blocker
> Labels: phoenix
> Fix For: 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6
>
> Attachments: 15698-suggest.txt, 15698.v1.txt, 15698.v2.txt,
> 15698.v3.txt, 15698.v4.txt, 15698.v4.txt, 15698.v5.txt, HBASE-15698.1.patch
>
>
> Before HBase-1.2, the Increment TimeRange set on the client was serialized
> over to the server. As of HBase 1.2, this appears to no longer be true, as my
> preIncrement coprocessor always gets HConstants.LATEST_TIMESTAMP as the value
> of increment.getTimeRange().getMax() regardless of what the client has
> specified.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
