[jira] [Commented] (HBASE-15931) Add log for long-running tasks in AsyncProcess
[
https://issues.apache.org/jira/browse/HBASE-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311789#comment-15311789
]
Hudson commented on HBASE-15931:
FAILURE: Integrated in HBase-1.4 #188 (See
[https://builds.apache.org/job/HBase-1.4/188/])
HBASE-15931 Add log for long-running tasks in AsyncProcess (liyu: rev
36bb496e4809f1ae3146fb93439f4531d7fea657)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorImpl.java
*
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
> Add log for long-running tasks in AsyncProcess
> --
>
> Key: HBASE-15931
> URL: https://issues.apache.org/jira/browse/HBASE-15931
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15931.patch
>
>
> Currently if there's any long-tail tasks in a multi-action request like
> triggered by {{BufferedMutatorImpl#backgroundFlushCommits}}, we could see
> logging message like below
> {noformat}
> 2016-05-31 09:36:55,461 INFO [Thread-16]
> org.apache.hadoop.hbase.client.AsyncProcess: #28, waiting for some tasks to
> finish. Expected max=0, tasksInProgress=1
> {noformat}
> but there's no way to know detail of this long-tail, such as which
> table/region it's accessing. This JIRA aims at adding such log for easier
> debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14921) Memory optimizations
[ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311788#comment-15311788 ] ramkrishna.s.vasudevan commented on HBASE-14921: +1 to begin with CellArrayMap. > Memory optimizations > > > Key: HBASE-14921 > URL: https://issues.apache.org/jira/browse/HBASE-14921 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Anastasia Braginsky > Attachments: CellBlocksSegmentInMemStore.pdf, > CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch, > HBASE-14921-V02.patch, HBASE-14921-V03.patch, > IntroductiontoNewFlatandCompactMemStore.pdf > > > Memory optimizations including compressed format representation and offheap > allocations -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15888) Extend HBASE-12769 for bulk load data replication
[ https://issues.apache.org/jira/browse/HBASE-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-15888: -- Attachment: HBASE-15888.v1.patch Addressed the comment. > Extend HBASE-12769 for bulk load data replication > - > > Key: HBASE-15888 > URL: https://issues.apache.org/jira/browse/HBASE-15888 > Project: HBase > Issue Type: Task > Components: Replication >Affects Versions: 1.3.0 >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15888.patch, HBASE-15888.v1.patch > > > Include hfile-refs queue check and fix also in hbck -fixReplication command -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15174) Client Public API should not have PB objects in 2.0
[
https://issues.apache.org/jira/browse/HBASE-15174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311742#comment-15311742
]
ramkrishna.s.vasudevan commented on HBASE-15174:
I can upload the UT that i have to this JIRA.
> Client Public API should not have PB objects in 2.0
> ---
>
> Key: HBASE-15174
> URL: https://issues.apache.org/jira/browse/HBASE-15174
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 2.0.0
>
>
> Some more cleanup for the parent jira.
> We have leaked some PB structs in Admin (and possible other places).
> We should clean up these API before 2.0.
> Examples include:
> {code}
> AdminProtos.GetRegionInfoResponse.CompactionState getCompactionState(final
> TableName tableName)
> throws IOException;
>
>
> void snapshot(final String snapshotName,
> final TableName tableName,
> HBaseProtos.SnapshotDescription.Type type) throws IOException,
> SnapshotCreationException,
> IllegalArgumentException;
>
> MasterProtos.SnapshotResponse
> takeSnapshotAsync(HBaseProtos.SnapshotDescription snapshot)
> throws IOException, SnapshotCreationException;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15931) Add log for long-running tasks in AsyncProcess
[
https://issues.apache.org/jira/browse/HBASE-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-15931:
--
Fix Version/s: 1.4.0
2.0.0
Release Note:
After HBASE-15931, we will log more details for long-running tasks in
AsyncProcess#waitForMaximumCurrentTasks every 10 seconds, including:
1. Table name will be included in the tasks status log
2. On which regionserver(s) the tasks are runnning will be logged when less
than hbase.client.threshold.log.details tasks left, by default 10.
3. Against which regions the tasks are running will be logged when less than 2
tasks left.
> Add log for long-running tasks in AsyncProcess
> --
>
> Key: HBASE-15931
> URL: https://issues.apache.org/jira/browse/HBASE-15931
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-15931.patch
>
>
> Currently if there's any long-tail tasks in a multi-action request like
> triggered by {{BufferedMutatorImpl#backgroundFlushCommits}}, we could see
> logging message like below
> {noformat}
> 2016-05-31 09:36:55,461 INFO [Thread-16]
> org.apache.hadoop.hbase.client.AsyncProcess: #28, waiting for some tasks to
> finish. Expected max=0, tasksInProgress=1
> {noformat}
> but there's no way to know detail of this long-tail, such as which
> table/region it's accessing. This JIRA aims at adding such log for easier
> debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15858) Some region server group shell commands don't work
[
https://issues.apache.org/jira/browse/HBASE-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15858:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
Status: Resolved (was: Patch Available)
> Some region server group shell commands don't work
> --
>
> Key: HBASE-15858
> URL: https://issues.apache.org/jira/browse/HBASE-15858
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0
>
> Attachments: 15858.v1.patch, 15858.v2.patch, 15858.v3.patch
>
>
> [~cartershanklin] reported the following:
> {code}
> hbase(main):021:0> get_rsgroup 'test_group'
> GROUP INFORMATION
> ERROR: undefined local variable or method `group_admin' for
> #
> Here is some help for this command:
> Get a region server group's information.
> Example:
> hbase> get_rsgroup 'default'
> {code}
> {code}
> hbase(main):009:0> get_server_rsgroup
> 'hdp250-hbase-unsecure.example.com:16030'
> ERROR: undefined method `getGroupOfServer' for
> #
> Here is some help for this command:
> Get the group name the given region server is a member of.
> hbase> get_server_rsgroup 'server1:port1'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15931) Add log for long-running tasks in AsyncProcess
[
https://issues.apache.org/jira/browse/HBASE-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311697#comment-15311697
]
Yu Li commented on HBASE-15931:
---
Pushed into master and branch-1. Thanks [~stack] for review.
[~mantonov], mind if I push this into branch-1.3 also? Thanks.
> Add log for long-running tasks in AsyncProcess
> --
>
> Key: HBASE-15931
> URL: https://issues.apache.org/jira/browse/HBASE-15931
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15931.patch
>
>
> Currently if there's any long-tail tasks in a multi-action request like
> triggered by {{BufferedMutatorImpl#backgroundFlushCommits}}, we could see
> logging message like below
> {noformat}
> 2016-05-31 09:36:55,461 INFO [Thread-16]
> org.apache.hadoop.hbase.client.AsyncProcess: #28, waiting for some tasks to
> finish. Expected max=0, tasksInProgress=1
> {noformat}
> but there's no way to know detail of this long-tail, such as which
> table/region it's accessing. This JIRA aims at adding such log for easier
> debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15942) User with 'RWX' permission of the table should be able to describe the table
[ https://issues.apache.org/jira/browse/HBASE-15942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311693#comment-15311693 ] Guang Yang commented on HBASE-15942: The patch also contains a minor refactoring for AccessController#requirePermission, which might unnecessarily create some AuthResult instances. > User with 'RWX' permission of the table should be able to describe the table > > > Key: HBASE-15942 > URL: https://issues.apache.org/jira/browse/HBASE-15942 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Guang Yang >Priority: Minor > Attachments: HBASE-15942.v0.patch > > > Currently only the user who has 'ADMIN' or 'CREATE' permission could describe > the table (via shell), it brings some inconvenience and confusion to those > uses who have RWX permission since they couldn't describe the table to get > the schema. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15942) User with 'RWX' permission of the table should be able to describe the table
[ https://issues.apache.org/jira/browse/HBASE-15942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guang Yang updated HBASE-15942: --- Attachment: HBASE-15942.v0.patch > User with 'RWX' permission of the table should be able to describe the table > > > Key: HBASE-15942 > URL: https://issues.apache.org/jira/browse/HBASE-15942 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: Guang Yang >Priority: Minor > Attachments: HBASE-15942.v0.patch > > > Currently only the user who has 'ADMIN' or 'CREATE' permission could describe > the table (via shell), it brings some inconvenience and confusion to those > uses who have RWX permission since they couldn't describe the table to get > the schema. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9465) HLog entries are not pushed to peer clusters serially when region-move or RS failure in master cluster
[ https://issues.apache.org/jira/browse/HBASE-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311691#comment-15311691 ] Phil Yang commented on HBASE-9465: -- Hi all, I'm going to pick up this issue to make replication log pushing serially in region+peer level. The brief idea is to prevent pushing logs if there are any logs with smaller sequence id which are not pushed to this peer cluster, by saving some barrier/position information in hbase:meta table. We should consider various cases so I will post a designing doc these days. Thanks. > HLog entries are not pushed to peer clusters serially when region-move or RS > failure in master cluster > -- > > Key: HBASE-9465 > URL: https://issues.apache.org/jira/browse/HBASE-9465 > Project: HBase > Issue Type: Bug > Components: regionserver, Replication >Reporter: Honghua Feng > > When region-move or RS failure occurs in master cluster, the hlog entries > that are not pushed before region-move or RS-failure will be pushed by > original RS(for region move) or another RS which takes over the remained hlog > of dead RS(for RS failure), and the new entries for the same region(s) will > be pushed by the RS which now serves the region(s), but they push the hlog > entries of a same region concurrently without coordination. > This treatment can possibly lead to data inconsistency between master and > peer clusters: > 1. there are put and then delete written to master cluster > 2. due to region-move / RS-failure, they are pushed by different > replication-source threads to peer cluster > 3. if delete is pushed to peer cluster before put, and flush and > major-compact occurs in peer cluster before put is pushed to peer cluster, > the delete is collected and the put remains in peer cluster > In this scenario, the put remains in peer cluster, but in master cluster the > put is masked by the delete, hence data inconsistency between master and peer > clusters -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15942) User with 'RWX' permission of the table should be able to describe the table
Guang Yang created HBASE-15942: -- Summary: User with 'RWX' permission of the table should be able to describe the table Key: HBASE-15942 URL: https://issues.apache.org/jira/browse/HBASE-15942 Project: HBase Issue Type: Bug Components: Admin Reporter: Guang Yang Priority: Minor Currently only the user who has 'ADMIN' or 'CREATE' permission could describe the table (via shell), it brings some inconvenience and confusion to those uses who have RWX permission since they couldn't describe the table to get the schema. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15941) HBCK repair should not unsplit healthy splitted region
Stephen Yuan Jiang created HBASE-15941: -- Summary: HBCK repair should not unsplit healthy splitted region Key: HBASE-15941 URL: https://issues.apache.org/jira/browse/HBASE-15941 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 1.0.0 Reporter: Stephen Yuan Jiang Assignee: Stephen Yuan Jiang Currently HBCK design in branch-1 has a flaw when repair option (the -fixHdfsOverlaps option specifically) is specified: it would wrongly merge split region (by looking at HDFS, it thinks that there exists overlapped regions - parent region and daughter regions covers the same key range, of course). See HBASE-15940 for details. This JIRA tracks the improvement not-to-merge split region in HBCK repair. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15429) Add a split policy for busy regions
[
https://issues.apache.org/jira/browse/HBASE-15429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311654#comment-15311654
]
Hadoop QA commented on HBASE-15429:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 42s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
48s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
37s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 21s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 13s
{color} | {color:red} hbase-server generated 3 new + 0 unchanged - 0 fixed = 3
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 42s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 90m 30s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 138m 53s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.BusyRegionSplitPolicy.blockedRequestCount;
locked 66% of time Unsynchronized access at BusyRegionSplitPolicy.java:66%
[jira] [Commented] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311632#comment-15311632
]
Vladimir Rodionov commented on HBASE-12721:
---
Great, definitely. One request: can you enhance your scripts and containerize
HBase in a cluster (not single), [~dimaspivak]? Run X HBase nodes per server in
M node cluster?
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15939) Two shell test failures on master
[
https://issues.apache.org/jira/browse/HBASE-15939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15939:
---
Attachment: 15939.v1.txt
Patch v1 fixes the first error.
[~talat]:
Mind taking a look at the second error ?
Thanks
> Two shell test failures on master
> -
>
> Key: HBASE-15939
> URL: https://issues.apache.org/jira/browse/HBASE-15939
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 15939.v1.txt
>
>
> For master branch, I observe two shell test failures:
> {code}
> test_Shell::Shell_noninteractive_mode_should_throw(NonInteractiveTest):
> NameError: no method 'getTable' for arguments (org.jruby.RubyString) on
> #
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/hbase.rb:61:in `table'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/shell.rb:95:in `hbase_table'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/shell/commands.rb:62:in `table'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/shell/commands/create.rb:69:in
> `command'
> org/jruby/RubyKernel.java:2113:in `send'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/shell/commands.rb:34:in
> `command_safe'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/shell/commands.rb:107:in
> `translate_hbase_exceptions'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/shell/commands.rb:34:in
> `command_safe'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/shell.rb:148:in
> `internal_command'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/shell.rb:140:in `command'
> ./src/test/ruby/shell/noninteractive_test.rb:35:in
> `test_Shell::Shell_noninteractive_mode_should_throw'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each'
> {code}
> {code}
> 1) Error:
> test_tasksOnHost_should_return_tasks_list(Hbase::TaskMonitorTest):
> NativeException: java.net.ConnectException: Connection refused
> java/net/PlainSocketImpl.java:-2:in `socketConnect'
> java/net/AbstractPlainSocketImpl.java:339:in `doConnect'
> java/net/AbstractPlainSocketImpl.java:200:in `connectToAddress'
> java/net/AbstractPlainSocketImpl.java:182:in `connect'
> java/net/SocksSocketImpl.java:392:in `connect'
> java/net/Socket.java:579:in `connect'
> java/net/Socket.java:528:in `connect'
> sun/net/NetworkClient.java:180:in `doConnect'
> sun/net/www/http/HttpClient.java:432:in `openServer'
> sun/net/www/http/HttpClient.java:527:in `openServer'
> sun/net/www/http/HttpClient.java:211:in `'
> sun/net/www/http/HttpClient.java:308:in `New'
> sun/net/www/http/HttpClient.java:326:in `New'
> sun/net/www/protocol/http/HttpURLConnection.java:996:in `getNewHttpClient'
> sun/net/www/protocol/http/HttpURLConnection.java:932:in `plainConnect'
> sun/net/www/protocol/http/HttpURLConnection.java:850:in `connect'
> sun/net/www/protocol/http/HttpURLConnection.java:1300:in `getInputStream'
> java/net/URL.java:1037:in `openStream'
> org/codehaus/jackson/JsonFactory.java:935:in `_optimizedStreamFromURL'
> org/codehaus/jackson/JsonFactory.java:530:in `createJsonParser'
> org/codehaus/jackson/map/ObjectMapper.java:1840:in `readValue'
> /Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/taskmonitor.rb:104:in
> `tasksOnHost'
> ./src/test/ruby/hbase/taskmonitor_test.rb:33:in
> `test_tasksOnHost_should_return_tasks_list'
> org/jruby/RubyArray.java:1620:in `each'
> ./src/test/ruby/hbase/taskmonitor_test.rb:32:in
> `test_tasksOnHost_should_return_tasks_list'
> org/jruby/RubyProc.java:270:in `call'
> org/jruby/RubyKernel.java:2105:in `send'
> org/jruby/RubyArray.java:1620:in `each'
> org/jruby/RubyArray.java:1620:in `each'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15938) submit-patch.py: Don't crash if there are tests with same name. Refactor: Split out html template to separate file.
[
https://issues.apache.org/jira/browse/HBASE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311628#comment-15311628
]
Hadoop QA commented on HBASE-15938:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} pylint {color} | {color:blue} 0m 3s
{color} | {color:blue} Pylint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 25s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 14s
{color} | {color:red} Patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 50s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12807593/HBASE-15938.master.001.patch
|
| JIRA Issue | HBASE-15938 |
| Optional Tests | asflicense pylint |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cbb95cd |
| asflicense |
https://builds.apache.org/job/PreCommit-HBASE-Build/2087/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2087/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> submit-patch.py: Don't crash if there are tests with same name. Refactor:
> Split out html template to separate file.
> ---
>
> Key: HBASE-15938
> URL: https://issues.apache.org/jira/browse/HBASE-15938
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-15938.master.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15940) HBCK unnecessary moves reference files when a table has split region to fix non-existing overlap regions
[
https://issues.apache.org/jira/browse/HBASE-15940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311627#comment-15311627
]
Stephen Yuan Jiang commented on HBASE-15940:
One possible solution is that we really don't need to copy referenced files to
a new region during fixHdfsOverlaps - either the referenced file is orphaned;
or the real files that the referenced files point to are already in the new
region.
> HBCK unnecessary moves reference files when a table has split region to fix
> non-existing overlap regions
>
>
> Key: HBASE-15940
> URL: https://issues.apache.org/jira/browse/HBASE-15940
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 1.0.0
>Reporter: Stephen Yuan Jiang
>Assignee: Stephen Yuan Jiang
> Attachments: org.apache.hadoop.hbase.util.TestHBaseFsck-output.txt,
> repro-hbck-repair-healthy-splitted=region.patch
>
>
> When repair option (the -fixHdfsOverlaps option specifically) is specified
> against a table, if the table has splitted regions (both parent region and
> child regions exists with reference files), Hbck would wrongly think that
> there exists overlapped regions and try to merge them and fix it.
> This is by-design, as current implementation of Hbck uses HDFS as the trusted
> source without consulting META table.
> Here is the comments from one of unit tests:
> {code}
> // TODO: fixHdfsHoles does not work against splits, since the parent
> dir lingers on
> // for some time until children references are deleted. HBCK
> erroneously sees this as
> // overlapping regions
> {code}
> However, this is undesirable. when the reference files moved to a new
> region, the parent region would have no daugher regions and hence it could be
> cleaned up by CatalogJanitor. This would create real inconsistency:
> lingering reference files.
> Another bad consequence is that we would merge splitted regions back to one.
> Even it is undesirable, at least this would not cause more inconsistency.
> this JIRA would not try to solve this unsplit issue, as it requires bigger
> design change in Hbck.
> This JIRA is trying to address the potential lingering reference files
> issue, as multiple customers using branch-1 faced this issue in production.
> (workaround is that run major compaction on all split regions before run
> HBCK, this could take longer time and have production impact).
> Attached is the log and modified unit test to repro the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15940) HBCK unnecessary moves reference files when a table has split region to fix non-existing overlap regions
[
https://issues.apache.org/jira/browse/HBASE-15940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Yuan Jiang updated HBASE-15940:
---
Attachment: org.apache.hadoop.hbase.util.TestHBaseFsck-output.txt
repro-hbck-repair-healthy-splitted=region.patch
> HBCK unnecessary moves reference files when a table has split region to fix
> non-existing overlap regions
>
>
> Key: HBASE-15940
> URL: https://issues.apache.org/jira/browse/HBASE-15940
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 1.0.0
>Reporter: Stephen Yuan Jiang
>Assignee: Stephen Yuan Jiang
> Attachments: org.apache.hadoop.hbase.util.TestHBaseFsck-output.txt,
> repro-hbck-repair-healthy-splitted=region.patch
>
>
> When repair option (the -fixHdfsOverlaps option specifically) is specified
> against a table, if the table has splitted regions (both parent region and
> child regions exists with reference files), Hbck would wrongly think that
> there exists overlapped regions and try to merge them and fix it.
> This is by-design, as current implementation of Hbck uses HDFS as the trusted
> source without consulting META table.
> Here is the comments from one of unit tests:
> {code}
> // TODO: fixHdfsHoles does not work against splits, since the parent
> dir lingers on
> // for some time until children references are deleted. HBCK
> erroneously sees this as
> // overlapping regions
> {code}
> However, this is undesirable. when the reference files moved to a new
> region, the parent region would have no daugher regions and hence it could be
> cleaned up by CatalogJanitor. This would create real inconsistency:
> lingering reference files.
> Another bad consequence is that we would merge splitted regions back to one.
> Even it is undesirable, at least this would not cause more inconsistency.
> this JIRA would not try to solve this unsplit issue, as it requires bigger
> design change in Hbck.
> This JIRA is trying to address the potential lingering reference files
> issue, as multiple customers using branch-1 faced this issue in production.
> (workaround is that run major compaction on all split regions before run
> HBCK, this could take longer time and have production impact).
> Attached is the log and modified unit test to repro the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15940) HBCK unnecessary moves reference files when a table has split region to fix non-existing overlap regions
Stephen Yuan Jiang created HBASE-15940:
--
Summary: HBCK unnecessary moves reference files when a table has
split region to fix non-existing overlap regions
Key: HBASE-15940
URL: https://issues.apache.org/jira/browse/HBASE-15940
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 1.0.0
Reporter: Stephen Yuan Jiang
Assignee: Stephen Yuan Jiang
Attachments: org.apache.hadoop.hbase.util.TestHBaseFsck-output.txt,
repro-hbck-repair-healthy-splitted=region.patch
When repair option (the -fixHdfsOverlaps option specifically) is specified
against a table, if the table has splitted regions (both parent region and
child regions exists with reference files), Hbck would wrongly think that there
exists overlapped regions and try to merge them and fix it.
This is by-design, as current implementation of Hbck uses HDFS as the trusted
source without consulting META table.
Here is the comments from one of unit tests:
{code}
// TODO: fixHdfsHoles does not work against splits, since the parent dir
lingers on
// for some time until children references are deleted. HBCK erroneously
sees this as
// overlapping regions
{code}
However, this is undesirable. when the reference files moved to a new region,
the parent region would have no daugher regions and hence it could be cleaned
up by CatalogJanitor. This would create real inconsistency: lingering
reference files.
Another bad consequence is that we would merge splitted regions back to one.
Even it is undesirable, at least this would not cause more inconsistency. this
JIRA would not try to solve this unsplit issue, as it requires bigger design
change in Hbck.
This JIRA is trying to address the potential lingering reference files issue,
as multiple customers using branch-1 faced this issue in production.
(workaround is that run major compaction on all split regions before run HBCK,
this could take longer time and have production impact).
Attached is the log and modified unit test to repro the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15939) Two shell test failures on master
Ted Yu created HBASE-15939:
--
Summary: Two shell test failures on master
Key: HBASE-15939
URL: https://issues.apache.org/jira/browse/HBASE-15939
Project: HBase
Issue Type: Test
Reporter: Ted Yu
For master branch, I observe two shell test failures:
{code}
test_Shell::Shell_noninteractive_mode_should_throw(NonInteractiveTest):
NameError: no method 'getTable' for arguments (org.jruby.RubyString) on
#
/Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/hbase.rb:61:in `table'
/Users/tyu/trunk/hbase-shell/src/main/ruby/shell.rb:95:in `hbase_table'
/Users/tyu/trunk/hbase-shell/src/main/ruby/shell/commands.rb:62:in `table'
/Users/tyu/trunk/hbase-shell/src/main/ruby/shell/commands/create.rb:69:in
`command'
org/jruby/RubyKernel.java:2113:in `send'
/Users/tyu/trunk/hbase-shell/src/main/ruby/shell/commands.rb:34:in
`command_safe'
/Users/tyu/trunk/hbase-shell/src/main/ruby/shell/commands.rb:107:in
`translate_hbase_exceptions'
/Users/tyu/trunk/hbase-shell/src/main/ruby/shell/commands.rb:34:in
`command_safe'
/Users/tyu/trunk/hbase-shell/src/main/ruby/shell.rb:148:in
`internal_command'
/Users/tyu/trunk/hbase-shell/src/main/ruby/shell.rb:140:in `command'
./src/test/ruby/shell/noninteractive_test.rb:35:in
`test_Shell::Shell_noninteractive_mode_should_throw'
org/jruby/RubyProc.java:270:in `call'
org/jruby/RubyKernel.java:2105:in `send'
org/jruby/RubyArray.java:1620:in `each'
org/jruby/RubyArray.java:1620:in `each'
{code}
{code}
1) Error:
test_tasksOnHost_should_return_tasks_list(Hbase::TaskMonitorTest):
NativeException: java.net.ConnectException: Connection refused
java/net/PlainSocketImpl.java:-2:in `socketConnect'
java/net/AbstractPlainSocketImpl.java:339:in `doConnect'
java/net/AbstractPlainSocketImpl.java:200:in `connectToAddress'
java/net/AbstractPlainSocketImpl.java:182:in `connect'
java/net/SocksSocketImpl.java:392:in `connect'
java/net/Socket.java:579:in `connect'
java/net/Socket.java:528:in `connect'
sun/net/NetworkClient.java:180:in `doConnect'
sun/net/www/http/HttpClient.java:432:in `openServer'
sun/net/www/http/HttpClient.java:527:in `openServer'
sun/net/www/http/HttpClient.java:211:in `'
sun/net/www/http/HttpClient.java:308:in `New'
sun/net/www/http/HttpClient.java:326:in `New'
sun/net/www/protocol/http/HttpURLConnection.java:996:in `getNewHttpClient'
sun/net/www/protocol/http/HttpURLConnection.java:932:in `plainConnect'
sun/net/www/protocol/http/HttpURLConnection.java:850:in `connect'
sun/net/www/protocol/http/HttpURLConnection.java:1300:in `getInputStream'
java/net/URL.java:1037:in `openStream'
org/codehaus/jackson/JsonFactory.java:935:in `_optimizedStreamFromURL'
org/codehaus/jackson/JsonFactory.java:530:in `createJsonParser'
org/codehaus/jackson/map/ObjectMapper.java:1840:in `readValue'
/Users/tyu/trunk/hbase-shell/src/main/ruby/hbase/taskmonitor.rb:104:in
`tasksOnHost'
./src/test/ruby/hbase/taskmonitor_test.rb:33:in
`test_tasksOnHost_should_return_tasks_list'
org/jruby/RubyArray.java:1620:in `each'
./src/test/ruby/hbase/taskmonitor_test.rb:32:in
`test_tasksOnHost_should_return_tasks_list'
org/jruby/RubyProc.java:270:in `call'
org/jruby/RubyKernel.java:2105:in `send'
org/jruby/RubyArray.java:1620:in `each'
org/jruby/RubyArray.java:1620:in `each'
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15932) Shell test fails due to uninitialized constant
[
https://issues.apache.org/jira/browse/HBASE-15932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15932:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks for the reviews.
> Shell test fails due to uninitialized constant
> --
>
> Key: HBASE-15932
> URL: https://issues.apache.org/jira/browse/HBASE-15932
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0
>
> Attachments: 15932.v1.txt
>
>
> In master branch, shell tests fail due to IN_MEMORY_COMPACTION not being
> found:
> {code}
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 32.299 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups
> testRunShellTests(org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups)
> Time elapsed: 0.069 sec <<< ERROR!
> org.jruby.embed.EvalFailedException: (NameError) uninitialized constant
> Java::OrgApacheHadoopHbase::HConstants::IN_MEMORY_COMPACTION
> at
> org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:136)
> at
> org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1263)
> at
> org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1308)
> at
> org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups.testRunShellTests(TestShellRSGroups.java:101)
> Caused by: org.jruby.exceptions.RaiseException: (NameError) uninitialized
> constant Java::OrgApacheHadoopHbase::HConstants::IN_MEMORY_COMPACTION
> at org.jruby.RubyModule.const_missing(org/jruby/RubyModule.java:2647)
> at
> Module.const_missing(file:/home/jenkins/yetus-m2/hbase-master-1/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/META-INF/jruby.home/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2503)
> at
> Module.HBaseConstants(/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/hbase-shell/src/main/ruby/hbase.rb:42)
> at
> (Anonymous).(root)(/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/hbase-shell/src/main/ruby/hbase.rb:34)
> at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1062)
> at
> Kernel.require(/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/hbase-shell/src/main/ruby/hbase.rb:36)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311607#comment-15311607
]
Dima Spivak commented on HBASE-12721:
-
Please do! Here are some instructions to get started while I continue to hammer
out a proper patch:
# Install Docker 1.11 on a machine. I did the development and testing of this
under an Ubuntu 14.04 AWS instance with a 3.13.0-85 kernel (earlier Ubuntu
kernels might have problems), so something similar would be the surest way not
to run into issues. If you have a cloud instance at your disposal, you can
simplify setup by taking advantage of a framework script that I've temporarily
uploaded to Gist (I ran it through bit.ly because of the long raw URL, but I
won't be offended if you read through the script before running it). If using
the script, spin up your single cloud instance and then run {{curl -sL
http://bit.ly/20UntA8}}, which will install Docker, update the host's root
user's SSH keys to let you log into the containers we're going to start up, and
generally make your life easier. *Note that this script copies the SSH keys
from the Docker images we're creating into the {{~root/.ssh folder}} on your
host, so don't use this script on a machine with an existing SSH keypair in
{{~root/.ssh}}.*
# If you ran my setup script, run {{source /usr/local/bin/clusterdock.sh}} on
the host machine to enable use of {{clusterdock_run}}, a small helper function
that makes the framework super easy to use. _Instead of dealing with
virtualenvs for this Python project, we run it out of a Docker container_. If
you already had Docker installed and just want to source the script from Gist,
run
{noformat}
curl -sL http://bit.ly/1ROTmE3 | source /dev/stdin
{noformat}
# Stand up an HBase cluster. For the purpose of a quick demo, I uploaded images
I built with this framework to my personal Docker Hub acount. To get a running
4-node HBase cluster with Oracle Java 8u91, Hadoop 2.7.2, and the most recent
build of HBase {{master}} as of when I built it an hour ago, just type:
{noformat}
clusterdock_run ./bin/start_cluster apache_hbase --primary-node=node-1
--secondary-nodes='node-{2..4}' --hbase-version=master --start-services
{noformat}
That's it. Running that will pull down the necessary HBase image (also built
with this framework), setup some configs (i.e. you could have created a 5-node
cluster by changing the Bash-expandable argument to {{\-\-secondary-nodes}}),
and then start up services. Standard out will give information on what's
happening and how to access various web UIs, but you should also be able to SSH
to nodes by name (assuming you have the key-pairs setup) by using the hostnames
shown during startup. Other things to try:
- Stop and remove all running containers:
{noformat}
clusterdock_run ./bin/housekeeping nuke
{noformat}
- Build your own HBase cluster images:
{noformat}
clusterdock_run ./bin/build_cluster apache_hbase --hbase-version=master
--hbase-git-commit=master
{noformat}
- Learn more about options a particular script provides:
{noformat}
clusterdock_run ./bin/start_cluster -h
{noformat}
Let me know if you run into any issues!
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15938) submit-patch.py: Don't crash if there are tests with same name. Refactor: Split out html template to separate file.
[ https://issues.apache.org/jira/browse/HBASE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15938: - Status: Patch Available (was: Open) > submit-patch.py: Don't crash if there are tests with same name. Refactor: > Split out html template to separate file. > --- > > Key: HBASE-15938 > URL: https://issues.apache.org/jira/browse/HBASE-15938 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-15938.master.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15938) submit-patch.py: Don't crash if there are tests with same name. Refactor: Split out html template to separate file.
[ https://issues.apache.org/jira/browse/HBASE-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15938: - Attachment: HBASE-15938.master.001.patch > submit-patch.py: Don't crash if there are tests with same name. Refactor: > Split out html template to separate file. > --- > > Key: HBASE-15938 > URL: https://issues.apache.org/jira/browse/HBASE-15938 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-15938.master.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-15858) Some region server group shell commands don't work
[
https://issues.apache.org/jira/browse/HBASE-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311483#comment-15311483
]
Ted Yu edited comment on HBASE-15858 at 6/2/16 2:06 AM:
With the change, the following test passes:
{code}
Running org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.183 sec - in
org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups
{code}
The change is orthogonal to TestShell.
was (Author: yuzhih...@gmail.com):
With the change, the following test passes:
{code}
Running org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.183 sec - in
org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups
{code}
TestShell passes too.
> Some region server group shell commands don't work
> --
>
> Key: HBASE-15858
> URL: https://issues.apache.org/jira/browse/HBASE-15858
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15858.v1.patch, 15858.v2.patch, 15858.v3.patch
>
>
> [~cartershanklin] reported the following:
> {code}
> hbase(main):021:0> get_rsgroup 'test_group'
> GROUP INFORMATION
> ERROR: undefined local variable or method `group_admin' for
> #
> Here is some help for this command:
> Get a region server group's information.
> Example:
> hbase> get_rsgroup 'default'
> {code}
> {code}
> hbase(main):009:0> get_server_rsgroup
> 'hdp250-hbase-unsecure.example.com:16030'
> ERROR: undefined method `getGroupOfServer' for
> #
> Here is some help for this command:
> Get the group name the given region server is a member of.
> hbase> get_server_rsgroup 'server1:port1'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15938) submit-patch.py: Don't crash if there are tests with same name. Refactor: Split out html template to separate file.
Appy created HBASE-15938: Summary: submit-patch.py: Don't crash if there are tests with same name. Refactor: Split out html template to separate file. Key: HBASE-15938 URL: https://issues.apache.org/jira/browse/HBASE-15938 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15913) Sasl encryption doesn't work with AsyncRpcChannel
[ https://issues.apache.org/jira/browse/HBASE-15913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311579#comment-15311579 ] Colin Ma commented on HBASE-15913: -- [~ghelmling], can you help to check this patch? Thanks. > Sasl encryption doesn't work with AsyncRpcChannel > - > > Key: HBASE-15913 > URL: https://issues.apache.org/jira/browse/HBASE-15913 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Colin Ma >Assignee: Colin Ma > Fix For: 1.4.0 > > Attachments: HBASE-15913.001-branch-1.patch > > > The async RPC code in branch-1 will have problems with qop != auth. It's the > same as HBASE-15830. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15381) Implement a distributed MOB compaction by procedure
[ https://issues.apache.org/jira/browse/HBASE-15381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311570#comment-15311570 ] Jingcheng Du commented on HBASE-15381: -- Hi [~jmhsieh], do you mind another round of review? I've posted a new patch to RB. Thanks a lot! > Implement a distributed MOB compaction by procedure > --- > > Key: HBASE-15381 > URL: https://issues.apache.org/jira/browse/HBASE-15381 > Project: HBase > Issue Type: Improvement > Components: mob >Reporter: Jingcheng Du >Assignee: Jingcheng Du > Attachments: HBASE-15381-v2.patch, HBASE-15381-v3.patch, > HBASE-15381-v4.patch, HBASE-15381-v5.patch, HBASE-15381-v6.patch, > HBASE-15381.patch, mob distributed compaction design-v2.pdf, mob distributed > compaction design.pdf > > > In MOB, there is a periodical compaction which runs in HMaster (It can be > disabled by configuration), some small mob files are merged into bigger ones. > Now the compaction only runs in HMaster which is not efficient and might > impact the running of HMaster. In this JIRA, a distributed MOB compaction is > introduced, it is triggered by HMaster, but all the compaction jobs are > distributed to HRegionServers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak updated HBASE-12721:
Summary: Create Docker container cluster infrastructure to enable better
testing (was: Extend hbase_docker to handle more useful use cases)
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15937) Figure out retry limit and timing for replication queue table operations
Joseph created HBASE-15937: -- Summary: Figure out retry limit and timing for replication queue table operations Key: HBASE-15937 URL: https://issues.apache.org/jira/browse/HBASE-15937 Project: HBase Issue Type: Sub-task Reporter: Joseph ReplicationQueuesHBaseImpl will abort the server if any of its HBase Table writes/reads fails. We should figure out a reasonable retry limit and pause duration for these operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-15937) Figure out retry limit and timing for replication queue table operations
[ https://issues.apache.org/jira/browse/HBASE-15937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph reassigned HBASE-15937: -- Assignee: Joseph > Figure out retry limit and timing for replication queue table operations > > > Key: HBASE-15937 > URL: https://issues.apache.org/jira/browse/HBASE-15937 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > > ReplicationQueuesHBaseImpl will abort the server if any of its HBase Table > writes/reads fails. We should figure out a reasonable retry limit and pause > duration for these operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15429) Add a split policy for busy regions
[ https://issues.apache.org/jira/browse/HBASE-15429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashu Pachauri updated HBASE-15429: -- Attachment: HBASE-15429-V1.patch HBASE-15429-V1.patch: Removed the properties from HTableDescriptor. Modified the defaults to be more aggressive and added a description for the BusyRegionSplitPolicy class. > Add a split policy for busy regions > --- > > Key: HBASE-15429 > URL: https://issues.apache.org/jira/browse/HBASE-15429 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15429-V1.patch, HBASE-15429.patch > > > Currently, all region split policies are based on size. However, in certain > cases, it is a wise choice to make a split decision based on number of > requests to the region and split busy regions. > A crude metric is that if a region blocks writes often and throws > RegionTooBusyExceoption, it's probably a good idea to split it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15858) Some region server group shell commands don't work
[
https://issues.apache.org/jira/browse/HBASE-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311490#comment-15311490
]
Appy commented on HBASE-15858:
--
+1.
> Some region server group shell commands don't work
> --
>
> Key: HBASE-15858
> URL: https://issues.apache.org/jira/browse/HBASE-15858
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15858.v1.patch, 15858.v2.patch, 15858.v3.patch
>
>
> [~cartershanklin] reported the following:
> {code}
> hbase(main):021:0> get_rsgroup 'test_group'
> GROUP INFORMATION
> ERROR: undefined local variable or method `group_admin' for
> #
> Here is some help for this command:
> Get a region server group's information.
> Example:
> hbase> get_rsgroup 'default'
> {code}
> {code}
> hbase(main):009:0> get_server_rsgroup
> 'hdp250-hbase-unsecure.example.com:16030'
> ERROR: undefined method `getGroupOfServer' for
> #
> Here is some help for this command:
> Get the group name the given region server is a member of.
> hbase> get_server_rsgroup 'server1:port1'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15858) Some region server group shell commands don't work
[
https://issues.apache.org/jira/browse/HBASE-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311483#comment-15311483
]
Ted Yu commented on HBASE-15858:
With the change, the following test passes:
{code}
Running org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.183 sec - in
org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups
{code}
TestShell passes too.
> Some region server group shell commands don't work
> --
>
> Key: HBASE-15858
> URL: https://issues.apache.org/jira/browse/HBASE-15858
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15858.v1.patch, 15858.v2.patch, 15858.v3.patch
>
>
> [~cartershanklin] reported the following:
> {code}
> hbase(main):021:0> get_rsgroup 'test_group'
> GROUP INFORMATION
> ERROR: undefined local variable or method `group_admin' for
> #
> Here is some help for this command:
> Get a region server group's information.
> Example:
> hbase> get_rsgroup 'default'
> {code}
> {code}
> hbase(main):009:0> get_server_rsgroup
> 'hdp250-hbase-unsecure.example.com:16030'
> ERROR: undefined method `getGroupOfServer' for
> #
> Here is some help for this command:
> Get the group name the given region server is a member of.
> hbase> get_server_rsgroup 'server1:port1'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15936) Sort out how to push to apache/ repositories on Docker Hub
[ https://issues.apache.org/jira/browse/HBASE-15936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311452#comment-15311452 ] Dima Spivak commented on HBASE-15936: - Adding link to INFRA JIRA. > Sort out how to push to apache/ repositories on Docker Hub > -- > > Key: HBASE-15936 > URL: https://issues.apache.org/jira/browse/HBASE-15936 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Dima Spivak >Assignee: Dima Spivak > Labels: docker > > It looks like there's already a [Docker Hub username for Apache > stuff|https://hub.docker.com/r/apache/]. We should figure out how to use it > to push the Docker images we use/build for testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15932) Shell test fails due to uninitialized constant
[
https://issues.apache.org/jira/browse/HBASE-15932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311451#comment-15311451
]
Hudson commented on HBASE-15932:
FAILURE: Integrated in HBase-Trunk_matrix #968 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/968/])
HBASE-15932 Shell test fails due to uninitialized constant (tedyu: rev
cbb95cd3a9bf9a9f8558560ae58f4061a73f15a8)
* hbase-shell/src/main/ruby/hbase.rb
* hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
> Shell test fails due to uninitialized constant
> --
>
> Key: HBASE-15932
> URL: https://issues.apache.org/jira/browse/HBASE-15932
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0
>
> Attachments: 15932.v1.txt
>
>
> In master branch, shell tests fail due to IN_MEMORY_COMPACTION not being
> found:
> {code}
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 32.299 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups
> testRunShellTests(org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups)
> Time elapsed: 0.069 sec <<< ERROR!
> org.jruby.embed.EvalFailedException: (NameError) uninitialized constant
> Java::OrgApacheHadoopHbase::HConstants::IN_MEMORY_COMPACTION
> at
> org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:136)
> at
> org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1263)
> at
> org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1308)
> at
> org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups.testRunShellTests(TestShellRSGroups.java:101)
> Caused by: org.jruby.exceptions.RaiseException: (NameError) uninitialized
> constant Java::OrgApacheHadoopHbase::HConstants::IN_MEMORY_COMPACTION
> at org.jruby.RubyModule.const_missing(org/jruby/RubyModule.java:2647)
> at
> Module.const_missing(file:/home/jenkins/yetus-m2/hbase-master-1/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/META-INF/jruby.home/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2503)
> at
> Module.HBaseConstants(/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/hbase-shell/src/main/ruby/hbase.rb:42)
> at
> (Anonymous).(root)(/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/hbase-shell/src/main/ruby/hbase.rb:34)
> at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1062)
> at
> Kernel.require(/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/hbase-shell/src/main/ruby/hbase.rb:36)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15932) Shell test fails due to uninitialized constant
[
https://issues.apache.org/jira/browse/HBASE-15932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311439#comment-15311439
]
Appy commented on HBASE-15932:
--
+1
Sad that the test was failing because of this unused constant. If you like,
replace the full {{HColumnDescriptor::IN_MEMORY_COMPACTION}} classpath in
admin.rb with this constant.
Also this change is no-op wrt to this failure. But good to make it public since
all other variables are.
{noformat}
public static final String IN_MEMORY_COMPACTION
{noformat}
> Shell test fails due to uninitialized constant
> --
>
> Key: HBASE-15932
> URL: https://issues.apache.org/jira/browse/HBASE-15932
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0
>
> Attachments: 15932.v1.txt
>
>
> In master branch, shell tests fail due to IN_MEMORY_COMPACTION not being
> found:
> {code}
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 32.299 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups
> testRunShellTests(org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups)
> Time elapsed: 0.069 sec <<< ERROR!
> org.jruby.embed.EvalFailedException: (NameError) uninitialized constant
> Java::OrgApacheHadoopHbase::HConstants::IN_MEMORY_COMPACTION
> at
> org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:136)
> at
> org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1263)
> at
> org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1308)
> at
> org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups.testRunShellTests(TestShellRSGroups.java:101)
> Caused by: org.jruby.exceptions.RaiseException: (NameError) uninitialized
> constant Java::OrgApacheHadoopHbase::HConstants::IN_MEMORY_COMPACTION
> at org.jruby.RubyModule.const_missing(org/jruby/RubyModule.java:2647)
> at
> Module.const_missing(file:/home/jenkins/yetus-m2/hbase-master-1/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/META-INF/jruby.home/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2503)
> at
> Module.HBaseConstants(/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/hbase-shell/src/main/ruby/hbase.rb:42)
> at
> (Anonymous).(root)(/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/hbase-shell/src/main/ruby/hbase.rb:34)
> at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1062)
> at
> Kernel.require(/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/hbase-shell/src/main/ruby/hbase.rb:36)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15936) Sort out how to push to apache/ repositories on Docker Hub
[ https://issues.apache.org/jira/browse/HBASE-15936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311437#comment-15311437 ] Dima Spivak commented on HBASE-15936: - Or perhaps we have both? Let me reach out to Apache Infra to see if we can have access to the public Docker Hub creds to push our releases and a separate less-public registry for testing stuff. > Sort out how to push to apache/ repositories on Docker Hub > -- > > Key: HBASE-15936 > URL: https://issues.apache.org/jira/browse/HBASE-15936 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Dima Spivak >Assignee: Dima Spivak > Labels: docker > > It looks like there's already a [Docker Hub username for Apache > stuff|https://hub.docker.com/r/apache/]. We should figure out how to use it > to push the Docker images we use/build for testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15858) Some region server group shell commands don't work
[
https://issues.apache.org/jira/browse/HBASE-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311402#comment-15311402
]
Appy commented on HBASE-15858:
--
TestShell is bad right now (http://hbase.x10host.com/flaky-tests/), so it
doesn't run as part of pre-commit.
maybe test it locally?
> Some region server group shell commands don't work
> --
>
> Key: HBASE-15858
> URL: https://issues.apache.org/jira/browse/HBASE-15858
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15858.v1.patch, 15858.v2.patch, 15858.v3.patch
>
>
> [~cartershanklin] reported the following:
> {code}
> hbase(main):021:0> get_rsgroup 'test_group'
> GROUP INFORMATION
> ERROR: undefined local variable or method `group_admin' for
> #
> Here is some help for this command:
> Get a region server group's information.
> Example:
> hbase> get_rsgroup 'default'
> {code}
> {code}
> hbase(main):009:0> get_server_rsgroup
> 'hdp250-hbase-unsecure.example.com:16030'
> ERROR: undefined method `getGroupOfServer' for
> #
> Here is some help for this command:
> Get the group name the given region server is a member of.
> hbase> get_server_rsgroup 'server1:port1'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15936) Sort out how to push to apache/ repositories on Docker Hub
[
https://issues.apache.org/jira/browse/HBASE-15936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311376#comment-15311376
]
Dima Spivak commented on HBASE-15936:
-
So my thoughts were that we can reserve the simply-named tags for official
releases, but then take liberties with longer tag names for our use if it means
simplicity in pushing and pulling images (and document the difference
extensively). e.g. {{docker pull apache/hbase}} could pull down a Docker-ized
version of the latest HBase repo, but {{docker pull
apache/hbase:centos6.6_branch1.2}} could grab down the image needed for testing
of distributed clusters in Docker.
> Sort out how to push to apache/ repositories on Docker Hub
> --
>
> Key: HBASE-15936
> URL: https://issues.apache.org/jira/browse/HBASE-15936
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Dima Spivak
>Assignee: Dima Spivak
> Labels: docker
>
> It looks like there's already a [Docker Hub username for Apache
> stuff|https://hub.docker.com/r/apache/]. We should figure out how to use it
> to push the Docker images we use/build for testing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15936) Sort out how to push to apache/ repositories on Docker Hub
[ https://issues.apache.org/jira/browse/HBASE-15936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311371#comment-15311371 ] Nick Dimiduk commented on HBASE-15936: -- Shouldn't a publicly-visible Apache repository be for official releases? > Sort out how to push to apache/ repositories on Docker Hub > -- > > Key: HBASE-15936 > URL: https://issues.apache.org/jira/browse/HBASE-15936 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Dima Spivak >Assignee: Dima Spivak > Labels: docker > > It looks like there's already a [Docker Hub username for Apache > stuff|https://hub.docker.com/r/apache/]. We should figure out how to use it > to push the Docker images we use/build for testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15858) Some region server group shell commands don't work
[
https://issues.apache.org/jira/browse/HBASE-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311361#comment-15311361
]
Ted Yu commented on HBASE-15858:
I ran the following with patch v3:
mvn clean install -DskipTests -DHBasePatchProcess -Dhadoop-two.version=2.4.1
Didn't see any error.
> Some region server group shell commands don't work
> --
>
> Key: HBASE-15858
> URL: https://issues.apache.org/jira/browse/HBASE-15858
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15858.v1.patch, 15858.v2.patch, 15858.v3.patch
>
>
> [~cartershanklin] reported the following:
> {code}
> hbase(main):021:0> get_rsgroup 'test_group'
> GROUP INFORMATION
> ERROR: undefined local variable or method `group_admin' for
> #
> Here is some help for this command:
> Get a region server group's information.
> Example:
> hbase> get_rsgroup 'default'
> {code}
> {code}
> hbase(main):009:0> get_server_rsgroup
> 'hdp250-hbase-unsecure.example.com:16030'
> ERROR: undefined method `getGroupOfServer' for
> #
> Here is some help for this command:
> Get the group name the given region server is a member of.
> hbase> get_server_rsgroup 'server1:port1'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15862) Backup - Delete- Restore does not restore deleted data
[ https://issues.apache.org/jira/browse/HBASE-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-15862: -- Attachment: HBASE-15862-v1.patch Patch v1. cc: [~tedyu] > Backup - Delete- Restore does not restore deleted data > -- > > Key: HBASE-15862 > URL: https://issues.apache.org/jira/browse/HBASE-15862 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-15862-v1.patch > > > This was discovered during testing. If we delete row after full backup and > perform immediately restore, the deleted row still remains deleted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15727) Canary Tool for Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311348#comment-15311348
]
Hadoop QA commented on HBASE-15727:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 26s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 86m 24s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 128m 37s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12807523/HBASE-15727.v5.patch |
| JIRA Issue | HBASE-15727 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cbb95cd |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/2083/testReport/ |
| modules | C:
[jira] [Commented] (HBASE-15858) Some region server group shell commands don't work
[
https://issues.apache.org/jira/browse/HBASE-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311332#comment-15311332
]
Hadoop QA commented on HBASE-15858:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 3s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 3s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 24s
{color} | {color:red} Patch causes 12 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 33s
{color} | {color:red} Patch causes 7 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 17s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.5.1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 15s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 18s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12807545/15858.v3.patch |
| JIRA Issue | HBASE-15858 |
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux pietas.apache.org 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT
Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cbb95cd |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/2085/testReport/ |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2085/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Some region server group shell commands don't work
> --
>
> Key: HBASE-15858
> URL: https://issues.apache.org/jira/browse/HBASE-15858
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15858.v1.patch, 15858.v2.patch, 15858.v3.patch
>
>
> [~cartershanklin] reported the following:
> {code}
> hbase(main):021:0> get_rsgroup 'test_group'
> GROUP INFORMATION
> ERROR: undefined local variable or method `group_admin' for
> #
> Here is some help for this command:
> Get a region server group's information.
> Example:
> hbase> get_rsgroup 'default'
> {code}
> {code}
> hbase(main):009:0> get_server_rsgroup
> 'hdp250-hbase-unsecure.example.com:16030'
> ERROR: undefined method `getGroupOfServer' for
> #
> Here
[jira] [Created] (HBASE-15936) Sort out how to push to apache/ repositories on Docker Hub
Dima Spivak created HBASE-15936: --- Summary: Sort out how to push to apache/ repositories on Docker Hub Key: HBASE-15936 URL: https://issues.apache.org/jira/browse/HBASE-15936 Project: HBase Issue Type: Sub-task Components: build Reporter: Dima Spivak Assignee: Dima Spivak It looks like there's already a [Docker Hub username for Apache stuff|https://hub.docker.com/r/apache/]. We should figure out how to use it to push the Docker images we use/build for testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13467) Prototype using GRPC as IPC mechanism
[ https://issues.apache.org/jira/browse/HBASE-13467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311310#comment-15311310 ] Louis Ryan commented on HBASE-13467: We're working on getting the GRPC 1.0 release out at which point Ill update my GRPC server impl. This will be done in conjunction with some of the zero-copy buffer support we've done in protobuf that stack is aware of. Probably mid-July. If someone has cycles I can share what I have so far > Prototype using GRPC as IPC mechanism > - > > Key: HBASE-13467 > URL: https://issues.apache.org/jira/browse/HBASE-13467 > Project: HBase > Issue Type: Improvement > Components: API >Affects Versions: 2.0.0 >Reporter: Louis Ryan >Priority: Minor > > GRPC provide an RPC layer for protocol buffers on top of Netty 4/5. This > could be used to replace the current internal implementation. > GRPC supports some advanced features like streaming, async, flow-control, > cancellation & timeout which might be useful > Will prototype on GitHub here if folks are interested > https://github.com/louiscryan/hbase -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12721) Extend hbase_docker to handle more useful use cases
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311299#comment-15311299
]
Andrew Purtell commented on HBASE-12721:
Nice! I'd like to try it out. Tomorrow
> Extend hbase_docker to handle more useful use cases
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15727) Canary Tool for Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311295#comment-15311295
]
Ted Yu commented on HBASE-15727:
lgtm
Minor comments:
{code}
109 * outputs some information about failure of latency.
{code}
failure of latency -> failure or latency
{code}
1161 } catch (InterruptedException e) {
{code}
Please restore interrupt status.
> Canary Tool for Zookeeper
> -
>
> Key: HBASE-15727
> URL: https://issues.apache.org/jira/browse/HBASE-15727
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: churro morales
>Assignee: churro morales
> Attachments: HBASE-15727-v1.patch, HBASE-15727-v2.patch,
> HBASE-15727-v3.patch, HBASE-15727.patch, HBASE-15727.v4.patch,
> HBASE-15727.v5.patch
>
>
> It would be nice to have the canary tool also monitor zookeeper. Something
> simple like doing a getData() call on zookeeper.znode.parent
> It would be nice to create clients for every instance in the quorum such that
> you could monitor overloaded or poor behaving instances.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13467) Prototype using GRPC as IPC mechanism
[ https://issues.apache.org/jira/browse/HBASE-13467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311285#comment-15311285 ] Vincent Poon commented on HBASE-13467: -- Any update on this? I was looking into using gRPC for streaming replication data, but it'd be nicer to have a full gRpcServer instead of doing a one-off for replication. > Prototype using GRPC as IPC mechanism > - > > Key: HBASE-13467 > URL: https://issues.apache.org/jira/browse/HBASE-13467 > Project: HBase > Issue Type: Improvement > Components: API >Affects Versions: 2.0.0 >Reporter: Louis Ryan >Priority: Minor > > GRPC provide an RPC layer for protocol buffers on top of Netty 4/5. This > could be used to replace the current internal implementation. > GRPC supports some advanced features like streaming, async, flow-control, > cancellation & timeout which might be useful > Will prototype on GitHub here if folks are interested > https://github.com/louiscryan/hbase -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15727) Canary Tool for Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311264#comment-15311264 ] Enis Soztutar commented on HBASE-15727: --- +1. Thanks. > Canary Tool for Zookeeper > - > > Key: HBASE-15727 > URL: https://issues.apache.org/jira/browse/HBASE-15727 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15727-v1.patch, HBASE-15727-v2.patch, > HBASE-15727-v3.patch, HBASE-15727.patch, HBASE-15727.v4.patch, > HBASE-15727.v5.patch > > > It would be nice to have the canary tool also monitor zookeeper. Something > simple like doing a getData() call on zookeeper.znode.parent > It would be nice to create clients for every instance in the quorum such that > you could monitor overloaded or poor behaving instances. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15858) Some region server group shell commands don't work
[
https://issues.apache.org/jira/browse/HBASE-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15858:
---
Assignee: Ted Yu
Status: Patch Available (was: Open)
> Some region server group shell commands don't work
> --
>
> Key: HBASE-15858
> URL: https://issues.apache.org/jira/browse/HBASE-15858
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15858.v1.patch, 15858.v2.patch, 15858.v3.patch
>
>
> [~cartershanklin] reported the following:
> {code}
> hbase(main):021:0> get_rsgroup 'test_group'
> GROUP INFORMATION
> ERROR: undefined local variable or method `group_admin' for
> #
> Here is some help for this command:
> Get a region server group's information.
> Example:
> hbase> get_rsgroup 'default'
> {code}
> {code}
> hbase(main):009:0> get_server_rsgroup
> 'hdp250-hbase-unsecure.example.com:16030'
> ERROR: undefined method `getGroupOfServer' for
> #
> Here is some help for this command:
> Get the group name the given region server is a member of.
> hbase> get_server_rsgroup 'server1:port1'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15858) Some region server group shell commands don't work
[
https://issues.apache.org/jira/browse/HBASE-15858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15858:
---
Attachment: 15858.v3.patch
> Some region server group shell commands don't work
> --
>
> Key: HBASE-15858
> URL: https://issues.apache.org/jira/browse/HBASE-15858
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
> Attachments: 15858.v1.patch, 15858.v2.patch, 15858.v3.patch
>
>
> [~cartershanklin] reported the following:
> {code}
> hbase(main):021:0> get_rsgroup 'test_group'
> GROUP INFORMATION
> ERROR: undefined local variable or method `group_admin' for
> #
> Here is some help for this command:
> Get a region server group's information.
> Example:
> hbase> get_rsgroup 'default'
> {code}
> {code}
> hbase(main):009:0> get_server_rsgroup
> 'hdp250-hbase-unsecure.example.com:16030'
> ERROR: undefined method `getGroupOfServer' for
> #
> Here is some help for this command:
> Get the group name the given region server is a member of.
> hbase> get_server_rsgroup 'server1:port1'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12721) Extend hbase_docker to handle more useful use cases
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311246#comment-15311246
]
Dima Spivak commented on HBASE-12721:
-
This is ready to go. I've put together a lightweight Python framework that uses
{{docker-py}} to orchestrate starting multi-container HBase clusters, handle
configurations, and successfully run the tests in hbase-it. Let me get a Review
Board up.
> Extend hbase_docker to handle more useful use cases
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15887) Report Log Additions and Removals in Builds
[
https://issues.apache.org/jira/browse/HBASE-15887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311226#comment-15311226
]
Hadoop QA commented on HBASE-15887:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 5s
{color} | {color:blue} Precommit patch detected. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for
instructions. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 3s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m 6s
{color} | {color:red} The applied patch generated 1 new + 416 unchanged - 0
fixed = 417 total (was 416) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 29s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 6s {color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12806010/HBASE-15887-v1.txt |
| JIRA Issue | HBASE-15887 |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux asf910.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cbb95cd |
| shellcheck | v0.3.3 (This is an old version that has serious bugs. Consider
upgrading.) |
| shellcheck |
https://builds.apache.org/job/PreCommit-HBASE-Build/2084/artifact/patchprocess/diff-patch-shellcheck.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2084/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Report Log Additions and Removals in Builds
> ---
>
> Key: HBASE-15887
> URL: https://issues.apache.org/jira/browse/HBASE-15887
> Project: HBase
> Issue Type: New Feature
> Components: build
>Affects Versions: master
>Reporter: Clay B.
>Priority: Trivial
> Fix For: master
>
> Attachments: HBASE-15887-v1.txt
>
>
> It would be very nice for the Apache Yetus verifications of HBase patches to
> report log item addition and deletions.
> This is not my idea but [~mbm] asked if we could modify the personality for
> reporting log additions and removals yesterday at an [HBase meetup at Splice
> machine|http://www.meetup.com/hbaseusergroup/events/230547750/] as [~aw]
> presented Apache Yetus for building HBase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15887) Report Log Additions and Removals in Builds
[ https://issues.apache.org/jira/browse/HBASE-15887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311151#comment-15311151 ] Hadoop QA commented on HBASE-15887: --- (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HBASE-Build/2084/console in case of problems. > Report Log Additions and Removals in Builds > --- > > Key: HBASE-15887 > URL: https://issues.apache.org/jira/browse/HBASE-15887 > Project: HBase > Issue Type: New Feature > Components: build >Affects Versions: master >Reporter: Clay B. >Priority: Trivial > Fix For: master > > Attachments: HBASE-15887-v1.txt > > > It would be very nice for the Apache Yetus verifications of HBase patches to > report log item addition and deletions. > This is not my idea but [~mbm] asked if we could modify the personality for > reporting log additions and removals yesterday at an [HBase meetup at Splice > machine|http://www.meetup.com/hbaseusergroup/events/230547750/] as [~aw] > presented Apache Yetus for building HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15809) Basic Replication WebUI
[ https://issues.apache.org/jira/browse/HBASE-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311150#comment-15311150 ] huaxiang sun commented on HBASE-15809: -- Appy is going to work on it. > Basic Replication WebUI > --- > > Key: HBASE-15809 > URL: https://issues.apache.org/jira/browse/HBASE-15809 > Project: HBase > Issue Type: New Feature > Components: Replication, UI >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Appy >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15809-v0.patch, HBASE-15809-v0.png, > HBASE-15809-v1.patch > > > At the moment the only way to have some insight on replication from the webui > is looking at zkdump and metrics. > the basic information useful to get started debugging are: peer information > and the view of WALs offsets for each peer. > https://reviews.apache.org/r/47275/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15809) Basic Replication WebUI
[ https://issues.apache.org/jira/browse/HBASE-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-15809: - Assignee: Appy (was: huaxiang sun) > Basic Replication WebUI > --- > > Key: HBASE-15809 > URL: https://issues.apache.org/jira/browse/HBASE-15809 > Project: HBase > Issue Type: New Feature > Components: Replication, UI >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: Appy >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15809-v0.patch, HBASE-15809-v0.png, > HBASE-15809-v1.patch > > > At the moment the only way to have some insight on replication from the webui > is looking at zkdump and metrics. > the basic information useful to get started debugging are: peer information > and the view of WALs offsets for each peer. > https://reviews.apache.org/r/47275/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15887) Report Log Additions and Removals in Builds
[ https://issues.apache.org/jira/browse/HBASE-15887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clay B. updated HBASE-15887: Fix Version/s: master Affects Version/s: master Status: Patch Available (was: Open) Try uploading patch again. > Report Log Additions and Removals in Builds > --- > > Key: HBASE-15887 > URL: https://issues.apache.org/jira/browse/HBASE-15887 > Project: HBase > Issue Type: New Feature > Components: build >Affects Versions: master >Reporter: Clay B. >Priority: Trivial > Fix For: master > > Attachments: HBASE-15887-v1.txt > > > It would be very nice for the Apache Yetus verifications of HBase patches to > report log item addition and deletions. > This is not my idea but [~mbm] asked if we could modify the personality for > reporting log additions and removals yesterday at an [HBase meetup at Splice > machine|http://www.meetup.com/hbaseusergroup/events/230547750/] as [~aw] > presented Apache Yetus for building HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15887) Report Log Additions and Removals in Builds
[ https://issues.apache.org/jira/browse/HBASE-15887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311140#comment-15311140 ] Enis Soztutar commented on HBASE-15887: --- [~busbey] want to take a look? > Report Log Additions and Removals in Builds > --- > > Key: HBASE-15887 > URL: https://issues.apache.org/jira/browse/HBASE-15887 > Project: HBase > Issue Type: New Feature > Components: build >Reporter: Clay B. >Priority: Trivial > Attachments: HBASE-15887-v1.txt > > > It would be very nice for the Apache Yetus verifications of HBase patches to > report log item addition and deletions. > This is not my idea but [~mbm] asked if we could modify the personality for > reporting log additions and removals yesterday at an [HBase meetup at Splice > machine|http://www.meetup.com/hbaseusergroup/events/230547750/] as [~aw] > presented Apache Yetus for building HBase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15883) Adding WAL files and tracking offsets in HBase.
[
https://issues.apache.org/jira/browse/HBASE-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311141#comment-15311141
]
Hadoop QA commented on HBASE-15883:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 49s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 50s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 33s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 40s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 3m 31s {color}
| {color:red} hbase-client-jdk1.8.0 with JDK v1.8.0 generated 6 new + 0
unchanged - 0 fixed = 6 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 4m 39s {color}
| {color:red} hbase-client-jdk1.7.0_79 with JDK v1.7.0_79 generated 6 new + 0
unchanged - 0 fixed = 6 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 35s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 22s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 4m 49s
{color} | {color:red} hbase-client-jdk1.7.0_79 with JDK v1.7.0_79 generated 1
new + 13 unchanged - 0 fixed = 14 total (was 13) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 49s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 94m 31s
{color} | {color:green}
[jira] [Assigned] (HBASE-15935) Add concurrency support for Generator and Verify
[ https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph reassigned HBASE-15935: -- Assignee: Joseph > Add concurrency support for Generator and Verify > > > Key: HBASE-15935 > URL: https://issues.apache.org/jira/browse/HBASE-15935 > Project: HBase > Issue Type: Sub-task > Components: integration tests >Reporter: Joseph >Assignee: Joseph >Priority: Minor > > Adding support for concurrent generation and verification. Will largely be > copying over the changes from > https://github.com/keith-turner/goraci/commit/9ae4323c3e3ae6bbc4746bcc313a97893c0368c9 > into the HBase version of the test -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15935) Add concurrency support for Generator and Verify
Joseph created HBASE-15935: -- Summary: Add concurrency support for Generator and Verify Key: HBASE-15935 URL: https://issues.apache.org/jira/browse/HBASE-15935 Project: HBase Issue Type: Sub-task Reporter: Joseph Priority: Minor Adding support for concurrent generation and verification. Will largely be copying over the changes from https://github.com/keith-turner/goraci/commit/9ae4323c3e3ae6bbc4746bcc313a97893c0368c9 into the HBase version of the test -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15727) Canary Tool for Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-15727: --- Attachment: HBASE-15727.v5.patch [~enis] here is the patch with the port fix > Canary Tool for Zookeeper > - > > Key: HBASE-15727 > URL: https://issues.apache.org/jira/browse/HBASE-15727 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15727-v1.patch, HBASE-15727-v2.patch, > HBASE-15727-v3.patch, HBASE-15727.patch, HBASE-15727.v4.patch, > HBASE-15727.v5.patch > > > It would be nice to have the canary tool also monitor zookeeper. Something > simple like doing a getData() call on zookeeper.znode.parent > It would be nice to create clients for every instance in the quorum such that > you could monitor overloaded or poor behaving instances. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15809) Basic Replication WebUI
[ https://issues.apache.org/jira/browse/HBASE-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311094#comment-15311094 ] huaxiang sun commented on HBASE-15809: -- I will work on this one, thanks. > Basic Replication WebUI > --- > > Key: HBASE-15809 > URL: https://issues.apache.org/jira/browse/HBASE-15809 > Project: HBase > Issue Type: New Feature > Components: Replication, UI >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: huaxiang sun >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15809-v0.patch, HBASE-15809-v0.png, > HBASE-15809-v1.patch > > > At the moment the only way to have some insight on replication from the webui > is looking at zkdump and metrics. > the basic information useful to get started debugging are: peer information > and the view of WALs offsets for each peer. > https://reviews.apache.org/r/47275/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-15809) Basic Replication WebUI
[ https://issues.apache.org/jira/browse/HBASE-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun reassigned HBASE-15809: Assignee: huaxiang sun > Basic Replication WebUI > --- > > Key: HBASE-15809 > URL: https://issues.apache.org/jira/browse/HBASE-15809 > Project: HBase > Issue Type: New Feature > Components: Replication, UI >Affects Versions: 2.0.0 >Reporter: Matteo Bertozzi >Assignee: huaxiang sun >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-15809-v0.patch, HBASE-15809-v0.png, > HBASE-15809-v1.patch > > > At the moment the only way to have some insight on replication from the webui > is looking at zkdump and metrics. > the basic information useful to get started debugging are: peer information > and the view of WALs offsets for each peer. > https://reviews.apache.org/r/47275/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15888) Extend HBASE-12769 for bulk load data replication
[
https://issues.apache.org/jira/browse/HBASE-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311092#comment-15311092
]
Ted Yu commented on HBASE-15888:
Looks good overall.
{code}
172 private void fixUnDeletedHfileRefsQueue() throws IOException {
{code}
Please make 'f' in Hfile uppercase to be consistent with other method name.
> Extend HBASE-12769 for bulk load data replication
> -
>
> Key: HBASE-15888
> URL: https://issues.apache.org/jira/browse/HBASE-15888
> Project: HBase
> Issue Type: Task
> Components: Replication
>Affects Versions: 1.3.0
>Reporter: Ashish Singhi
>Assignee: Ashish Singhi
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15888.patch
>
>
> Include hfile-refs queue check and fix also in hbck -fixReplication command
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15727) Canary Tool for Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311076#comment-15311076
]
Hadoop QA commented on HBASE-15727:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 85m 14s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 127m 8s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12807492/HBASE-15727.v4.patch |
| JIRA Issue | HBASE-15727 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cbb95cd |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/2081/testReport/ |
| modules | C:
[jira] [Comment Edited] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[
https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311070#comment-15311070
]
Jurriaan Mous edited comment on HBASE-15921 at 6/1/16 8:42 PM:
---
{quote}
This has to be this way? (Lots of abstract classes...)
public abstract class AsyncRegionServerCallable extends
AbstractRegionServerCallable
It probably has to be given you are coming into a convoluted hierarchy that has
accreted over a long period of time. Was just wondering if could have a
shallower hierarchy. No issue if can't be done easily... or has to wait till
later after you've gotten your async client in.
Or, you just moved this existing class out to its own file?
{quote}
I have to add the abstract classes right now to share code between the sync and
async variants. As usage of the sync variants are replaced with async variants
we can remove the abstract versions again.
bq. AsyncTable returns Future only? Not CompletableFuture? Consumers won't be
able to consume AsyncTable in an event-driven way? We need a callback?
This is indeed as [~Apache9] mentions the Netty and now also HBase variant with
event driven methods.
bq. Why we let out EventExecutor? Especially given it a netty class. Can we
contain the fact that the implementation is betty-based?
At the moment I use the event executors for (new/failed/succes) promise
creation. I can take a fresh look at it to see if I can contain it more but am
not sure if it is possible.
bq. In successful future it has a class comment "25 * A Failed Response
future"
Will fix it.
bq. Any chance of a note on how the PromiseKeepers work?
Yes I will. It is the async way to combine multiple calls. So it replaces the
batch callback in places.
bq. The replacement of HTable by TableImpl comes later?
It is already replaced at getTable in Connection in this patch. I want to make
sure the performance is ok and then I find a solution for the remaining HTable
refs.
Created review: https://reviews.apache.org/r/48152/
was (Author: jurmous):
{quote}
This has to be this way? (Lots of abstract classes...)
public abstract class AsyncRegionServerCallable extends
AbstractRegionServerCallable
It probably has to be given you are coming into a convoluted hierarchy that has
accreted over a long period of time. Was just wondering if could have a
shallower hierarchy. No issue if can't be done easily... or has to wait till
later after you've gotten your async client in.
Or, you just moved this existing class out to its own file?
{quote}
I have to add the abstract classes right now to share code between the sync and
async variants. As usage of the sync variants are replaced with async variants
we can remove the abstract versions again.
bq. AsyncTable returns Future only? Not CompletableFuture? Consumers won't be
able to consume AsyncTable in an event-driven way? We need a callback?
This is indeed as [~Apache9] mentions the Netty variant with event driven
methods.
bq. Why we let out EventExecutor? Especially given it a netty class. Can we
contain the fact that the implementation is betty-based?
At the moment I use the event executors for (new/failed/succes) promise
creation. I can take a fresh look at it to see if I can contain it more but am
not sure if it is possible.
bq. In successful future it has a class comment "25 * A Failed Response
future"
Will fix it.
bq. Any chance of a note on how the PromiseKeepers work?
Yes I will. It is the async way to combine multiple calls. So it replaces the
batch callback in places.
bq. The replacement of HTable by TableImpl comes later?
It is already replaced at getTable in Connection in this patch. I want to make
sure the performance is ok and then I find a solution for the remaining HTable
refs.
Created review: https://reviews.apache.org/r/48152/
> Add first AsyncTable impl and create TableImpl based on it
> --
>
> Key: HBASE-15921
> URL: https://issues.apache.org/jira/browse/HBASE-15921
> Project: HBase
> Issue Type: Improvement
>Reporter: Jurriaan Mous
>Assignee: Jurriaan Mous
> Attachments: HBASE-15921.patch, HBASE-15921.v1.patch
>
>
> First we create an AsyncTable interface with implementation without the Scan
> functionality. Those will land in a separate patch since they need a refactor
> of existing scans.
> Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl
> internally and should be a bit faster because it does jump through less hoops
> to do ProtoBuf transportation. This way we can run all existing tests on the
> AsyncTableImpl to guarantee its quality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[
https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311070#comment-15311070
]
Jurriaan Mous commented on HBASE-15921:
---
{quote}
This has to be this way? (Lots of abstract classes...)
public abstract class AsyncRegionServerCallable extends
AbstractRegionServerCallable
It probably has to be given you are coming into a convoluted hierarchy that has
accreted over a long period of time. Was just wondering if could have a
shallower hierarchy. No issue if can't be done easily... or has to wait till
later after you've gotten your async client in.
Or, you just moved this existing class out to its own file?
{quote}
I have to add the abstract classes right now to share code between the sync and
async variants. As usage of the sync variants are replaced with async variants
we can remove the abstract versions again.
bq. AsyncTable returns Future only? Not CompletableFuture? Consumers won't be
able to consume AsyncTable in an event-driven way? We need a callback?
This is indeed as [~Apache9] mentions the Netty variant with event driven
methods.
bq. Why we let out EventExecutor? Especially given it a netty class. Can we
contain the fact that the implementation is betty-based?
At the moment I use the event executors for (new/failed/succes) promise
creation. I can take a fresh look at it to see if I can contain it more but am
not sure if it is possible.
bq. In successful future it has a class comment "25 * A Failed Response
future"
Will fix it.
bq. Any chance of a note on how the PromiseKeepers work?
Yes I will. It is the async way to combine multiple calls. So it replaces the
batch callback in places.
bq. The replacement of HTable by TableImpl comes later?
It is already replaced at getTable in Connection in this patch. I want to make
sure the performance is ok and then I find a solution for the remaining HTable
refs.
Created review: https://reviews.apache.org/r/48152/
> Add first AsyncTable impl and create TableImpl based on it
> --
>
> Key: HBASE-15921
> URL: https://issues.apache.org/jira/browse/HBASE-15921
> Project: HBase
> Issue Type: Improvement
>Reporter: Jurriaan Mous
>Assignee: Jurriaan Mous
> Attachments: HBASE-15921.patch, HBASE-15921.v1.patch
>
>
> First we create an AsyncTable interface with implementation without the Scan
> functionality. Those will land in a separate patch since they need a refactor
> of existing scans.
> Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl
> internally and should be a bit faster because it does jump through less hoops
> to do ProtoBuf transportation. This way we can run all existing tests on the
> AsyncTableImpl to guarantee its quality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
[ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311064#comment-15311064 ] Andrew Purtell commented on HBASE-11625: If you could do that 1.1 patch without HBASE-11927 I might be able to take the result a further delta back into 0.98 - at least there's a possibility. > Reading datablock throws "Invalid HFile block magic" and can not switch to > hdfs checksum > - > > Key: HBASE-11625 > URL: https://issues.apache.org/jira/browse/HBASE-11625 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.21, 0.98.4, 0.98.5, 1.0.1.1, 1.0.3 >Reporter: qian wang >Assignee: Appy > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, > HBASE-11625-branch-1-v1.patch, HBASE-11625-branch-1.2-v1.patch, > HBASE-11625-branch-1.2-v2.patch, HBASE-11625-branch-1.2-v3.patch, > HBASE-11625-branch-1.2-v4.patch, HBASE-11625-master-v2.patch, > HBASE-11625-master-v3.patch, HBASE-11625-master.patch, HBASE-11625.patch, > correct-hfile, corrupted-header-hfile > > > when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it > could happen file corruption but it only can switch to hdfs checksum > inputstream till validateBlockChecksum(). If the datablock's header corrupted > when b = new HFileBlock(),it throws the exception "Invalid HFile block magic" > and the rpc call fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15893) Get object
[ https://issues.apache.org/jira/browse/HBASE-15893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311046#comment-15311046 ] Elliott Clark commented on HBASE-15893: --- You don't need to abstract it out if you never make the promises. > Get object > -- > > Key: HBASE-15893 > URL: https://issues.apache.org/jira/browse/HBASE-15893 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar > Attachments: HBASE-15893.HBASE-14850.v1.patch > > > Patch for creating Get objects. Get objects can be passed to the Table > implementation to fetch results for a given row. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15727) Canary Tool for Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311040#comment-15311040 ] Enis Soztutar commented on HBASE-15727: --- Indeed. Otherwise, if client port is not 2181, this will not work. > Canary Tool for Zookeeper > - > > Key: HBASE-15727 > URL: https://issues.apache.org/jira/browse/HBASE-15727 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15727-v1.patch, HBASE-15727-v2.patch, > HBASE-15727-v3.patch, HBASE-15727.patch, HBASE-15727.v4.patch > > > It would be nice to have the canary tool also monitor zookeeper. Something > simple like doing a getData() call on zookeeper.znode.parent > It would be nice to create clients for every instance in the quorum such that > you could monitor overloaded or poor behaving instances. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
[ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311005#comment-15311005 ] Appy commented on HBASE-11625: -- Should be able to backport without it too. Just that would have been easier with it. Is it fine if i put a patch by eow? > Reading datablock throws "Invalid HFile block magic" and can not switch to > hdfs checksum > - > > Key: HBASE-11625 > URL: https://issues.apache.org/jira/browse/HBASE-11625 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.21, 0.98.4, 0.98.5, 1.0.1.1, 1.0.3 >Reporter: qian wang >Assignee: Appy > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, > HBASE-11625-branch-1-v1.patch, HBASE-11625-branch-1.2-v1.patch, > HBASE-11625-branch-1.2-v2.patch, HBASE-11625-branch-1.2-v3.patch, > HBASE-11625-branch-1.2-v4.patch, HBASE-11625-master-v2.patch, > HBASE-11625-master-v3.patch, HBASE-11625-master.patch, HBASE-11625.patch, > correct-hfile, corrupted-header-hfile > > > when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it > could happen file corruption but it only can switch to hdfs checksum > inputstream till validateBlockChecksum(). If the datablock's header corrupted > when b = new HFileBlock(),it throws the exception "Invalid HFile block magic" > and the rpc call fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15893) Get object
[ https://issues.apache.org/jira/browse/HBASE-15893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311012#comment-15311012 ] Enis Soztutar commented on HBASE-15893: --- bq. That's how bytes are passed back. If we want to have something like slice that's what IOBuf's are. That is fine if all you do is get with a key for a single cell. We still need a way to iterate over (rowKey,column+ts -> value) mappings within a given row, as a result of Get, or iterate over the results of Scan. Even levelDB's scan iterator is Slice based, however, unlike levelDB where there is only key, we have rowKey+cf+column+ts to represent somehow. https://github.com/google/leveldb/blob/master/include/leveldb/iterator.h#L65 bq. The only difference between cell and key value is what accessors are there. Cell was a hack to get around removing some that made promises around data layout. KV was a bad idea in the first place. It ties the underlying representation of the in-memory data model to a very specific representation. We should not make the same mistake and make the implementation know the returned representation format in the new client. bq. We have no such inconveniences around having already promised a static data layout, hence the cell/kv difference isn't needed. We should not "promise" any specific in-memory representation hence we still need a way to abstract an interface so that you can iterate over a single Get results, and still can inspect the column names, timestamps, etc. It does not necessarily be cell versus KV, but we need the abstraction in place. > Get object > -- > > Key: HBASE-15893 > URL: https://issues.apache.org/jira/browse/HBASE-15893 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar > Attachments: HBASE-15893.HBASE-14850.v1.patch > > > Patch for creating Get objects. Get objects can be passed to the Table > implementation to fetch results for a given row. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
[ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310990#comment-15310990 ] Enis Soztutar commented on HBASE-11625: --- Yep, I'd be -1 for HBASE-11927 in branch-1.1. This cannot be backported without it? > Reading datablock throws "Invalid HFile block magic" and can not switch to > hdfs checksum > - > > Key: HBASE-11625 > URL: https://issues.apache.org/jira/browse/HBASE-11625 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.21, 0.98.4, 0.98.5, 1.0.1.1, 1.0.3 >Reporter: qian wang >Assignee: Appy > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, > HBASE-11625-branch-1-v1.patch, HBASE-11625-branch-1.2-v1.patch, > HBASE-11625-branch-1.2-v2.patch, HBASE-11625-branch-1.2-v3.patch, > HBASE-11625-branch-1.2-v4.patch, HBASE-11625-master-v2.patch, > HBASE-11625-master-v3.patch, HBASE-11625-master.patch, HBASE-11625.patch, > correct-hfile, corrupted-header-hfile > > > when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it > could happen file corruption but it only can switch to hdfs checksum > inputstream till validateBlockChecksum(). If the datablock's header corrupted > when b = new HFileBlock(),it throws the exception "Invalid HFile block magic" > and the rpc call fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15928) hbase backup delete command does not remove backup root dir from hdfs
[
https://issues.apache.org/jira/browse/HBASE-15928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15928:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> hbase backup delete command does not remove backup root dir from hdfs
> -
>
> Key: HBASE-15928
> URL: https://issues.apache.org/jira/browse/HBASE-15928
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Labels: backup
> Attachments: 15928.v1.txt
>
>
> [~romil.choksi] reported the following bug.
> hbase backup delete command successfully deletes backup
> {code}
> hbase@hbase-backup-test-5:~> hbase backup delete backup_1464217940560
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Delete backup failed: no information found for backupID=delete
> 2016-05-26 01:44:40,077 INFO [main] impl.BackupUtil: No data has been found
> in
> hdfs://hbase-backup-test-5.openstacklocal:8020/user/hbase/backup_1464217940560/default/t1.
> 2016-05-26 01:44:40,081 INFO [main] impl.BackupUtil: No data has been found
> in
> hdfs://hbase-backup-test-5.openstacklocal:8020/user/hbase/backup_1464217940560/default/t2.
> 2016-05-26 01:44:40,085 INFO [main] impl.BackupUtil: No data has been found
> in
> hdfs://hbase-backup-test-5.openstacklocal:8020/user/hbase/backup_1464217940560/default/t3.
> Delete backup for backupID=backup_1464217940560 completed.
> {code}
> Listing the backup directory of the backup that was just deleted
> {code}
> hbase@hbase-backup-test-5:~> hdfs dfs -ls /user/hbase
> Found 37 items
> drwx-- - hbase hbase 0 2016-05-25 23:13 /user/hbase/.staging
> drwxr-xr-x - hbase hbase 0 2016-05-24 19:42
> /user/hbase/backup_1464047611132
>
> drwxr-xr-x - hbase hbase 0 2016-05-25 23:08
> /user/hbase/backup_1464217727296
> drwxr-xr-x - hbase hbase 0 2016-05-26 01:44
> /user/hbase/backup_1464217940560
> {code}
> Backup root dir still exists
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15933) NullPointerException may be thrown from SimpleRegionNormalizer#getRegionSize()
[
https://issues.apache.org/jira/browse/HBASE-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310979#comment-15310979
]
Ted Yu commented on HBASE-15933:
Planning to commit tomorrow morning if there is no further review comment.
> NullPointerException may be thrown from SimpleRegionNormalizer#getRegionSize()
> --
>
> Key: HBASE-15933
> URL: https://issues.apache.org/jira/browse/HBASE-15933
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15933.v1.txt
>
>
> The following was observed in master log:
> {code}
> 2016-05-26 00:09:54,716 INFO [AM.ZK.Worker-pool2-t6917] master.RegionStates:
> Transition {7669ce6bec40dd44bb2388ff4d610b7b state=CLOSED, ts=1464221394716,
> server=x.net,60020,1463615066136} to {7669ce6bec40dd44bb2388ff4d610b7b
> state=OFFLINE, ts=1464221394716, server=x.net,60020,1463615066136}
> 2016-05-26 00:09:54,717 ERROR [y.net,6,1463614725717_ChoreService_1]
> normalizer.RegionNormalizerChore: Caught error
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.normalizer.SimpleRegionNormalizer.getRegionSize(SimpleRegionNormalizer.java:174)
> at
> org.apache.hadoop.hbase.master.normalizer.SimpleRegionNormalizer.computePlanForTable(SimpleRegionNormalizer.java:99)
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1406)
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> {code}
> DEBUG was not turned on.
> Looks like race condition where the Map returned by getRegionsLoad() no
> longer contained certain region.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
[ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310972#comment-15310972 ] Nick Dimiduk commented on HBASE-11625: -- HBASE-11927 looks pretty invasive for a patch release. I'm getting cold feet now [~appy]. Maybe you or [~enis] could make a case for it? > Reading datablock throws "Invalid HFile block magic" and can not switch to > hdfs checksum > - > > Key: HBASE-11625 > URL: https://issues.apache.org/jira/browse/HBASE-11625 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.21, 0.98.4, 0.98.5, 1.0.1.1, 1.0.3 >Reporter: qian wang >Assignee: Appy > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, > HBASE-11625-branch-1-v1.patch, HBASE-11625-branch-1.2-v1.patch, > HBASE-11625-branch-1.2-v2.patch, HBASE-11625-branch-1.2-v3.patch, > HBASE-11625-branch-1.2-v4.patch, HBASE-11625-master-v2.patch, > HBASE-11625-master-v3.patch, HBASE-11625-master.patch, HBASE-11625.patch, > correct-hfile, corrupted-header-hfile > > > when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it > could happen file corruption but it only can switch to hdfs checksum > inputstream till validateBlockChecksum(). If the datablock's header corrupted > when b = new HFileBlock(),it throws the exception "Invalid HFile block magic" > and the rpc call fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15933) NullPointerException may be thrown from SimpleRegionNormalizer#getRegionSize()
[
https://issues.apache.org/jira/browse/HBASE-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310971#comment-15310971
]
Hadoop QA commented on HBASE-15933:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 2s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 50s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 127m 45s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 181m 55s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12807466/15933.v1.txt |
| JIRA Issue | HBASE-15933 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf910.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
[ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310967#comment-15310967 ] Appy commented on HBASE-11625: -- Okay. But first, it requires backporting HBASE-11927. > Reading datablock throws "Invalid HFile block magic" and can not switch to > hdfs checksum > - > > Key: HBASE-11625 > URL: https://issues.apache.org/jira/browse/HBASE-11625 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.21, 0.98.4, 0.98.5, 1.0.1.1, 1.0.3 >Reporter: qian wang >Assignee: Appy > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, > HBASE-11625-branch-1-v1.patch, HBASE-11625-branch-1.2-v1.patch, > HBASE-11625-branch-1.2-v2.patch, HBASE-11625-branch-1.2-v3.patch, > HBASE-11625-branch-1.2-v4.patch, HBASE-11625-master-v2.patch, > HBASE-11625-master-v3.patch, HBASE-11625-master.patch, HBASE-11625.patch, > correct-hfile, corrupted-header-hfile > > > when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it > could happen file corruption but it only can switch to hdfs checksum > inputstream till validateBlockChecksum(). If the datablock's header corrupted > when b = new HFileBlock(),it throws the exception "Invalid HFile block magic" > and the rpc call fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15727) Canary Tool for Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310938#comment-15310938
]
churro morales commented on HBASE-15727:
[~enis] were you thinking about something more along the lines of this:
{code} new
ConnectStringParser(ZkConfig.getZKQuorumServersStringFromHbaseConfig(conf))
{code}
> Canary Tool for Zookeeper
> -
>
> Key: HBASE-15727
> URL: https://issues.apache.org/jira/browse/HBASE-15727
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: churro morales
>Assignee: churro morales
> Attachments: HBASE-15727-v1.patch, HBASE-15727-v2.patch,
> HBASE-15727-v3.patch, HBASE-15727.patch, HBASE-15727.v4.patch
>
>
> It would be nice to have the canary tool also monitor zookeeper. Something
> simple like doing a getData() call on zookeeper.znode.parent
> It would be nice to create clients for every instance in the quorum such that
> you could monitor overloaded or poor behaving instances.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-15473) Documentation for the usage of hbase dataframe user api (JSON, Avro, etc)
[ https://issues.apache.org/jira/browse/HBASE-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiqing Yang reassigned HBASE-15473: Assignee: Weiqing Yang > Documentation for the usage of hbase dataframe user api (JSON, Avro, etc) > - > > Key: HBASE-15473 > URL: https://issues.apache.org/jira/browse/HBASE-15473 > Project: HBase > Issue Type: Sub-task > Components: documentation, spark >Reporter: Zhan Zhang >Assignee: Weiqing Yang >Priority: Blocker > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15727) Canary Tool for Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310923#comment-15310923 ] Enis Soztutar commented on HBASE-15727: --- Looks good. Does this work with different zk client ports? We are parsing the quorum string, but in HBase unfortunately, quorum string does not contain the port usually. > Canary Tool for Zookeeper > - > > Key: HBASE-15727 > URL: https://issues.apache.org/jira/browse/HBASE-15727 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15727-v1.patch, HBASE-15727-v2.patch, > HBASE-15727-v3.patch, HBASE-15727.patch, HBASE-15727.v4.patch > > > It would be nice to have the canary tool also monitor zookeeper. Something > simple like doing a getData() call on zookeeper.znode.parent > It would be nice to create clients for every instance in the quorum such that > you could monitor overloaded or poor behaving instances. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15934) Integration Test with Concurrent Read-Writes
Joseph created HBASE-15934: -- Summary: Integration Test with Concurrent Read-Writes Key: HBASE-15934 URL: https://issues.apache.org/jira/browse/HBASE-15934 Project: HBase Issue Type: Test Components: integration tests Reporter: Joseph Assignee: Joseph Write an integration test similar to Test-Big-Linked-List that runs many concurrent read/writes to better simulate a real workload. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
[ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310880#comment-15310880 ] Enis Soztutar commented on HBASE-11625: --- [~appy] mind committing to 1.1 as well? > Reading datablock throws "Invalid HFile block magic" and can not switch to > hdfs checksum > - > > Key: HBASE-11625 > URL: https://issues.apache.org/jira/browse/HBASE-11625 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.21, 0.98.4, 0.98.5, 1.0.1.1, 1.0.3 >Reporter: qian wang >Assignee: Appy > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, > HBASE-11625-branch-1-v1.patch, HBASE-11625-branch-1.2-v1.patch, > HBASE-11625-branch-1.2-v2.patch, HBASE-11625-branch-1.2-v3.patch, > HBASE-11625-branch-1.2-v4.patch, HBASE-11625-master-v2.patch, > HBASE-11625-master-v3.patch, HBASE-11625-master.patch, HBASE-11625.patch, > correct-hfile, corrupted-header-hfile > > > when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it > could happen file corruption but it only can switch to hdfs checksum > inputstream till validateBlockChecksum(). If the datablock's header corrupted > when b = new HFileBlock(),it throws the exception "Invalid HFile block magic" > and the rpc call fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15883) Adding WAL files and tracking offsets in HBase.
[ https://issues.apache.org/jira/browse/HBASE-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15883: --- Status: Patch Available (was: Open) > Adding WAL files and tracking offsets in HBase. > > > Key: HBASE-15883 > URL: https://issues.apache.org/jira/browse/HBASE-15883 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15883.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15883) Adding WAL files and tracking offsets in HBase.
[ https://issues.apache.org/jira/browse/HBASE-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15883: --- Attachment: (was: HBASE-15883.patch) > Adding WAL files and tracking offsets in HBase. > > > Key: HBASE-15883 > URL: https://issues.apache.org/jira/browse/HBASE-15883 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15883.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15883) Adding WAL files and tracking offsets in HBase.
[ https://issues.apache.org/jira/browse/HBASE-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-15883: --- Attachment: HBASE-15883.patch > Adding WAL files and tracking offsets in HBase. > > > Key: HBASE-15883 > URL: https://issues.apache.org/jira/browse/HBASE-15883 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Joseph >Assignee: Joseph > Attachments: HBASE-15883.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15727) Canary Tool for Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-15727: --- Attachment: (was: HBASE-15727.v4.patch) > Canary Tool for Zookeeper > - > > Key: HBASE-15727 > URL: https://issues.apache.org/jira/browse/HBASE-15727 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15727-v1.patch, HBASE-15727-v2.patch, > HBASE-15727-v3.patch, HBASE-15727.patch, HBASE-15727.v4.patch > > > It would be nice to have the canary tool also monitor zookeeper. Something > simple like doing a getData() call on zookeeper.znode.parent > It would be nice to create clients for every instance in the quorum such that > you could monitor overloaded or poor behaving instances. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15727) Canary Tool for Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-15727: --- Attachment: HBASE-15727.v4.patch > Canary Tool for Zookeeper > - > > Key: HBASE-15727 > URL: https://issues.apache.org/jira/browse/HBASE-15727 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15727-v1.patch, HBASE-15727-v2.patch, > HBASE-15727-v3.patch, HBASE-15727.patch, HBASE-15727.v4.patch > > > It would be nice to have the canary tool also monitor zookeeper. Something > simple like doing a getData() call on zookeeper.znode.parent > It would be nice to create clients for every instance in the quorum such that > you could monitor overloaded or poor behaving instances. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15727) Canary Tool for Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-15727: --- Attachment: HBASE-15727.v4.patch rebased > Canary Tool for Zookeeper > - > > Key: HBASE-15727 > URL: https://issues.apache.org/jira/browse/HBASE-15727 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15727-v1.patch, HBASE-15727-v2.patch, > HBASE-15727-v3.patch, HBASE-15727.patch, HBASE-15727.v4.patch > > > It would be nice to have the canary tool also monitor zookeeper. Something > simple like doing a getData() call on zookeeper.znode.parent > It would be nice to create clients for every instance in the quorum such that > you could monitor overloaded or poor behaving instances. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15933) NullPointerException may be thrown from SimpleRegionNormalizer#getRegionSize()
[
https://issues.apache.org/jira/browse/HBASE-15933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310836#comment-15310836
]
Stephen Yuan Jiang commented on HBASE-15933:
LGTM
> NullPointerException may be thrown from SimpleRegionNormalizer#getRegionSize()
> --
>
> Key: HBASE-15933
> URL: https://issues.apache.org/jira/browse/HBASE-15933
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15933.v1.txt
>
>
> The following was observed in master log:
> {code}
> 2016-05-26 00:09:54,716 INFO [AM.ZK.Worker-pool2-t6917] master.RegionStates:
> Transition {7669ce6bec40dd44bb2388ff4d610b7b state=CLOSED, ts=1464221394716,
> server=x.net,60020,1463615066136} to {7669ce6bec40dd44bb2388ff4d610b7b
> state=OFFLINE, ts=1464221394716, server=x.net,60020,1463615066136}
> 2016-05-26 00:09:54,717 ERROR [y.net,6,1463614725717_ChoreService_1]
> normalizer.RegionNormalizerChore: Caught error
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.normalizer.SimpleRegionNormalizer.getRegionSize(SimpleRegionNormalizer.java:174)
> at
> org.apache.hadoop.hbase.master.normalizer.SimpleRegionNormalizer.computePlanForTable(SimpleRegionNormalizer.java:99)
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1406)
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> {code}
> DEBUG was not turned on.
> Looks like race condition where the Map returned by getRegionsLoad() no
> longer contained certain region.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15928) hbase backup delete command does not remove backup root dir from hdfs
[
https://issues.apache.org/jira/browse/HBASE-15928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310806#comment-15310806
]
Vladimir Rodionov commented on HBASE-15928:
---
Looks OK to me.
> hbase backup delete command does not remove backup root dir from hdfs
> -
>
> Key: HBASE-15928
> URL: https://issues.apache.org/jira/browse/HBASE-15928
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Labels: backup
> Attachments: 15928.v1.txt
>
>
> [~romil.choksi] reported the following bug.
> hbase backup delete command successfully deletes backup
> {code}
> hbase@hbase-backup-test-5:~> hbase backup delete backup_1464217940560
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Delete backup failed: no information found for backupID=delete
> 2016-05-26 01:44:40,077 INFO [main] impl.BackupUtil: No data has been found
> in
> hdfs://hbase-backup-test-5.openstacklocal:8020/user/hbase/backup_1464217940560/default/t1.
> 2016-05-26 01:44:40,081 INFO [main] impl.BackupUtil: No data has been found
> in
> hdfs://hbase-backup-test-5.openstacklocal:8020/user/hbase/backup_1464217940560/default/t2.
> 2016-05-26 01:44:40,085 INFO [main] impl.BackupUtil: No data has been found
> in
> hdfs://hbase-backup-test-5.openstacklocal:8020/user/hbase/backup_1464217940560/default/t3.
> Delete backup for backupID=backup_1464217940560 completed.
> {code}
> Listing the backup directory of the backup that was just deleted
> {code}
> hbase@hbase-backup-test-5:~> hdfs dfs -ls /user/hbase
> Found 37 items
> drwx-- - hbase hbase 0 2016-05-25 23:13 /user/hbase/.staging
> drwxr-xr-x - hbase hbase 0 2016-05-24 19:42
> /user/hbase/backup_1464047611132
>
> drwxr-xr-x - hbase hbase 0 2016-05-25 23:08
> /user/hbase/backup_1464217727296
> drwxr-xr-x - hbase hbase 0 2016-05-26 01:44
> /user/hbase/backup_1464217940560
> {code}
> Backup root dir still exists
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15931) Add log for long-running tasks in AsyncProcess
[
https://issues.apache.org/jira/browse/HBASE-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310786#comment-15310786
]
stack commented on HBASE-15931:
---
+1
> Add log for long-running tasks in AsyncProcess
> --
>
> Key: HBASE-15931
> URL: https://issues.apache.org/jira/browse/HBASE-15931
> Project: HBase
> Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15931.patch
>
>
> Currently if there's any long-tail tasks in a multi-action request like
> triggered by {{BufferedMutatorImpl#backgroundFlushCommits}}, we could see
> logging message like below
> {noformat}
> 2016-05-31 09:36:55,461 INFO [Thread-16]
> org.apache.hadoop.hbase.client.AsyncProcess: #28, waiting for some tasks to
> finish. Expected max=0, tasksInProgress=1
> {noformat}
> but there's no way to know detail of this long-tail, such as which
> table/region it's accessing. This JIRA aims at adding such log for easier
> debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15584) Revisit handling of BackupState#CANCELLED
[
https://issues.apache.org/jira/browse/HBASE-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310758#comment-15310758
]
Ted Yu commented on HBASE-15584:
Suppose the procID / backupID mapping is written in the hbase:backup table
first, after that RPC call returns to the client.
This would endure the loss of RPC, right ?
> Revisit handling of BackupState#CANCELLED
> -
>
> Key: HBASE-15584
> URL: https://issues.apache.org/jira/browse/HBASE-15584
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Priority: Minor
>
> During review of HBASE-15411, Enis made the following point:
> {code}
> nobody puts the backup in cancelled state. setCancelled() is not used. So if
> I abort a backup, who writes to the system table the new state?
> Not sure whether this is a phase 1 patch issue or due to this patch. We can
> open a new jira and address it there if you do not want to do it in this
> patch.
> Also maybe this should be named ABORTED rather than CANCELLED.
> {code}
> This issue is to decide whether this state should be kept (e.g. through
> notification from procedure V2 framework in response to abortion).
> If it is to be kept, the state should be renamed ABORTED.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
[ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310759#comment-15310759 ] Nick Dimiduk commented on HBASE-11625: -- This backports to 1.1? > Reading datablock throws "Invalid HFile block magic" and can not switch to > hdfs checksum > - > > Key: HBASE-11625 > URL: https://issues.apache.org/jira/browse/HBASE-11625 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.21, 0.98.4, 0.98.5, 1.0.1.1, 1.0.3 >Reporter: qian wang >Assignee: Appy > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2 > > Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, > HBASE-11625-branch-1-v1.patch, HBASE-11625-branch-1.2-v1.patch, > HBASE-11625-branch-1.2-v2.patch, HBASE-11625-branch-1.2-v3.patch, > HBASE-11625-branch-1.2-v4.patch, HBASE-11625-master-v2.patch, > HBASE-11625-master-v3.patch, HBASE-11625-master.patch, HBASE-11625.patch, > correct-hfile, corrupted-header-hfile > > > when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it > could happen file corruption but it only can switch to hdfs checksum > inputstream till validateBlockChecksum(). If the datablock's header corrupted > when b = new HFileBlock(),it throws the exception "Invalid HFile block magic" > and the rpc call fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15584) Revisit handling of BackupState#CANCELLED
[
https://issues.apache.org/jira/browse/HBASE-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310751#comment-15310751
]
Ted Yu commented on HBASE-15584:
Recapping previous discussion from
https://reviews.apache.org/r/45149/#comment189160 :
>From myself:
{quote}
HMaster#abortBackup() looks up the procedure Id by backup Id and aborts the
procedure.
Using Future#cancel() is geared towards Java client.
After admin issues backup command (and receiving backupId), the backup process
may be quite long. Another admin command (say through shell) would be used to
cancel the backup. Since backupTables() API returns backupId to user, providing
API to cancel by backupId is more user friendly.
The user can still list procedures and cancel the selected backup procedure.
{quote}
>From Enis:
{quote}
if we save the procId in the BackupContext and do persist that in the system
table before returning back to the client with the backupId, then the client
API can accept abort(backupId) then it will look up backupId -> procId and call
abort(procId). Just a suggestion.
{quote}
>From Matteo:
{quote}
I still don't understand why we need two system that do the same thing.
the procId does already what the backupId does.
with the difference that we have already a mechanism to list the procedures in
progress and abort them. and I don't see that for backup yet.
assume the case. your client dies before being able to recive the rpc response
from the master or before being able to write the id. how do you look that up?
with the procedure you can already do it. I don't see the point to write the
same thing for backup and then have to do a reverse jump to abort the procedure
when you ask to abort the backup
{quote}
> Revisit handling of BackupState#CANCELLED
> -
>
> Key: HBASE-15584
> URL: https://issues.apache.org/jira/browse/HBASE-15584
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Priority: Minor
>
> During review of HBASE-15411, Enis made the following point:
> {code}
> nobody puts the backup in cancelled state. setCancelled() is not used. So if
> I abort a backup, who writes to the system table the new state?
> Not sure whether this is a phase 1 patch issue or due to this patch. We can
> open a new jira and address it there if you do not want to do it in this
> patch.
> Also maybe this should be named ABORTED rather than CANCELLED.
> {code}
> This issue is to decide whether this state should be kept (e.g. through
> notification from procedure V2 framework in response to abortion).
> If it is to be kept, the state should be renamed ABORTED.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15884) NPE in StoreFileScanner#skipKVsNewerThanReadpoint during reverse scan
[
https://issues.apache.org/jira/browse/HBASE-15884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310747#comment-15310747
]
Hudson commented on HBASE-15884:
ABORTED: Integrated in HBase-0.98-matrix #350 (See
[https://builds.apache.org/job/HBase-0.98-matrix/350/])
HBASE-15884 NPE in StoreFileScanner#skipKVsNewerThanReadpoint during (apurtell:
rev 67dc17240edffdb048d63ee13e12bbcac9d51bfc)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
> NPE in StoreFileScanner#skipKVsNewerThanReadpoint during reverse scan
> -
>
> Key: HBASE-15884
> URL: https://issues.apache.org/jira/browse/HBASE-15884
> Project: HBase
> Issue Type: Bug
> Components: Scanners
>Affects Versions: 2.0.0, 1.2.1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15884-1.patch
>
>
> Here is a part of {{skipKVsNewerThanReadpoint}} method:
> {noformat}
> hfs.next();
> setCurrentCell(hfs.getKeyValue());
> if (this.stopSkippingKVsIfNextRow
> && getComparator().compareRows(cur.getRowArray(),
> cur.getRowOffset(),
> cur.getRowLength(), startKV.getRowArray(),
> startKV.getRowOffset(),
> startKV.getRowLength()) > 0) {
> {noformat}
> If hfs has no more KVs, cur will be set to Null and on on the next step will
> throw NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15473) Documentation for the usage of hbase dataframe user api (JSON, Avro, etc)
[ https://issues.apache.org/jira/browse/HBASE-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated HBASE-15473: --- Assignee: (was: Zhan Zhang) > Documentation for the usage of hbase dataframe user api (JSON, Avro, etc) > - > > Key: HBASE-15473 > URL: https://issues.apache.org/jira/browse/HBASE-15473 > Project: HBase > Issue Type: Sub-task > Components: documentation, spark >Reporter: Zhan Zhang >Priority: Blocker > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
