Re: Please congratulate our new PMC Chair Misty Stanley-Jones

[Jimmy Xiang]

Congrats! Misty!!

On Fri, Sep 22, 2017 at 7:16 AM, Pankaj kr  wrote:

> Congratulations Misty..!! :)
>
>
> -Pankaj-
>
>
> -Original Message-
> From: Andrew Purtell [mailto:apurt...@apache.org]
> Sent: Friday, September 22, 2017 3:08 AM
> To: dev@hbase.apache.org; u...@hbase.apache.org
> Subject: Please congratulate our new PMC Chair Misty Stanley-Jones
>
> At today's meeting of the Board, Special Resolution B changing the HBase
> project Chair to Misty Stanley-Jones was passed unanimously.
>
> Please join me in congratulating Misty on her new role!
>
> ​(If you need any help or advice please don't hesitate to ping me, Misty,
> but I suspect you'll do just fine and won't need it.)​
>
>
> --
> Best regards,
> Andrew
>

Re: [ANNOUNCE] New HBase committer Huaxiang Sun

[Jimmy Xiang]

Congrats!

On Mon, Jun 19, 2017 at 5:02 PM, Stephen Jiang  wrote:
> Congrats and welcome to the team!
>
> Stephen
>
> On Mon, Jun 19, 2017 at 1:43 PM, Mike Drob  wrote:
>
>> Great work, Huaxiang!
>>
>> On Mon, Jun 19, 2017 at 2:30 PM, Sean Busbey  wrote:
>>
>> > On behalf of the Apache HBase PMC, I am pleased to announce that
>> > Huaxiang Sun has accepted the PMC's invitation to become a committer
>> > on the project. We appreciate all of Huaxiang's great work thus far
>> > and look forward to continued involvement.
>> >
>> > Please join me in congratulating Huaxiang!
>> >
>>

[jira] [Created] (HBASE-17625) Slow to enable a table

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-17625:
---

 Summary: Slow to enable a table
 Key: HBASE-17625
 URL: https://issues.apache.org/jira/browse/HBASE-17625
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang


Tried to enable a table with 10k+ regions, it takes more time to generate the 
plan than do the actual assignment. This is so embarrassing. :)

It turns out that it took quite some time to get the top HDFS block locations 
in registering regions when creating the Cluster object. There is no new region 
server, why do we need such info when trying to retain assignment?

Is the region availability thing related to region replica? Can we avoid such 
penalty if  region replica in not needed?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [ANNOUNCE] Dima Spivak joins the Apache HBase PMC

[Jimmy Xiang]

Congrats!!

On Wed, Aug 31, 2016 at 1:33 PM, Aleksandr Shulman  wrote:
> Congrats Dima!!
>
> On Wed, Aug 31, 2016 at 12:37 PM, Andrew Purtell 
> wrote:
>
>> On behalf of the Apache HBase PMC I am pleased to announce that Dima Spivak
>> has accepted our invitation to become a committer and PMC member on the
>> Apache HBase project. Dima has been an active contributor for some time,
>> particularly in development and contribution of release tooling that all of
>> our RMs now use, such as the API compatibility checker. Dima has also been
>> active in testing and voting on release candidates. Release voting is
>> important to project health and momentum and demonstrates interest and
>> capability above and beyond just committing. We wish to recognize this and
>> make those release votes binding. Please join me in thanking Dima for his
>> contributions to date and anticipation of many more contributions.
>>
>> Welcome to the HBase project, Dima!
>>
>> --
>> Best regards,
>>
>>- Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
>
>
>
> --
> Best Regards,
>
> Aleks Shulman
> 847.814.5804
> Cloudera

Re: [DISCUSS] No regions on Master node in 2.0

[Jimmy Xiang]

One thing I'd like to say is that it makes the master startup much
more simpler and realiable to put system tables on master.

Even if proc-v2 can solve the problem, it makes things complicated,
right? I prefer to be sure that meta is always available, in a
consistent state.

If we really need to split meta, we should have an option for most
users to have just one meta region, and keep it on master.


On Fri, Apr 8, 2016 at 8:03 AM, Matteo Bertozzi  wrote:
> # Without meta on master, we double assign and lose data.
>
> I doubt meta on master solve this problem.
> This has more to do on the fact that balancer, assignment, split, merge
> are disjoint operations that are not aware of each other.
> also those operation in general consist of multiple steps and if the master
> crashes you may end up in an inconsistent state.
>
> this is what proc-v2 should solve. since we are aware of each operation
> there is no chance of double assignment and similar by design.
>
> The master doesn't need the full meta to operate properly
> it just need the "state" (at which point of the operation am I).
> which is the wal of proc-v2. given that we can split meta or meta
> remote without any problem. since we only have 1 update to meta to
> update the location when the assignment is completed.
>
> also at the moment the master has a copy of the information in meta.
> a map with the RegionInfo, state and locations. but we are still doing
> a query on meta instead of using that local map directly.
> if we move meta on master we can remove that extra copy, but that
> will tight together meta and master making impossible to offload meta, if
> we need to.
>
>
> In my opinion with the new assignment you have all the main problem solved.
> we can keep regions on master as we have now,
> so you can configure it to get more performance (avoid the remote rpc).
> but our design should allow meta to be split and to be hosted somewhere
> else.
>
> Matteo
>
>
> On Fri, Apr 8, 2016 at 2:08 AM, 张铎  wrote:
>
>> Agree on the performance concerns. IMO we should not hurt the performance
>> of small(maybe normal?) clusters when scaling for huge clusters.
>> And I also agree that the current implementation which allows Master to
>> carry system regions is not good(sorry for the chinglish...). At least, it
>> makes the master startup really complicated.
>>
>> So IMO, we should let the master process or master machine to also carry
>> system regions, but in another way. Start another RS instance on the same
>> machine or in the same JVM? Or build a new storage based on the procedure
>> store and convert it to a normal table when it is too large?
>>
>> Thanks.
>>
>> 2016-04-08 16:42 GMT+08:00 Elliott Clark :
>>
>> > # Without meta on master, we double assign and lose data.
>> >
>> > That is currently a fact that I have seen over and over on multiple
>> loaded
>> > clusters. Some abstract clean up of deployment vs losing data is a
>> > no-brainer for me. Master assignment, region split, region merge are all
>> > risky, and all places that HBase can lose data. Meta being hosted on the
>> > master makes communication easier and less flakey. Running ITBLL on a
>> loop
>> > that creates a new table every time, and without meta on master
>> everything
>> > will fail pretty reliably in ~2 days. With meta on master things pass
>> MUCH
>> > more.
>> >
>> > # Master hosting the system tables locates the system tables as close as
>> > possible to the machine that will be mutating the data.
>> >
>> > Data locality is something that we all work for. Short circuit local
>> reads,
>> > Caching blocks in jvm, etc. Bringing data closer to the interested party
>> > has a long history of making things faster and better. Master is in
>> charge
>> > of just about all mutations of all systems tables. It's in charge of
>> > changing meta, changing acls, creating new namespaces, etc. So put the
>> > memstore as close as possible to the system that's going to mutate meta.
>> >
>> > # If you want to make meta faster then moving it to other regionservers
>> > makes things worse.
>> >
>> > Meta can get pretty hot. Putting it with other regions that clients will
>> be
>> > trying to access makes everything worse. It means that meta is competing
>> > with user requests. If meta gets served and other requests don't, causing
>> > more requests to meta; or requests to user regions get served and other
>> > clients get starved.
>> > At FB we've seen read throughput to meta doubled or more by swapping it
>> to
>> > master. Writes to meta are also much faster since there's no rpc hop, no
>> > queueing, to fighting with reads. So far it has been the single biggest
>> > thing to make meta faster.
>> >
>> >
>> > On Thu, Apr 7, 2016 at 10:11 PM, Stack  wrote:
>> >
>> > > I would like to start a discussion on whether Master should be carrying
>> > > regions or not. No hurry. I see this thread going on a 

Re: [ANNOUNCE] New HBase committer Esteban Gutierrez

[Jimmy Xiang]

Congrats!!

On Tue, Jun 30, 2015 at 5:15 PM, Andrew Purtell apurt...@apache.org wrote:

 On behalf of the Apache HBase PMC, I am pleased to announce that Esteban
 Gutierrez has accepted the PMC's invitation to become a committer on the
 project. We appreciate all of Esteban's generous contributions thus far and
 look forward to his continued involvement.

 Congratulations and welcome, Esteban!

 --
 Best regards,

- Andy

 Problems worthy of attack prove their worth by hitting back. - Piet Hein
 (via Tom White)

[jira] [Created] (HBASE-12572) Meta flush hangs

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12572:
---

 Summary: Meta flush hangs
 Key: HBASE-12572
 URL: https://issues.apache.org/jira/browse/HBASE-12572
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Jimmy Xiang


Not sure if this is still an issue with the latest branch 1 code. I ran into 
this with branch 1 commit: 0.99.2-SNAPSHOT, 
revision=290749fc56d07461441bd532f62d70f562eee588.

Jstack shows lots of scanners blocked at close.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12555) Region mover should not try to move regions to master

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12555:
---

 Summary: Region mover should not try to move regions to master
 Key: HBASE-12555
 URL: https://issues.apache.org/jira/browse/HBASE-12555
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang


If meta and master is co-located, master is a region server. Region mover 
script may try to move regions to the master which will fail since load 
balancer doesn't allow that. The script should be fixed not to move regions to 
master.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-12453) Make region available once it's open

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-12453.
-
Resolution: Invalid

 Make region available once it's open
 

 Key: HBASE-12453
 URL: https://issues.apache.org/jira/browse/HBASE-12453
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang

 Currently (in trunk, with zk-less assignment), a region is available to 
 serving requests only after RS notifies the master the region is open, and 
 the meta is updated with the new location. We may be able to do better than 
 this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12453) Make region available once it's open

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12453:
---

 Summary: Make region available once it's open
 Key: HBASE-12453
 URL: https://issues.apache.org/jira/browse/HBASE-12453
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


Currently (in trunk, with zk-less assignment), a region is available to serving 
requests only after RS notifies the master the region is open, and the meta is 
updated with the new location. We may be able to do better than this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12216) Lower closed region logging level

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12216:
---

 Summary: Lower closed region logging level
 Key: HBASE-12216
 URL: https://issues.apache.org/jira/browse/HBASE-12216
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
 Fix For: 2.0.0


There are quite some ERROR messages in the log, which sounds some problems but 
actually they are not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12206) NPE in RSRpcServices

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12206:
---

 Summary: NPE in RSRpcServices
 Key: HBASE-12206
 URL: https://issues.apache.org/jira/browse/HBASE-12206
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


Looks leases is null, which is possible since the region server is not open 
yet. Will add a check.

{noformat}
2014-10-08 08:38:17,985 ERROR [B.defaultRpcServer.handler=0,queue=0,port=20020] 
ipc.RpcServer: Unexpected throwable object
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:1957)
at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30422)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2020)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
at 
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
at java.lang.Thread.run(Thread.java:724)
2014-10-08 08:38:17,988 DEBUG [B.defaultRpcServer.handler=0,queue=0,port=20020] 
ipc.RpcServer: B.defaultRpcServer.handler=0,queue=0,port=20020: callId: 645 
service: ClientService methodName: Scan size: 22 connection: 10.20.212.36:53810
java.io.IOException
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2054)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
at 
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:1957)
at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30422)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2020)
... 4 more
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12209) NPE in HRegionServer#getLastSequenceId

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12209:
---

 Summary: NPE in HRegionServer#getLastSequenceId
 Key: HBASE-12209
 URL: https://issues.apache.org/jira/browse/HBASE-12209
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
 Fix For: 2.0.0, 0.99.1


The region server got the logging splitting task, but the master is gone.
{noformat}
2014-10-08 08:31:22,089 ERROR [RS_LOG_REPLAY_OPS-a2428:20020-1] 
executor.EventHandler: Caught throwable while processing event RS_LOG_REPLAY
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.getLastSequenceId(HRegionServer.java:2113)
at 
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:317)
at 
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:218)
at 
org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:103)
at 
org.apache.hadoop.hbase.regionserver.handler.HLogSplitterHandler.process(HLogSplitterHandler.java:72)
at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-11838) Enable PREFIX_TREE in integration tests

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-11838.
-
   Resolution: Fixed
Fix Version/s: 0.99.1
   2.0.0
 Hadoop Flags: Reviewed

With HBASE-11728 and HBASE-12078, ITBLL with PREFIX_TREE encoding works fine 
for me now. Integrated the patch to branch 1 and master. ITBLL tests all 
supported data encodings from now on. Thanks.

 Enable PREFIX_TREE in integration tests
 ---

 Key: HBASE-11838
 URL: https://issues.apache.org/jira/browse/HBASE-11838
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
 Fix For: 2.0.0, 0.99.1

 Attachments: hbase-11838.patch


 HBASE-11728 fixed a PREFIX_TREE encoding bug. Let's try to enable the 
 encoding in integration tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12196) SSH should retry in case failed to assign regions

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12196:
---

 Summary: SSH should retry in case failed to assign regions
 Key: HBASE-12196
 URL: https://issues.apache.org/jira/browse/HBASE-12196
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 2.0.0, 0.99.1


If there is only master alive, all regionservers are down, SSH can't find a 
plan to assign user regions. In this case, SSH should retry.

{noformat}
2014-10-07 14:05:18,310 ERROR [MASTER_SERVER_OPERATIONS-a2424:20020-2] 
executor.EventHandler: Caught throwable while processing event M_SERVER_SHUTDOWN
java.io.IOException: Unable to determine a plan to assign region(s)
at 
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1411)
at 
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:272)
at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12184) ServerShutdownHandler throws NPE

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12184:
---

 Summary: ServerShutdownHandler throws NPE
 Key: HBASE-12184
 URL: https://issues.apache.org/jira/browse/HBASE-12184
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 2.0.0, 0.99.1


{noformat}
2014-10-06 16:59:22,219 ERROR [MASTER_SERVER_OPERATIONS-a2424:20020-2] 
executor.EventHandler: Caught throwable while processing event M_SERVER_SHUTDOWN
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:190)
at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12175) Can't create table

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12175:
---

 Summary: Can't create table
 Key: HBASE-12175
 URL: https://issues.apache.org/jira/browse/HBASE-12175
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang


Trying to create a table from hbase shell and couldn't get region assigned:

{noformat}
^Gdefault^R^Dtest
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2213)
at 
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1879)
Caused by: java.lang.IllegalArgumentException: Illegal character 10 at 0. 
Namespaces can only contain 'alphanumeric characters': i.e. [a-zA-Z_0-9]:
^Gdefault^R^Dtest
at 
org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:215)
at 
org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:204)
at org.apache.hadoop.hbase.TableName.init(TableName.java:302)
at 
org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:339)
at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:460)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-12175) Can't create table

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-12175.
-
Resolution: Invalid

Could be my env issue.

 Can't create table
 --

 Key: HBASE-12175
 URL: https://issues.apache.org/jira/browse/HBASE-12175
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang

 Trying to create a table from hbase shell and couldn't get region assigned:
 {noformat}
 ^Gdefault^R^Dtest
 at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2213)
 at 
 org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1879)
 Caused by: java.lang.IllegalArgumentException: Illegal character 10 at 0. 
 Namespaces can only contain 'alphanumeric characters': i.e. [a-zA-Z_0-9]:
 ^Gdefault^R^Dtest
 at 
 org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:215)
 at 
 org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:204)
 at org.apache.hadoop.hbase.TableName.init(TableName.java:302)
 at 
 org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:339)
 at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:460)
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12167) NPE in AssignmentManager

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12167:
---

 Summary: NPE in AssignmentManager
 Key: HBASE-12167
 URL: https://issues.apache.org/jira/browse/HBASE-12167
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


If we can't find a region plan, we should check.

{noformat}
2014-10-02 18:36:27,719 ERROR [MASTER_SERVER_OPERATIONS-a2424:20020-0] 
executor.EventHandler: Caught throwable while processing event M_SERVER_SHUTDOWN
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1417)
at 
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1409)
at 
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:271)
at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12119) Master regionserver web UI NOT_FOUND

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12119:
---

 Summary: Master regionserver web UI NOT_FOUND
 Key: HBASE-12119
 URL: https://issues.apache.org/jira/browse/HBASE-12119
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


Although the active master is configured to host regions, I got Error 404 
NOT_FOUND.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12122) Try not to assign user regions to master all the time

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12122:
---

 Summary: Try not to assign user regions to master all the time
 Key: HBASE-12122
 URL: https://issues.apache.org/jira/browse/HBASE-12122
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
 Fix For: 2.0.0, 0.99.1


Load balancer does a good job not to assign regions of tables not configured to 
put on the active master. However, if there is no other region server, it still 
assigns users regions to the master. This happens when all normal region 
servers are crashed and recovering.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12124) Closed region could stay closed if master stops at bad time

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12124:
---

 Summary: Closed region could stay closed if master stops at bad 
time
 Key: HBASE-12124
 URL: https://issues.apache.org/jira/browse/HBASE-12124
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 2.0.0, 0.99.1


This applies to RPC-based region assignment only.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-12129) ProtobufLogWriter throws NPE

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-12129.
-
   Resolution: Duplicate
Fix Version/s: HBASE-12074

Right. Close it as Duplicate.

 ProtobufLogWriter throws NPE
 

 Key: HBASE-12129
 URL: https://issues.apache.org/jira/browse/HBASE-12129
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
 Fix For: HBASE-12074


 {noformat}
 2014-09-30 14:47:53,170 INFO  [sync.2] wal.FSHLog: Slow sync cost: 161 ms, 
 current pipeline: []
 2014-09-30 14:47:53,171 ERROR [sync.0] wal.FSHLog: Error syncing, request 
 close of hlog
 java.io.IOException: java.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:170)
 at 
 org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1302)
 at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.NullPointerException
 at 
 org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:167)
 ... 2 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-11970) BlockCache would be instantiated when Master doesn't host regions

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-11970.
-
Resolution: Implemented

Fixed this in HBASE-12034. For master that doesn't host any region, block cache 
is not instantiated.

 BlockCache would be instantiated when Master doesn't host regions
 -

 Key: HBASE-11970
 URL: https://issues.apache.org/jira/browse/HBASE-11970
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Ted Yu
Assignee: Jimmy Xiang

 HMaster ctor calls HRegionServer ctor which calls CacheConfig ctor where 
 CacheConfig.instantiateBlockCache() would create BlockCache.
 This happens even if Master is configured not to host regions.
 This was observed when I tried to answer question from the thread titled:
 'The default hbase.regionserver.info.port doesn't work'



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12005) Split/merge fails if master restarts before PONR

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-12005:
---

 Summary: Split/merge fails if master restarts before PONR
 Key: HBASE-12005
 URL: https://issues.apache.org/jira/browse/HBASE-12005
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


This applies to RPC based assignment. The root cause is that we don't persist 
the state of the new region(s). If we do persist it, we need to clean it up if 
the split/merge fails.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-11991) Region states may be out of sync

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11991:
---

 Summary: Region states may be out of sync
 Key: HBASE-11991
 URL: https://issues.apache.org/jira/browse/HBASE-11991
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


Region states could be out of sync under a rare scenario. The regions hosted by 
a server could be wrong.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-11976) Server startcode is not checked for some open region request

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11976:
---

 Summary: Server startcode is not checked for some open region 
request
 Key: HBASE-11976
 URL: https://issues.apache.org/jira/browse/HBASE-11976
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


I got the following failure yesterday. It looks like the master and the 
regionserver are both restarted. In retrying sending open region request, the 
region server failed to check the server start code.
 
{noformat}
2014-09-14 19:36:45,565 ERROR 
[B.defaultRpcServer.handler=24,queue=0,port=20020] 
master.AssignmentManager: Failed to transition region from 
{2706f577540a7d1b53b5a8f66178fbf2 state=PENDING_OPEN, ts=1410748604803, 
server=a2428.halxg.cloudera.com,20020,1410746518223} on OPENED by 
a2428.halxg.cloudera.com,20020,1410748599408: 2706f577540a7d1b53b5a8f66178fbf2 
is not opening on a2428.halxg.cloudera.com,20020,1410748599408
ABORTING region server a2428.halxg.cloudera.com,20020,1410748599408: Exception 
running postOpenDeployTasks; region=2706f577540a7d1b53b5a8f66178fbf2
{noformat}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-11959) TestAssignmentManagerOnCluster is falky

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11959:
---

 Summary: TestAssignmentManagerOnCluster is falky
 Key: HBASE-11959
 URL: https://issues.apache.org/jira/browse/HBASE-11959
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


https://builds.apache.org/job/HBase-1.0/178/testReport/





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-11960) Provide a sample to show how to use Thrift client authentication

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11960:
---

 Summary: Provide a sample to show how to use Thrift client 
authentication
 Key: HBASE-11960
 URL: https://issues.apache.org/jira/browse/HBASE-11960
 Project: HBase
  Issue Type: Task
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


Now Thrift server can authenticate clients. However, many people asked how to 
use it. Although we have some info in the refguide, it doesn't mention how to 
change the client to turn it on. A sample should help.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-11961) Document region state transitions

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11961:
---

 Summary: Document region state transitions
 Key: HBASE-11961
 URL: https://issues.apache.org/jira/browse/HBASE-11961
 Project: HBase
  Issue Type: Task
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


Document the region state transitions in the refguide.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-11959) TestAssignmentManagerOnCluster is flaky

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-11959.
-
   Resolution: Fixed
Fix Version/s: 0.99.1
 Hadoop Flags: Reviewed

I tested related tests 10 times locally and they are good with this patch. 
Integrated into branch 1. Thanks.

 TestAssignmentManagerOnCluster is flaky
 ---

 Key: HBASE-11959
 URL: https://issues.apache.org/jira/browse/HBASE-11959
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.99.1

 Attachments: hbase-11959.patch


 https://builds.apache.org/job/HBase-1.0/178/testReport/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-11880) NPE in MasterStatusServlet

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11880:
---

 Summary: NPE in MasterStatusServlet
 Key: HBASE-11880
 URL: https://issues.apache.org/jira/browse/HBASE-11880
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


{quote}
2014-09-02 10:03:04,807 ERROR [82061158@qtp-1041323834-0] mortbay.log: 
/master-status
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.master.MasterStatusServlet.getMetaLocationOrNull(MasterStatusServlet.java:91)
at 
org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:67)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at 
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at 
org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at 
org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1345)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at 
org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at 
org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at 
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at 
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at 
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at 
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at 
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at 
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at 
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at 
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at 
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at 
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
{quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Thrift client authentication skipped for HBase thrift server

[Jimmy Xiang]

Kashif,

HBASE-11349/HBASE-11474 is indeed to authenticate Thrift clients using
Kerberos. Is this what you are looking for?
For Thrift server authentication, it is already there. Please refer to the
hbase book
http://hbase.apache.org/book/security.html#hbase.secure.configuration
Section 8.1.4 for more details.

Thanks,
Jimmy


On Sun, Aug 31, 2014 at 11:28 PM, Kashif Jawed Siddiqui kashi...@huawei.com
 wrote:

 Hi All,

 As per current implementation done for
 https://issues.apache.org/jira/i#browse/HBASE-11349  
 https://issues.apache.org/jira/i#browse/HBASE-11474 ,

 The authentication mechanism using  Kerberos principal for Thrift server
 with HBase is perfectly fine.



 But for clients communicating to HBase via thrift server does not handle
 the security mechanism.

 Any unauthenticated client can access HBase via thrift server. The thrift
 sever can act as a backdoor entry for skipping the security 
 authentication.

 It will be better if thrift clients can also be authenticated through some
 mechanism like Kerberos or IP restriction,etc



 Let us discuss on mechanism for thrift client authentication that can be
 implemented.



 ***
 This e-mail and attachments contain confidential information from HUAWEI,
 which is intended only for the person or entity whose address is listed
 above. Any use of the information contained herein in any way (including,
 but not limited to, total or partial disclosure, reproduction, or
 dissemination) by persons other than the intended recipient's) is
 prohibited. If you receive this e-mail in error, please notify the sender
 by phone or email immediately and delete it!

[jira] [Created] (HBASE-11866) TestDistributedLogSplitting is flaky

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11866:
---

 Summary: TestDistributedLogSplitting is flaky
 Key: HBASE-11866
 URL: https://issues.apache.org/jira/browse/HBASE-11866
 Project: HBase
  Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


{quote}
org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running 
yet
at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:831)
at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.getOnlineRegion(RSRpcServices.java:1044)
at 
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getOnlineRegions(ProtobufUtil.java:1723)
at 
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.findRSToKill(TestDistributedLogSplitting.java:1608)
at 
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testMetaRecoveryInZK(TestDistributedLogSplitting.java:1133)
{quote}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11867) TestSplitLogManager.testUnassignedTimeout is flaky

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11867:
---

 Summary: TestSplitLogManager.testUnassignedTimeout is flaky
 Key: HBASE-11867
 URL: https://issues.apache.org/jira/browse/HBASE-11867
 Project: HBase
  Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


{quote}
junit.framework.AssertionFailedError: Waiting timed out after [3,200] msec
at junit.framework.Assert.fail(Assert.java:57)
at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:146)
at 
org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3463)
at 
org.apache.hadoop.hbase.master.TestSplitLogManager.waitForCounter(TestSplitLogManager.java:241)
at 
org.apache.hadoop.hbase.master.TestSplitLogManager.waitForCounter(TestSplitLogManager.java:234)
at 
org.apache.hadoop.hbase.master.TestSplitLogManager.testUnassignedTimeout(TestSplitLogManager.java:483)
{quote}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11838) Enable PREFIX_TREE in integration tests

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11838:
---

 Summary: Enable PREFIX_TREE in integration tests
 Key: HBASE-11838
 URL: https://issues.apache.org/jira/browse/HBASE-11838
 Project: HBase
  Issue Type: Test
  Components: test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


HBASE-11728 fixed a PREFIX_TREE encoding bug. Let's try to enable the encoding 
in integration tests.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11756) Some assignment notifications are not triggered

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11756:
---

 Summary: Some assignment notifications are not triggered
 Key: HBASE-11756
 URL: https://issues.apache.org/jira/browse/HBASE-11756
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang


In AssignmentManager, notifications are triggered when regions are 
opened/closed. However, notifications are not always triggered. For example, 
when a server is dead, regions are closed implicitly, and no notifications are 
triggered.

Is it a better place to trigger such events in RegionStates?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11760) Tighten up region state transition

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11760:
---

 Summary: Tighten up region state transition
 Key: HBASE-11760
 URL: https://issues.apache.org/jira/browse/HBASE-11760
 Project: HBase
  Issue Type: Improvement
  Components: Region Assignment
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 2.0.0


When a regionserver reports to master a region transition, we should check the 
current region state to be exactly what we expect.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11732) No need to force unassign a region

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11732:
---

 Summary: No need to force unassign a region
 Key: HBASE-11732
 URL: https://issues.apache.org/jira/browse/HBASE-11732
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Affects Versions: 2.0.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


Clean up force region unassignment.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11725) Backport failover checking change to 1.0

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11725:
---

 Summary: Backport failover checking change to 1.0
 Key: HBASE-11725
 URL: https://issues.apache.org/jira/browse/HBASE-11725
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


In HBASE-11611, we fixed a failover checking bug. We need to backport it to 
branch 1.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11727) Assignment wait time error in case of ServerNotRunningYetException

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11727:
---

 Summary: Assignment wait time error in case of 
ServerNotRunningYetException
 Key: HBASE-11727
 URL: https://issues.apache.org/jira/browse/HBASE-11727
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


{quote}
maxWaitTime = this.server.getConfiguration().
  getLong(hbase.regionserver.rpc.startup.waittime, 6);
{quote}

It should add the current time.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-11728.
-

Resolution: Duplicate

I think this is a duplicate. However, the information you put here is very 
useful.

 Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
 --

 Key: HBASE-11728
 URL: https://issues.apache.org/jira/browse/HBASE-11728
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.96.1.1, 0.98.4
 Environment: ubuntu12 
 hadoop-2.2.0
 Hbase-0.96.1.1
 SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Priority: Critical
   Original Estimate: 72h
  Remaining Estimate: 72h

 In Scan case, i prepare some data as beflow:
 Table Desc (Using the prefix-tree encoding) :
 'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE', 
 TTL = '15552000'}
 and i put 5 rows as:
 (RowKey , Qualifier, Value)
 'a-b-0-0', 'qf_1', 'c1-value'
 'a-b-A-1', 'qf_1', 'c1-value'
 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
 'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
 so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the 
 corret result:
 Test 1: 
 Scan scan = new Scan();
 scan.setStartRow(a-b-A-1.getBytes());
 scan.setStopRow(a-b-A-1:.getBytes());
 --
 'a-b-A-1', 'qf_1', 'c1-value'
 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
 and then i try next , scan to addColumn
 Test2:
 Scan scan = new Scan();
 scan.addColumn(Bytes.toBytes(cf_1) ,  Bytes.toBytes(qf_2));
 scan.setStartRow(a-b-A-1.getBytes());
 scan.setStopRow(a-b-A-1:.getBytes());
 --
 except:
 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
 but actually i got nonthing. Then i update the addColumn for 
 scan.addColumn(Bytes.toBytes(cf_1) ,  Bytes.toBytes(qf_1)); and i got the 
 expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
 then i do more testing...  i update the case to modify the startRow greater 
 than the 'a-b-A-1' 
 Test3:
 Scan scan = new Scan();
 scan.setStartRow(a-b-A-1-.getBytes());
 scan.setStopRow(a-b-A-1:.getBytes());
 --
 except:
 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
 but actually  i got nothing again. i modify the start row greater than 
 'a-b-A-1-1402329600-1402396277'
 Scan scan = new Scan();
 scan.setStartRow(a-b-A-1-140239.getBytes());
 scan.setStopRow(a-b-A-1:.getBytes());
 and i got the expect row as well:
 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
 So, i think it may be a bug in the prefix-tree encoding.It happens after the 
 data flush to the storefile, and it's ok when the data in mem-store.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Reopened] (HBASE-11728) Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang reopened HBASE-11728:
-


 Some data miss when scan using PREFIX_TREE DATA-BLOCK-ENCODING
 --

 Key: HBASE-11728
 URL: https://issues.apache.org/jira/browse/HBASE-11728
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.96.1.1, 0.98.4
 Environment: ubuntu12 
 hadoop-2.2.0
 Hbase-0.96.1.1
 SUN-JDK(1.7.0_06-b24)
Reporter: wuchengzhi
Priority: Critical
   Original Estimate: 72h
  Remaining Estimate: 72h

 In Scan case, i prepare some data as beflow:
 Table Desc (Using the prefix-tree encoding) :
 'prefix_tree_test', {NAME = 'cf_1', DATA_BLOCK_ENCODING = 'PREFIX_TREE', 
 TTL = '15552000'}
 and i put 5 rows as:
 (RowKey , Qualifier, Value)
 'a-b-0-0', 'qf_1', 'c1-value'
 'a-b-A-1', 'qf_1', 'c1-value'
 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
 'a-b-B-2-1402397300-1402416535', 'qf_2', 'c2-value-3'
 so i try to scan the rowKey between 'a-b-A-1' and 'a-b-A-1:' , i and got the 
 corret result:
 Test 1: 
 Scan scan = new Scan();
 scan.setStartRow(a-b-A-1.getBytes());
 scan.setStopRow(a-b-A-1:.getBytes());
 --
 'a-b-A-1', 'qf_1', 'c1-value'
 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
 and then i try next , scan to addColumn
 Test2:
 Scan scan = new Scan();
 scan.addColumn(Bytes.toBytes(cf_1) ,  Bytes.toBytes(qf_2));
 scan.setStartRow(a-b-A-1.getBytes());
 scan.setStopRow(a-b-A-1:.getBytes());
 --
 except:
 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
 but actually i got nonthing. Then i update the addColumn for 
 scan.addColumn(Bytes.toBytes(cf_1) ,  Bytes.toBytes(qf_1)); and i got the 
 expected result 'a-b-A-1', 'qf_1', 'c1-value' as well.
 then i do more testing...  i update the case to modify the startRow greater 
 than the 'a-b-A-1' 
 Test3:
 Scan scan = new Scan();
 scan.setStartRow(a-b-A-1-.getBytes());
 scan.setStopRow(a-b-A-1:.getBytes());
 --
 except:
 'a-b-A-1-1402329600-1402396277', 'qf_2', 'c2-value'
 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
 but actually  i got nothing again. i modify the start row greater than 
 'a-b-A-1-1402329600-1402396277'
 Scan scan = new Scan();
 scan.setStartRow(a-b-A-1-140239.getBytes());
 scan.setStopRow(a-b-A-1:.getBytes());
 and i got the expect row as well:
 'a-b-A-1-1402397227-1402415999', 'qf_2', 'c2-value-2'
 So, i think it may be a bug in the prefix-tree encoding.It happens after the 
 data flush to the storefile, and it's ok when the data in mem-store.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11709) TestMasterShutdown can fail sometime

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11709:
---

 Summary: TestMasterShutdown can fail sometime 
 Key: HBASE-11709
 URL: https://issues.apache.org/jira/browse/HBASE-11709
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


This applies to 1.0 and master, not previous versions.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Failing region replica tests in trunk

[Jimmy Xiang]

HBase master jenkins is finally blue again. Let's keep it blue.

https://builds.apache.org/job/HBase-TRUNK/5380/

Thanks,
Jimmy


On Wed, Aug 6, 2014 at 3:24 PM, Stack st...@duboce.net wrote:

 On Wed, Aug 6, 2014 at 3:21 PM, Devaraj Das d...@hortonworks.com wrote:

  Yeah the replica tests issues are due to turning on the zk-less
  assignment by default.
 
  If HBASE-11611 addresses the issues to do with zk-less assignments +
  replicas, let's wait for that.
 
 
 Grand. Thanks DD,
 St.Ack

[jira] [Created] (HBASE-11703) Meta region state could be corrupted

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11703:
---

 Summary: Meta region state could be corrupted
 Key: HBASE-11703
 URL: https://issues.apache.org/jira/browse/HBASE-11703
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


Internal meta region state could be corrupted if the meta is not on master:

1. the meta region server (not master) shuts down,
2. meta SSH offlines it without updating the dead server's region list,
3. meta is transitioned to pending_open and the previous server (the dead 
server) of meta is lost,
4. meta is assigned somewhere else without updating its previous server,
5. normal SSH processes the dead server and offlines all of it's dead regions 
including the meta, so the meta internal state is corrupted



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11687) Cancel postOpenDeployTasks if region opening is cancelled

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11687:
---

 Summary: Cancel postOpenDeployTasks if region opening is cancelled
 Key: HBASE-11687
 URL: https://issues.apache.org/jira/browse/HBASE-11687
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 1.0.0


With ZK-less region assignment, if region opening is in postOpenDeployTasks 
step and the opening is cancelled, the region server will abort because it 
can't report the transition to the master. We should cancel postOpenDeployTasks 
instead. At least, there is no need to abort.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11689) Tack meta in transition

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11689:
---

 Summary: Tack meta in transition
 Key: HBASE-11689
 URL: https://issues.apache.org/jira/browse/HBASE-11689
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Reporter: Jimmy Xiang


With ZK-less region assignment, user regions in transition are tracked in meta. 
We need a way to track meta in transition too.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: regionserver does not start in same machine as master

[Jimmy Xiang]

In 2.0, a master is also a region server now. So it uses the region server
ports. If you want to start a pseudo cluster, you can take a look
HBASE-11575. For a distributed cluster, if you want to start a master and a
regionserver on the same machine, you need to specify the port in the
command line (-D option). The port configuration in the configuration file
is shared by both the master and the region server.


On Fri, Aug 1, 2014 at 10:11 AM, abhishek1015 abhishek1...@gmail.com
wrote:

 Hello

 I am using hbase 2.0.0-SNAPSHOT with hadoop 2.2.0. I have two node cluster.
 After configuring the cluster, when i start the hbase cluster using
 start-hbase.sh command, every thing starts except the HRegionServer in same
 machine as HMaster.

 I see following error in HRegionServer logs. Issue seems to be caused by
 port conflict. I changed the regionserver port to different number. But,
 same error reoccurring.

 Thanks for any help.

 2014-08-01 17:58:07,154 ERROR [main] regionserver.HRegionServerCommandLine:
 Region server exiting
 java.lang.RuntimeException: Failed construction of Regionserver: class
 org.apache.hadoop.hbase.regionserver.HRegionServer
 at

 org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2371)
 at

 org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:64)
 at

 org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
 at

 org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
 at

 org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2386)
 Caused by: java.lang.reflect.InvocationTargetException
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
 Method)
 at

 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
 at

 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
 at

 org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2369)
 Caused by: java.net.BindException: Problem binding to
 sceplus-vm41.almaden.ibm.com/9.1.143.51:16021 : Address already in use
 at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2352)
 at
 org.apache.hadoop.hbase.ipc.RpcServer$Listener.init(RpcServer.java:525)
 at
 org.apache.hadoop.hbase.ipc.RpcServer.init(RpcServer.java:1882)
 at

 org.apache.hadoop.hbase.regionserver.RSRpcServices.init(RSRpcServices.java:732)
 at

 org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:540)
 at

 org.apache.hadoop.hbase.regionserver.HRegionServer.init(HRegionServer.java:461)
 ... 10 more

 Abhishek



 --
 View this message in context:
 http://apache-hbase.679495.n3.nabble.com/regionserver-does-not-start-in-same-machine-as-master-tp4062203.html
 Sent from the HBase Developer mailing list archive at Nabble.com.

[jira] [Created] (HBASE-11631) Wait a little till server is online in assigning meta

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11631:
---

 Summary: Wait a little till server is online in assigning meta
 Key: HBASE-11631
 URL: https://issues.apache.org/jira/browse/HBASE-11631
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


In assigning the meta to a regionserver not the master, the regionserver could 
have not processed the reportForDuty response yet. This happens a lot in unit 
tests. It's better to wait a little bit.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11610) Enhance remote meta updates

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11610:
---

 Summary: Enhance remote meta updates
 Key: HBASE-11610
 URL: https://issues.apache.org/jira/browse/HBASE-11610
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang


Currently, if the meta region is on a regionserver instead of the master, meta 
update is synchronized on one HTable instance. We should be able to do better.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11611) Clean up ZK-based region assignment

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11611:
---

 Summary: Clean up ZK-based region assignment
 Key: HBASE-11611
 URL: https://issues.apache.org/jira/browse/HBASE-11611
 Project: HBase
  Issue Type: Improvement
  Components: Region Assignment
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 2.0.0


We can clean up the ZK-based region assignment code and use the ZK-less one in 
the master branch, to make the code easier to understand and maintain.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11604) Disable co-locating meta/master by default

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11604:
---

 Summary: Disable co-locating meta/master by default
 Key: HBASE-11604
 URL: https://issues.apache.org/jira/browse/HBASE-11604
 Project: HBase
  Issue Type: Task
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 1.0.0


To avoid possible confusing, it's better to keep the original deployment scheme 
in 1.0. ZK-less region assignment is off by default in 1.0 already. We should, 
by default, not assign any region to master or backup master.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11606) Enable ZK-less region assignment by default

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11606:
---

 Summary: Enable ZK-less region assignment by default
 Key: HBASE-11606
 URL: https://issues.apache.org/jira/browse/HBASE-11606
 Project: HBase
  Issue Type: Task
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
 Fix For: 2.0.0


Let's enable ZK-less region assignment by default in the master branch.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-11565) Stale connection could stay for a while

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-11565.
-

   Resolution: Fixed
Fix Version/s: 2.0.0
   0.94.22
   0.98.5
   0.96.3
   0.99.0
 Hadoop Flags: Reviewed

Thanks. Integrated into branch 0.94 and up

 Stale connection could stay for a while
 ---

 Key: HBASE-11565
 URL: https://issues.apache.org/jira/browse/HBASE-11565
 Project: HBase
  Issue Type: Bug
  Components: Client
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.99.0, 0.96.3, 0.98.5, 0.94.22, 2.0.0

 Attachments: hbase-11565-0.98.patch, hbase-11565-v1.patch


 In RpcClient, we cache the connection to each region server. When the 
 connection goes bad, it stays in the cache till it's removed. Before it's 
 removed, new calls will try to use it and just fail. The connection is a 
 thread. It could be stuck in trying to receive some response. Before this 
 receiving thread times out, it won't remove itself from the cache.
 It will be better to interrupt the receiving thread and let it clean up 
 sooner.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11565) Stale connection could stay for a while

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11565:
---

 Summary: Stale connection could stay for a while
 Key: HBASE-11565
 URL: https://issues.apache.org/jira/browse/HBASE-11565
 Project: HBase
  Issue Type: Bug
  Components: Client
Reporter: Jimmy Xiang


In RpcClient, we cache the connection to each region server. When the 
connection goes bad, it stays in the cache till it's removed. Before it's 
removed, new calls will try to use it and just fail. The connection is a 
thread. It could be stuck in trying to receive some response. Before this 
receiving thread times out, it won't remove itself from the cache.

It will be better to interrupt the receiving thread and let it clean up sooner.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11525) Region server holding in region states is out of sync with meta

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11525:
---

 Summary: Region server holding in region states is out of sync 
with meta
 Key: HBASE-11525
 URL: https://issues.apache.org/jira/browse/HBASE-11525
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


In RegionStates, we remove a region from the region list a region server hosts 
once the region is offline. However, in meta, we do this only when the region 
is assigned to a new region server. We should keep them in sync so that we can 
claim they are consistent.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11530) RegionStates adds regions to wrong servers

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11530:
---

 Summary: RegionStates adds regions to wrong servers
 Key: HBASE-11530
 URL: https://issues.apache.org/jira/browse/HBASE-11530
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Attachments: hbase-11530.patch

In RegionStates.java, we have (line 332)
{quote}
addToServerHoldings(serverName, hri);
{quote}

It should be

{quote}
addToServerHoldings(lastHost, hri);
{quote}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11474) [Thrift2] support authentication/impersonation

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11474:
---

 Summary: [Thrift2] support authentication/impersonation
 Key: HBASE-11474
 URL: https://issues.apache.org/jira/browse/HBASE-11474
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


This is to do the same as HBASE-11349 for Thrift2.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11378) TableMapReduceUtil overwrites user supplied options for multiple tables/scaners job

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11378:
---

 Summary: TableMapReduceUtil overwrites user supplied options for 
multiple tables/scaners job
 Key: HBASE-11378
 URL: https://issues.apache.org/jira/browse/HBASE-11378
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.99.0, 0.98.4


In TableMapReduceUtil#initTableMapperJob, we have

HBaseConfiguration.addHbaseResources(job.getConfiguration());

It should use merge instead. Otherwise, user supplied options are overwritten.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11370) SSH doesn't need to scan meta if not using ZK for assignment

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11370:
---

 Summary: SSH doesn't need to scan meta if not using ZK for 
assignment
 Key: HBASE-11370
 URL: https://issues.apache.org/jira/browse/HBASE-11370
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


If we don't use ZK for assignment, the meta content should be the same as that 
in memory. So we should be able to avoid a meta scan.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11349) [Thrift] support authentication/impersonation

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11349:
---

 Summary: [Thrift] support authentication/impersonation
 Key: HBASE-11349
 URL: https://issues.apache.org/jira/browse/HBASE-11349
 Project: HBase
  Issue Type: Improvement
  Components: security, Thrift
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


Thrift server can access HBase as a fixed authenticated user. However, we don't 
authenticate thrift clients. It will be great if the thrift server can 
authenticate clients, and support impersonation.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: [VOTE] Merge branch HBASE-10070 to trunk

[Jimmy Xiang]

zk-less assignment will be off by default.

co-location of meta and master is on by default now. We can turn it off if
needed.

Thanks,
Jimmy



On Thu, Jun 12, 2014 at 2:12 PM, Mikhail Antonov olorinb...@gmail.com
wrote:

 So how would the defaults and dependencies look amongst these 3, once
 committed to 0.99?

  - co-location of meta and master
  - zk-less assignment
 - timeline-consistent replicas?

 As I understand -
  -  All 3 are off by default
  -  zk-less assignments and region replicas to not be turned on at the same
 time
  -  zk-less assignments require meta co-location
  -  region replicas don't care about meta co-location?

 -Mikhail


 2014-06-12 14:07 GMT-07:00 Jonathan Hsieh j...@cloudera.com:

  sgtm.
 
 
  On Mon, Jun 9, 2014 at 10:46 AM, Enis Söztutar enis@gmail.com
 wrote:
 
   Thanks Jon.
  
   I think we'll try the rebase approach first. I'll start the effort
 today
   and see how far along I can get with that. Rebasing each patch might
 be a
   bit more work actually. If this turns out painful, we'll revert to the
   merge approach similar to what you describe.
  
   Enis
  
  
   On Sat, Jun 7, 2014 at 7:41 AM, Jonathan Hsieh j...@cloudera.com
 wrote:
  
On Fri, Jun 6, 2014 at 5:05 PM, Enis Söztutar enis@gmail.com
   wrote:
   
 My preference would be to do the rebase-then-merge style (last
 style
  in
 your comment). For each patch, I am hoping for all the changes
  between
 committed version and rebased version to be mechanical. I like
  having a
 linear history with explicit commit-to-patch mapping.

 If the changes are of a mechanical nature and that each patch
  compiles
and
passes unit tests along the way, then rebase-then-merge is perfectly
  fine
by me.
   
Even though snapshots were fairly orthogonal to the rest of the
  codebase,
when we did merges we had problems maintaining the compiles and
 passes
   with
every commit invariants when we tried rebase-then-merge approach.
   After
each rebase we would end up doing a bunch of work and still end up
   failing
unit tests.   In that case we (jesse, matteo, myself) went to the
 merge
approach.   We actually merged into the snapshot branch a few times
   fixing
things due to changes in trunk that broke parts of the snapshots
  branch.
Here's a more accurate picture of what the commit history looked like
  in
git (we dev'ed in git and in end had to recommit this all in svn).
   
m  (essentially empty merge patch).
|\
t |  (trunk patch with no impact)
| s6* (patch to fix newly introduced problem)
|/|
t |
t*|   (trunk patch that broke snapshots again)
| s5* (branch bug fixes due to merge)
| s4* (mechanical fixups due to the merge)
|/|
t |
t |
t |
| s3
| s2
| s1
|/
t
t
   
This approach was captured in github and had the added benefit of
preserving exact history, and having known good points preserving the
compile/unit test invariants on trunk.  (unfortunately, some of this
history was lost when we ported the git history over to svn, but that
   isn't
a problem any more).If you want to see the whole thing, look at
 my
   git
repo:
   
git remote add jmhsieh g...@github.com:jmhsieh/hbase.git
git log --oneline --graph --color jmhsieh/hbase-7290
   
   
Can we do history-preserving merge with the first style? I can do the
 rebases and upload interdiff if that is big so that we can compare
  on a
 per-patch basis.


It is possible.
   
   
   
   
   
 Enis


 On Fri, Jun 6, 2014 at 4:36 PM, Jonathan Hsieh j...@cloudera.com
   wrote:

  When we merged snapshots branch in we did this:
 
  t = trunk commit
  s = snapshot branch commit
  m = merge point.
 
  During work:
  t
  t
  t
  |  s3
  |  s2
  |  s1
  |/
  t
  t
 
  During after merge:
  m  (essentially empty merge patch).
  t \
  t |
  t |
  | s4* (fixups due to the merge)
  | s3
  | s2
  | s1
  |/
  t
  t
 
  Does your proposal mean you are going to do this instead?
 
  s3* (modified due to rebase)
  s2* (modified due to rebase)
  s1
  t
  t
  t
  | s (now dead hbase-10070 branch)
  | s
  | s
  | s
  |/
  t
  t
 
  If it is the then we should probably have a review of the
 modified
 patches.
   If it the same as the snapshot merge, then we just need a review
  of
the
  merge point delta (as well as a full review).
 
  Personally I prefer the merge for these large feature branches --
  it
  guarantees that each commit is compilable, and reflects what you
  guys
 have
  been testing for a while.  If you go with the last approach you
  might
 have
  stuff broken, and in the mainline commit path.
 
  Jon.
 

[jira] [Created] (HBASE-11328) testMoveRegion could fail

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11328:
---

 Summary: testMoveRegion could fail
 Key: HBASE-11328
 URL: https://issues.apache.org/jira/browse/HBASE-11328
 Project: HBase
  Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


TestAssignmentManagerOnCluster#testMoveRegion could try to move a region to a 
server not online, and fail.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11319) No need for favored node mapping initialization to scan meta

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11319:
---

 Summary: No need for favored node mapping initialization to scan 
meta
 Key: HBASE-11319
 URL: https://issues.apache.org/jira/browse/HBASE-11319
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


During a clean re/start, favored node mapping initialization scans meta for an 
assignment snapshot. We can avoid this scan since such info is already loaded 
into region states. For small cluster, this is not a big deal. However, if 
there are lots of regions, the scan is better not to do.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: [VOTE] Merge branch HBASE-10070 to trunk

[Jimmy Xiang]

I will set zk-less assignment off by default so that your change is not
affected. But zk-less assignment won't support replica, at least for now.


On Mon, Jun 9, 2014 at 11:01 AM, Enis Söztutar enis@gmail.com wrote:

 BTW, I would really prefer to finish the merge before HBASE-11059. The
 replica assignment has been working with the new patches and zk-based
 assignment for many months.

 Enis


 On Fri, Jun 6, 2014 at 7:07 PM, Jimmy Xiang jxi...@cloudera.com wrote:

  First of all, I have not looked into the patches recently. I remember
 there
  are some changes to the public interface. I was wondering if it is
 backward
  compatible. Enis mentioned that it's rolling upgradable. Just want to
  confirm if it is backward compatible. For existing applications, do they
  need to recompile?
 
  Second, there should be some conflict with HBASE-11059 I am working on
 now,
  which is almost finished. How should we resolve this issue? ZK-less
 region
  assignment doesn't know region replica yet.
 
  Thanks,
  Jimmy
 
 
 
 
  On Fri, Jun 6, 2014 at 6:12 PM, Jeffrey Zhong jzh...@hortonworks.com
  wrote:
 
  
   +1. It brings goodies such as region replica, cross region server
   scanget, anti-affinity of regions, and pave the way for sync region
   replication.
  
   Thanks,
   -Jeffrey
  
   On 6/6/14 2:42 PM, Devaraj Das d...@hortonworks.com wrote:
  
   +1
   
   On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell apurt...@apache.org
   wrote:
+1, thanks Enis
   
   
On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar enis@gmail.com
   wrote:
   
Sorry, I was mostly out for HadoopSummit.
   
Yes, the git flow would be very similar to what you propose:
   
 $ git checkout HBASE-10070
 $ git rebase --ignore-date master
 (fixups, git add, git rebase --continue, etc, etc, etc)
 $ git checkout master
   
 $ git push origin HBASE-10070 HBASE-10070-rebase-date
   (optionally)
 $ git merge HBASE-10070
   
We can either go --ignore-date or not depending on what we want. If
   needed
I am fine with pushing the rebased master branch for review to main
   repo
before the merge to another branch. If not, I can just rebase the
   branch
locally and merge + push to main repo.
   
Creating final patches and attaching them to jira might be
  cumbersome.
   If
we do the rebased-branch on repo, we might not need it. But if we
  need
   that
for review, I can do it.
   
Thanks,
Enis
   
   
   
On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell 
 apurt...@apache.org
  
wrote:
   
 I realize this is a vote thread but I need a satisfactory answer
 to
   the
 below inquiries before feeling comfortable casting a vote. Or
  perhaps
that
 means we need to cancel this vote and move back to discussion.


 On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell 
  apurt...@apache.org
   
 wrote:

  Also after the merge process is completed, do you plan to use
 git
  format-patch to break out the per-JIRA changes into updated
   patches for
  those JIRAs representing in effect the final commit?
 
 
  On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell
   apurt...@apache.org
  wrote:
 
 
  On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar 
 e...@apache.org
wrote:
 
  This VOTE is for merging back the remaining changes in branch
 to
trunk.
 If
  passes, we will rebase the branch on top of current trunk, in
   which
we
  will
  keep the commit-per-issue log history. After that we will do
 a
   git
 merge
  for the branch keeping the history clean and not squashing
 the
 commits. I
  expect rebasing to be straightforward, however with some
 manual
 conflict
  resolution. After the merge we'll keep running the tests to
  make
   sure
  everything is ok.
 
 
  Just to clarify that would look something like this:
 
   $ git checkout HBASE-10070
   $ git rebase --ignore-date master
   (fixups, git add, git rebase --continue, etc, etc, etc)
   $ git checkout master
   $ git merge HBASE-10070
 
  ?
 
  That sounds good to me, the final merge should be a fast
 forward
merge.
 
  Use of ' --ignore-date' could be mildly controversial. It's
 not
strictly
  necessary because the commits for 10070 will appear grouped in
history,
 but
  then dates on commits will be discontiguous in that section of
history.
 I
  suggest using that option so the order of commits and dates
 sort
   the
 same
  on master.
 
 
  On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar 
 e...@apache.org
wrote:
 
  Hi,
 
  Last week we started some discussion[4] for merging branch
 hbase-10070[1]
  into trunk. It seems like the consensus there is to do the
  merge
sooner
  rather than later

Re: [VOTE] Merge branch HBASE-10070 to trunk

[Jimmy Xiang]

First of all, I have not looked into the patches recently. I remember there
are some changes to the public interface. I was wondering if it is backward
compatible. Enis mentioned that it's rolling upgradable. Just want to
confirm if it is backward compatible. For existing applications, do they
need to recompile?

Second, there should be some conflict with HBASE-11059 I am working on now,
which is almost finished. How should we resolve this issue? ZK-less region
assignment doesn't know region replica yet.

Thanks,
Jimmy




On Fri, Jun 6, 2014 at 6:12 PM, Jeffrey Zhong jzh...@hortonworks.com
wrote:


 +1. It brings goodies such as region replica, cross region server
 scanget, anti-affinity of regions, and pave the way for sync region
 replication.

 Thanks,
 -Jeffrey

 On 6/6/14 2:42 PM, Devaraj Das d...@hortonworks.com wrote:

 +1
 
 On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell apurt...@apache.org
 wrote:
  +1, thanks Enis
 
 
  On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar enis@gmail.com
 wrote:
 
  Sorry, I was mostly out for HadoopSummit.
 
  Yes, the git flow would be very similar to what you propose:
 
   $ git checkout HBASE-10070
   $ git rebase --ignore-date master
   (fixups, git add, git rebase --continue, etc, etc, etc)
   $ git checkout master
 
   $ git push origin HBASE-10070 HBASE-10070-rebase-date
 (optionally)
   $ git merge HBASE-10070
 
  We can either go --ignore-date or not depending on what we want. If
 needed
  I am fine with pushing the rebased master branch for review to main
 repo
  before the merge to another branch. If not, I can just rebase the
 branch
  locally and merge + push to main repo.
 
  Creating final patches and attaching them to jira might be cumbersome.
 If
  we do the rebased-branch on repo, we might not need it. But if we need
 that
  for review, I can do it.
 
  Thanks,
  Enis
 
 
 
  On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell apurt...@apache.org
  wrote:
 
   I realize this is a vote thread but I need a satisfactory answer to
 the
   below inquiries before feeling comfortable casting a vote. Or perhaps
  that
   means we need to cancel this vote and move back to discussion.
  
  
   On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell apurt...@apache.org
 
   wrote:
  
Also after the merge process is completed, do you plan to use git
format-patch to break out the per-JIRA changes into updated
 patches for
those JIRAs representing in effect the final commit?
   
   
On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell
 apurt...@apache.org
wrote:
   
   
On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar e...@apache.org
  wrote:
   
This VOTE is for merging back the remaining changes in branch to
  trunk.
   If
passes, we will rebase the branch on top of current trunk, in
 which
  we
will
keep the commit-per-issue log history. After that we will do a
 git
   merge
for the branch keeping the history clean and not squashing the
   commits. I
expect rebasing to be straightforward, however with some manual
   conflict
resolution. After the merge we'll keep running the tests to make
 sure
everything is ok.
   
   
Just to clarify that would look something like this:
   
 $ git checkout HBASE-10070
 $ git rebase --ignore-date master
 (fixups, git add, git rebase --continue, etc, etc, etc)
 $ git checkout master
 $ git merge HBASE-10070
   
?
   
That sounds good to me, the final merge should be a fast forward
  merge.
   
Use of ' --ignore-date' could be mildly controversial. It's not
  strictly
necessary because the commits for 10070 will appear grouped in
  history,
   but
then dates on commits will be discontiguous in that section of
  history.
   I
suggest using that option so the order of commits and dates sort
 the
   same
on master.
   
   
On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar e...@apache.org
  wrote:
   
Hi,
   
Last week we started some discussion[4] for merging branch
   hbase-10070[1]
into trunk. It seems like the consensus there is to do the merge
  sooner
rather than later.
   
   
We had branched hbase-10070 in Feb out of trunk[5]. The branch
  contains
55
jiras committed[2]. Out of these 55, 15 has already been
 committed to
trunk
and backported to hbase-10070 branch[3].
   
This VOTE is for merging back the remaining changes in branch to
  trunk.
If
passes, we will rebase the branch on top of current trunk, in
 which
  we
will
keep the commit-per-issue log history. After that we will do a
 git
   merge
for the branch keeping the history clean and not squashing the
   commits. I
expect rebasing to be straightforward, however with some manual
   conflict
resolution. After the merge we'll keep running the tests to make
 sure
everything is ok.
   
An overview of the changes, and the status of the work can be
 found
   under

Re: [VOTE] Merge branch HBASE-10070 to trunk

[Jimmy Xiang]

Yes, the state for assignment is in the meta row.


On Fri, Jun 6, 2014 at 7:21 PM, Enis Söztutar enis@gmail.com wrote:

 On Fri, Jun 6, 2014 at 7:07 PM, Jimmy Xiang jxi...@cloudera.com wrote:

  First of all, I have not looked into the patches recently. I remember
 there
  are some changes to the public interface. I was wondering if it is
 backward
  compatible. Enis mentioned that it's rolling upgradable. Just want to
  confirm if it is backward compatible. For existing applications, do they
  need to recompile?
 

 I think you are referring to the patch for
 https://issues.apache.org/jira/browse/HBASE-10347. It changes HRI to add
 the replicaId and deprecates all client-facing interfaces using HRL.
 RegionLocations replaces that, but locating regions are no longer
 client-facing interfaces. Existing applications might have to re-compile,
 but we do not have to have binary-compatibility guarantee between 0.98 and
 1.0. AFAIK, non of the latest major releases had that guarantee to the
 previous major version (0.98 - 0.96 - 0.94).


 
  Second, there should be some conflict with HBASE-11059 I am working on
 now,
  which is almost finished. How should we resolve this issue? ZK-less
 region
  assignment doesn't know region replica yet.
 

 I have not looked at your patch yet. The region replica assignment does not
 change the assignment mechanics on zookeeper. So assigning a secondary
 region works without any changes there. However region replicas do not have
 explicit meta locations (they share the same meta row). I am not sure where
 you keep the state for assignment. Is it in the meta row?


 
  Thanks,
  Jimmy
 
 
 
 
  On Fri, Jun 6, 2014 at 6:12 PM, Jeffrey Zhong jzh...@hortonworks.com
  wrote:
 
  
   +1. It brings goodies such as region replica, cross region server
   scanget, anti-affinity of regions, and pave the way for sync region
   replication.
  
   Thanks,
   -Jeffrey
  
   On 6/6/14 2:42 PM, Devaraj Das d...@hortonworks.com wrote:
  
   +1
   
   On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell apurt...@apache.org
   wrote:
+1, thanks Enis
   
   
On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar enis@gmail.com
   wrote:
   
Sorry, I was mostly out for HadoopSummit.
   
Yes, the git flow would be very similar to what you propose:
   
 $ git checkout HBASE-10070
 $ git rebase --ignore-date master
 (fixups, git add, git rebase --continue, etc, etc, etc)
 $ git checkout master
   
 $ git push origin HBASE-10070 HBASE-10070-rebase-date
   (optionally)
 $ git merge HBASE-10070
   
We can either go --ignore-date or not depending on what we want. If
   needed
I am fine with pushing the rebased master branch for review to main
   repo
before the merge to another branch. If not, I can just rebase the
   branch
locally and merge + push to main repo.
   
Creating final patches and attaching them to jira might be
  cumbersome.
   If
we do the rebased-branch on repo, we might not need it. But if we
  need
   that
for review, I can do it.
   
Thanks,
Enis
   
   
   
On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell 
 apurt...@apache.org
  
wrote:
   
 I realize this is a vote thread but I need a satisfactory answer
 to
   the
 below inquiries before feeling comfortable casting a vote. Or
  perhaps
that
 means we need to cancel this vote and move back to discussion.


 On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell 
  apurt...@apache.org
   
 wrote:

  Also after the merge process is completed, do you plan to use
 git
  format-patch to break out the per-JIRA changes into updated
   patches for
  those JIRAs representing in effect the final commit?
 
 
  On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell
   apurt...@apache.org
  wrote:
 
 
  On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar 
 e...@apache.org
wrote:
 
  This VOTE is for merging back the remaining changes in branch
 to
trunk.
 If
  passes, we will rebase the branch on top of current trunk, in
   which
we
  will
  keep the commit-per-issue log history. After that we will do
 a
   git
 merge
  for the branch keeping the history clean and not squashing
 the
 commits. I
  expect rebasing to be straightforward, however with some
 manual
 conflict
  resolution. After the merge we'll keep running the tests to
  make
   sure
  everything is ok.
 
 
  Just to clarify that would look something like this:
 
   $ git checkout HBASE-10070
   $ git rebase --ignore-date master
   (fixups, git add, git rebase --continue, etc, etc, etc)
   $ git checkout master
   $ git merge HBASE-10070
 
  ?
 
  That sounds good to me, the final merge should be a fast
 forward
merge.
 
  Use of ' --ignore-date' could be mildly controversial. It's

Re: DISCUSSION: 1.0.0

[Jimmy Xiang]

For ZK-less assignment, I prefer it to go in before 1.0.  It's backward
compatible and rolling-upgradable.

I am doing more testing now.

Thanks,
Jimmy


On Mon, Jun 2, 2014 at 12:54 PM, Mikhail Antonov olorinb...@gmail.com
wrote:

 On ZK abstraction...

 What date tentatively we're talking as a release date for 1.0.0?

 I think Enis is right that technically as long as we're just changing
 private interfaces, we can do part in one release and the rest in
 subsequent one, yet also I agree with Andrew that having ZK
 partially-separated is a bit odd. On other consideration is that we can't
 do anything non-rolling-upgradable before 1.0 is out, IIUC.

 I would say - let's aim to have not-too-intrusive refactoring work as part
 of 1.0 (non-intrusive means - not changing control flow through ZK or data
 stored in ZooKeeper). If we end up in in situation when ZK abstraction is
 on the critical part for release - let's reduce the scope and leave certain
 areas (like replication queues) for the subsequent release. And from 1.0
 onwards - we can start redesigning the way we store shared state and tackle
 multiple active master/region replicas.

 Thoughts?

 Jimmy - what about ZK-less assignment, will it go in before 1.0?

 -Mikhail





 2014-06-02 11:49 GMT-07:00 Andrew Purtell apurt...@apache.org:

  On Mon, Jun 2, 2014 at 11:43 AM, Enis Söztutar enis@gmail.com
 wrote:
 
   On Mon, Jun 2, 2014 at 10:25 AM, Andrew Purtell apurt...@apache.org
   wrote:
  
An email from JIRA reminds me that we should also have the ZooKeeper
related refactoring complete in 1.0 before releasing it. That work is
pretty far along and needs all bits in place to be useful.
   
  
   Agreed that it will be good to get this completed. However, they are
  mostly
   internal interfaces and I am not sure whether all the changes required
  will
   be done in time. We can continue on this even after 1.0, no?
 
 
  ​I think it would be weird to have a release where we are partially on
 the
  way to plugging ZooKeeper in and out but not all the way, so that's not
  actually possible.​
 
 
 
  --
  Best regards,
 
 - Andy
 
  Problems worthy of attack prove their worth by hitting back. - Piet Hein
  (via Tom White)
 



 --
 Thanks,
 Michael Antonov

[jira] [Created] (HBASE-11279) Block cache could be disabled by mistake

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11279:
---

 Summary: Block cache could be disabled by mistake
 Key: HBASE-11279
 URL: https://issues.apache.org/jira/browse/HBASE-11279
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


There is a weird test failure:

{noformat}
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at 
org.apache.hadoop.hbase.io.hfile.TestPrefetch.readStoreFile(TestPrefetch.java:96)
at 
org.apache.hadoop.hbase.io.hfile.TestPrefetch.testPrefetch(TestPrefetch.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{noformat}

Looked into it, and found it could be because CacheConfig tries to compare a 
float with a long. Probably we should do this instead:

{noformat}
-if (cachePercentage == 0L) {
+if (cachePercentage = 0.0001f) {
{noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-11236) Last flushed sequence id is ignored by ServerManager

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-11236.
-

Resolution: Cannot Reproduce

Don't see it any more. Close it.

 Last flushed sequence id is ignored by ServerManager
 

 Key: HBASE-11236
 URL: https://issues.apache.org/jira/browse/HBASE-11236
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang

 I got lots of error messages like this:
 {quote}
 2014-05-22 08:58:59,793 DEBUG [RpcServer.handler=1,port=20020] 
 master.ServerManager: RegionServer 
 a2428.halxg.cloudera.com,20020,1400742071109 indicates a last flushed 
 sequence id (numberOfStores=9, numberOfStorefiles=2, 
 storefileUncompressedSizeMB=517, storefileSizeMB=517, 
 compressionRatio=1., memstoreSizeMB=0, storefileIndexSizeMB=0, 
 readRequestsCount=0, writeRequestsCount=0, rootIndexSizeKB=34, 
 totalStaticIndexSizeKB=381, totalStaticBloomSizeKB=0, totalCompactingKVs=0, 
 currentCompactedKVs=0, compactionProgressPct=NaN) that is less than the 
 previous last flushed sequence id (605446) for region 
 IntegrationTestBigLinkedList, 
 �A��*t�^FU�2��0,1400740489477.a44d3e309b5a7e29355f6faa0d3a4095. Ignoring.
 {quote}
 RegionLoad.toString doesn't print out the last flushed sequence id passed in. 
 Why is it less than the previous one?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: We have our first victim

[Jimmy Xiang]

+1. I was confused.


On Fri, May 23, 2014 at 10:22 AM, Andrew Purtell apurt...@apache.orgwrote:

 Ram pushed up a commit as a new branch 'trunk' instead of to 'master'. I
 will file an infra ticket to nuke the new branch 'trunk' as this is going
 to confuse people. Objections?

 --
 Best regards,

- Andy

 Problems worthy of attack prove their worth by hitting back. - Piet Hein
 (via Tom White)

Re: We have our first victim

[Jimmy Xiang]

Cool, thanks.

How about the merge? I know it's a different thing.
I was wondering if we have permission to do a force-push.


On Fri, May 23, 2014 at 11:39 AM, Andrew Purtell apurt...@apache.orgwrote:

 In any event the offending branch is gone. You can garbage collect the
 stale remote ref with 'git remote prune origin' if you have it (or
 substitute 'origin' for whatever you may have changed the ASF remote to if
 you renamed it).


 On Fri, May 23, 2014 at 11:31 AM, Andrew Purtell apurt...@apache.org
 wrote:

  I tried that before but it failed. I tried it again just now for sanity
  and it was allowed. I'm not sure if I mistyped or if a setting on the
 repo
  was changed as Jake has since assigned the infra ticket to himself.
 
 
  On Fri, May 23, 2014 at 11:25 AM, Todd Lipcon t...@cloudera.com wrote:
 
  Do you need an infra JIRA or can you just git push origin :trunk to
 delete
  it? (note the colon)
 
 
  On Fri, May 23, 2014 at 11:09 AM, Andrew Purtell apurt...@apache.org
  wrote:
 
   Ok, no objections so INFRA-7800
  
  
   On Fri, May 23, 2014 at 10:29 AM, Ted Yu yuzhih...@gmail.com wrote:
  
'trunk' should be removed in git repo.
   
   
On Fri, May 23, 2014 at 10:24 AM, Jimmy Xiang jxi...@cloudera.com
   wrote:
   
 +1. I was confused.


 On Fri, May 23, 2014 at 10:22 AM, Andrew Purtell 
  apurt...@apache.org
 wrote:

  Ram pushed up a commit as a new branch 'trunk' instead of to
   'master'.
I
  will file an infra ticket to nuke the new branch 'trunk' as this
  is
going
  to confuse people. Objections?
 
  --
  Best regards,
 
 - Andy
 
  Problems worthy of attack prove their worth by hitting back. -
  Piet
Hein
  (via Tom White)
 

   
  
  
  
   --
   Best regards,
  
  - Andy
  
   Problems worthy of attack prove their worth by hitting back. - Piet
 Hein
   (via Tom White)
  
 
 
 
  --
  Todd Lipcon
  Software Engineer, Cloudera
 
 
 
 
  --
  Best regards,
 
 - Andy
 
  Problems worthy of attack prove their worth by hitting back. - Piet Hein
  (via Tom White)
 



 --
 Best regards,

- Andy

 Problems worthy of attack prove their worth by hitting back. - Piet Hein
 (via Tom White)

Re: We have our first victim

[Jimmy Xiang]

I see. It's fine with me.


On Fri, May 23, 2014 at 11:49 AM, Andrew Purtell apurt...@apache.orgwrote:

 I advocate leaving it in place. We've had several commits to trunk after,
 from N. It's not worth the pain.


 On Fri, May 23, 2014 at 11:45 AM, Jimmy Xiang jxi...@cloudera.com wrote:

  Cool, thanks.
 
  How about the merge? I know it's a different thing.
  I was wondering if we have permission to do a force-push.
 
 
  On Fri, May 23, 2014 at 11:39 AM, Andrew Purtell apurt...@apache.org
  wrote:
 
   In any event the offending branch is gone. You can garbage collect the
   stale remote ref with 'git remote prune origin' if you have it (or
   substitute 'origin' for whatever you may have changed the ASF remote to
  if
   you renamed it).
  
  
   On Fri, May 23, 2014 at 11:31 AM, Andrew Purtell apurt...@apache.org
   wrote:
  
I tried that before but it failed. I tried it again just now for
 sanity
and it was allowed. I'm not sure if I mistyped or if a setting on the
   repo
was changed as Jake has since assigned the infra ticket to himself.
   
   
On Fri, May 23, 2014 at 11:25 AM, Todd Lipcon t...@cloudera.com
  wrote:
   
Do you need an infra JIRA or can you just git push origin :trunk to
   delete
it? (note the colon)
   
   
On Fri, May 23, 2014 at 11:09 AM, Andrew Purtell 
 apurt...@apache.org
wrote:
   
 Ok, no objections so INFRA-7800


 On Fri, May 23, 2014 at 10:29 AM, Ted Yu yuzhih...@gmail.com
  wrote:

  'trunk' should be removed in git repo.
 
 
  On Fri, May 23, 2014 at 10:24 AM, Jimmy Xiang 
  jxi...@cloudera.com
 wrote:
 
   +1. I was confused.
  
  
   On Fri, May 23, 2014 at 10:22 AM, Andrew Purtell 
apurt...@apache.org
   wrote:
  
Ram pushed up a commit as a new branch 'trunk' instead of to
 'master'.
  I
will file an infra ticket to nuke the new branch 'trunk' as
  this
is
  going
to confuse people. Objections?
   
--
Best regards,
   
   - Andy
   
Problems worthy of attack prove their worth by hitting
 back. -
Piet
  Hein
(via Tom White)
   
  
 



 --
 Best regards,

- Andy

 Problems worthy of attack prove their worth by hitting back. -
 Piet
   Hein
 (via Tom White)

   
   
   
--
Todd Lipcon
Software Engineer, Cloudera
   
   
   
   
--
Best regards,
   
   - Andy
   
Problems worthy of attack prove their worth by hitting back. - Piet
  Hein
(via Tom White)
   
  
  
  
   --
   Best regards,
  
  - Andy
  
   Problems worthy of attack prove their worth by hitting back. - Piet
 Hein
   (via Tom White)
  
 



 --
 Best regards,

- Andy

 Problems worthy of attack prove their worth by hitting back. - Piet Hein
 (via Tom White)

[jira] [Created] (HBASE-11236) Last flushed sequence id is ignored by ServerManager

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11236:
---

 Summary: Last flushed sequence id is ignored by ServerManager
 Key: HBASE-11236
 URL: https://issues.apache.org/jira/browse/HBASE-11236
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


HRegion.lastFlushSeqId is set to -1 at the beginning. So the first value master 
gets is a really a huge number instead since it is a uint64. That's why all 
valid last flushed sequence ids are ignored by the server manager.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: VOTE: Move to GIT

[Jimmy Xiang]

+1


On Mon, May 19, 2014 at 9:24 AM, Andrew Purtell apurt...@apache.org wrote:

 +1


 On Mon, May 19, 2014 at 8:32 AM, Stack st...@duboce.net wrote:

  Following up on the old DISCUSSION thread on moving to git [1], lets vote
  (INFRA needs to see a vote).
 
  Move to GIT?
 
  [  ] +1 on yes, lets move to GIT
  [  ] 0 on don't care
  [  ] -1 on stay w/ current SVN setup.
 
  I'm +1.
 
  St.Ack
  P.S. The Mighty Talat Uyarer, who volunteered to run the migration a
 while
  ago is back with a vengeance and up for the job.
 
  1. http://search-hadoop.com/m/WmXbV15hqqq
 



 --
 Best regards,

- Andy

 Problems worthy of attack prove their worth by hitting back. - Piet Hein
 (via Tom White)

[jira] [Resolved] (HBASE-11197) Region could remain unassigned if regionserver crashes

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-11197.
-

Resolution: Invalid

I think it's an issue in my local patch/branch. Close it as Invalid.

 Region could remain unassigned if regionserver crashes
 --

 Key: HBASE-11197
 URL: https://issues.apache.org/jira/browse/HBASE-11197
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang

 When looking into test failure: 
 testVisibilityLabelsOnKillingOfRSContainingLabelsTable
 and find this is what has happened:
 1. try to assign a region a region server;
 2. master creates a znode, and send an openRegion request to the rs;
 3. rs gets the request and sends back a response, then crashed;
 4. try to assign the region again with forceNewPlan = true;
 5. since the region is in transition, master tries to close it and get region 
 server stopped exception;
 6. master offlines the region and removes it from transition; but can't 
 assign the region since the dead server is not processed;
 7. now SSH finally kicks in, tries to assign this region again;
 8. SSH will fail to assign it since the znode is there already.
 We should clean up the znode in force offline a region.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11197) Region could remain unassigned if regionserver crashes

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11197:
---

 Summary: Region could remain unassigned if regionserver crashes
 Key: HBASE-11197
 URL: https://issues.apache.org/jira/browse/HBASE-11197
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


When looking into test failure: 
testVisibilityLabelsOnKillingOfRSContainingLabelsTable

and find this is what has happened:

1. try to assign a region a region server;
2. master creates a znode, and send an openRegion request to the rs;
3. rs gets the request and sends back a response, then crashed;
4. try to assign the region again with forceNewPlan = true;
5. since the region is in transition, master tries to close it and get region 
server stopped exception;
6. master offlines the region and removes it from transition; but can't assign 
the region since the dead server is not processed;
7. now SSH finally kicks in, tries to assign this region again;
8. SSH will fail to assign it since the znode is there already.

We should clean up the znode in force offline a region.




--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: scan filtering for certain rows by column family

[Jimmy Xiang]

After you create the scan object, you can use addFamily to specify the
families you are interested.


On Tue, May 6, 2014 at 4:29 PM, cnpeyton cnpey...@hotmail.com wrote:

 I want to do a scan of a table and only have rows that contain certain
 columns families be returned from the scan.  For example, lets say i have 2
 column families a, b, and c.  I want to set a scan up with a filter that
 would only give me rows that have data within column families a and c.  For
 example, if row 2 has only data in column family b, then it should not be
 returned.



 --
 View this message in context:
 http://apache-hbase.679495.n3.nabble.com/scan-filtering-for-certain-rows-by-column-family-tp4058919.html
 Sent from the HBase Developer mailing list archive at Nabble.com.

[jira] [Reopened] (HBASE-10923) Control where to put meta region

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang reopened HBASE-10923:
-


Re-open it. We need a way for the master to do the old-style deployment.

 Control where to put meta region
 

 Key: HBASE-10923
 URL: https://issues.apache.org/jira/browse/HBASE-10923
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang

 There is a concern on placing meta regions on the master, as in the comments 
 of HBASE-10569. I was thinking we should have a configuration for a load 
 balancer to decide where to put it.  Adjusting this configuration we can 
 control whether to put the meta on master, or other region server.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: HBase returns NoServerForRegionException

[Jimmy Xiang]

Does your application B call Configuration.addDefaultResource() somewhere?

Thanks,
Jimmy


On Mon, Apr 28, 2014 at 10:42 AM, Hotec04 hao.l...@dish.com wrote:

 Thank you Ted for looking into it.

 The hbase version is cdh4.5.0 and the api is hbase0.94.6-cdh4.5.0. The
 maven
 dependencies on hadoop and hbase are using hadoop-client: 2.0.0-cdh-4.5.0
 and hbase0.94.6-cdh4.5.0.

 Sincerely



 --
 View this message in context:
 http://apache-hbase.679495.n3.nabble.com/HBase-returns-NoServerForRegionException-tp4058667p4058678.html
 Sent from the HBase Developer mailing list archive at Nabble.com.

[jira] [Resolved] (HBASE-10923) Control where to put meta region

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-10923.
-

Resolution: Won't Fix

Close it as Won't Fix. Let's keep meta together with master for now. 

 Control where to put meta region
 

 Key: HBASE-10923
 URL: https://issues.apache.org/jira/browse/HBASE-10923
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang

 There is a concern on placing meta regions on the master, as in the comments 
 of HBASE-10569. I was thinking we should have a configuration for a load 
 balancer to decide where to put it.  Adjusting this configuration we can 
 control whether to put the meta on master, or other region server.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-9611) Remove empty ChecksumType.java in hbase-server

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-9611.


Resolution: Implemented

Can't find this class in hbase-server any more. Could have been removed already.

 Remove empty ChecksumType.java in hbase-server
 --

 Key: HBASE-9611
 URL: https://issues.apache.org/jira/browse/HBASE-9611
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor

 I could not build my hbase-server test classes with it.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-9063) TestAssignmentManagerOnCluster.testSSHWhenDisablingTableRegionsInOpeningOrPendingOpenState fails

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-9063.


Resolution: Fixed

 TestAssignmentManagerOnCluster.testSSHWhenDisablingTableRegionsInOpeningOrPendingOpenState
  fails
 

 Key: HBASE-9063
 URL: https://issues.apache.org/jira/browse/HBASE-9063
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: stack
Assignee: Jimmy Xiang
 Attachments: trunk-9063.patch


 https://builds.apache.org/job/hbase-0.95-on-hadoop2/200/testReport/org.apache.hadoop.hbase.master/TestAssignmentManagerOnCluster/testSSHWhenDisablingTableRegionsInOpeningOrPendingOpenState/
 {code}java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.regionOnline(AssignmentManager.java:1314)
   at 
 org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster.testSSHWhenDisablingTableRegionsInOpeningOrPendingOpenState(TestAssignmentManagerOnCluster.java:482)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at 
 org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74){code}
 Hope you don't mind my assigning it to you Jimmy.  Thought you might be 
 interested.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11059) ZK-less region assignment

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11059:
---

 Summary: ZK-less region assignment
 Key: HBASE-11059
 URL: https://issues.apache.org/jira/browse/HBASE-11059
 Project: HBase
  Issue Type: Improvement
  Components: master, Region Assignment
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang


It seems that most people don't like region assignment with ZK (HBASE-5487), 
which causes many uncertainties. This jira is to support ZK-less region 
assignment. We need to make sure this patch doesn't break backward 
compatibility/rolling upgrade.




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11047) Remove TimeoutMontior

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-11047:
---

 Summary: Remove TimeoutMontior
 Key: HBASE-11047
 URL: https://issues.apache.org/jira/browse/HBASE-11047
 Project: HBase
  Issue Type: Improvement
  Components: Region Assignment
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


With HBASE-8002, the TimeoutMonitor is disabled by default. Lately, we haven't 
see much problem of region assignments during integration testing with CM. I 
was thinking it may be time to remove the timeout monitor now?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10973) CatalogTracker can share ZK connection with regionserver

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-10973:
---

 Summary: CatalogTracker can share ZK connection with regionserver
 Key: HBASE-10973
 URL: https://issues.apache.org/jira/browse/HBASE-10973
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


Currently, CatalogTracker has its own ZK connection even when it is used in the 
server side. It could share one connection with the server.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-10973) CatalogTracker can share ZK connection with regionserver

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-10973.
-

Resolution: Later

Looked into it. Current implementation makes it very hard to fix this now.

 CatalogTracker can share ZK connection with regionserver
 

 Key: HBASE-10973
 URL: https://issues.apache.org/jira/browse/HBASE-10973
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor

 Currently, CatalogTracker has its own ZK connection even when it is used in 
 the server side. It could share one connection with the server.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10976) Start CatalogTracker after cluster ID is available

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-10976:
---

 Summary: Start CatalogTracker after cluster ID is available
 Key: HBASE-10976
 URL: https://issues.apache.org/jira/browse/HBASE-10976
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Trivial


CatalogTracker is started before cluster ID is created now. So the HConnection 
doesn't have the cluster information.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10949) Meta scan could hang

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-10949:
---

 Summary: Meta scan could hang
 Key: HBASE-10949
 URL: https://issues.apache.org/jira/browse/HBASE-10949
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Priority: Critical
 Fix For: 0.99.0


When I play with ITBLL (from trunk tip), sometimes, meta scan hangs when the 
cluster is rolling restarted. When this happens, the master takes about 1000% 
of CPU. It looks like there is an infinite loop somewhere.  The logs show 
nothing interesting except some meta scanner RPC calls timed out. Jstask shows 
the 10 high QoS RPC handlers are busy with meta scanning.

However, if I run it again without HBASE-10018, things are fine. I suspect 
there is something to do with the small/reverse scan.

By the way, I see this problem even with log replay off and hfile version = 2.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Reopened] (HBASE-10897) On master start, deadlock if refresh UI

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang reopened HBASE-10897:
-


With this fix, the master web UI is not available while it's finishing the 
initialization. Perhaps we just need to wait till meta is assigned? Let me take 
another look.

 On master start, deadlock if refresh UI
 ---

 Key: HBASE-10897
 URL: https://issues.apache.org/jira/browse/HBASE-10897
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: stack
Assignee: Jimmy Xiang
 Fix For: 0.99.0

 Attachments: hbase-10897.patch


 Playing w/ MTTR recovery on trunk, master starting up deadlocked:
 Waiting to finish active master initialization:
 {code}
 ActiveMasterManager daemon prio=10 tid=0x7fafb5dc3800 nid=0x5fb5 
 waiting for monitor entry [0x7faf8f57d000]
java.lang.Thread.State: BLOCKED (on object monitor)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveZooKeeperWatcher(ConnectionManager.java:1683)
 - waiting to lock 0x00064ab4b9a8 (a java.lang.Object)
 at 
 org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:53)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1029)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:989)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:830)
 at 
 org.apache.hadoop.hbase.client.ConnectionAdapter.getRegionLocation(ConnectionAdapter.java:305)
 at 
 org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:77)
 at 
 org.apache.hadoop.hbase.client.ScannerCallable.prepare(ScannerCallable.java:118)
 at 
 org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:101)
 at 
 org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:264)
 at 
 org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:169)
 at 
 org.apache.hadoop.hbase.client.ClientScanner.init(ClientScanner.java:164)
 at 
 org.apache.hadoop.hbase.client.ClientScanner.init(ClientScanner.java:107)
 at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:766)
 at 
 org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:539)
 at 
 org.apache.hadoop.hbase.catalog.MetaReader.fullScanOfMeta(MetaReader.java:140)
 at 
 org.apache.hadoop.hbase.catalog.MetaMigrationConvertingToPB.isMetaTableUpdated(MetaMigrationConvertingToPB.java:164)
 at 
 org.apache.hadoop.hbase.catalog.MetaMigrationConvertingToPB.updateMetaIfNecessary(MetaMigrationConvertingToPB.java:131)
 at 
 org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:567)
 at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:147)
 at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1242)
 at java.lang.Thread.run(Thread.java:744)
 {code}
 ... but the master servlet has the lock while trying to access master:
 {code}
 686004346@qtp-2101021459-0 daemon prio=10 tid=0x7fafb5d2a800 nid=0x5fb1 
 waiting on condition [0x7faf8f87f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
 at java.lang.Thread.sleep(Native Method)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1562)
 - locked 0x00064ab4b9a8 (a java.lang.Object)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1597)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1805)
 - locked 0x00064ab4b9a8 (a java.lang.Object)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.listTables(ConnectionManager.java:2481)
 at 
 org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:321)
 at 
 org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__userTables(MasterStatusTmplImpl.java:530)
 at 
 org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:255)
 at 
 org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:382)
 at 
 org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:372

[jira] [Created] (HBASE-10938) Use shortcut connection for namespace manager

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-10938:
---

 Summary: Use shortcut connection for namespace manager
 Key: HBASE-10938
 URL: https://issues.apache.org/jira/browse/HBASE-10938
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
 Fix For: 0.99.0


In case the namespace table is assigned to the master, TableNamespaceManager 
should use the shortcut connection.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-10938) Use shortcut connection for namespace manager

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-10938.
-

   Resolution: Won't Fix
Fix Version/s: (was: 0.99.0)

No need to fix it now.

 Use shortcut connection for namespace manager
 -

 Key: HBASE-10938
 URL: https://issues.apache.org/jira/browse/HBASE-10938
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor

 In case the namespace table is assigned to the master, TableNamespaceManager 
 should use the shortcut connection.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10923) Control where to put meta region

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-10923:
---

 Summary: Control where to put meta region
 Key: HBASE-10923
 URL: https://issues.apache.org/jira/browse/HBASE-10923
 Project: HBase
  Issue Type: Improvement
Reporter: Jimmy Xiang


There is a concern on placing meta regions on the master, as in the comments of 
HBASE-10569. I was thinking we should have a configuration for a load balancer 
to decide where to put it.  Adjusting this configuration we can control whether 
to put the meta on master, or other region server.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-10816) CatalogTracker abortable usage

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-10816.
-

Resolution: Won't Fix

I looked into it, and prefer not to do anything for now.

 CatalogTracker abortable usage
 --

 Key: HBASE-10816
 URL: https://issues.apache.org/jira/browse/HBASE-10816
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor

 In reviewing patch for HBASE-10569, Stack pointed out some existing issue 
 with CatalogTracker. I looked into it and I think the abortable usage can be 
 improved.
 * If ZK is null, when a new one is created, the abortable could be null. We 
 need consider this.
 * The throwableAborter is to abort the process in case some ZK exception in 
 MetaRegionTracker. In case the tracker is in a server, we don't need to do 
 this, we can use the server as the abortable. In case the tracker is in a 
 client, we can just abort the connection. Right?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10911) ServerShutdownHandler#toString shows meaningless message

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-10911:
---

 Summary: ServerShutdownHandler#toString shows meaningless message
 Key: HBASE-10911
 URL: https://issues.apache.org/jira/browse/HBASE-10911
 Project: HBase
  Issue Type: Improvement
  Components: master
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor


SSH#toString returns the master server name, which is not so interesting. It's 
better to show the dead server's name instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-10897) On master start, deadlock if refresh UI

[Jimmy Xiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang resolved HBASE-10897.
-

   Resolution: Fixed
Fix Version/s: 0.99.0
 Assignee: Jimmy Xiang
 Hadoop Flags: Reviewed

Committed to trunk (as-is). Thanks Stack for the review, and finding out the 
issue.

 On master start, deadlock if refresh UI
 ---

 Key: HBASE-10897
 URL: https://issues.apache.org/jira/browse/HBASE-10897
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.99.0
Reporter: stack
Assignee: Jimmy Xiang
 Fix For: 0.99.0

 Attachments: hbase-10897.patch


 Playing w/ MTTR recovery on trunk, master starting up deadlocked:
 Waiting to finish active master initialization:
 {code}
 ActiveMasterManager daemon prio=10 tid=0x7fafb5dc3800 nid=0x5fb5 
 waiting for monitor entry [0x7faf8f57d000]
java.lang.Thread.State: BLOCKED (on object monitor)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveZooKeeperWatcher(ConnectionManager.java:1683)
 - waiting to lock 0x00064ab4b9a8 (a java.lang.Object)
 at 
 org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:53)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1029)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:989)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:830)
 at 
 org.apache.hadoop.hbase.client.ConnectionAdapter.getRegionLocation(ConnectionAdapter.java:305)
 at 
 org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:77)
 at 
 org.apache.hadoop.hbase.client.ScannerCallable.prepare(ScannerCallable.java:118)
 at 
 org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:101)
 at 
 org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:264)
 at 
 org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:169)
 at 
 org.apache.hadoop.hbase.client.ClientScanner.init(ClientScanner.java:164)
 at 
 org.apache.hadoop.hbase.client.ClientScanner.init(ClientScanner.java:107)
 at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:766)
 at 
 org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:539)
 at 
 org.apache.hadoop.hbase.catalog.MetaReader.fullScanOfMeta(MetaReader.java:140)
 at 
 org.apache.hadoop.hbase.catalog.MetaMigrationConvertingToPB.isMetaTableUpdated(MetaMigrationConvertingToPB.java:164)
 at 
 org.apache.hadoop.hbase.catalog.MetaMigrationConvertingToPB.updateMetaIfNecessary(MetaMigrationConvertingToPB.java:131)
 at 
 org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:567)
 at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:147)
 at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1242)
 at java.lang.Thread.run(Thread.java:744)
 {code}
 ... but the master servlet has the lock while trying to access master:
 {code}
 686004346@qtp-2101021459-0 daemon prio=10 tid=0x7fafb5d2a800 nid=0x5fb1 
 waiting on condition [0x7faf8f87f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
 at java.lang.Thread.sleep(Native Method)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1562)
 - locked 0x00064ab4b9a8 (a java.lang.Object)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1597)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1805)
 - locked 0x00064ab4b9a8 (a java.lang.Object)
 at 
 org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.listTables(ConnectionManager.java:2481)
 at 
 org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:321)
 at 
 org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__userTables(MasterStatusTmplImpl.java:530)
 at 
 org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:255)
 at 
 org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:382)
 at 
 org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:372

[jira] [Created] (HBASE-10873) Control number of regions assigned to backup masters

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-10873:
---

 Summary: Control number of regions assigned to backup masters
 Key: HBASE-10873
 URL: https://issues.apache.org/jira/browse/HBASE-10873
 Project: HBase
  Issue Type: Improvement
  Components: Balancer
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.99.0


By default, a backup master is treated just like another regionserver. So it 
can host as many regions as other regionserver does. When the backup master 
becomes the active one, region balancer needs to move those user regions on 
this master to other region servers. To minimize the impact, it's better not to 
assign too many regions on backup masters. It may not be good to leave the 
backup masters idle and not host any region either.

We should make this adjustable so that users can control how many regions to 
assign to each backup master.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10874) Use just one Jetty server for master web UI

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-10874:
---

 Summary: Use just one Jetty server for master web UI
 Key: HBASE-10874
 URL: https://issues.apache.org/jira/browse/HBASE-10874
 Project: HBase
  Issue Type: Improvement
  Components: master, UI
Reporter: Jimmy Xiang


In HBase-10815, we introduced a separate Jetty server to redirect requests to 
original master web UI port to the new shared info server. The reason for the 
new Jetty server is because our InfoServer extends from the deprecated hadoop 
HttpServer that doesn't support listening to multiple ports. We should either 
have our own info server, or use the new version of HttpServer (if it supports 
listening to multiple ports). So that we have just one Jetty server for web UI.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10851) Wait for regionservers to join the cluster

[Jimmy Xiang (JIRA)]

Jimmy Xiang created HBASE-10851:
---

 Summary: Wait for regionservers to join the cluster
 Key: HBASE-10851
 URL: https://issues.apache.org/jira/browse/HBASE-10851
 Project: HBase
  Issue Type: Bug
Reporter: Jimmy Xiang


With HBASE-10569, if regionservers are started a while after the master, all 
regions will be assigned to the master.  That may not be what users expect.

A work-around is to always start regionservers before masters.

I was wondering if the master can wait a little for other regionservers to join.



--
This message was sent by Atlassian JIRA
(v6.2#6252)