[jira] [Created] (HBASE-19336) Improve rsgroup to allow assign all tables within a specified namespace from one group to another
xinxin fan created HBASE-19336:
--
Summary: Improve rsgroup to allow assign all tables within a
specified namespace from one group to another
Key: HBASE-19336
URL: https://issues.apache.org/jira/browse/HBASE-19336
Project: HBase
Issue Type: Improvement
Components: rsgroup
Affects Versions: 2.0.0-alpha-4
Reporter: xinxin fan
Currently, use can only assign tables within a namespace from one group to
another by writing all table names in move_tables_rsgroup command. Allowing to
assign all tables within a specifed namespace by only wirting namespace name is
useful.
Usage as follows:
{code:java}
hbase(main):055:0> move_tables_rsgroup 'default',['@ns1']
Took 2.2211 seconds
{code}
{code:java}
hbase(main):051:0* move_servers_tables_rsgroup
'rsgroup1',['hbase39.lt.163.org:60020'],['@ns1','@ns2','table3']
Took 15.3710 seconds
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19335) Fix waitUntilAllRegionsAssigned
Appy created HBASE-19335:
Summary: Fix waitUntilAllRegionsAssigned
Key: HBASE-19335
URL: https://issues.apache.org/jira/browse/HBASE-19335
Project: HBase
Issue Type: Bug
Reporter: Appy
Assignee: Appy
Found when debugging flaky test TestRegionObserverInterface#testRecovery.
In the end, the test does the following:
- Kills the RS
- Waits for all regions to be assigned
- Some validation (unrelated)
- Cleanup: delete table.
{noformat}
cluster.killRegionServer(rs1.getRegionServer().getServerName());
Threads.sleep(1000); // Let the kill soak in.
util.waitUntilAllRegionsAssigned(tableName);
LOG.info("All regions assigned");
verifyMethodResult(SimpleRegionObserver.class,
new String[] { "getCtPreReplayWALs", "getCtPostReplayWALs",
"getCtPreWALRestore",
"getCtPostWALRestore", "getCtPrePut", "getCtPostPut" },
tableName, new Integer[] { 1, 1, 2, 2, 0, 0 });
} finally {
util.deleteTable(tableName);
table.close();
}
}
{noformat}
However, looking at test logs, found that we had overlapping Assigns with
Unassigns. As a result, regions ended up 'stuck in RIT' and the test timeout.
Assigns were from the ServerCrashRecovery and Unassigns were from the
deleteTable cleanup.
Which begs the question, why did HBTU.waitUntilAllRegionsAssigned(tableName)
not wait until recovery was complete.
Answer: Looks like that function is only meant for sunny scenarios but not for
crashes. It iterates over meta and just [checks for *some value* in the server
column|https://github.com/apache/hbase/blob/cdc2bb17ff38dcbd273cf501aea565006e995a06/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java#L3421]
which is obviously present and equal to the server that was just killed.
This bug must be affecting other fault tolerance tests too and fixing it may
fix more than just one test, hopefully.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19334) User.runAsLoginUser not work in AccessController because it use a short circuited connection
Guanghao Zhang created HBASE-19334: -- Summary: User.runAsLoginUser not work in AccessController because it use a short circuited connection Key: HBASE-19334 URL: https://issues.apache.org/jira/browse/HBASE-19334 Project: HBase Issue Type: Bug Reporter: Guanghao Zhang -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19329) hbase regionserver log output error (quota)
[ https://issues.apache.org/jira/browse/HBASE-19329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser resolved HBASE-19329. Resolution: Invalid Please use the user mailing list for help in debugging your system. JIRA is not the place for this. https://hbase.apache.org/mail-lists.html > hbase regionserver log output error (quota) > > > Key: HBASE-19329 > URL: https://issues.apache.org/jira/browse/HBASE-19329 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: gehaijiang > > 2017-11-16 02:50:33,474 WARN > [blackstone064030,16020,1510632966258_ChoreService_1] quotas.QuotaCache: > Unable to read user from quota table > org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 3 > actions: Table 'hbase:quota' was not found, got: hbase:namespace.: 3 times, > servers with issues: null > , > at > org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:330) > at > org.apache.hadoop.hbase.quotas.QuotaUtil.fetchUserQuotas(QuotaUtil.java:155) > at > org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$3.fetchEntries(QuotaCache.java:256) > at > org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:290) > at > org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchUserQuotaState(QuotaCache.java:248) > at > org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:213) > 2017-11-16 02:55:33,453 WARN > [blackstone064030,16020,1510632966258_ChoreService_1] quotas.QuotaCache: > Unable to read namespace from quota table > org.apache.hadoop.hbase.TableNotFoundException: Table 'hbase:quota' was not > found, got: hbase:namespace. > at > org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:330) > at > org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:220) > at > org.apache.hadoop.hbase.quotas.QuotaUtil.fetchNamespaceQuotas(QuotaUtil.java:207) > at > org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$1.fetchEntries(QuotaCache.java:226) > at > org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:290) > at > org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchNamespaceQuotaState(QuotaCache.java:218) > at > org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:211) > 2017-11-16 02:55:33,488 WARN > [blackstone064030,16020,1510632966258_ChoreService_1] quotas.QuotaCache: > Unable to read table from quota table > org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed > 47 actions: Table 'hbase:quota' was not found, got: hbase:namespace.: 47 > times, servers with issues: nu > ll, -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19333) Consider exposing ExportSnapshot#getSnapshotFiles through POJO class
[
https://issues.apache.org/jira/browse/HBASE-19333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-19333.
---
Resolution: Incomplete
Resolving as incomplete/invalid.
> Consider exposing ExportSnapshot#getSnapshotFiles through POJO class
>
>
> Key: HBASE-19333
> URL: https://issues.apache.org/jira/browse/HBASE-19333
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>
> In the thread,
> http://search-hadoop.com/m/HBase/YGbbUxY9FyU74X?subj=Re+Deleting+and+cleaning+old+snapshots+exported+to+S3
> , Timothy mentioned that he used reflection to get to
> ExportSnapshot#getSnapshotFiles().
> {code}
> private static List> getSnapshotFiles(final
> Configuration conf,
> final FileSystem fs, final Path snapshotDir) throws IOException {
> {code}
> SnapshotFileInfo is protobuf.
> We should consider exposing the API by replacing the protobuf class with POJO
> class.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19333) Consider exposing ExportSnapshot#getSnapshotFiles through POJO class
Ted Yu created HBASE-19333:
--
Summary: Consider exposing ExportSnapshot#getSnapshotFiles through
POJO class
Key: HBASE-19333
URL: https://issues.apache.org/jira/browse/HBASE-19333
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
In the thread,
http://search-hadoop.com/m/HBase/YGbbUxY9FyU74X?subj=Re+Deleting+and+cleaning+old+snapshots+exported+to+S3
, Timothy mentioned that he used reflection to get to
ExportSnapshot#getSnapshotFiles().
{code}
private static List> getSnapshotFiles(final
Configuration conf,
final FileSystem fs, final Path snapshotDir) throws IOException {
{code}
SnapshotFileInfo is protobuf.
We should consider exposing the API by replacing the protobuf class with POJO
class.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19332) DumpReplicationQueues misreports total WAL size
Gary Helmling created HBASE-19332: - Summary: DumpReplicationQueues misreports total WAL size Key: HBASE-19332 URL: https://issues.apache.org/jira/browse/HBASE-19332 Project: HBase Issue Type: Bug Components: Replication Reporter: Gary Helmling Assignee: Gary Helmling Priority: Trivial DumpReplicationQueues uses an int to collect the total WAL size for a queue. Predictably, this overflows much of the time. Let's use a long instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19321) ZKAsyncRegistry ctor would hang when zookeeper cluster is not available
[
https://issues.apache.org/jira/browse/HBASE-19321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-19321.
Resolution: Fixed
> ZKAsyncRegistry ctor would hang when zookeeper cluster is not available
> ---
>
> Key: HBASE-19321
> URL: https://issues.apache.org/jira/browse/HBASE-19321
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Guoquan Wu
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19321.master.001.patch
>
>
> From
> https://builds.apache.org/job/HBASE-Flaky-Tests/23477/testReport/junit/org.apache.hadoop.hbase.client/TestAdmin2/testCheckHBaseAvailableWithoutCluster/
> :
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 30
> milliseconds
> at
> org.apache.hadoop.hbase.client.TestAdmin2.testCheckHBaseAvailableWithoutCluster(TestAdmin2.java:573)
> {code}
> It seems this started hanging after HBASE-19313
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19331) Region start-key/end-key corruption in Hbase meta table
Shamith kumar created HBASE-19331: - Summary: Region start-key/end-key corruption in Hbase meta table Key: HBASE-19331 URL: https://issues.apache.org/jira/browse/HBASE-19331 Project: HBase Issue Type: Bug Components: Region Assignment Affects Versions: 0.98.8 Environment: Reproduced on HBase 0.98.8 on hadoop-2 Reporter: Shamith kumar when a region split happens on a key with trailing byte equals zero, the end key of the first resulting region and and start key of the second resulting region in meta table gets corrupted. Here is the link to code to reproduce this issue https://bitbucket.org/flytxt/hbase-meta-corruption-test -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19214) Align scan in Table and AsyncTable
[ https://issues.apache.org/jira/browse/HBASE-19214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-19214. --- Resolution: Won't Fix > Align scan in Table and AsyncTable > -- > > Key: HBASE-19214 > URL: https://issues.apache.org/jira/browse/HBASE-19214 > Project: HBase > Issue Type: Sub-task > Components: API, scan >Affects Versions: 2.0.0-alpha-4 >Reporter: Peter Somogyi >Priority: Minor > Fix For: 2.0.0 > > > Both interfaces include getScanner methods with the same signature. > Additionally, AsyncTable has scan and scanAll methods which are not part of > Table interface. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-19301) Provide way for CPs to create short circuited connection with custom configurations
[ https://issues.apache.org/jira/browse/HBASE-19301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-19301: --- Let me reopen till we address [~zghaobac] comment. > Provide way for CPs to create short circuited connection with custom > configurations > --- > > Key: HBASE-19301 > URL: https://issues.apache.org/jira/browse/HBASE-19301 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19301.patch, HBASE-19301_V2.patch, > HBASE-19301_V2.patch > > > Over in HBASE-18359 we have discussions for this. > Right now HBase provide getConnection() in RegionCPEnv, MasterCPEnv etc. But > this returns a pre created connection (per server). This uses the configs at > hbase-site.xml at that server. > Phoenix needs creating connection in CP with some custom configs. Having this > custom changes in hbase-site.xml is harmful as that will affect all > connections been created at that server. > This issue is for providing an overloaded getConnection(Configuration) API -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-19321) ZKAsyncRegistry ctor would hang when zookeeper cluster is not available
[
https://issues.apache.org/jira/browse/HBASE-19321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reopened HBASE-19321:
Guoquan:
TestAcidGuarantees started to timeout (again).
Check flaky test dashbaord:
https://builds.apache.org/job/HBASE-Flaky-Tests/23484/
> ZKAsyncRegistry ctor would hang when zookeeper cluster is not available
> ---
>
> Key: HBASE-19321
> URL: https://issues.apache.org/jira/browse/HBASE-19321
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Guoquan Wu
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19321.master.001.patch
>
>
> From
> https://builds.apache.org/job/HBASE-Flaky-Tests/23477/testReport/junit/org.apache.hadoop.hbase.client/TestAdmin2/testCheckHBaseAvailableWithoutCluster/
> :
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 30
> milliseconds
> at
> org.apache.hadoop.hbase.client.TestAdmin2.testCheckHBaseAvailableWithoutCluster(TestAdmin2.java:573)
> {code}
> It seems this started hanging after HBASE-19313
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19330) Remove duplicated dependency from hbase-rest
Peter Somogyi created HBASE-19330: - Summary: Remove duplicated dependency from hbase-rest Key: HBASE-19330 URL: https://issues.apache.org/jira/browse/HBASE-19330 Project: HBase Issue Type: Bug Components: dependencies Affects Versions: 2.0.0-alpha-4 Reporter: Peter Somogyi Assignee: Peter Somogyi Priority: Trivial Fix For: 2.0.0-beta-1 In hbase-rest module hbase-hadoop-compat dependency is listed twice. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19329) hbase regionserver log output error (quota)
gehaijiang created HBASE-19329: -- Summary: hbase regionserver log output error (quota) Key: HBASE-19329 URL: https://issues.apache.org/jira/browse/HBASE-19329 Project: HBase Issue Type: Bug Affects Versions: 1.3.1 Reporter: gehaijiang 2017-11-16 02:50:33,474 WARN [blackstone064030,16020,1510632966258_ChoreService_1] quotas.QuotaCache: Unable to read user from quota table org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 3 actions: Table 'hbase:quota' was not found, got: hbase:namespace.: 3 times, servers with issues: null , at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:330) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchUserQuotas(QuotaUtil.java:155) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$3.fetchEntries(QuotaCache.java:256) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:290) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchUserQuotaState(QuotaCache.java:248) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:213) 2017-11-16 02:55:33,453 WARN [blackstone064030,16020,1510632966258_ChoreService_1] quotas.QuotaCache: Unable to read namespace from quota table org.apache.hadoop.hbase.TableNotFoundException: Table 'hbase:quota' was not found, got: hbase:namespace. at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:330) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:220) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchNamespaceQuotas(QuotaUtil.java:207) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$1.fetchEntries(QuotaCache.java:226) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:290) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchNamespaceQuotaState(QuotaCache.java:218) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:211) 2017-11-16 02:55:33,488 WARN [blackstone064030,16020,1510632966258_ChoreService_1] quotas.QuotaCache: Unable to read table from quota table org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 47 actions: Table 'hbase:quota' was not found, got: hbase:namespace.: 47 times, servers with issues: nu ll, -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19328) Remove asked if splittable log messages
Balazs Meszaros created HBASE-19328:
---
Summary: Remove asked if splittable log messages
Key: HBASE-19328
URL: https://issues.apache.org/jira/browse/HBASE-19328
Project: HBase
Issue Type: Task
Components: proc-v2
Affects Versions: 3.0.0
Reporter: Balazs Meszaros
Priority: Minor
I have found this log message in HBase log:
{code}
2017-11-22 11:16:54,133 INFO
[RpcServer.priority.FPBQ.Fifo.handler=5,queue=0,port=52586]
regionserver.HRegion(1309): ASKED IF SPLITTABLE true
0a66d6e20801eec2c6cd1204fedde592
java.lang.Throwable: LOGGING: REMOVE
at
org.apache.hadoop.hbase.regionserver.HRegion.isSplittable(HRegion.java:1310)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegionInfo(RSRpcServices.java:1665)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:28159)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:325)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:305)
{code}
Still we need this?
It was introduced in commit {{dc1065a85}} by [~stack] and [~mbertozzi].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19327) Update the import layout for formatter
Chia-Ping Tsai created HBASE-19327: -- Summary: Update the import layout for formatter Key: HBASE-19327 URL: https://issues.apache.org/jira/browse/HBASE-19327 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai HBASE-19262 changed the import order rule - the shaded* imports is in separate group now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19326) Remove dead servers from rsgroup
Guangxu Cheng created HBASE-19326:
-
Summary: Remove dead servers from rsgroup
Key: HBASE-19326
URL: https://issues.apache.org/jira/browse/HBASE-19326
Project: HBase
Issue Type: New Feature
Components: rsgroup
Affects Versions: 3.0.0, 2.0.0-beta-2
Reporter: Guangxu Cheng
Assignee: Guangxu Cheng
In HBASE-18131, we add an hbase shell command {{clear_deadservers}} to clear
deadserver list in ServerManager.
But rsgroup still contains these dead servers, so we should also remove dead
servers from the group information.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-16137) Fix findbugs warning introduced by hbase-14730
[ https://issues.apache.org/jira/browse/HBASE-16137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-16137. --- Resolution: Won't Fix > Fix findbugs warning introduced by hbase-14730 > -- > > Key: HBASE-16137 > URL: https://issues.apache.org/jira/browse/HBASE-16137 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0, 1.3.0 >Reporter: huaxiang sun >Assignee: Peter Somogyi >Priority: Minor > > From stack: > "Lads. This patch makes for a new findbugs warning: > https://builds.apache.org/job/PreCommit-HBASE-Build/2390/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html > If you are good w/ the code, i can fix the findbugs warning... just say." -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19325) Pass a list of server name to postClearDeadServers
Guangxu Cheng created HBASE-19325:
-
Summary: Pass a list of server name to postClearDeadServers
Key: HBASE-19325
URL: https://issues.apache.org/jira/browse/HBASE-19325
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0-beta-2
Reporter: Guangxu Cheng
Assignee: Guangxu Cheng
Over on the tail of HBASE-18131. [~chia7712] said
{quote}
(Revisiting the AccessController remind me of this issue)
Could we remove the duplicate code on the server side? Why not pass a list of
server name to postClearDeadServers and postListDeadServers?
{quote}
The duplicate code has been removed in HBASE-19131.Now Pass a list of server
name to postClearDeadServers
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
