[jira] [Created] (HBASE-25026) Create a metric to track scans that have no start row and/or stop row
ramkrishna.s.vasudevan created HBASE-25026: -- Summary: Create a metric to track scans that have no start row and/or stop row Key: HBASE-25026 URL: https://issues.apache.org/jira/browse/HBASE-25026 Project: HBase Issue Type: Improvement Affects Versions: 3.0.0-alpha-1, 2.4.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan A metric that indicates how many of the scan requests were without start row and/or stop row. Generally such queries may be wrongly written or may require better schema design and those may be some queries doing some sanity check to verify if their actual application logic has done the necessary updates and the all the expected rows are processed. We do have some logs at the RPC layer to see what queries take time but nothing as a metric. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] Snapshot meta table or system tables
On Mon, Sep 14, 2020 at 19:24 Sean Busbey wrote: > meta isn't the only system table and not all system tables have a > > representation in HDFS. i.e. the ACL table. > +1 On Mon, Sep 14, 2020 at 5:13 PM Nick Dimiduk wrote: > > > > > > Is it really a good idea to restore to a previous copy of meta? Wouldn't > it > > > be better to fix meta using hbck2? The only thing we know for certain > about > > > a past meta state is it's in the past. > > > > > > On Sat, Sep 12, 2020 at 9:27 AM Sean Busbey wrote: > > > > > > > Now that we have snapshots with TTLs, it would be helpful if we had > > > > automated periodic snapshots of system tables that e.g. hbck2 could > use to > > > > do a forcible restore in cases of severe system damage. > > > > > > > > On Sat, Sep 12, 2020, 08:49 张铎(Duo Zhang) > wrote: > > > > > > > > > When reviewing the snapshot related code, I found that we have some > code > > > > > for treating meta table. For example: > > > > > > > > > > > > > > > > > > > > https://github.com/apache/hbase/blob/e5ca9adc54f9f580f85d21d38217afa97aa79d68/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java#L195 > > > > > > > > > > But when I tried to actually snapshot meta from client(in a UT), I > got > > > > the > > > > > following error message: > > > > > > > > > > System table snapshots are not allowed > > > > > > > > > > It is thrown from this method: > > > > > > > > > > > > > > > > > > > > https://github.com/apache/hbase/blob/e5ca9adc54f9f580f85d21d38217afa97aa79d68/hbase-client/src/main/java/org/apache/hadoop/hbase/snapshot/ClientSnapshotDescriptionUtils.java#L42 > > > > > > > > > > So I'm wondering whether we want to support snapshotting system > tables or > > > > > not. > > > > > > > > > > In general, at least for meta table, we can not disable it nor > delete it, > > > > > so even if we could support snapshotting it, we can not restore the > > > > > snapshot... > > > > > > > > > > Thoughts? Thanks. > > > > > > > > > > > > > > > > > -- > > Sean > >
Re: [DISCUSS] Snapshot meta table or system tables
meta isn't the only system table and not all system tables have a representation in HDFS. i.e. the ACL table. On Mon, Sep 14, 2020 at 5:13 PM Nick Dimiduk wrote: > > Is it really a good idea to restore to a previous copy of meta? Wouldn't it > be better to fix meta using hbck2? The only thing we know for certain about > a past meta state is it's in the past. > > On Sat, Sep 12, 2020 at 9:27 AM Sean Busbey wrote: > > > Now that we have snapshots with TTLs, it would be helpful if we had > > automated periodic snapshots of system tables that e.g. hbck2 could use to > > do a forcible restore in cases of severe system damage. > > > > On Sat, Sep 12, 2020, 08:49 张铎(Duo Zhang) wrote: > > > > > When reviewing the snapshot related code, I found that we have some code > > > for treating meta table. For example: > > > > > > > > > > > https://github.com/apache/hbase/blob/e5ca9adc54f9f580f85d21d38217afa97aa79d68/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java#L195 > > > > > > But when I tried to actually snapshot meta from client(in a UT), I got > > the > > > following error message: > > > > > > System table snapshots are not allowed > > > > > > It is thrown from this method: > > > > > > > > > > > https://github.com/apache/hbase/blob/e5ca9adc54f9f580f85d21d38217afa97aa79d68/hbase-client/src/main/java/org/apache/hadoop/hbase/snapshot/ClientSnapshotDescriptionUtils.java#L42 > > > > > > So I'm wondering whether we want to support snapshotting system tables or > > > not. > > > > > > In general, at least for meta table, we can not disable it nor delete it, > > > so even if we could support snapshotting it, we can not restore the > > > snapshot... > > > > > > Thoughts? Thanks. > > > > > -- Sean
Re: [DISCUSS] Snapshot meta table or system tables
Is it really a good idea to restore to a previous copy of meta? Wouldn't it be better to fix meta using hbck2? The only thing we know for certain about a past meta state is it's in the past. On Sat, Sep 12, 2020 at 9:27 AM Sean Busbey wrote: > Now that we have snapshots with TTLs, it would be helpful if we had > automated periodic snapshots of system tables that e.g. hbck2 could use to > do a forcible restore in cases of severe system damage. > > On Sat, Sep 12, 2020, 08:49 张铎(Duo Zhang) wrote: > > > When reviewing the snapshot related code, I found that we have some code > > for treating meta table. For example: > > > > > > > https://github.com/apache/hbase/blob/e5ca9adc54f9f580f85d21d38217afa97aa79d68/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java#L195 > > > > But when I tried to actually snapshot meta from client(in a UT), I got > the > > following error message: > > > > System table snapshots are not allowed > > > > It is thrown from this method: > > > > > > > https://github.com/apache/hbase/blob/e5ca9adc54f9f580f85d21d38217afa97aa79d68/hbase-client/src/main/java/org/apache/hadoop/hbase/snapshot/ClientSnapshotDescriptionUtils.java#L42 > > > > So I'm wondering whether we want to support snapshotting system tables or > > not. > > > > In general, at least for meta table, we can not disable it nor delete it, > > so even if we could support snapshotting it, we can not restore the > > snapshot... > > > > Thoughts? Thanks. > > >
[jira] [Resolved] (HBASE-25002) Create simple pattern matching query for retrieving metrics matching the pattern
[ https://issues.apache.org/jira/browse/HBASE-25002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-25002. Hadoop Flags: Reviewed Resolution: Fixed > Create simple pattern matching query for retrieving metrics matching the > pattern > > > Key: HBASE-25002 > URL: https://issues.apache.org/jira/browse/HBASE-25002 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.2.5 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Currently we allow querying a metric with a specific metric name. But > generally say under the MBean sub=Tables under a server we might have 'n' > number of tables and we might be interested in a specific metric across all > tables. > So in such cases giving a simple pattern based query will help so that we can > retrieve all metrics that follow that pattern. > The other side effect is that we can also reduce the size of the json we pull > from the server to the querying client. Which are generally smaller scripts > which might not be able to process the big sized response JSON. > Thanks [~anoopsamjohn] for the suggestion. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25022) Remove 'hbase.testing.nocluster' config
[ https://issues.apache.org/jira/browse/HBASE-25022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25022. --- Hadoop Flags: Reviewed Resolution: Fixed Pushed to master and branch-2. Thanks [~zghao] for reviewing. > Remove 'hbase.testing.nocluster' config > --- > > Key: HBASE-25022 > URL: https://issues.apache.org/jira/browse/HBASE-25022 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is only used at two places in tests and easy to be removed, so let's just > remove it to make our code cleaner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25025) [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp
Reid Chan created HBASE-25025: - Summary: [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp Key: HBASE-25025 URL: https://issues.apache.org/jira/browse/HBASE-25025 Project: HBase Issue Type: Improvement Reporter: Reid Chan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25024) [Flake Test][branch-1] TestClientOperationInterrupt#testInterrupt50Percent
Reid Chan created HBASE-25024: - Summary: [Flake Test][branch-1] TestClientOperationInterrupt#testInterrupt50Percent Key: HBASE-25024 URL: https://issues.apache.org/jira/browse/HBASE-25024 Project: HBase Issue Type: Improvement Reporter: Reid Chan Fix For: 1.7.0 Sometimes threads could finish faster than interrupt() gets called. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25016) Should close ResultScanner in MetaTableAccessor.scanByRegionEncodedName
[ https://issues.apache.org/jira/browse/HBASE-25016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani resolved HBASE-25016. -- Resolution: Fixed > Should close ResultScanner in MetaTableAccessor.scanByRegionEncodedName > --- > > Key: HBASE-25016 > URL: https://issues.apache.org/jira/browse/HBASE-25016 > Project: HBase > Issue Type: Bug > Components: master, meta >Reporter: Duo Zhang >Assignee: niuyulin >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.4.0, 2.3.2 > > > It will cause resource leak. > Th good news is that this method is only used for cluster maintainance, such > as assign or unassign region manually, so it is less hurt. Anyway, we should > fix it ASAP. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25023) NPE while shutdown master node
lujie created HBASE-25023:
-
Summary: NPE while shutdown master node
Key: HBASE-25023
URL: https://issues.apache.org/jira/browse/HBASE-25023
Project: HBase
Issue Type: Bug
Reporter: lujie
while shutdown the master node, we can see the exception:
{code:java}
2020-09-14 06:48:29,530 ERROR [PEWorker-16] procedure2.ProcedureExecutor:
CODE-BUG: Uncaught runtime exception: pid=111, ppid=64, state=RUNNABLE,
locked=true; org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.assignment.RegionRemoteProcedureBase.execute(RegionRemoteProcedureBase.java:276)
at
org.apache.hadoop.hbase.master.assignment.RegionRemoteProcedureBase.execute(RegionRemoteProcedureBase.java:58)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:962)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1648)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:78)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1965)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
