[jira] [Created] (HBASE-25402) HBase UI: sorting order by start key or end key is not considering empty start key/end key
Ajeet Rai created HBASE-25402: - Summary: HBase UI: sorting order by start key or end key is not considering empty start key/end key Key: HBASE-25402 URL: https://issues.apache.org/jira/browse/HBASE-25402 Project: HBase Issue Type: Bug Components: UI Affects Versions: 2.2.3 Reporter: Ajeet Rai HBase UI: sorting order by start key or end key is not considering empty start key/end key. Steps: 1: Open any table page having multiple regions 2: click on sort button of start key/end key 3: Observed that sorting doesn't consider start and end key. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25371) When openRegion fails during initial verification(before initializing and setting seq num), exception is observed during region close.
Ajeet Rai created HBASE-25371: - Summary: When openRegion fails during initial verification(before initializing and setting seq num), exception is observed during region close. Key: HBASE-25371 URL: https://issues.apache.org/jira/browse/HBASE-25371 Project: HBase Issue Type: Bug Affects Versions: 2.2.3 Reporter: Ajeet Rai When openRegion fails during initial verification(before initializing and setting seq num), exception is observed during region close: 2020-12-03 16:34:47,133 ERROR [RS_OPEN_REGION-regionserver/AA:16040-0] handler.OpenRegionHandler: Failed open of region=ns2:testtable4,15,1606912406234.cd386135276b7d3c57416df3666e4aea.2020-12-03 16:34:47,133 ERROR [RS_OPEN_REGION-regionserver/blrphispra01054:16040-0] handler.OpenRegionHandler: Failed open of region=ns2:testtable4,15,1606912406234.cd386135276b7d3c57416df3666e4aea.java.io.IOException: The new max sequence id 1 is less than the old max sequence id 7134 at org.apache.hadoop.hbase.wal.WALSplitUtil.writeRegionSequenceIdFile(WALSplitUtil.java:418) at org.apache.hadoop.hbase.regionserver.HRegion.writeRegionCloseMarker(HRegion.java:1253) at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1793) at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1606) at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1552) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7522) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7467) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7439) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7397) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7348) at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:286) at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:111) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25301) NPE while running balance_rsgroup if any split region is present.
Ajeet Rai created HBASE-25301: - Summary: NPE while running balance_rsgroup if any split region is present. Key: HBASE-25301 URL: https://issues.apache.org/jira/browse/HBASE-25301 Project: HBase Issue Type: Bug Components: rsgroup Affects Versions: 2.2.3 Reporter: Ajeet Rai NPE is thrown in below scenario: 1: Create rsgroup pgroup 2: Add two RS to pgroup 3: create two table with 15 region each 4: kill 1 RS and start again 5: disable t1 6: run balancer and balance_rsgroup 'pgroup' and observe it is working fine 7: now split one of the region of t2 8: run balance_rsgroup 'pgroup' and observe NPE: ERROR: java.io.IOException at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:517) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318) Caused by: java.lang.NullPointerException at java.util.TreeMap.put(TreeMap.java:563) at org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.correctAssignments(RSGroupBasedLoadBalancer.java:347) at org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.balanceCluster(RSGroupBasedLoadBalancer.java:140) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.balanceRSGroup(RSGroupAdminServer.java:531) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.balanceRSGroup(RSGroupAdminEndpoint.java:302) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16306) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:1023) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:458) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25077) [HBCK2]hbck.jsp page loading fails, logs NPE in master log.
Ajeet Rai created HBASE-25077: - Summary: [HBCK2]hbck.jsp page loading fails, logs NPE in master log. Key: HBASE-25077 URL: https://issues.apache.org/jira/browse/HBASE-25077 Project: HBase Issue Type: Bug Affects Versions: 2.2.3 Reporter: Ajeet Rai While testing with large data, it is observed that some time hbck page was not accessible and NPE was thrown in log. 【Test step】Excat Steps are not known. Observed this while executing below steps 1: Test data: 9K regions,~300 in my test setup 2: run list inconsistency with run option 3: run hbck_chore and catalogjanitor_run multiple times 4: Also run balancer so that region movement is happening. 5: Observe that HBCK page not available 2020-09-21 20:05:31,706 WARN [qtp1698322791-94] server.HttpChannel: /hbck.jsp java.lang.NullPointerException at org.apache.hadoop.hbase.generated.master.hbck_jsp.formatServerName(hbck_jsp.java:37) at org.apache.hadoop.hbase.generated.master.hbck_jsp._jspService(hbck_jsp.java:144) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:844) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24977) [Alter hbase:meta]: Making meta read only should not be be allowed
Ajeet Rai created HBASE-24977:
-
Summary: [Alter hbase:meta]: Making meta read only should not be
be allowed
Key: HBASE-24977
URL: https://issues.apache.org/jira/browse/HBASE-24977
Project: HBase
Issue Type: Bug
Components: meta
Affects Versions: 2.2.3
Reporter: Ajeet Rai
After HBASE-23055, Alter of Meta schema is allowed.
After this change meta table can be made Read only table which is not correct
as per my view,because it will impact majority of the functionality of HBase.
alter 'hbase:meta',\{METHOD=>'table_att', READONLY=>true}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24951) HBCK2: extra characters "\x00" are added in endkey while creating new region during fixHdfsOrphan operations
Ajeet Rai created HBASE-24951:
-
Summary: HBCK2: extra characters "\x00" are added in endkey while
creating new region during fixHdfsOrphan operations
Key: HBASE-24951
URL: https://issues.apache.org/jira/browse/HBASE-24951
Project: HBase
Issue Type: Bug
Components: hbck2
Affects Versions: 2.2.3
Reporter: Ajeet Rai
extra characters "\x00" are added in endkey while creating new region during
fixHdfsOrphan operations
See the log
ERROR: Found inconsistency in table mob_table
14:19:06.946 [main] INFO org.apache.hbase.hbck1.HBaseFsck - Attempting to
handle orphan hdfs dir:
hdfs://hacluster/hbase/data/default/mob_table/156d2bf587ee03a7afc53cbf00ebce38
14:19:06.974 [main] WARN org.apache.hbase.hbck1.HBaseFsck - Problem reading
orphan file
HdfsNamedFileStatus\{path=hdfs://hacluster/hbase/data/default/mob_table/156d2bf587ee03a7afc53cbf00ebce38/recovered.edits/4.seqid;
isDirectory=false; length=0; replication=3; blocksize=268435456;
modification_time=1598246777496; access_time=1598246777487; owner=hdfs;
group=hadoop; permission=rw-r--r--; isSymlink=false; hasAcl=false;
isEncrypted=false; isErasureCoded=false}, skipping
14:19:06.975 [main] INFO org.apache.hbase.hbck1.HBaseFsck - Min max keys are :
[889091ff744069cab08dc605d162a8d3-6482, 9996535e07258a7bbfd8b132435c5962-1235)
14:19:06.975 [main] INFO org.apache.hbase.hbck1.HBaseFsck - Creating new region
: {ENCODED => 364959ef65602934940f627c7a303eaf, NAME =>
'mob_table,889091ff744069cab08dc605d162a8d3-6482,1598249946975.364959ef65602934940f627c7a303eaf.',
STARTKEY => '889091ff744069cab08dc605d162a8d3-6482', ENDKEY =>
'9996535e07258a7bbfd8b132435c5962-1235*\x00'*}
As you can see from log extra characters "\x00" are added In ENDKEY which is
wrong
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24917) HBase RSGroup: Table region details under RS Groups are in the onder fromat like HBase 1.x
Ajeet Rai created HBASE-24917: - Summary: HBase RSGroup: Table region details under RS Groups are in the onder fromat like HBase 1.x Key: HBASE-24917 URL: https://issues.apache.org/jira/browse/HBASE-24917 Project: HBase Issue Type: Bug Components: rsgroup Affects Versions: 2.2.3 Reporter: Ajeet Rai HBase RSGroup: Table region details under RS Groups are in the older format like HBase 1.x. in HBase2, Master UI will contain table details ||Namespace||Name||State||Regions||Description|| ||OPEN||OPENING||CLOSED||CLOSING||OFFLINE||FAILED||SPLIT||Other|| but under Regroup, table region details are different which are same as in HBase1.x ||Namespace||Table||Stats||Online Regions||Offline Regions||Failed Regions||Split Regions||Other Regions||Description|| -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24916) HBCK2 Holes:Wrong region detail in HBCK report a start key of a region which is missing in meta is empty
Ajeet Rai created HBASE-24916: - Summary: HBCK2 Holes:Wrong region detail in HBCK report a start key of a region which is missing in meta is empty Key: HBASE-24916 URL: https://issues.apache.org/jira/browse/HBASE-24916 Project: HBase Issue Type: Bug Components: hbck2 Affects Versions: 2.2.3 Reporter: Ajeet Rai HBCK2 Holes:Wrong region detail in HBCK report a start key of a region which is missing in meta is empty. for hole, it reports two region where hole is present. Scenario where first region is missing for eg: in below scenario r1 is missing in meta start key end key region empty 10 r1 10 20 r2 20 30 r2 In our test hbck report contains two region where first region belongs to another table and second region is r2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24754) Bulk load performance is degraded in HBase 2
Ajeet Rai created HBASE-24754: - Summary: Bulk load performance is degraded in HBase 2 Key: HBASE-24754 URL: https://issues.apache.org/jira/browse/HBASE-24754 Project: HBase Issue Type: Bug Components: Performance Affects Versions: 2.2.3 Reporter: Ajeet Rai in our Test,It is observed that Bulk load performance is degraded in HBase 2 . Test Input: 1: Table with 500 region 2: data =2 TB 3: Cluster: 7 node(2 master+5 Region Server) HBase 2 took 10% more time then HBase 1.3 where test input is same for both cluster -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24481) HBase Rest: Request for region detail of a table which doesn't exits is success(200 success code) instead of 403
Ajeet Rai created HBASE-24481: - Summary: HBase Rest: Request for region detail of a table which doesn't exits is success(200 success code) instead of 403 Key: HBASE-24481 URL: https://issues.apache.org/jira/browse/HBASE-24481 Project: HBase Issue Type: Bug Affects Versions: 2.2.3 Reporter: Ajeet Rai Request for region detail of a table which doesn't exits is success(200 success code) however response doesn't have any region data. However response for a request of schema detail for a table which doesn't exits, is 404(Not found) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24135) HBase RS Group
Ajeet Rai created HBASE-24135: - Summary: HBase RS Group Key: HBASE-24135 URL: https://issues.apache.org/jira/browse/HBASE-24135 Project: HBase Issue Type: Bug Affects Versions: 2.2.3 Reporter: Ajeet Rai IF RS group is enabled then Unable to get table state error is thrown while creating a new table Stpes: 1: Make sure RS group feature is enabled 2: Create a table say usertable200 3: Check master log and observe below exception is thrown org.apache.hadoop.hbase.master.TableStateManager$TableStateNotFoundException: usertable2000 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-22342) space quota usage is calculated almost double then actual usage in case of Split /Merge of region
Ajeet Rai created HBASE-22342: - Summary: space quota usage is calculated almost double then actual usage in case of Split /Merge of region Key: HBASE-22342 URL: https://issues.apache.org/jira/browse/HBASE-22342 Project: HBase Issue Type: Bug Reporter: Ajeet Rai space quota usage is calculated double for some time because it is considering the usage of the parent region report received earlier. This is normal behaviour because *hbase.master.quotas.region.report.retention.millis* is set as 10 minutes.It's not calculating the usage from the store files, it's taking the usage from the earlier report send to the master by the parent region server till 10 minutes. So after 10 minutes, it stops considering the earlier region report and quota is not violated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22086) space quota issue: deleting snapshot doesn't update the usage of table
Ajeet Rai created HBASE-22086: - Summary: space quota issue: deleting snapshot doesn't update the usage of table Key: HBASE-22086 URL: https://issues.apache.org/jira/browse/HBASE-22086 Project: HBase Issue Type: Bug Reporter: Ajeet Rai space quota issue: deleting snapshot doesn't update the usage of table Steps: 1: set_quota TYPE => SPACE, TABLE => 'bugatti', LIMIT => '7M', POLICY => NO_WRITES_COMPACTIONS 2: ./hbase pe --table="bugatti" --nomapred --rows=200 sequentialWrite 10 3: ./hbase pe --table="bugatti" --nomapred --rows=200 sequentialWrite 10 4: snapshot 'bugatti','bugatti_snapshot' 5: ./hbase pe --table="bugatti" --nomapred --rows=200 sequentialWrite 10 6: major_compact 'bugatti' 7: delete_snapshot 'bugatti_snapshot' now check the usage and observe that it is not getting updated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22054) Space Quota: Compaction is not working for super user in case of NO_WRITES_COMPACTIONS
Ajeet Rai created HBASE-22054:
-
Summary: Space Quota: Compaction is not working for super user in
case of NO_WRITES_COMPACTIONS
Key: HBASE-22054
URL: https://issues.apache.org/jira/browse/HBASE-22054
Project: HBase
Issue Type: Bug
Reporter: Ajeet Rai
Space Quota: Compaction is not working for super user. Compaction command is
issued successfully at client but actually compaction is not happening.
In debug log below message is printed:
as an active space quota violation policy disallows compaction.
Reference:
[https://lists.apache.org/thread.html/d09aa7abaacf1f0be9d59fa9260515ddc0c17ac0aba9cc0f2ac569bf@%3Cuser.hbase.apache.org%3E]
Actually in requestCompactionInternal method of CompactSplit class ,there is
no check for super user and compcations are disallowed
{noformat}
RegionServerSpaceQuotaManager spaceQuotaManager =
this.server.getRegionServerSpaceQuotaManager();
if (spaceQuotaManager != null &&
spaceQuotaManager.areCompactionsDisabled(region.getTableDescriptor().getTableName()))
{
String reason = "Ignoring compaction request for " + region +
" as an active space quota violation " + " policy disallows
compactions.";
tracker.notExecuted(store, reason);
completeTracker.completed(store);
LOG.debug(reason);
return;
}
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-22014) Space Quota: If some reasons are offline then quota usage calculation is not happening.
Ajeet Rai created HBASE-22014: - Summary: Space Quota: If some reasons are offline then quota usage calculation is not happening. Key: HBASE-22014 URL: https://issues.apache.org/jira/browse/HBASE-22014 Project: HBase Issue Type: Bug Reporter: Ajeet Rai Space Quota: If some reasons are offline then quota usage calculation is not happening. Steps: Scenario1: 1: create 'testmultireason','info0',SPLITS => ['100','200','300','400','500','600','700','800','900','1000'] 2: unassign '3f356ccac99fba4e12741a57f79a18a7' // close one of the region 3: set_quota TYPE => SPACE, TABLE => 'testmultireason', LIMIT => '3M', POLICY => NO_INSERTS 4: put some data 5: observe that after some time data usage and state is also shown but usage is 0 only even if data has crossed quota limit Scenario2: 1: set_quota TYPE => SPACE, TABLE => 'testmultireason2', LIMIT => '3M', POLICY => NO_INSERTS 2: create 'testmultireason2','info0',SPLITS => ['100','200','300','400','500','600','700','800','900','1000'] 3: unassign '3b6e28e0060596e35814673ef4941d4c' //close one of the region 4: put 4 mb data and observe that policy is in violation 5: Observe after some time policy is changed from violation to In Observance -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22013) Space Quota: Space Quota Issue: If a table is created with region replica then quota calculation is not happening
Ajeet Rai created HBASE-22013: - Summary: Space Quota: Space Quota Issue: If a table is created with region replica then quota calculation is not happening Key: HBASE-22013 URL: https://issues.apache.org/jira/browse/HBASE-22013 Project: HBase Issue Type: Bug Reporter: Ajeet Rai Space Quota: Space Quota Issue: If a table is created with region replica then quota calculation is not happening Steps: 1: Create a table with 100 regions with region replica 3 2: Observe that 'hbase:quota' table doesn't have entry of usage for this table So In UI only policy Limit and Policy is shown but not Usage and State. Reason: It looks like File system utilization core is sending data of 100 reasons but not the size of region replicas. But in quota observer chore, it is considering total region(actual regions+ replica reasons) So the ratio of reported regions is less then configured percentRegionsReportedThreshold. SO quota calculation is not happening -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22012) Space Quota: Policy state is getting changed from disable to Observance after sometime automatically.
Ajeet Rai created HBASE-22012: - Summary: Space Quota: Policy state is getting changed from disable to Observance after sometime automatically. Key: HBASE-22012 URL: https://issues.apache.org/jira/browse/HBASE-22012 Project: HBase Issue Type: Bug Reporter: Ajeet Rai pace Quota: Policy state is getting changed from disable to Observance after sometime automatically. Steps: 1: Create a table with space quota policy as Disable 2: Put some data so that table state is in space quota violation 3: So observe that table state is in violation 4: Now wait for some time 5: Observe that after some time table state is changing to to Observance however table is still disabled -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21892) If a new table is created inside a namespace then NS quota violation is not working for first time
Ajeet Rai created HBASE-21892: - Summary: If a new table is created inside a namespace then NS quota violation is not working for first time Key: HBASE-21892 URL: https://issues.apache.org/jira/browse/HBASE-21892 Project: HBase Issue Type: Bug Reporter: Ajeet Rai If a new table is created inside a namespace then NS quota violation is not working for first time. Steps to reproduce: 1: Set a space quota at NameSpace level 2: create a table and add insert data such that NS quota is violated 3: Try to load data in table again and observe that quota is applicable 4: Now create another table under same NS and put some data 5: Observe that data is inserted in new table which should not happen due to namespace quota is already violated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21893) Space quota: Usage is not getting calculated correctly if snapshot is created on a table then deleted and created again
Ajeet Rai created HBASE-21893: - Summary: Space quota: Usage is not getting calculated correctly if snapshot is created on a table then deleted and created again Key: HBASE-21893 URL: https://issues.apache.org/jira/browse/HBASE-21893 Project: HBase Issue Type: Bug Reporter: Ajeet Rai *Steps to reproduce:* 1: ./hbase pe --table="bugatti" --nomapred --rows=400 sequentialWrite 10 (will put 4 mb data) 2: set_quota TYPE => SPACE, TABLE => 'bugatti', LIMIT => '7M', POLICY => NO_WRITES_COMPACTIONS 3: snapshot 'bugatti','bugatti_snapshot' 4: disable 'bugatti' 5: drop 'bugatti' 6: create 'bugatti','info0' 7: set_quota TYPE => SPACE, TABLE => 'bugatti', LIMIT => '5M', POLICY => NO_WRITES_COMPACTIONS 8: scan 'bugatti' >> Observe that no data here and original snapshot size was 4 MB. but current >> usage is shown as 8 MB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21891) New space quota policy doesn't take effect if quota policy is changed after violation
Ajeet Rai created HBASE-21891: - Summary: New space quota policy doesn't take effect if quota policy is changed after violation Key: HBASE-21891 URL: https://issues.apache.org/jira/browse/HBASE-21891 Project: HBase Issue Type: Bug Reporter: Ajeet Rai *Steps to reproduce* 1: set_quota TYPE => SPACE, TABLE => 'test25', LIMIT => '2M', POLICY => NO_WRITES 2: ./hbase pe --table="test25" --nomapred --rows=300 sequentialWrite 10 3: Observe that after some time data usage is 3 mb and policy is in violation 4: now try to insert some data again in the table and observe that operation fails due to NoWritesViolationPolicyEnforcement 5: Now change the quota policy set_quota TYPE => SPACE, TABLE => 'test25', LIMIT => '2M', POLICY => NO_WRITES_COMPACTIONS 6: Now again try to insert data once new policy takes effect 7: Observe that still operation fails but because of old policy and not new policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
