[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16912003#comment-16912003 ] Sakthi commented on HBASE-22013: Looks like TestSpaceQuotas wasn't run (because it's in the flakey list). However I have run the test locally & verified that it passes. Let me push this in [~shardulsingh] > SpaceQuota utilization issue with region replicas enabled > - > > Key: HBASE-22013 > URL: https://issues.apache.org/jira/browse/HBASE-22013 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Assignee: Shardul Singh >Priority: Major > Labels: Quota, Space > Fix For: 2.1.0 > > Attachments: HBASE-22013.master.001.patch, > HBASE-22013.master.002.patch > > > Space Quota: Space Quota Issue: If a table is created with region replica > then quota calculation is not happening > Steps: > 1: Create a table with 100 regions with region replica 3 > 2: Observe that 'hbase:quota' table doesn't have entry of usage for this > table So In UI only policy Limit and Policy is shown but not Usage and State. > Reason: > It looks like File system utilization core is sending data of 100 reasons > but not the size of region replicas. > But in quota observer chore, it is considering total region(actual regions+ > replica reasons) > So the ratio of reported regions is less then configured > percentRegionsReportedThreshold. > SO quota calculation is not happening -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911124#comment-16911124 ] Shardul Singh commented on HBASE-22013: --- Hi [~jatsakthi], can we push this in if everything is fine?..thanks :) > SpaceQuota utilization issue with region replicas enabled > - > > Key: HBASE-22013 > URL: https://issues.apache.org/jira/browse/HBASE-22013 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Assignee: Shardul Singh >Priority: Major > Labels: Quota, Space > Fix For: 2.1.0 > > Attachments: HBASE-22013.master.001.patch, > HBASE-22013.master.002.patch > > > Space Quota: Space Quota Issue: If a table is created with region replica > then quota calculation is not happening > Steps: > 1: Create a table with 100 regions with region replica 3 > 2: Observe that 'hbase:quota' table doesn't have entry of usage for this > table So In UI only policy Limit and Policy is shown but not Usage and State. > Reason: > It looks like File system utilization core is sending data of 100 reasons > but not the size of region replicas. > But in quota observer chore, it is considering total region(actual regions+ > replica reasons) > So the ratio of reported regions is less then configured > percentRegionsReportedThreshold. > SO quota calculation is not happening -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[
https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908987#comment-16908987
]
HBase QA commented on HBASE-22013:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
30s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
30s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 3s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}138m
30s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}191m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/785/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22013 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12977753/HBASE-22013.master.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7ee49f535d87 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 94af65163b |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/785/testReport/ |
| Max. process+thread count | 4585 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/785/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
> SpaceQuota utilization issue with
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908718#comment-16908718 ] Shardul Singh commented on HBASE-22013: --- Thanks [~jatsakthi] for reviewing the patch and sorry for the naming convention mistake. As told, I have handled the review comments given by you and have uploaded a new patch. > SpaceQuota utilization issue with region replicas enabled > - > > Key: HBASE-22013 > URL: https://issues.apache.org/jira/browse/HBASE-22013 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Assignee: Shardul Singh >Priority: Major > Labels: Quota, Space > Fix For: 2.1.0 > > Attachments: HBASE-22013.master.001.patch, > HBASE-22013.master.002.patch > > > Space Quota: Space Quota Issue: If a table is created with region replica > then quota calculation is not happening > Steps: > 1: Create a table with 100 regions with region replica 3 > 2: Observe that 'hbase:quota' table doesn't have entry of usage for this > table So In UI only policy Limit and Policy is shown but not Usage and State. > Reason: > It looks like File system utilization core is sending data of 100 reasons > but not the size of region replicas. > But in quota observer chore, it is considering total region(actual regions+ > replica reasons) > So the ratio of reported regions is less then configured > percentRegionsReportedThreshold. > SO quota calculation is not happening -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[
https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907307#comment-16907307
]
HBase QA commented on HBASE-22013:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
31s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
59s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 50s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}177m
17s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}236m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/771/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22013 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12977131/HBASE-22013.master.v001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 18be99afdcf5 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2
18:22:20 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 27ed2ac071 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/771/testReport/ |
| Max. process+thread count | 4553 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/771/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
> SpaceQuota utilization issue with
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[
https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907244#comment-16907244
]
Sakthi commented on HBASE-22013:
The patch looks good. Few comments:
1. In the below fn in SpaceQuotaHelperForTests:
{code:java}
TableName createTableWithRegions(Admin admin, int numRegions){code}
The parameter admin is never used. Let's use that rather than
testutil.getAdmin()?
2. Also, let's add few test cases for 0 replicas?
3. I think it's better to wait for the tables to enter violation(with timeout)
before verifying for violation. Or in a simpler way - just do a thread sleep
for 5 secs before verification to avoid flakeyness of the tests.
[~shardulsingh]
> SpaceQuota utilization issue with region replicas enabled
> -
>
> Key: HBASE-22013
> URL: https://issues.apache.org/jira/browse/HBASE-22013
> Project: HBase
> Issue Type: Bug
>Reporter: Ajeet Rai
>Assignee: Shardul Singh
>Priority: Major
> Labels: Quota, Space
> Fix For: 2.1.0
>
> Attachments: HBASE-22013.master.v001.patch
>
>
> Space Quota: Space Quota Issue: If a table is created with region replica
> then quota calculation is not happening
> Steps:
> 1: Create a table with 100 regions with region replica 3
> 2: Observe that 'hbase:quota' table doesn't have entry of usage for this
> table So In UI only policy Limit and Policy is shown but not Usage and State.
> Reason:
> It looks like File system utilization core is sending data of 100 reasons
> but not the size of region replicas.
> But in quota observer chore, it is considering total region(actual regions+
> replica reasons)
> So the ratio of reported regions is less then configured
> percentRegionsReportedThreshold.
> SO quota calculation is not happening
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[
https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907236#comment-16907236
]
Sakthi commented on HBASE-22013:
Thanks for putting up a patch [~shardulsingh]. Let's follow the naming
convention of the patches here according to
[http://hbase.apache.org/book.html#submitting.patches.create] . In particular:
_Patch name should be as follows to adhere to Yetus' naming convention:_
_{{(JIRA).(branch name).(patch number).patch}}_
_For eg. HBASE-11625.master.001.patch, HBASE-X.branch-1.2.0005.patch, etc._
Could you please update the name of the patch accordingly, while I review your
patch? Thanks.
> SpaceQuota utilization issue with region replicas enabled
> -
>
> Key: HBASE-22013
> URL: https://issues.apache.org/jira/browse/HBASE-22013
> Project: HBase
> Issue Type: Bug
>Reporter: Ajeet Rai
>Assignee: Shardul Singh
>Priority: Major
> Labels: Quota, Space
> Fix For: 2.1.0
>
> Attachments: HBASE-22013.master.v001.patch
>
>
> Space Quota: Space Quota Issue: If a table is created with region replica
> then quota calculation is not happening
> Steps:
> 1: Create a table with 100 regions with region replica 3
> 2: Observe that 'hbase:quota' table doesn't have entry of usage for this
> table So In UI only policy Limit and Policy is shown but not Usage and State.
> Reason:
> It looks like File system utilization core is sending data of 100 reasons
> but not the size of region replicas.
> But in quota observer chore, it is considering total region(actual regions+
> replica reasons)
> So the ratio of reported regions is less then configured
> percentRegionsReportedThreshold.
> SO quota calculation is not happening
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906878#comment-16906878 ] Shardul Singh commented on HBASE-22013: --- Thanks :) [~Uma Murugesan] > SpaceQuota utilization issue with region replicas enabled > - > > Key: HBASE-22013 > URL: https://issues.apache.org/jira/browse/HBASE-22013 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Assignee: Shardul Singh >Priority: Major > Labels: Quota, Space > Attachments: HBASE-22013.master.v001.patch > > > Space Quota: Space Quota Issue: If a table is created with region replica > then quota calculation is not happening > Steps: > 1: Create a table with 100 regions with region replica 3 > 2: Observe that 'hbase:quota' table doesn't have entry of usage for this > table So In UI only policy Limit and Policy is shown but not Usage and State. > Reason: > It looks like File system utilization core is sending data of 100 reasons > but not the size of region replicas. > But in quota observer chore, it is considering total region(actual regions+ > replica reasons) > So the ratio of reported regions is less then configured > percentRegionsReportedThreshold. > SO quota calculation is not happening -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903770#comment-16903770 ] Shardul Singh commented on HBASE-22013: --- Submitting the initial patch. Please review. > SpaceQuota utilization issue with region replicas enabled > - > > Key: HBASE-22013 > URL: https://issues.apache.org/jira/browse/HBASE-22013 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Assignee: Uma Maheswari >Priority: Major > Labels: Quota, Space > Attachments: HBASE-22013.master.v001.patch > > > Space Quota: Space Quota Issue: If a table is created with region replica > then quota calculation is not happening > Steps: > 1: Create a table with 100 regions with region replica 3 > 2: Observe that 'hbase:quota' table doesn't have entry of usage for this > table So In UI only policy Limit and Policy is shown but not Usage and State. > Reason: > It looks like File system utilization core is sending data of 100 reasons > but not the size of region replicas. > But in quota observer chore, it is considering total region(actual regions+ > replica reasons) > So the ratio of reported regions is less then configured > percentRegionsReportedThreshold. > SO quota calculation is not happening -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903590#comment-16903590 ] Shardul Singh commented on HBASE-22013: --- Hi, Since this issue is opened for a long time. Can I work on this jira issue if you are not working on it? [~Uma Murugesan] > SpaceQuota utilization issue with region replicas enabled > - > > Key: HBASE-22013 > URL: https://issues.apache.org/jira/browse/HBASE-22013 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Assignee: Uma Maheswari >Priority: Major > Labels: Quota, Space > > Space Quota: Space Quota Issue: If a table is created with region replica > then quota calculation is not happening > Steps: > 1: Create a table with 100 regions with region replica 3 > 2: Observe that 'hbase:quota' table doesn't have entry of usage for this > table So In UI only policy Limit and Policy is shown but not Usage and State. > Reason: > It looks like File system utilization core is sending data of 100 reasons > but not the size of region replicas. > But in quota observer chore, it is considering total region(actual regions+ > replica reasons) > So the ratio of reported regions is less then configured > percentRegionsReportedThreshold. > SO quota calculation is not happening -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832046#comment-16832046 ] Sakthi commented on HBASE-22013: Can we change the status? > SpaceQuota utilization issue with region replicas enabled > - > > Key: HBASE-22013 > URL: https://issues.apache.org/jira/browse/HBASE-22013 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Assignee: Uma Maheswari >Priority: Major > Labels: Quota, Space > > Space Quota: Space Quota Issue: If a table is created with region replica > then quota calculation is not happening > Steps: > 1: Create a table with 100 regions with region replica 3 > 2: Observe that 'hbase:quota' table doesn't have entry of usage for this > table So In UI only policy Limit and Policy is shown but not Usage and State. > Reason: > It looks like File system utilization core is sending data of 100 reasons > but not the size of region replicas. > But in quota observer chore, it is considering total region(actual regions+ > replica reasons) > So the ratio of reported regions is less then configured > percentRegionsReportedThreshold. > SO quota calculation is not happening -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[
https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800705#comment-16800705
]
Josh Elser commented on HBASE-22013:
{quote}Can you provide permission to work on the issue,so that I can assign
this to me?
{quote}
Done.
> SpaceQuota utilization issue with region replicas enabled
> -
>
> Key: HBASE-22013
> URL: https://issues.apache.org/jira/browse/HBASE-22013
> Project: HBase
> Issue Type: Bug
>Reporter: Ajeet Rai
>Assignee: Uma Maheswari
>Priority: Major
>
> Space Quota: Space Quota Issue: If a table is created with region replica
> then quota calculation is not happening
> Steps:
> 1: Create a table with 100 regions with region replica 3
> 2: Observe that 'hbase:quota' table doesn't have entry of usage for this
> table So In UI only policy Limit and Policy is shown but not Usage and State.
> Reason:
> It looks like File system utilization core is sending data of 100 reasons
> but not the size of region replicas.
> But in quota observer chore, it is considering total region(actual regions+
> replica reasons)
> So the ratio of reported regions is less then configured
> percentRegionsReportedThreshold.
> SO quota calculation is not happening
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800416#comment-16800416 ] Uma Maheswari commented on HBASE-22013: --- [~elserj] Quotaobserverchore is taking the replicated region also for calculation We can make getNumRegions to return only number of non-replicated regions problem Can you provide permission to work on the issue,so that I can assign this to me? > SpaceQuota utilization issue with region replicas enabled > - > > Key: HBASE-22013 > URL: https://issues.apache.org/jira/browse/HBASE-22013 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Priority: Major > > Space Quota: Space Quota Issue: If a table is created with region replica > then quota calculation is not happening > Steps: > 1: Create a table with 100 regions with region replica 3 > 2: Observe that 'hbase:quota' table doesn't have entry of usage for this > table So In UI only policy Limit and Policy is shown but not Usage and State. > Reason: > It looks like File system utilization core is sending data of 100 reasons > but not the size of region replicas. > But in quota observer chore, it is considering total region(actual regions+ > replica reasons) > So the ratio of reported regions is less then configured > percentRegionsReportedThreshold. > SO quota calculation is not happening -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[
https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790573#comment-16790573
]
Josh Elser commented on HBASE-22013:
{noformat}
int numRegionsInTable = getNumRegions(table) //return 2 regions (1
replica+actual region){noformat}
So, the problem is that {{getNumRegions}}() is returning the wrong number of
regions. That's a much clearer explanation of the problem. Please update this
Jira issue to reflect that.
What is the problem with this method? Do you intend to provide a test case and
patch to fix the problem?
> SpaceQuota utilization issue with region replicas enabled
> -
>
> Key: HBASE-22013
> URL: https://issues.apache.org/jira/browse/HBASE-22013
> Project: HBase
> Issue Type: Bug
>Reporter: Ajeet Rai
>Priority: Major
>
> Space Quota: Space Quota Issue: If a table is created with region replica
> then quota calculation is not happening
> Steps:
> 1: Create a table with 100 regions with region replica 3
> 2: Observe that 'hbase:quota' table doesn't have entry of usage for this
> table So In UI only policy Limit and Policy is shown but not Usage and State.
> Reason:
> It looks like File system utilization core is sending data of 100 reasons
> but not the size of region replicas.
> But in quota observer chore, it is considering total region(actual regions+
> replica reasons)
> So the ratio of reported regions is less then configured
> percentRegionsReportedThreshold.
> SO quota calculation is not happening
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[
https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790249#comment-16790249
]
Uma Maheswari commented on HBASE-22013:
---
FileSystemUtilizationChore-doesnot calculate the size of Region Replica
Consider the scenario with table created with 1 region and 1 replica
In our case,
FileSystemUtilizationChore -reports the size of 1 region only,skipping the size
of Region Replica
+QuotaObserverChore:+
hbase.master.quotas.observer.report.percent=0.95
_//percentRegionsReportedThreshold_
In filterInsufficientlyReportedTables() method,
{noformat}
int numRegionsInTable = getNumRegions(table) //return 2 regions (1
replica+actual region)
ratioReported = ((double) reportedRegionsInQuota) / numRegionsInTable
//ratioReported =0.5 which is less than percentRegionsReportedThreshold
{noformat}
Master waits for replication region report also and usage is not calculated
QuotaObserverChore is considering Replicated regions for calculation
> SpaceQuota utilization issue with region replicas enabled
> -
>
> Key: HBASE-22013
> URL: https://issues.apache.org/jira/browse/HBASE-22013
> Project: HBase
> Issue Type: Bug
>Reporter: Ajeet Rai
>Priority: Major
>
> Space Quota: Space Quota Issue: If a table is created with region replica
> then quota calculation is not happening
> Steps:
> 1: Create a table with 100 regions with region replica 3
> 2: Observe that 'hbase:quota' table doesn't have entry of usage for this
> table So In UI only policy Limit and Policy is shown but not Usage and State.
> Reason:
> It looks like File system utilization core is sending data of 100 reasons
> but not the size of region replicas.
> But in quota observer chore, it is considering total region(actual regions+
> replica reasons)
> So the ratio of reported regions is less then configured
> percentRegionsReportedThreshold.
> SO quota calculation is not happening
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
