[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[ https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880290#comment-15880290 ] Hudson commented on HBASE-17647: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2556 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2556/]) HBASE-17647 OffheapKeyValue#heapSize() implementation is wrong. (anoopsamjohn: rev ff045cab84ada47ae2d10015649cca8f6edb76ac) * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java * (edit) hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/PrefixTreeSeeker.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NoLimitScannerContext.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScannerContext.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java > OffheapKeyValue#heapSize() implementation is wrong > -- > > Key: HBASE-17647 > URL: https://issues.apache.org/jira/browse/HBASE-17647 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-17647.patch, HBASE-17647_V2.patch, > HBASE-17647_V3.patch > > > We consider the key and data lengths also even though the data is actually in > off heap area. We should correct it. > The impact will be at ScannerContext limit tracking where we use heapSize of > cells to account the result size. So my proposal is to consider the cells > length and heap size in Limit tracking and accounting. We have a > maxResultSize which defaults to 2MB. When the sum of all cell's data size > reaches 'maxResultSize' OR the sum of all cell's heap size reaches > 'maxResultSize' , we need to send back the RPC response -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[ https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879145#comment-15879145 ] stack commented on HBASE-17647: --- bq. Ya old 'size' is what dataSize now. dataSize might be in on heap or off heap.. heapSize is tracks total heap space occupied by all cells. This includes cell POJO heap overheads and if a cell is on heap, the total length of this cells key + value which resides in on heap area any way. Add this in as a comment on commit. +1 > OffheapKeyValue#heapSize() implementation is wrong > -- > > Key: HBASE-17647 > URL: https://issues.apache.org/jira/browse/HBASE-17647 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-17647.patch, HBASE-17647_V2.patch, > HBASE-17647_V3.patch > > > We consider the key and data lengths also even though the data is actually in > off heap area. We should correct it. > The impact will be at ScannerContext limit tracking where we use heapSize of > cells to account the result size. So my proposal is to consider the cells > length and heap size in Limit tracking and accounting. We have a > maxResultSize which defaults to 2MB. When the sum of all cell's data size > reaches 'maxResultSize' OR the sum of all cell's heap size reaches > 'maxResultSize' , we need to send back the RPC response -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[
https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879046#comment-15879046
]
Hadoop QA commented on HBASE-17647:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 19s
{color} | {color:red} hbase-prefix-tree in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 13s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 1s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 24s
{color} | {color:green} hbase-prefix-tree in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m 23s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 146m 42s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12854001/HBASE-17647_V3.patch |
| JIRA Issue | HBASE-17647 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 0ec42ed67cd7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f037f23 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[ https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878626#comment-15878626 ] Anoop Sam John commented on HBASE-17647: Thanks for the review Stack.. Now we have ByteBufferKeyValue which can be backed by both on heap as well as off heap BB. So the change in the heapSize calc. Ya I will rename the params and method as needed.. Changed some places and missed some may be. Ya old 'size' is what dataSize now. dataSize might be in on heap or off heap.. heapSize is tracks total heap space occupied by all cells. This includes cell POJO heap overheads and if a cell is on heap, the total length of this cells key + value which resides in on heap area any way. > OffheapKeyValue#heapSize() implementation is wrong > -- > > Key: HBASE-17647 > URL: https://issues.apache.org/jira/browse/HBASE-17647 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-17647.patch, HBASE-17647_V2.patch > > > We consider the key and data lengths also even though the data is actually in > off heap area. We should correct it. > The impact will be at ScannerContext limit tracking where we use heapSize of > cells to account the result size. So my proposal is to consider the cells > length and heap size in Limit tracking and accounting. We have a > maxResultSize which defaults to 2MB. When the sum of all cell's data size > reaches 'maxResultSize' OR the sum of all cell's heap size reaches > 'maxResultSize' , we need to send back the RPC response -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[
https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877532#comment-15877532
]
stack commented on HBASE-17647:
---
How'd you figure this one?
255 if (this.buf.hasArray()) {
256 return ClassSize.align(FIXED_OVERHEAD + length);
257 }
258 return ClassSize.align(FIXED_OVERHEAD);
Could we have better names for params so clear what the different sizes are:
void setSizeProgress(long sizeProgress, long heapSizeProgress) {
... for example, you have better naming here: void
incrementSizeProgress(long dataSize, long heapSize) {
It looks like 'size' is 'data' size and it is everything and then heap is less
than 'data':
189 void setSizeProgress(long sizeProgress, long heapSizeProgress) {
Is it written up somewhere how data relates to heap? It is not in this patch.
Otherwise looks good [~anoop.hbase] How I know the calculation is being doine
right? Thanks sir.
> OffheapKeyValue#heapSize() implementation is wrong
> --
>
> Key: HBASE-17647
> URL: https://issues.apache.org/jira/browse/HBASE-17647
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-17647.patch, HBASE-17647_V2.patch
>
>
> We consider the key and data lengths also even though the data is actually in
> off heap area. We should correct it.
> The impact will be at ScannerContext limit tracking where we use heapSize of
> cells to account the result size. So my proposal is to consider the cells
> length and heap size in Limit tracking and accounting. We have a
> maxResultSize which defaults to 2MB. When the sum of all cell's data size
> reaches 'maxResultSize' OR the sum of all cell's heap size reaches
> 'maxResultSize' , we need to send back the RPC response
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[
https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874159#comment-15874159
]
Hadoop QA commented on HBASE-17647:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 29s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 19s
{color} | {color:red} hbase-prefix-tree in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 16s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 3s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s
{color} | {color:green} hbase-prefix-tree in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 118m 36s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
57s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 173m 1s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12853516/HBASE-17647_V2.patch |
| JIRA Issue | HBASE-17647 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux dbdd4a12ec8b 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d08bafa |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[
https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15865991#comment-15865991
]
Ted Yu commented on HBASE-17647:
{code}
* Progress towards the size limit has been made. Increment internal
tracking of size progress
*/
- void incrementSizeProgress(long size) {
+ void incrementSizeProgress(long size, long heapSize) {
{code}
We now have two sizes. Please name the existing size parameter so that its
meaning is clear.
Modify javadoc accordingly.
{code}
long getSizeProgress() {
{code}
I think the above method should be renamed to match increment operations (now
that you add getHeapSizeProgress).
> OffheapKeyValue#heapSize() implementation is wrong
> --
>
> Key: HBASE-17647
> URL: https://issues.apache.org/jira/browse/HBASE-17647
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-17647.patch
>
>
> We consider the key and data lengths also even though the data is actually in
> off heap area. We should correct it.
> The impact will be at ScannerContext limit tracking where we use heapSize of
> cells to account the result size. So my proposal is to consider the cells
> length and heap size in Limit tracking and accounting. We have a
> maxResultSize which defaults to 2MB. When the sum of all cell's data size
> reaches 'maxResultSize' OR the sum of all cell's heap size reaches
> 'maxResultSize' , we need to send back the RPC response
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[
https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15865742#comment-15865742
]
Hadoop QA commented on HBASE-17647:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 22s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 20s
{color} | {color:red} hbase-prefix-tree in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 59s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 51s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s
{color} | {color:green} hbase-prefix-tree in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 96m 5s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 144m 21s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852539/HBASE-17647.patch |
| JIRA Issue | HBASE-17647 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 3073cdd64468 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b2217d1 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Buil
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[ https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15865729#comment-15865729 ] Anoop Sam John commented on HBASE-17647: Yep only for master. But addition of new fields/methods might be ok I guess. > OffheapKeyValue#heapSize() implementation is wrong > -- > > Key: HBASE-17647 > URL: https://issues.apache.org/jira/browse/HBASE-17647 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-17647.patch > > > We consider the key and data lengths also even though the data is actually in > off heap area. We should correct it. > The impact will be at ScannerContext limit tracking where we use heapSize of > cells to account the result size. So my proposal is to consider the cells > length and heap size in Limit tracking and accounting. We have a > maxResultSize which defaults to 2MB. When the sum of all cell's data size > reaches 'maxResultSize' OR the sum of all cell's heap size reaches > 'maxResultSize' , we need to send back the RPC response -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
[
https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15865725#comment-15865725
]
Ted Yu commented on HBASE-17647:
ScannerContext is marked:
{code}
@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.COPROC)
@InterfaceStability.Evolving
{code}
Is this change for master branch only ?
> OffheapKeyValue#heapSize() implementation is wrong
> --
>
> Key: HBASE-17647
> URL: https://issues.apache.org/jira/browse/HBASE-17647
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-17647.patch
>
>
> We consider the key and data lengths also even though the data is actually in
> off heap area. We should correct it.
> The impact will be at ScannerContext limit tracking where we use heapSize of
> cells to account the result size. So my proposal is to consider the cells
> length and heap size in Limit tracking and accounting. We have a
> maxResultSize which defaults to 2MB. When the sum of all cell's data size
> reaches 'maxResultSize' OR the sum of all cell's heap size reaches
> 'maxResultSize' , we need to send back the RPC response
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
