[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528778#comment-13528778 ] Matt Corgan commented on HBASE-7319: oops, sorry! i may have some sub-tasks that don't fit under your title though. fix as you see fit Lars. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528784#comment-13528784 ] Lars Hofhansl commented on HBASE-7319: -- I think the issue I filed would be a subtask under this one. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950614#comment-13950614 ] ramkrishna.s.vasudevan commented on HBASE-7319: --- Once HBASE-10531 is in, will start with this. Replacing BRs in place of ByteBuffers preferably needs this change first. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[
https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962064#comment-13962064
]
Hadoop QA commented on HBASE-7319:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639008/HBASE-7319.patch
against trunk revision .
ATTACHMENT ID: 12639008
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 45 new
or modified tests.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 7
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+return Bytes.equals(left.getQualifierArray(),
left.getQualifierOffset(), left.getQualifierLength(),
+int len = KeyValue.writeByteArray(buffer, boffset, row, roffset, rlength,
family, foffset, flength,
+ (nextJoinedKv != null && CellUtil.matchingRow(nextJoinedKv,
currentRow, offset, length))
+ ||
(this.joinedHeap.requestSeek(KeyValueUtil.createFirstOnRow(currentRow, offset,
length),
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
org.apache.hadoop.hbase.regionserver.TestReversibleScanners
org.apache.hadoop.hbase.client.TestFromClientSide
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9215//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9215//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9215//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9215//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9215//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9215//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9215//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9215//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9215//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9215//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9215//console
This message is automatically generated.
> Extend Cell usage through read path
> ---
>
> Key: HBASE-7319
> URL: https://issues.apache.org/jira/browse/HBASE-7319
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, Performance, regionserver, Scanners
>Reporter: Matt Corgan
> Attachments: HBASE-7319.patch
>
>
> Umbrella issue for eliminating Cell copying.
> The Cell interface allows us to work with a reference to underlying bytes in
> the block cache without copying each Cell into consecutive bytes in an array
> (KeyValue).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962591#comment-13962591 ] stack commented on HBASE-7319: -- Tell us more [~ram_krish] about this 'ScanQueryMatch still uses KV. With this patch it would help BufferedDataBlockEncoders to use Cell and directly use it in comparison and during Kv.next() and kv.peek().' Will this be fixed in a follow up? Or these are not changeable? > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan > Attachments: HBASE-7319.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962594#comment-13962594 ] ramkrishna.s.vasudevan commented on HBASE-7319: --- ScanQueryMatcher is changable. For now it will still work with KeyValues. Will take it up in a follow up issue. It is a core area and so wanted to do it separately after making sure that these changes does not affect the other read areas. For the DBE part see HBASE-10801 and the patch attached there. This change combined with HBASE-10801 would speed up reads using DBE because we could avoid the deep copy while doing getKeyValue. See https://issues.apache.org/jira/browse/HBASE-10801?focusedCommentId=13941459&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13941459. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan > Attachments: HBASE-7319.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[
https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962598#comment-13962598
]
stack commented on HBASE-7319:
--
Patch is looking good.
These seem a little strange:
public static KeyValue createFirstOnRow(final byte [] row) {
-return createFirstOnRow(row, HConstants.LATEST_TIMESTAMP);
+return KeyValueUtil.createFirstOnRow(row, HConstants.LATEST_TIMESTAMP);
}
Why not change the caller so it uses the KVU directly rather than have KV
reference KVU?
Same for createFirstOnRow, etc.
I skimmed the patch. Radical. It looks great Ram.
> Extend Cell usage through read path
> ---
>
> Key: HBASE-7319
> URL: https://issues.apache.org/jira/browse/HBASE-7319
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, Performance, regionserver, Scanners
>Reporter: Matt Corgan
> Attachments: HBASE-7319.patch
>
>
> Umbrella issue for eliminating Cell copying.
> The Cell interface allows us to work with a reference to underlying bytes in
> the block cache without copying each Cell into consecutive bytes in an array
> (KeyValue).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962605#comment-13962605 ] ramkrishna.s.vasudevan commented on HBASE-7319: --- Thanks for the review Stack. There are some test failures. Will udpate your comments too. Is it ok to rename getKeyValue() to getCell() in all the KeyValueScanner, HFileScanner etc? I have not done in the patch, but will it make sense? > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan > Attachments: HBASE-7319.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[
https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962681#comment-13962681
]
Hadoop QA commented on HBASE-7319:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639144/HBASE-7319_1.patch
against trunk revision .
ATTACHMENT ID: 12639144
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 63 new
or modified tests.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestMultiParallel
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9219//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9219//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9219//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9219//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9219//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9219//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9219//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9219//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9219//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9219//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9219//console
This message is automatically generated.
> Extend Cell usage through read path
> ---
>
> Key: HBASE-7319
> URL: https://issues.apache.org/jira/browse/HBASE-7319
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, Performance, regionserver, Scanners
>Reporter: Matt Corgan
> Attachments: HBASE-7319.patch, HBASE-7319_1.patch
>
>
> Umbrella issue for eliminating Cell copying.
> The Cell interface allows us to work with a reference to underlying bytes in
> the block cache without copying each Cell into consecutive bytes in an array
> (KeyValue).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962752#comment-13962752 ] ramkrishna.s.vasudevan commented on HBASE-7319: --- Test failure seems unrelated. Passes locally. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan > Attachments: HBASE-7319.patch, HBASE-7319_1.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963142#comment-13963142 ] Andrew Purtell commented on HBASE-7319: --- I spent about an hour with this patch today. Refactoring looks good. I like where we are now using static methods for comparisons. As transitional changes these look ok to me. I see some follow up issues have been filed already. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan > Attachments: HBASE-7319.patch, HBASE-7319_1.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963774#comment-13963774 ] ramkrishna.s.vasudevan commented on HBASE-7319: --- Will commit this patch unless objections. If there are more places to change will raise subsequent JIRAs to do that so that finally we can ensure that we deal with Cells rather than KVs. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-7319.patch, HBASE-7319_1.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963780#comment-13963780 ] stack commented on HBASE-7319: -- [~ram_krish] Let me retry the hadoopqa to see if the failure related for sure or not. Good on you. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-7319.patch, HBASE-7319_1.patch, HBASE-7319_1.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[
https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963870#comment-13963870
]
Hadoop QA commented on HBASE-7319:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639340/HBASE-7319_1.patch
against trunk revision .
ATTACHMENT ID: 12639340
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 63 new
or modified tests.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9227//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9227//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9227//console
This message is automatically generated.
> Extend Cell usage through read path
> ---
>
> Key: HBASE-7319
> URL: https://issues.apache.org/jira/browse/HBASE-7319
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, Performance, regionserver, Scanners
>Reporter: Matt Corgan
>Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-7319.patch, HBASE-7319_1.patch, HBASE-7319_1.patch
>
>
> Umbrella issue for eliminating Cell copying.
> The Cell interface allows us to work with a reference to underlying bytes in
> the block cache without copying each Cell into consecutive bytes in an array
> (KeyValue).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963944#comment-13963944 ] ramkrishna.s.vasudevan commented on HBASE-7319: --- Will commit this patch? [~anoop.hbase] you have any comments on this? > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-7319.patch, HBASE-7319_1.patch, HBASE-7319_1.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[
https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963957#comment-13963957
]
Anoop Sam John commented on HBASE-7319:
---
Oh.. It is a big patch..Just started going through it Ram. One quick comment
{code}
KeyValue kv = KeyValueUtil.ensureKeyValue(kvs[i]);
- if (kv.matchingColumn(family,qualifier)) {
+ if (CellUtil.matchingColumn(kv, family,qualifier)) {
{code}
Now when u use CellUtil why still need KeyValueUtil.ensureKeyValue(kvs[i])?
This is there in some places.
> Extend Cell usage through read path
> ---
>
> Key: HBASE-7319
> URL: https://issues.apache.org/jira/browse/HBASE-7319
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, Performance, regionserver, Scanners
>Reporter: Matt Corgan
>Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-7319.patch, HBASE-7319_1.patch, HBASE-7319_1.patch
>
>
> Umbrella issue for eliminating Cell copying.
> The Cell interface allows us to work with a reference to underlying bytes in
> the block cache without copying each Cell into consecutive bytes in an array
> (KeyValue).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963962#comment-13963962 ] ramkrishna.s.vasudevan commented on HBASE-7319: --- It is not needed. In my other patches I will remove most of the places where we try to use KeyValueUtil.ensureKeyValue(). If something is there in this current patch I can remove that too. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan >Assignee: ramkrishna.s.vasudevan > Attachments: HBASE-7319.patch, HBASE-7319_1.patch, HBASE-7319_1.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[
https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963979#comment-13963979
]
Anoop Sam John commented on HBASE-7319:
---
Yes some 4, 5 places I have seen Ram. You can correct on commit.
{code}
-public KeyValue getKeyValue() {
+public Cell getKeyValue() {
{code}
We need rename it as getCell() now?
Other than these looks very good Ram.
> Extend Cell usage through read path
> ---
>
> Key: HBASE-7319
> URL: https://issues.apache.org/jira/browse/HBASE-7319
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, Performance, regionserver, Scanners
>Reporter: Matt Corgan
>Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-7319.patch, HBASE-7319_1.patch, HBASE-7319_1.patch
>
>
> Umbrella issue for eliminating Cell copying.
> The Cell interface allows us to work with a reference to underlying bytes in
> the block cache without copying each Cell into consecutive bytes in an array
> (KeyValue).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13964061#comment-13964061 ] ramkrishna.s.vasudevan commented on HBASE-7319: --- Thanks for the review Stack, Andrew and Anoop. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-7319.patch, HBASE-7319_1.patch, > HBASE-7319_1.patch, HBASE-7319_2.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13964255#comment-13964255 ] Hudson commented on HBASE-7319: --- SUCCESS: Integrated in HBase-TRUNK #5073 (See [https://builds.apache.org/job/HBase-TRUNK/5073/]) HBASE-7319-Extend Cell usage through read path (Ram) (ramkrishna: rev 1585945) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Result.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnPrefixFilter.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnRangeFilter.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/DependentColumnFilter.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FirstKeyValueMatchingQualifiersFilter.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/MultipleColumnPrefixFilter.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueExcludeFilter.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueFilter.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java * /hbase/trunk/hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/row/data/TestRowDataDeeper.java * /hbase/trunk/hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/row/data/TestRowDataTrivial.java * /hbase/trunk/hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/row/data/TestRowDataTrivialWithTags.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/Reference.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileScanner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultMemStore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/GetClosestRowBeforeTracker.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueScanner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NonLazyKeyValueScanner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NonReversedNonLazyKeyValueScanner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedKeyValueHeap.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedRegionScannerImpl.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedStoreScanner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeMultiFileWriter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/ParallelSeekHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/trunk/hbase
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13964932#comment-13964932 ] stack commented on HBASE-7319: -- [~ram_krish] You got [~anoop.hbase]'s comment above? That seemed like a good one to do. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-7319.patch, HBASE-7319_1.patch, > HBASE-7319_1.patch, HBASE-7319_2.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13964940#comment-13964940 ] ramkrishna.s.vasudevan commented on HBASE-7319: --- [~saint@gmail.com] You mean the comment to change to getCell? See my comment here https://issues.apache.org/jira/browse/HBASE-7319?focusedCommentId=13962605&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13962605. As I did not get a reply though can do it in a follow on JIRA (if needed). If it is the other comment I have done it already in the patch applied. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-7319.patch, HBASE-7319_1.patch, > HBASE-7319_1.patch, HBASE-7319_2.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[
https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13964984#comment-13964984
]
stack commented on HBASE-7319:
--
[~ram_krish] Follow-on is fine. Was talking about this:
-public KeyValue getKeyValue() {
+public Cell getKeyValue() {
and
We need rename it as getCell() now?
> Extend Cell usage through read path
> ---
>
> Key: HBASE-7319
> URL: https://issues.apache.org/jira/browse/HBASE-7319
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, Performance, regionserver, Scanners
>Reporter: Matt Corgan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.99.0
>
> Attachments: HBASE-7319.patch, HBASE-7319_1.patch,
> HBASE-7319_1.patch, HBASE-7319_2.patch
>
>
> Umbrella issue for eliminating Cell copying.
> The Cell interface allows us to work with a reference to underlying bytes in
> the block cache without copying each Cell into consecutive bytes in an array
> (KeyValue).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-7319) Extend Cell usage through read path
[ https://issues.apache.org/jira/browse/HBASE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13965081#comment-13965081 ] ramkrishna.s.vasudevan commented on HBASE-7319: --- I would rename to getCell() in another JIRA after the pending things are done. > Extend Cell usage through read path > --- > > Key: HBASE-7319 > URL: https://issues.apache.org/jira/browse/HBASE-7319 > Project: HBase > Issue Type: Umbrella > Components: Compaction, Performance, regionserver, Scanners >Reporter: Matt Corgan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.99.0 > > Attachments: HBASE-7319.patch, HBASE-7319_1.patch, > HBASE-7319_1.patch, HBASE-7319_2.patch > > > Umbrella issue for eliminating Cell copying. > The Cell interface allows us to work with a reference to underlying bytes in > the block cache without copying each Cell into consecutive bytes in an array > (KeyValue). -- This message was sent by Atlassian JIRA (v6.2#6252)
