[
https://issues.apache.org/jira/browse/PHOENIX-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15102935#comment-15102935
]
ASF GitHub Bot commented on PHOENIX-2417:
-----------------------------------------
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/147#discussion_r49925693
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java
---
@@ -510,14 +532,18 @@ private static String toString(List<byte[]> gps) {
endRegionKey = regionInfo.getEndKey();
keyOffset =
ScanUtil.getRowKeyOffset(regionInfo.getStartKey(), endRegionKey);
}
- while (guideIndex < gpsSize
- && (Bytes.compareTo(currentGuidePost =
gps.get(guideIndex), endKey) <= 0 || endKey.length == 0)) {
- Scan newScan = scanRanges.intersectScan(scan, currentKey,
currentGuidePost, keyOffset, false);
- scans = addNewScan(parallelScans, scans, newScan,
currentGuidePost, false, regionLocation);
- currentKey = currentGuidePost;
- guideIndex++;
- }
- Scan newScan = scanRanges.intersectScan(scan, currentKey,
endKey, keyOffset, true);
+ try {
+ while (guideIndex < gpsSize &&
(currentGuidePost.compareTo(endKey) <= 0 || endKey.length == 0)) {
+ Scan newScan = scanRanges.intersectScan(scan,
currentKeyBytes, currentGuidePostBytes, keyOffset,
+ false);
+ scans = addNewScan(parallelScans, scans, newScan,
currentGuidePostBytes, false, regionLocation);
+ currentKeyBytes = currentGuidePost.copyBytes();
+ currentGuidePost = CodecUtils.decode(decoder, input);
+ currentGuidePostBytes = currentGuidePost.copyBytes();
--- End diff --
And here
> Compress memory used by row key byte[] of guideposts
> ----------------------------------------------------
>
> Key: PHOENIX-2417
> URL: https://issues.apache.org/jira/browse/PHOENIX-2417
> Project: Phoenix
> Issue Type: Sub-task
> Reporter: James Taylor
> Assignee: Ankit Singhal
> Fix For: 4.7.0
>
> Attachments: PHOENIX-2417.patch, PHOENIX-2417_encoder.diff,
> PHOENIX-2417_v2_wip.patch
>
>
> We've found that smaller guideposts are better in terms of minimizing any
> increase in latency for point scans. However, this increases the amount of
> memory significantly when caching the guideposts on the client. Guidepost are
> equidistant row keys in the form of raw byte[] which are likely to have a
> large percentage of their leading bytes in common (as they're stored in
> sorted order. We should use a simple compression technique to mitigate this.
> I noticed that Apache Parquet has a run length encoding - perhaps we can use
> that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
