[jira] [Created] (HBASE-20480) Multiple one-time cell objects are allocated and de-allocated when working with CCM
Anastasia Braginsky created HBASE-20480: --- Summary: Multiple one-time cell objects are allocated and de-allocated when working with CCM Key: HBASE-20480 URL: https://issues.apache.org/jira/browse/HBASE-20480 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky We believe that the cause for some read performance degradation while working with CellChunkMap (CCM). Multiple one-time cell objects are allocated and de-allocated when performing multiple reads and working with CCM MemStore. We have a couple of ideas for solution. More details will follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19506) Support variable sized chunks from ChunkCreator
Anastasia Braginsky created HBASE-19506: --- Summary: Support variable sized chunks from ChunkCreator Key: HBASE-19506 URL: https://issues.apache.org/jira/browse/HBASE-19506 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky When CellChunkMap is created it allocates a special index chunk (or chunks) where array of cell-representations is stored. When the number of cell-representations is small, it is preferable to allocate a chunk smaller than a default value which is 2MB. On the other hand, those "non-standard size" chunks can not be used in pool. On-demand allocations in off-heap are costly. So this JIRA is about to investigate the trade of between memory usage and the final performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19282) CellChunkMap Benchmarking and User Interface
Anastasia Braginsky created HBASE-19282: --- Summary: CellChunkMap Benchmarking and User Interface Key: HBASE-19282 URL: https://issues.apache.org/jira/browse/HBASE-19282 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky We have made some experiments how working with CellChunkMap (CCM) influences the performance when running on-heap and off-heap. Based on those results it is suggested to tie the MSLAB usage (off-heap or on-heap) with CCM index usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19133) Transfer big cells or upserted/appended cells into MSLAB upon flattening to CellChunkMap
Anastasia Braginsky created HBASE-19133: --- Summary: Transfer big cells or upserted/appended cells into MSLAB upon flattening to CellChunkMap Key: HBASE-19133 URL: https://issues.apache.org/jira/browse/HBASE-19133 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky CellChunkMap Segment index requires all cell data to be written in the MSLAB Chunks. Eventhough MSLAB is enabled, cells bigger than chunk size or upserted/incremented/appended cells are still allocated on the JVM stack. If such cells are found in the process of flattening into CellChunkMap (in-memory-flush) they need to be copied into MSLAB. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18748) Cache pre-warming upon replication
Anastasia Braginsky created HBASE-18748: --- Summary: Cache pre-warming upon replication Key: HBASE-18748 URL: https://issues.apache.org/jira/browse/HBASE-18748 Project: HBase Issue Type: New Feature Reporter: Anastasia Braginsky HBase's cluster replication is very important and widely used feature. Let's assume primary cluster is replicated to secondary (backup) cluster using the WAL of the primary cluster to propagate the changes. Let's also assume the secondary cluster is a target for failover when needed and should become primary when needed. We suggest improving the way the HBase cluster failover works today. Namely, upon failover, the backup RS's cache is cold. Warming it up to the right working set takes many minutes. The suggested solution is to selectively replay read requests at the backup - namely, those reads that caused cache-ins at the primary. We intend to use WAL replication as transport protocol (hopefully, as black box), and of course add custom replay callbacks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18375) The pool chunks from ChunkCreator are deallocated while in pool because there is no reference to them
Anastasia Braginsky created HBASE-18375: --- Summary: The pool chunks from ChunkCreator are deallocated while in pool because there is no reference to them Key: HBASE-18375 URL: https://issues.apache.org/jira/browse/HBASE-18375 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky Because MSLAB list of chunks was changed to list of chunk IDs, the chunks returned back to pool can be deallocated by JVM because there is no reference to them. The solution is to protect pool chunks from GC by the strong map of ChunkCreator introduced by HBASE-18010. Will prepare the patch today. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18251) Remove unnecessary traversing to the first and last keys in the CellSet
Anastasia Braginsky created HBASE-18251:
---
Summary: Remove unnecessary traversing to the first and last keys
in the CellSet
Key: HBASE-18251
URL: https://issues.apache.org/jira/browse/HBASE-18251
Project: HBase
Issue Type: Bug
Reporter: Anastasia Braginsky
The implementation of finding the first and last keys in the CellSet is as
following:
{code}
public Cell first() {
return this.delegatee.get(this.delegatee.firstKey());
}
public Cell last() {
return this.delegatee.get(this.delegatee.lastKey());
}
{code}
Recall we have Cell to Cell mapping, therefore the methods bringing the
first/last key, which allready return Cell. Thus no need to waist time on the
get() method for the same Cell.
Fix: return just the first/lastKey(), should be at least twice more effective.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18232) Add variable size chunks to the MSLAB
Anastasia Braginsky created HBASE-18232: --- Summary: Add variable size chunks to the MSLAB Key: HBASE-18232 URL: https://issues.apache.org/jira/browse/HBASE-18232 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky Add possibility to create a variable size chunks of memory, so any cell (of any size) can reside on a chunk. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18056) Change CompactingMemStore in BASIC mode to merge multiple segments in pipeline
Anastasia Braginsky created HBASE-18056: --- Summary: Change CompactingMemStore in BASIC mode to merge multiple segments in pipeline Key: HBASE-18056 URL: https://issues.apache.org/jira/browse/HBASE-18056 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky Under HBASE-16417 it was decided that CompactingMemStore in BASIC mode should merge multiple ImmutableSegments in CompactionPipeline. Basic+Merge actually demonstrated reduction in GC, alongside improvement in other metrics. However, the limit on the number of segments in pipeline is still set to 30. Under this JIRA it should be changed to 1, as it was tested under HBASE-16417. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
Anastasia Braginsky created HBASE-18010: --- Summary: Connect CellChunkMap to be used for flattening in CompactingMemStore Key: HBASE-18010 URL: https://issues.apache.org/jira/browse/HBASE-18010 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky The CellChunkMap helps to create a new type of ImmutableSegment, where the index (CellSet's delegatee) is going to be CellChunkMap. No big cells or upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (HBASE-17377) MemStoreChunkAllocator
[ https://issues.apache.org/jira/browse/HBASE-17377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky resolved HBASE-17377. - Resolution: Duplicate Duplicate of HBASE-16438 > MemStoreChunkAllocator > -- > > Key: HBASE-17377 > URL: https://issues.apache.org/jira/browse/HBASE-17377 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > Attachments: MemStoreChunkAllocator.pdf > > > Refactoring the separation between MemStoreChunkPool and (new) > MemStoreChunkAllocator. The latter allocates chunks either from heap or from > pool and assigns them the Chunk IDs. MemStoreChunkAllocator stores the > mapping between chunks and their IDs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17765) Reviving the merge possibility in the CompactingMemStore
Anastasia Braginsky created HBASE-17765: --- Summary: Reviving the merge possibility in the CompactingMemStore Key: HBASE-17765 URL: https://issues.apache.org/jira/browse/HBASE-17765 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky According to the new performance results presented in the HBASE-16417 we see that the read latency of the 90th percentile of the BASIC policy is too big due to the need to traverse through too many segments in the pipeline. In this JIRA we correct the bug in the merge sizing calculations and allow pipeline size threshold to be a configurable parameter. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17662) Disable in-memory flush when eplaying from WAL
Anastasia Braginsky created HBASE-17662: --- Summary: Disable in-memory flush when eplaying from WAL Key: HBASE-17662 URL: https://issues.apache.org/jira/browse/HBASE-17662 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky When replaying the edits from WAL, the region's updateLock is not taken, because a single threaded action is assumed. However, the thread-safeness of the in-memory flush of CompactingMemStore is based on taking the region's updateLock. The in-memory flush can be skipped in the replay time (anyway everything is flushed to disk just after the replay). Therefore it is acceptable to just skip the in-memory flush action while the updates come as part of replay from WAL. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17492) Fix the compacting memstore part in hbase shell ruby script
Anastasia Braginsky created HBASE-17492: --- Summary: Fix the compacting memstore part in hbase shell ruby script Key: HBASE-17492 URL: https://issues.apache.org/jira/browse/HBASE-17492 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky Make the MemoryCompaction enum, not an internal class of HColumnDescriptor, but an external class. This enum is later used in the ruby script and the ruby script doesn't accept this internal class and proceeds with an error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17377) MemStoreChunkAllocator
Anastasia Braginsky created HBASE-17377: --- Summary: MemStoreChunkAllocator Key: HBASE-17377 URL: https://issues.apache.org/jira/browse/HBASE-17377 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky Refactoring the separation between MemStoreChunkPool and (new) MemStoreChunkAllocator. The latter allocates chunks either from heap or from pool and assigns them the Chunk IDs. MemStoreChunkAllocator stores the mapping between chunks and their IDs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17373) Reverse the order of snapshot creation in the CompactingMemStore
Anastasia Braginsky created HBASE-17373: --- Summary: Reverse the order of snapshot creation in the CompactingMemStore Key: HBASE-17373 URL: https://issues.apache.org/jira/browse/HBASE-17373 Project: HBase Issue Type: Bug Reporter: Anastasia Braginsky In CompactingMemStore both in BASIC and EAGER cases when snapshot is created the segments are first removed from the pipeline then added to the snapshot. This is the opposite to what is done in the DefaultMemStore where the snapshot is firstly created with the active segment and only after the active segment is refreshed. This JIRA is about to reverse the order in CompactingMemStore and to make all MemStores to behave the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17081) Flush the entire CompactingMemStore content to disk
Anastasia Braginsky created HBASE-17081: --- Summary: Flush the entire CompactingMemStore content to disk Key: HBASE-17081 URL: https://issues.apache.org/jira/browse/HBASE-17081 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky Assignee: Anastasia Braginsky Part of CompactingMemStore's memory is held by an active segment, and another part is divided between immutable segments in the compacting pipeline. Upon flush-to-disk request we want to flush all of it to disk, in contrast to flushing only tail of the compacting pipeline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16608) Introducing the ability to merge ImmutableSegments without copy-compaction or SQM usage
[ https://issues.apache.org/jira/browse/HBASE-16608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Braginsky resolved HBASE-16608. - Resolution: Fixed > Introducing the ability to merge ImmutableSegments without copy-compaction or > SQM usage > --- > > Key: HBASE-16608 > URL: https://issues.apache.org/jira/browse/HBASE-16608 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 2.0.0 > > Attachments: HBASE-16417-V02.patch, HBASE-16417-V04.patch, > HBASE-16417-V06.patch, HBASE-16417-V07.patch, HBASE-16417-V08.patch, > HBASE-16417-V10.patch, HBASE-16608-Final.patch, HBASE-16608-Final.patch, > HBASE-16608-V01.patch, HBASE-16608-V03.patch, HBASE-16608-V04.patch, > HBASE-16608-V08.patch, HBASE-16608-V09.patch, HBASE-16608-V09.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16608) Introducing the ability to merge ImmutableSegments without copy-compaction or SQM usage
Anastasia Braginsky created HBASE-16608: --- Summary: Introducing the ability to merge ImmutableSegments without copy-compaction or SQM usage Key: HBASE-16608 URL: https://issues.apache.org/jira/browse/HBASE-16608 Project: HBase Issue Type: Sub-task Reporter: Anastasia Braginsky -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16421) Introducing the CellChunkMap as a new additional indes in the MemStore
Anastasia Braginsky created HBASE-16421: --- Summary: Introducing the CellChunkMap as a new additional indes in the MemStore Key: HBASE-16421 URL: https://issues.apache.org/jira/browse/HBASE-16421 Project: HBase Issue Type: Umbrella Reporter: Anastasia Braginsky -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions
Anastasia Braginsky created HBASE-16417: --- Summary: In-Memory MemStore Policy for Flattening and Compactions Key: HBASE-16417 URL: https://issues.apache.org/jira/browse/HBASE-16417 Project: HBase Issue Type: Improvement Reporter: Anastasia Braginsky -- This message was sent by Atlassian JIRA (v6.3.4#6332)
