[ 
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15420309#comment-15420309
 ] 

Anastasia Braginsky commented on HBASE-14921:
---------------------------------------------

bq. Makes sense. So were you able to observe the problems that we highlighted 
in the previous discussions - We got two problems one is the invalid negative 
size for memstore and the 'Too many open files'?
Were you able to get this patch running with the PE tool and the configs that 
we had updated in the earlier discussions?

We are yet unable to use the PE tool. Even on DefaultMemStore it doesn't work 
properly. We are currently running with YCSB, stressing the writes. We didn't 
get the exceptions that you mentioned, neither "negative size" nor "too many 
open files".
However, we were able to see the problem of "too many open files" without 
getting the exception. From here, the quick fix of limiting the number of 
segments in the pipeline. This to be managed by a proper policy later. 

Later today I am going to publish the patch where we have instrumentation 
measuring the time it takes to do the speculative scan. Will you be able to 
give it a run on your PE tool? You can by the way take a look if you are still 
seeing the "negative size" exception. This one (if still exists) can later be 
investigated while dealing with the sizes issues.

> Memory optimizations
> --------------------
>
>                 Key: HBASE-14921
>                 URL: https://issues.apache.org/jira/browse/HBASE-14921
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: Eshcar Hillel
>            Assignee: Anastasia Braginsky
>         Attachments: CellBlocksSegmentInMemStore.pdf, 
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch, 
> HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch, 
> HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch, 
> HBASE-14921-V06-CAO.patch, HBASE-14921-V08-CAO.patch, 
> HBASE-14921-V09-CAO.patch, InitialCellArrayMapEvaluation.pdf, 
> IntroductiontoNewFlatandCompactMemStore.pdf, MemStoreSizes.pdf, 
> MemstoreItrCountissue.patch, NewCompactingMemStoreFlow.pptx
>
>
> Memory optimizations including compressed format representation and offheap 
> allocations



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to