[
https://issues.apache.org/jira/browse/HBASE-23223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Wang updated HBASE-23223:
-------------------------------
Description:
The offsetLock of BucketCache use SoftReference for its
ReentrantReadWriteLock,when bucketCache is big,it will cost too much time in Gc
ref-proc.
eg :
2019-09-29T01:55:45.186+0800: 365222.053:
[GC remark
2019-09-29T01:55:45.186+0800: 365222.053:
[Finalize Marking, 0.0016327 secs]
2019-09-29T01:55:45.188+0800: 365222.054:
[GC ref-proc
2019-09-29T01:55:45.188+0800: 365222.054: [SoftReference, 1264586 refs,
0.3151392 secs]
2019-09-29T01:55:45.503+0800: 365222.370: [WeakReference, 4317 refs, 0.0024381
secs]
2019-09-29T01:55:45.505+0800: 365222.372: [FinalReference, 9791 refs, 0.0037445
secs]
2019-09-29T01:55:45.509+0800: 365222.376: [PhantomReference, 0 refs, 1963 refs,
0.0018941 secs]
2019-09-29T01:55:45.511+0800: 365222.378: [JNI Weak Reference, 0.0001156 secs]
, 1.4554361 secs]
2019-09-29T01:55:46.643+0800: 365223.510:
[Unloading, 0.0211370 secs]
, 1.4851728 secs]
I think the strong reference is a better choice in some case ,so i changed it
and get good effect on my live cluster:
2019-10-26T02:29:10.248+0800: 131255.447:
[GC remark
2019-10-26T02:29:10.248+0800: 131255.447:
[Finalize Marking, 0.0011426 secs]
2019-10-26T02:29:10.249+0800: 131255.448:
[GC ref-proc
2019-10-26T02:29:10.249+0800: 131255.448: [SoftReference, 213 refs, 0.0002216
secs]
2019-10-26T02:29:10.249+0800: 131255.448: [WeakReference, 2423 refs, 0.0007833
secs]
2019-10-26T02:29:10.250+0800: 131255.449: [FinalReference, 4991 refs,
0.0061449 secs]
2019-10-26T02:29:10.256+0800: 131255.455: [PhantomReference, 0 refs, 7940
refs, 0.0011711 secs]
2019-10-26T02:29:10.257+0800: 131255.456: [JNI Weak Reference, 0.0000579 secs]
, 0.0086389 secs]
2019-10-26T02:29:10.257+0800: 131255.456:
[Unloading, 0.0123417 secs]
, 0.0294484 secs]
Strong reference will always hold the mem,so it maybe not fit for some case
that have not enough mem, so i make it configable.
was:
The offsetLock of BucketCache use SoftReference for its
ReentrantReadWriteLock,when bucketCache is big,it will cost too much time in Gc
ref-proc.
eg : 2019-09-29T01:55:45.186+0800: 365222.053: [GC remark
2019-09-29T01:55:45.186+0800: 365222.053: [Finalize Marking, 0.0016327 secs]
2019-09-29T01:55:45.188+0800: 365222.054: [GC ref-proc
2019-09-29T01:55:45.188+0800: 365222.054: [SoftReference, 1264586 refs,
0.3151392 secs] 2019-09-29T01:55:45.503+0800: 365222.370: [WeakReference, 4317
refs, 0.0024381 secs] 2019-09-29T01:55:45.505+0800: 365222.372:
[FinalReference, 9791 refs, 0.0037445 secs] 2019-09-29T01:55:45.509+0800:
365222.376: [PhantomReference, 0 refs, 1963 refs, 0.0018941 secs]
2019-09-29T01:55:45.511+0800: 365222.378: [JNI Weak Reference, 0.0001156 secs]
, 1.4554361 secs] 2019-09-29T01:55:46.643+0800: 365223.510: [Unloading,
0.0211370 secs] , 1.4851728 secs]
I think the strong reference is a better choice in some case ,so i changed it
and get good effect on my live cluster:
2019-10-26T02:29:10.248+0800: 131255.447:
[GC remark
2019-10-26T02:29:10.248+0800: 131255.447:
[Finalize Marking, 0.0011426 secs]
2019-10-26T02:29:10.249+0800: 131255.448:
[GC ref-proc
2019-10-26T02:29:10.249+0800: 131255.448: [SoftReference, 213 refs, 0.0002216
secs]
2019-10-26T02:29:10.249+0800: 131255.448: [WeakReference, 2423 refs, 0.0007833
secs]
2019-10-26T02:29:10.250+0800: 131255.449: [FinalReference, 4991 refs, 0.0061449
secs]
2019-10-26T02:29:10.256+0800: 131255.455: [PhantomReference, 0 refs, 7940 refs,
0.0011711 secs]
2019-10-26T02:29:10.257+0800: 131255.456: [JNI Weak Reference, 0.0000579 secs]
, 0.0086389 secs]
2019-10-26T02:29:10.257+0800: 131255.456:
[Unloading, 0.0123417 secs]
, 0.0294484 secs]
Strong reference will always hold the mem,so it maybe not fit for some case
that have not enough mem, so i make it configable.
> Support the offsetLock of bucketCache to use strong reference
> --------------------------------------------------------------
>
> Key: HBASE-23223
> URL: https://issues.apache.org/jira/browse/HBASE-23223
> Project: HBase
> Issue Type: Improvement
> Components: BucketCache
> Reporter: Zheng Wang
> Assignee: Zheng Wang
> Priority: Major
>
> The offsetLock of BucketCache use SoftReference for its
> ReentrantReadWriteLock,when bucketCache is big,it will cost too much time in
> Gc ref-proc.
> eg :
> 2019-09-29T01:55:45.186+0800: 365222.053:
> [GC remark
> 2019-09-29T01:55:45.186+0800: 365222.053:
> [Finalize Marking, 0.0016327 secs]
> 2019-09-29T01:55:45.188+0800: 365222.054:
> [GC ref-proc
> 2019-09-29T01:55:45.188+0800: 365222.054: [SoftReference, 1264586 refs,
> 0.3151392 secs]
> 2019-09-29T01:55:45.503+0800: 365222.370: [WeakReference, 4317 refs,
> 0.0024381 secs]
> 2019-09-29T01:55:45.505+0800: 365222.372: [FinalReference, 9791 refs,
> 0.0037445 secs]
> 2019-09-29T01:55:45.509+0800: 365222.376: [PhantomReference, 0 refs, 1963
> refs, 0.0018941 secs]
> 2019-09-29T01:55:45.511+0800: 365222.378: [JNI Weak Reference, 0.0001156 secs]
> , 1.4554361 secs]
> 2019-09-29T01:55:46.643+0800: 365223.510:
> [Unloading, 0.0211370 secs]
> , 1.4851728 secs]
>
> I think the strong reference is a better choice in some case ,so i changed
> it and get good effect on my live cluster:
>
> 2019-10-26T02:29:10.248+0800: 131255.447:
> [GC remark
> 2019-10-26T02:29:10.248+0800: 131255.447:
> [Finalize Marking, 0.0011426 secs]
> 2019-10-26T02:29:10.249+0800: 131255.448:
> [GC ref-proc
> 2019-10-26T02:29:10.249+0800: 131255.448: [SoftReference, 213 refs,
> 0.0002216 secs]
> 2019-10-26T02:29:10.249+0800: 131255.448: [WeakReference, 2423 refs,
> 0.0007833 secs]
> 2019-10-26T02:29:10.250+0800: 131255.449: [FinalReference, 4991 refs,
> 0.0061449 secs]
> 2019-10-26T02:29:10.256+0800: 131255.455: [PhantomReference, 0 refs, 7940
> refs, 0.0011711 secs]
> 2019-10-26T02:29:10.257+0800: 131255.456: [JNI Weak Reference, 0.0000579
> secs]
> , 0.0086389 secs]
> 2019-10-26T02:29:10.257+0800: 131255.456:
> [Unloading, 0.0123417 secs]
> , 0.0294484 secs]
>
> Strong reference will always hold the mem,so it maybe not fit for some case
> that have not enough mem, so i make it configable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
