[
https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072375#comment-17072375
]
Michael Stack commented on HBASE-24051:
---------------------------------------
bq. I don't think you can pin your hopes on someone else thinking about
perfection...
Sorry. What you mean by above? You mean client doing manual unbuffer calls will
never be satisfactory?
bq. So the hbase side as a client needs to be as thoughtful as possible.
Agree.
bq. I don't know why hbase keeps those inputstreams after load, just
unbuffer....
They were added to address an old plethora of CLOSE_WAIT issues, a common
complaint. Do the CLOSE_WAITs return do you know if we apply this patch?
bq. vI'll keep watching.
Do you mean, apply the patch and you'll watch how it performs (or something
else?)
bq. Another easy way to avoid errors is to add "implements CanUnbuffer" to the
DFSStripedInputStream
Would this be a HDFS change?
Was interested in the use case that has hbase run on top of an EC FS.
Thanks
> When the data directory is set to EC,the amount of storefiles reaches a
> relatively high level,heap Occupancy Percent is above heap occupancy alarm
> watermark,resulting in frequent gc and eventual failure
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-24051
> URL: https://issues.apache.org/jira/browse/HBASE-24051
> Project: HBase
> Issue Type: Bug
> Affects Versions: 2.0.1
> Reporter: shenshengli
> Priority: Major
> Attachments: 1.png
>
>
> When the data directory is set to EC, during the continuous load process,
> regionserver is found down ,to view the log:
> regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below
> the heap occupancy alarm watermark (0.95)
> regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap
> occupancy alarm watermark (0.95)
> [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine
> (eg GC): pause of approximately 1254ms
> GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms
> GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms
> In particular, GC pauses occurred 283 times in half an hour.
> The memory analysis found that most of the memory footprint was generated by
> HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer
> See HDFS-14308，but after the fix, hbase did not return to normal.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
