[
https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561837#comment-13561837
]
Devaraj Das commented on HBASE-5930:
------------------------------------
[~lhofhansl], yeah what [~enis] meant IMHO is that the latency from the
client's point of view would improve when deferred flush is used for the
mutations. Also, we considered the case that users would most likely not want
to skip WAL if we promise them that there wouldn't be latency issues (maybe on
a busy cluster). But yeah, it'd not make a difference on the overall IOPS in
the cluster...
[~nkeywal], generally agree with you that we should not remove the skipWal
option without giving it a real good thought and before considering more use
cases. And, yes the idea of randomizing the flushes across regionservers sounds
good. I'll think up how to incorporate that.
[~yuzhih...@gmail.com], good catch on the instantiation :) I was focusing on
getting the logic right; forgot to instantiate the chore. I'd prefer to leave
the shouldFlush() signature as is (it's a matter of implementation that the
shouldFlush method implementation is using the same constant underneath but it
could be very well a different constant or shouldFlush implementation could be
different sometime when this constant is not even used..).
> Periodically flush the Memstore?
> --------------------------------
>
> Key: HBASE-5930
> URL: https://issues.apache.org/jira/browse/HBASE-5930
> Project: HBase
> Issue Type: Improvement
> Reporter: Lars Hofhansl
> Assignee: Devaraj Das
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5930-1.patch, 5930-wip.patch
>
>
> A colleague of mine ran into an interesting issue.
> He inserted some data with the WAL disabled, which happened to fit in the
> aggregate Memstores memory.
> Two weeks later he a had problem with the HDFS cluster, which caused the
> region servers to abort. He found that his data was lost. Looking at the log
> we found that the Memstores were not flushed at all during these two weeks.
> Should we have an option to flush memstores periodically. There are obvious
> downsides to this, like many small storefiles, etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
