[
https://issues.apache.org/jira/browse/HBASE-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427130#comment-13427130
]
Lars George commented on HBASE-6497:
------------------------------------
The goal in designing a proper HBase schema is to maximize heap usage across
all regions, which can lead to the situation where the WALs (aka HLog's) are
required to be kept for a considerable amount of time.
The last iteration on WAL properties added a configurable block size, as well
as threshold percentage to roll the log before it completely fills the single
HDFS block (see HBASE-1394).
I am questioning if this is still in issue, maybe even in the light of recent
improvements on log performance, for example HBASE-5699 and HBASE-4608.
At the least, I would like to figure out, if we should increase the WAL size to
512MB, to avoid getting into early flushing situations, impacting the overall
I/O. Isn't HBASE-1364 helping to split larger logs (though not the logs
themselves but distributed across the region servers obviously). I am not sure
if the log splitting prefers block local nodes first, so that there is no
remote reading though.
Questions:
# Is there a need to keep the logs small (typically 64-128 depending on the
HDFS config)?
# Should we go multiple blocks?
# Do we still need the logroll multiplier?
# Should we increase the maxlogs number (default is 32)?
> Revisit HLog sizing and roll parameters
> ---------------------------------------
>
> Key: HBASE-6497
> URL: https://issues.apache.org/jira/browse/HBASE-6497
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: Lars George
>
> The last major update to the HLog sizing and roll features were done in
> HBASE-1394. I am proposing to revisit these settings to overcome recent
> issues where the HLog becomes a major bottleneck.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
