[
https://issues.apache.org/jira/browse/HDFS-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Xiaoqiao resolved HDFS-14511.
--------------------------------
Resolution: Duplicate
> FSEditlog write both Quorum Journal and Local disk by default in HA using QJM
> scenario
> --------------------------------------------------------------------------------------
>
> Key: HDFS-14511
> URL: https://issues.apache.org/jira/browse/HDFS-14511
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode, qjm
> Reporter: He Xiaoqiao
> Assignee: He Xiaoqiao
> Priority: Major
>
> Recently, I meet case about FSEditLog in HA using QJM scenario. NameNode
> enter suspended state and can not process other RPC requests any more.
> The root cause is load of local disk is very high, it will block edit log
> recored flush local, then following RPC request will occupy all RPC handlers
> since #FSEditLog write edit log record to both FileJournal (which is local
> directory located at the same as FsImage) and QuorumJournal in proper order
> by default and no configuration to switch off FileJournal. However local edit
> log is not used any time soon.
> More detailed information, the location where edit log write to is decided by
> configuration items 'dfs.namenode.shared.edits.dir' and
> 'dfs.namenode.name.dir' (since 'dfs.namenode.edits.dir' is deprecated item,
> if not set it will be overrided/replaced by 'dfs.namenode.name.dir' where
> fsimage located.) by default. So JournalSet = QuorumJournal (SharedEditsDirs,
> set by 'dfs.namenode.shared.edits.dir') + FileJournal (LocalStorageEditsDirs,
> set by 'dfs.namenode.name.dir' by default). Another side, these two config
> items have to set in HA with QJM.
> In one word, edit log is double write to both QJM and local disk by default
> and no way to turn off local write with current implementation. I propose we
> should offer some choice or turn off local edit log write by default in HA
> using QJM for users.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
