[
https://issues.apache.org/jira/browse/HDFS-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16847249#comment-16847249
]
He Xiaoqiao commented on HDFS-14511:
------------------------------------
[~shv] offer one valid solution that set 'dfs.namenode.shared.edits.dir' and
'dfs.namenode.edits.dir' both point to Quorum Journal, then FSEditlog will not
write local disk, it is trick but indeed feasible solution.
{code:java}
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://qjn1:8485;qjn2:8485;qjn3:8485/myCluster</value>
</property>
<property>
<name>dfs.namenode.edits.dir</name>
<value>qjournal://qjn1:8485;qjn2:8485;qjn3:8485/myCluster</value>
</property>
{code}
I think we should update code logic and solve this issue completely.
> FSEditlog write both Quorum Journal and Local disk by default in HA using QJM
> scenario
> --------------------------------------------------------------------------------------
>
> Key: HDFS-14511
> URL: https://issues.apache.org/jira/browse/HDFS-14511
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode, qjm
> Reporter: He Xiaoqiao
> Assignee: He Xiaoqiao
> Priority: Major
>
> Recently, I meet case about FSEditLog in HA using QJM scenario. NameNode
> enter suspended state and can not process other RPC requests any more.
> The root cause is load of local disk is very high, it will block edit log
> recored flush local, then following RPC request will occupy all RPC handlers
> since #FSEditLog write edit log record to both FileJournal (which is local
> directory located at the same as FsImage) and QuorumJournal in proper order
> by default and no configuration to switch off FileJournal. However local edit
> log is not used any time soon.
> More detailed information, the location where edit log write to is decided by
> configuration items 'dfs.namenode.shared.edits.dir' and
> 'dfs.namenode.name.dir' (since 'dfs.namenode.edits.dir' is deprecated item,
> if not set it will be overrided/replaced by 'dfs.namenode.name.dir' where
> fsimage located.) by default. So JournalSet = QuorumJournal (SharedEditsDirs,
> set by 'dfs.namenode.shared.edits.dir') + FileJournal (LocalStorageEditsDirs,
> set by 'dfs.namenode.name.dir' by default). Another side, these two config
> items have to set in HA with QJM.
> In one word, edit log is double write to both QJM and local disk by default
> and no way to turn off local write with current implementation. I propose we
> should offer some choice or turn off local edit log write by default in HA
> using QJM for users.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
