[
https://issues.apache.org/jira/browse/HDFS-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13013822#comment-13013822
]
Matt Foley commented on HDFS-1798:
----------------------------------
Design sketch:
The objective is to speed up cluster startup even more. It is our premise that
while SafeMode is in force, the namespace is prevented from changing in any way
important to saveNamespace(), even while processing block reports. Is anyone
aware of exceptions to this? Is this also true for the DatanodeImage info?
If this premise is valid, then the following steps should work:
- in FSNamesystem.initialize(), split the functionality of loadFSImage to
separate the read and write phases. Move up the instantiation of SafeMode to
just after FSImage read, but before FSImage write.
- add a new feature to SafeModeInfo allowing it to be "pinned" to a condition,
so it cannot leave SafeMode until the condition is satisfied
- in saveNamespace() where it currently launches the FSImageSaver threads and
waits for them, instead launch one master thread which:
* enters a "try" context
* if SafeMode is on, "pin" it on
* launches and waits for the FSImageSaver threads
* in the "finally" block, releases the SafeMode condition
- if SafeMode was on and was successfully "pinned", then saveNamespace() does
not wait for the master thread to complete.
- all threads are DaemonThreads to ease termination in case of problems.
Comments welcome.
> saveNamespace should background the writing of the FSImage files when in
> SafeMode
> ---------------------------------------------------------------------------------
>
> Key: HDFS-1798
> URL: https://issues.apache.org/jira/browse/HDFS-1798
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Matt Foley
>
> Even with parallel writes of the FSImage files (HDFS-1071), large clusters
> still typically spend 5 minutes writing the files. When in SafeMode, it
> should be safe to background these writes instead of waiting for them. To
> enable this, the namenode must be prevented from leaving SafeMode until the
> writes are done.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
