[
https://issues.apache.org/jira/browse/HDFS-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103527#comment-14103527
]
Uma Maheswara Rao G commented on HDFS-6871:
-------------------------------------------
Nice catch Yi!
Thanks a lot for working on fixing. I have one comment.
Just skipping the logsync might have issue with NN crash.
With overwrite true, existing file deleted and removed the blocks from memory.
All this delete op is under fsnlock as it is in startFIle fsn lock.
Once it release fsnlock, NN might be ready for scheduling the block deletions
to DN regarding to deleted file. Just before logsync if NN schedules block
deletions and NN crash without logsync.
Then on restart of NN, it will get the deleted file back as it did not persist
the edits, but DN would have deleted the blocks already. Then file metadata
exist but blocks will be missed. This condition is not right for NN state.
Lets delete blocks only after logsync.
> Improve NameNode performance when creating file
> -------------------------------------------------
>
> Key: HDFS-6871
> URL: https://issues.apache.org/jira/browse/HDFS-6871
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode, performance
> Reporter: Yi Liu
> Assignee: Yi Liu
> Priority: Critical
> Fix For: 2.6.0
>
> Attachments: HDFS-6871.001.patch, HDFS-6871.002.patch
>
>
> Creating file with overwrite flag will cause NN fall into flush edit logs and
> block other requests if the file exists.
> When we create a file with overwrite flag (default is true) in HDFS, NN will
> remove original file if it exists. In FSNamesystem#startFileInternal, NN
> already holds the write lock, it calls {{deleteInt}} if the file exists,
> there is logSync in {{deleteInt}}. So in this case, logSync is under write
> lock, it will heavily affect the NN performance.
> We should ignore the force logSync in {{deleteInt}} in this case.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
