[
https://issues.apache.org/jira/browse/HDFS-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12965663#action_12965663
]
dhruba borthakur commented on HDFS-1508:
----------------------------------------
Thanks Sanjay and Konstantin for looking at this one.
First the use case: putting the namenode in safemode causes existing
applications to fail. This is a severe problem for us. If you are using hdfs
for running map-reduce jobs, then putting the namenode is safemode means that
tasks fail immediately. A reduce task that has been running for a long long
time will fail and has to start all over again. If you are running hbase on
hdfs, then hundreds of hbase region servers will die when the namenode goes
into safemode.
@Sanjay: I cluster that runs hbase typically has very few files, less than
100K files. It takes a few seconds to run the savenamespace command. I can
generalize: if a user is running hbase on hdfs, then it makes more sense to
make savenamespace wait for a few seconds (via the read/write lock) rather than
writing special case code in the hbase region servers to handle
SafeModeException. As far as backward compatibility is concerned, i can add a
"-f" option to indicate "do the savenamespace even if namenode is not in
safemode", but still think that this is an option that everybody will use.
Can one of you explain why we always required savenamespace to have namenode is
safemode? isn't it always better to stall the workload rather to fail the
workload?
@Konstantin: can you pl explain the precise problem you have in mind? The
rollFSImage() call acquires the FSNamesystem writelock, so it cannot race with
saveNamespace. Moreover, saveNamespace truncates the edits log and removes
edits.new while rollFSImage will exit out if it does not find edits.new.
> Ability to do savenamespace without being in safemode
> -----------------------------------------------------
>
> Key: HDFS-1508
> URL: https://issues.apache.org/jira/browse/HDFS-1508
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: savenamespaceWithoutSafemode.txt
>
>
> In the current code, the administrator can run savenamespace only after
> putting the namenode in safemode. This means that applications that are
> writing to HDFS encounters errors because the NN is in safemode. We would
> like to allow saveNamespace even when the namenode is not in safemode.
> The savenamespace command already acquires the FSNamesystem writelock. There
> is no need to require that the namenode is in safemode too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
