[
https://issues.apache.org/jira/browse/HDFS-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12797276#action_12797276
]
Konstantin Shvachko commented on HDFS-161:
------------------------------------------
It is true if a directory listed in {{dfs.datanode.data.dir}} (formerly
{{dfs.data.dir}}) does not exist, it will be automatically created by the
data-node.
I traced this behavior back to where the history goes: it has always been like
that. The idea is to make data-nodes autonomous, because on large clusters it
is hard to keep track of each and every one of them.
This contradicts the description to field {{dfs.datanode.data.dir}} saying "...
Directories that do not exist are ignored.", which also goes far back in
history.
I'd propose to update the description and change the documentation as far back
in versions as we possible could.
> DataNode will create missing dfs.data.dir directories contrary to docs
> ----------------------------------------------------------------------
>
> Key: HDFS-161
> URL: https://issues.apache.org/jira/browse/HDFS-161
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, documentation
> Reporter: Todd Lipcon
> Priority: Minor
>
> The documentation (and HADOOP-340) seems to indicate that if a data dir
> listed in dfs.data.dir does not exist, it will be ignored. In fact, if the
> hadoop user has access to (essentially) mkdir -p the data dir it will do so.
> I'd like to add another configuration parameter, perhaps dfs.data.dir.create
> with default true (to not break any existing tutorials). If this is set to
> false then data dirs that do not exist will not implicitly be created.
> The motivation for this is the same as HADOOP-340 - we have different disk
> configs in our cluster and hadoop is creating data dirs where we do not want
> them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
