[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105436#comment-13105436 ] Eric Yang commented on HADOOP-7599: --- Aaron, inclusion of the files are intentional. However, we probably want to put license on the files. I filed HADOOP-7641 to track the license changes. Thanks > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0, 0.23.0, 0.24.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-4.patch, HADOOP-7599-5.patch, > HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk-4.patch, HADOOP-7599-trunk-5.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[
https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105164#comment-13105164
]
Aaron T. Myers commented on HADOOP-7599:
I think that by including the contents of the {{templates/}} directories, this
patch introduced several release audit warnings. See [this for pre-commit
Jenkins
run|https://builds.apache.org/job/PreCommit-HADOOP-Build/186/artifact/trunk/hadoop-common-project/patchprocess/patchReleaseAuditProblems.txt]
for an example.
Was the inclusion of these files intentional? If so, should we just add the
relevant directories to RAT's exclude list(s)?
> Improve hadoop setup conf script to setup secure Hadoop cluster
> ---
>
> Key: HADOOP-7599
> URL: https://issues.apache.org/jira/browse/HADOOP-7599
> Project: Hadoop Common
> Issue Type: Bug
> Components: scripts
>Affects Versions: 0.20.203.0
> Environment: Java 6, RHEL 5.6
>Reporter: Eric Yang
>Assignee: Eric Yang
> Fix For: 0.20.205.0, 0.23.0, 0.24.0
>
> Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch,
> HADOOP-7599-3.patch, HADOOP-7599-4.patch, HADOOP-7599-5.patch,
> HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch,
> HADOOP-7599-trunk-4.patch, HADOOP-7599-trunk-5.patch,
> HADOOP-7599-trunk.patch, HADOOP-7599.patch
>
>
> Setting up a secure Hadoop cluster requires a lot of manual setup. The
> motivation of this jira is to provide setup scripts to automate setup secure
> Hadoop cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103559#comment-13103559 ] Hudson commented on HADOOP-7599: Integrated in Hadoop-Hdfs-trunk #792 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/792/]) HADOOP-7599. Script improvements to setup a secure Hadoop cluster. Contributed by Eric Yang. ddas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169986 Files : * /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-dist.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-datanode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-jobtracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-namenode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-tasktracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-create-user.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-hdfs.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-single-node.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-datanode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-jobtracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-namenode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-tasktracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/capacity-scheduler.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/commons-logging.properties * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/core-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-policy.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-queue-acls.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/taskcontroller.cfg > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0, 0.23.0, 0.24.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-4.patch, HADOOP-7599-5.patch, > HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk-4.patch, HADOOP-7599-trunk-5.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103236#comment-13103236 ] Hudson commented on HADOOP-7599: Integrated in Hadoop-Mapreduce-trunk #816 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/816/]) HADOOP-7599. Script improvements to setup a secure Hadoop cluster. Contributed by Eric Yang. ddas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169986 Files : * /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-dist.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-datanode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-jobtracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-namenode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-tasktracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-create-user.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-hdfs.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-single-node.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-datanode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-jobtracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-namenode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-tasktracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/capacity-scheduler.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/commons-logging.properties * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/core-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-policy.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-queue-acls.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/taskcontroller.cfg > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0, 0.23.0, 0.24.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-4.patch, HADOOP-7599-5.patch, > HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk-4.patch, HADOOP-7599-trunk-5.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103220#comment-13103220 ] Hudson commented on HADOOP-7599: Integrated in Hadoop-Mapreduce-trunk-Commit #890 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/890/]) HADOOP-7599. Script improvements to setup a secure Hadoop cluster. Contributed by Eric Yang. ddas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169986 Files : * /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-dist.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-datanode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-jobtracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-namenode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-tasktracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-create-user.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-hdfs.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-single-node.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-datanode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-jobtracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-namenode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-tasktracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/capacity-scheduler.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/commons-logging.properties * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/core-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-policy.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-queue-acls.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/taskcontroller.cfg > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0, 0.23.0, 0.24.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-4.patch, HADOOP-7599-5.patch, > HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk-4.patch, HADOOP-7599-trunk-5.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103219#comment-13103219 ] Hudson commented on HADOOP-7599: Integrated in Hadoop-Hdfs-trunk-Commit #955 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/955/]) HADOOP-7599. Script improvements to setup a secure Hadoop cluster. Contributed by Eric Yang. ddas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169986 Files : * /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-dist.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-datanode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-jobtracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-namenode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-tasktracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-create-user.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-hdfs.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-single-node.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-datanode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-jobtracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-namenode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-tasktracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/capacity-scheduler.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/commons-logging.properties * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/core-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-policy.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-queue-acls.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/taskcontroller.cfg > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0, 0.23.0, 0.24.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-4.patch, HADOOP-7599-5.patch, > HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk-4.patch, HADOOP-7599-trunk-5.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103218#comment-13103218 ] Hudson commented on HADOOP-7599: Integrated in Hadoop-Common-trunk-Commit #878 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/878/]) HADOOP-7599. Script improvements to setup a secure Hadoop cluster. Contributed by Eric Yang. ddas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169986 Files : * /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-dist.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-datanode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-jobtracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-namenode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/deb/init.d/hadoop-tasktracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-create-user.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-conf.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-hdfs.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/hadoop-setup-single-node.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-datanode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-jobtracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-namenode * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/rpm/init.d/hadoop-tasktracker * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/capacity-scheduler.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/commons-logging.properties * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/core-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-env.sh * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hadoop-policy.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/hdfs-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-queue-acls.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/mapred-site.xml * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/taskcontroller.cfg > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0, 0.23.0, 0.24.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-4.patch, HADOOP-7599-5.patch, > HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk-4.patch, HADOOP-7599-trunk-5.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103202#comment-13103202 ] Devaraj Das commented on HADOOP-7599: - I am going to commit the patch in 0.23. When Yarn stabilizes, we can raise another jira and do the appropriate fixes. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-4.patch, HADOOP-7599-5.patch, > HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk-4.patch, HADOOP-7599-trunk-5.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102373#comment-13102373 ] Devaraj Das commented on HADOOP-7599: - I committed the patch on branch-0.20-security. I am hesitant to commit the patch on trunk/0.23 yet since this patch needs some work to make Yarn setup work. Lets discuss.. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-4.patch, HADOOP-7599-5.patch, > HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk-4.patch, HADOOP-7599-trunk-5.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102208#comment-13102208 ] Hadoop QA commented on HADOOP-7599: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493942/HADOOP-7599-5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/161//console This message is automatically generated. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-4.patch, HADOOP-7599-5.patch, > HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk-4.patch, HADOOP-7599-trunk-5.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102183#comment-13102183 ] Hadoop QA commented on HADOOP-7599: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493931/HADOOP-7599-trunk-4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/160//console This message is automatically generated. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-4.patch, HADOOP-7599-trunk-2.patch, > HADOOP-7599-trunk-3.patch, HADOOP-7599-trunk-4.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101715#comment-13101715 ] Eric Yang commented on HADOOP-7599: --- In the patch, I put in the configuration properties that are not security related but are good configurations that are tested to work in large 0.20.2xx clusters. mapred.heartbeats.in.second - enable HADOOP:5784 mapreduce.tasktracker.outofband.heartbeat - enable MAPREDUCE:270 mapred.jobtracker.maxtasks.per.job - safe guard job tracker from running out of memory mapred.jobtracker.retirejob.check - 1 mapred.jobtracker.retirejob.interval - 0 mapred.map.tasks.speculative.execution - false mapred.reduce.tasks.speculative.execution - false mapred.tasktracker.tasks.sleeptime-before-sigkill - 250 > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101702#comment-13101702 ] Hadoop QA commented on HADOOP-7599: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493878/HADOOP-7599-3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/154//console This message is automatically generated. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101685#comment-13101685 ] Devaraj Das commented on HADOOP-7599: - Please remove the config properties that aren't generated by the scripts in this patch (bullet 12 in my first comment). I also noticed now that there are references to 'hadoop' group still present. Please replace them with the variable you defined for the special group. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-3.patch, HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk-3.patch, > HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101643#comment-13101643 ] Matt Foley commented on HADOOP-7599: Hi, I talked with Devaraj about item #9 in his comment [above|https://issues.apache.org/jira/browse/HADOOP-7599?focusedCommentId=13099188&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13099188]. I believe that for a simple setup what Eric is doing is okay, and I would like to have it in 205. Please go ahead and commit, and we can continue making it better in the next release, perhaps by using rpm and deb upgrade/update features as Devaraj mentioned. Thanks. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101636#comment-13101636 ] Hadoop QA commented on HADOOP-7599: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493865/HADOOP-7599-trunk-2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/153//console This message is automatically generated. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-2.patch, > HADOOP-7599-trunk-2.patch, HADOOP-7599-trunk.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101599#comment-13101599 ] Devaraj Das commented on HADOOP-7599: - Went over the patch. Some comments: 1. Don't chmod the keytab dir contents to 755. The keytab files should be owned by the user running the respective daemon, and 700ed. 2. On the bullet#9 in my last comment, you can do a check for empty config files (like if the strings '' and/or '' occurs, the config file is not empty). Not pretty but safer.. Long term, Hadoop could stop shipping the empty config files. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-trunk.patch, > HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101340#comment-13101340 ] Hadoop QA commented on HADOOP-7599: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493816/HADOOP-7599-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/151//console This message is automatically generated. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599-trunk.patch, > HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100968#comment-13100968 ] Hadoop QA commented on HADOOP-7599: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493726/HADOOP-7599-1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/150//console This message is automatically generated. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599-1.patch, HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13099198#comment-13099198 ] Eric Yang commented on HADOOP-7599: --- bq. 6. Why did we remove the call to hadoop-setup-config.sh from src/packages/hadoop-setup-conf.sh Previously, hadoop-setup-conf.sh refers to $PREFIX/libexec/hadoop-config.sh to source existing HADOOP environment variables. It is safer to make hadoop-setup-conf.sh independent from existing environment to avoid recursively append of environment variables, i.e. HADOOP_LOG_DIR=/var/log/hadoop/root/root. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13099188#comment-13099188 ] Eric Yang commented on HADOOP-7599: --- # I will polish this. # I will polish this. # I will polish this. # --datanodes and --tasktrackers are already optional # this is a bug fix. namenode-host is more widely used than namenode-url. It is better to make the name changes. This script was only working internally in RPM post installation script. Therefore, it does not break backward compatibility in this name changes. # I don't understand this. # I will polish this. # I will polish this. # By default, hadoop tar ball ships with empty configuration files. Consequencely, if --force flag is implemented, --force flag becomes mandatory in order to run the script. This is counter productive to have this flag because user will always use --force flag in order to run the setup script. User will take this for granted and --force loses it's ability to safe guard user from making mistakes. # I will polish this. # HADOOP_SECURE_DN_LOG_DIR is used in hadoop-daemon.sh # I will polish this. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098537#comment-13098537 ] Devaraj Das commented on HADOOP-7599: - Some comments: 1. The mapred system dir should be 700ed 2. The -format is always called by default in hadoop-setup-hdfs. Can we make the formatting done based on command line option (-format) provided by the user running the script. I really don't want implicit namenode formatting. 3. We could let the directory /mapred be group owned by the system group. 4. The --datanodes & --tasktrackers options - could they be made optional? They might be already optional. Please confirm. 5. Could we live with namenode-url instead of replacing it with namenode-host (ditto for jobtracker-url). I also see you changed a couple of other places in a backward incompatible way (like HADOOP_JT_HOST, taskcontroller ownership). Want to avoid incompatible changes. 6. Why did we remove the call to hadoop-setup-config.sh from src/packages/hadoop-setup-conf.sh 7. The group is hardcoded to 'hadoop' in a couple of places. Can we avoid that? 8. I see commented-out lines in hadoop-setup-conf.sh. Please remove those. 9. I don't think we should blindly overwrite the conf files in the config directory. We probably should warn & exit if the conf directory already has some files within. The user can probably use --force if he wants to avoid the warning. 10. All the components in the path leading up to taskcontroller.cfg has to be owned by root. Have this checked. 11. Where is HADOOP_SECURE_DN_LOG_DIR used? 12. A whole lot of configuration options that you added in the *-site.xml files are already there in *-default.xml. We don't need those. We only need the security related ones. Also, we don't want to play with the non-security configs like mapred.tasktracker.map.tasks.maximum. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7599) Improve hadoop setup conf script to setup secure Hadoop cluster
[ https://issues.apache.org/jira/browse/HADOOP-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095816#comment-13095816 ] Hadoop QA commented on HADOOP-7599: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12492693/HADOOP-7599.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/122//console This message is automatically generated. > Improve hadoop setup conf script to setup secure Hadoop cluster > --- > > Key: HADOOP-7599 > URL: https://issues.apache.org/jira/browse/HADOOP-7599 > Project: Hadoop Common > Issue Type: Bug > Components: scripts >Affects Versions: 0.20.203.0 > Environment: Java 6, RHEL 5.6 >Reporter: Eric Yang >Assignee: Eric Yang > Fix For: 0.20.205.0 > > Attachments: HADOOP-7599.patch > > > Setting up a secure Hadoop cluster requires a lot of manual setup. The > motivation of this jira is to provide setup scripts to automate setup secure > Hadoop cluster. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
