[
https://issues.apache.org/jira/browse/HADOOP-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14101361#comment-14101361
]
Allen Wittenauer commented on HADOOP-9902:
------------------------------------------
bq. bin/hadoop not longer checks for hdfs commands portmap and nfs3. Is this
intentional?
Yes. Those commands were never hooked into the hadoop command in the Apache
source that I saw... but I guess I could have missed one? In any case, I
didn't see a reason to have an explicit check for something that never existed
as a result, especially considering how much other, actually deprecated stuff
is there.
(It could be argued that for trunk all of these deprecations should be removed
since it's going to be a major release since they were put in. In other words,
they were in 1.x, deprecated in 2.x, and if this is going into 3.x, we could
remove them. There's some discussion on that in this jira.)
bq. hadoop-daemon.sh usage no longer prints --hosts optional paramter in usage;
this is intentional right?
Correct. --hosts only ever worked as far as back as I looked with
hadoop-daemons.sh (plural) and related commands. The --hosts in
hadoop-daemon.sh's (singular) usage was a very longstanding (and amusing) bug.
bq. Also does all daemons now support option status along with start and stop?
If those daemons use the shell daemon framework (hadoop_daemon_handler,
hadoop_secure_daemon_handler, etc) in hadoop-functions, yes. So, barring bugs
or different functionality in the Java code, this should cover all current
daemons started by yarn, hdfs, and mapred. This means kms, httpfs, etc, that
haven't been converted yet unfortunately do not. I've got another jira open to
rewrite those to use the new stuff.
To cover what I suspect is the future question, if one adds a daemon following
the pattern (daemon=true being the big one) to the current commands, that
daemon will get the status handling and more (stop, start, logs, pids, etc) for
free. This also means that if we add, e.g. 'restart', all daemons will get it
too. Consolidating all of this daemon handling makes this much much easier.
There is some other cleanup that should probably happen here to make it easier
to add new --daemon capability though. (e.g., changing hadoop_usage everywhere
is a pain.)
bq. locating HADOOP_PREFIX is repeated in bin/hadoop and hadoop-daemon.sh (this
can be optimized in a future patch)
It's intentional because we need to run through the initialization code to find
where the hdfs command lives. Totally agree it's ugly, but with the
hadoop-layout.sh code that was introduced in 0.21, we're sort of stuck here.
FWIW, mapred and yarn have the same ugliness.
bq. start-all.sh and stop-all.sh exits with warning. Why retain code after
that. Expect users to delete the exit in the beginning?
I started to clean this up but realized it could wait. So at some point, I plan
to clean this up and make it functional, esp wrt HADOOP-6590 and some...
tricks. ;) I didn't see any harm in leaving the code there for reference.
Plus, as you noticed, if someone wanted to make their own, they could pull it
out, delete those lines, and be on their way.
bq. hadoop_error is not used in some cases and still echo is used.
Correct. hadoop_error isn't defined yet in some situations so the script has
to echo to stderr manually. In particular, when the code is looking for
HADOOP_LIBEXEC_DIR and the location of hadoop-functions.sh... so that it can
define those functions. ;)
bq. hadoop-env.sh - we should document the GC configuration for max, min, young
generation starting and max size.
This should probably be a part of HADOOP-10950. I'm going to rework the
generic heap management to allow for setting Xms, get rid of JAVA_HEAP, etc.
Since this is another (but thankfully smaller) "touch everything" JIRA, it'd be
great if you could update that one with what you had in mind. (I think I know
what you have in mind, since I suspect this reflects upon the examples I put in
for NN, etc GC stuff. )
bq. hadoop_usage is in every script (I checked, it is).
Shame on you for ruining my easter egg... but your check wasn't very thorough.
;)
> Shell script rewrite
> --------------------
>
> Key: HADOOP-9902
> URL: https://issues.apache.org/jira/browse/HADOOP-9902
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
> Affects Versions: 3.0.0
> Reporter: Allen Wittenauer
> Assignee: Allen Wittenauer
> Labels: releasenotes
> Fix For: 3.0.0
>
> Attachments: HADOOP-9902-10.patch, HADOOP-9902-11.patch,
> HADOOP-9902-12.patch, HADOOP-9902-13-branch-2.patch, HADOOP-9902-13.patch,
> HADOOP-9902-14.patch, HADOOP-9902-2.patch, HADOOP-9902-3.patch,
> HADOOP-9902-4.patch, HADOOP-9902-5.patch, HADOOP-9902-6.patch,
> HADOOP-9902-7.patch, HADOOP-9902-8.patch, HADOOP-9902-9.patch,
> HADOOP-9902.patch, HADOOP-9902.txt, hadoop-9902-1.patch, more-info.txt
>
>
> Umbrella JIRA for shell script rewrite. See more-info.txt for more details.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
