> On Oct 6, 2017, at 5:51 PM, Eric Yang <ey...@hortonworks.com> wrote:
> yarn application -deploy –f spec.json
> yarn application -stop <service-name>
> yarn application -restart <service-name>
> yarn application -remove <service-name>
>
> and
>
> yarn application –list will display both application list from RM as well as
> docker services?
IMO, that makes much more sense. [*] I’m trying think of a reason why
I’d care if something was using this API or not. It’s not like users can’t run
whatever they want as part of their job now. The break out is really only
necessary so I have an idea if something is running that is using the REST API
daemon. But more on that later….
> I think the development team was concerned that command structure overload
> between batch applications and long running services. In my view, there is
> no difference, they are all applications. The only distinction is the
> launching and shutdown of services may be different from batch jobs. I think
> user can get used to these command structures without creating additional
> command grouping.
I pretty much agree. In fact, I’d love to see ‘yarn application’ even
replace ‘yarn jar’. One Interface To Rule Them All.
I was under the impression (and, maybe this was my misunderstanding. if
so, sorry) that “the goal” for this first pass was to integrate the existing
Apache Slider functionality into YARN. As it stands, I don’t think those goals
have been met. It doesn’t seem to be much different than just writing a shell
profile to call slider directly:
---
function yarn_subcommand_service
{
exec slider “$@“
}
----
(or whatever). Plus doing it this way, one gets the added benefit of the
SIGNIFICANTLY better documentation. (Seriously: well done that team)
From an outside perspective, the extra daemon for running the REST API
seems like when it should have clicked that the project is going off the rails
and missing the whole “integration” aspect. Integrating the REST API into the
RM from day one and the command separation would have also stuck out. If the RM
runs the REST API, it now becomes a problem of “how does a user launch more
than just a jar easily?” A problem that Hadoop has had since nearly day one.
Redefining the “application” subcommand sounds like a reasonable way to move
forward on that problem while also dropping the generic sounding "service"
subcommand.
But all that said, it feels like direct integration was avoided from
the beginning and I’m unclear as to why. Take this line from the quick start
documentation:
"Start all the hadoop components HDFS, YARN as usual.”
a) This sentence is pretty much a declaration that this feature
set isn’t part of “YARN”.
b) Minimally, this should link to ClusterSetup.
Anyway, yes, please work on removing all of these extra adoption
barriers and increased workload on admin teams with Yet Another Daemon to
monitor and collect metrics.
Thanks!
[*] - I’m reminded of a conversation I had with a PMC member year or three ago
about HDFS. They proudly almost defiantly stated that the HDFS command
structure is such because it resembles the protocols and that was great. Guess
what: users’ don’t care about how something is implemented, much less the
protocols that are used to drive it. They care about consistency, EOU, and all
those feel good things that make applications a joy to use. They have more
important stuff to do. Copying the protocols onto the command line only help
the person who wrote it and no one else. It’s hard not to walk away from
playing with YARN in this branch as exhibiting those same anti-user behaviors.
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
