[
https://issues.apache.org/jira/browse/BIGTOP-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122584#comment-14122584
]
Bruno Mahé commented on BIGTOP-1422:
------------------------------------
Thanks for adding more details on that proposal!
{quote} I think this community has enough leverage to start dictating the
industry what and how really Hadoop should be deploy and managed, right?{quote}
{quote}getting buy-in from Ambari: I don't see how they won't follow the lead
with all the benefits I just laid out (and more){quote}
Actually I would rather not dictate anything and would rather not assume that
people will find such solution awesome after the facts. Instead I would rather
lead by example and bring together multiple communities to foster cooperation
and consensus.
If we want buy in from other communities, let's invite them now into the
conversation to gather their thoughts and requirements so the solution can
really fit their needs and requirements. Their feedback and experience will be
valuable to your proposal and we can involve and make that community a part of
the solution.
I don't necessarily agree with the benefits layed out above but in any case,
this discussion is not about packages vs UPS/Parcel and we don't have to choose
one or the other. If people are interested in working in UPS, Apache Bigtop is
a natural host for such effort. The same applies to RPM, DEB or any other
package management system. So we are not going to deprecate one in favor of the
other. And as well, I would not tie UPS to a particular Apache Bigtop release
in order to let it mature appropriately and not block release trains.
Regarding the implementation idea shared above, what worries me is not to get
something working. Making a parcel system is not that difficult.
What worries me are mainly the little details and time and effort to get it.
Besides you, who would be interested in working on that? This is not a blocker,
since we are not time bound.
The other part is all the tooling and integration. Yum and co have an entire
ecosystem of tools available to sysadmins such as proxies and mirrors (creation
and use) so the cluster does not have to reach for the internet and gets the
bits faster, security integrated end to end from packages to repositories,
delta packages so you don't have to re-download entire packages for updates as
well as many other options and tools. And regarding the integration, RPM/YUM
and co also provide a host of basic tools enabling an easy integration such as
queries (what is being installed?). And tools like puppet directly integrate
commands for packages.
Your implementation does not list basic tools to manipulate UPS. For instance
doing a wget is a little bit too low level (and not convenient if you want to
have some sort of repositories). Or to help with rolling upgrade.
Similarly I would suggest to add some features in your implementation to extend
puppet DSL so people interact with UPS in an easy way. As you currently
describe, a puppet user would have to write recipes to download, expand and
move bits arounds just to call a single script (along with the proper
notify/require). Not very convenient.
Also how would you handle additional jars? What if I need to drop a new jar
into Apache Hadoop lib directory?
> Introduce Universal Packaging System for Hadoop distribution
> ------------------------------------------------------------
>
> Key: BIGTOP-1422
> URL: https://issues.apache.org/jira/browse/BIGTOP-1422
> Project: Bigtop
> Issue Type: New Feature
> Components: build, deployment
> Affects Versions: 0.8.0
> Reporter: Konstantin Boudnik
> Priority: Blocker
> Fix For: 1.0.0
>
>
> Looks like the time comes where Bigtop needs to step up and champion new
> revolutionary format for packaging of the Hadoop.
> New UPS provides completely relocatable archive-like media of distribution
> for the content and the services alike. Preparation of an UPS deliverables
> shouldn't require any special development tools nor particular SDL: only
> cross-platform archives and widely available command-line interpreters would
> be a per-requisite for the creation and installation.
> In our opinion, the separation of the services and content of the
> applications, e.g. .deb and .rpm, are an outdated paradigm, which complicates
> the deployment processes, upgrades, and overall system orchestration.
> Architectural proposal will follow shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
