[
https://issues.apache.org/jira/browse/BIGTOP-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126271#comment-14126271
]
Roman Shaposhnik commented on BIGTOP-1422:
------------------------------------------
Sorry for coming late to the party. Still I feel compelled to share
a few observations of the discussions that has happened so far.
Ultimately, while I appreciate [~cos] enthusiasm and I do agree
that a few use cases (such as rolling upgrades, side-by-side installs, etc)
are quite important to our customers, I need to remind us not
to commit the same grave mistake that Cloudera has committed with
Parcels.
Here it is in a nutshell: without Cloudera Manager parcels are
not only useless -- they become a liability. Suppose you install
your cluster via CM using parcels and then, later, decide to
migrate away to a different Configuration Management solution
(Puppet, Chef, etc.). Well, at that point you've just signed
yourself up for a very tedious and error-prone manual migration
process.
Now, granted, as a commercial piece of software CM has all the
right in the world to look at parcels as its own implementation
details. That is fine. What isn't fine at all is us, here in
Bigtop, essentially endorsing a vendor lock-in. [~cos] you
of all people should know way better than this!
With that in mind, let us actually take a broader outlook and
to [~bmahe] point try to define what is the actual problem
we are trying to solve here.
On a typical cluster here are the tree inter-related problems
that need to be solved:
# distributing appropriate (as defined by a node's role)
set of bits to all nodes in the cluster while preserving
BOM consistency (e.g. we can't mix a version of Pig from
one Bigtop release with a version of Hadoop from a
different one).
# distributing appropriate configuration files to all
nodes in the cluster.
# orchestrating state transitions of individual
services across the cluster
It goes without saying that #3 is a very thorny problem that
perhaps can be solved by systemd and etcd integration, but
it clearly lies outside of this JIRA. Recognizing this allows
us to simplify #1 and #2 though: we can treat them as a
simple matter of properly distributing immutable, innert
bits. After all, as long as multiple versions of these bits
can co-exist -- activating a particular one becomes one
of the issues that #3 deals with.
It is my strong personal belief that whatever we do #1 has
to integrate well with existing Linux packages. I would go
as far as even extending it to #2.
If there's enough interest in both -- I can post my thoughts
on how Bigtop RPM/DEB packaging can achieve both
at the same time. Let me know. We can even have this
discussion on list to make it more interactive.
> Introduce Universal Packaging System for Hadoop distribution
> ------------------------------------------------------------
>
> Key: BIGTOP-1422
> URL: https://issues.apache.org/jira/browse/BIGTOP-1422
> Project: Bigtop
> Issue Type: New Feature
> Components: build, deployment
> Affects Versions: 0.8.0
> Reporter: Konstantin Boudnik
> Priority: Blocker
> Fix For: 1.0.0
>
>
> Looks like the time comes where Bigtop needs to step up and champion new
> revolutionary format for packaging of the Hadoop.
> New UPS provides completely relocatable archive-like media of distribution
> for the content and the services alike. Preparation of an UPS deliverables
> shouldn't require any special development tools nor particular SDL: only
> cross-platform archives and widely available command-line interpreters would
> be a per-requisite for the creation and installation.
> In our opinion, the separation of the services and content of the
> applications, e.g. .deb and .rpm, are an outdated paradigm, which complicates
> the deployment processes, upgrades, and overall system orchestration.
> Architectural proposal will follow shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
