[
https://issues.apache.org/jira/browse/BIGTOP-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168059#comment-14168059
]
stanley shi edited comment on BIGTOP-1323 at 10/13/14 2:44 AM:
---------------------------------------------------------------
Thanks Roman, this make sense.
Just came to my mind: for example, for build hadoop, previously the protoc
version is 2.4.1 and now it's 2.5.0; they are not exactly compatible, how are
we going to handle this case?
was (Author: stanley_shi):
Thanks Roman, this make sense.
> provide an option to containerize Bigtop package build
> -------------------------------------------------------
>
> Key: BIGTOP-1323
> URL: https://issues.apache.org/jira/browse/BIGTOP-1323
> Project: Bigtop
> Issue Type: Improvement
> Components: build
> Affects Versions: 0.8.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.9.0
>
> Attachments: Dockerfile.sh
>
>
> Looking at the state of our Jenkins made me scream "I'm mad as Hell and I'm
> not going to take this anymore". The basic problem is simple: our build
> slaves are not actually managed by Puppet (even though we do have Puppet code
> to do that) and they seem to bitrot quite a bit (e.g. git clone from ASF
> repos on SLES and CentOS5 is totally borked).
> There's another, somewhat subtler, problem: currently we have to keep as many
> slaves as we've got platforms to support. This leads to a pretty poor
> utilization and if a slave goes down requires manual intervention.
> What I'd like to introduce in this JIRA is a notion of decoupling OS
> environment dependency from the base OS. The technology I'm proposing here is
> Docker (which really is just a prettier UI on top of Linux Containers). IOW,
> regardless of what OS you'd be running on your host -- as long as you've got
> docker you can have an exact replica of a target os running in a container.
> Here's my plan for simplifying the management aspect of what's going on.
> Note, that this is a complimentary plan. Whatever is happening (or not
> happening!) today on our Jenkins can remain there for as long as we need it.
> # make our build slaves be 100% uniform and fungible instances of CoreOS
> VMs https://coreos.com/ The benefit of CoreOS is pretty much 0 administration
> cost, quick boot up time, full integration with Docker (see bellow) and full
> integration with things like VirtualBox, etc (so that you can replicate
> exactly the kind of process that we will setup on the Jenkins).
> # make the actual builds happen in a container using CoreOS's integration
> with Docker. IOW, instead of running: make foo on the slave the actual
> command will look like: docker run -i -t -v `pwd`:/workspace:rw BUILD-ENV
> bash -c 'cd /workspace ; make foo'
> # add jobs to the jenkins that would maintain images of BUILD-ENV
> containers. Each image will be a base-line OS (RHEL, Ubuntu, etc) plus all
> the build dependencies of a given source package (but nothing else!).
> All of these 3 steps are pretty easy to automate and they do have an added
> benefit of being 100% replicatable on your workstation. IOW, as long as you
> have docker running on your host OS you can build packages for any OS with a
> simple command.
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
