Rich Freeman <rich0 <at> gentoo.org> writes:
> > Besides, if I want to deploy 50 systems for a cluster, one at a time in
> > parallel what do you recommend? via handbook? The modern diversity of
> > hardware options has rendered the gentoo handbook, dysfunctional,
> > at best, imho. ymmv.
> I have mixed feelings on this one.
YES. We all do. I just think the time has come for gentoo to offer
a variety of installation semantics. The hand book is valid. An installer
is valid. Using Ansible and such is valid. Clonezilla is valid [1].
Using scp or dd HD to HD is valid. There are no limits to valid
pathways. We should just get on with 'diversity of gentoo installs' and
be done with it. If the handbook is 95% of the new installs, so be it.
MY prediction is with other viable options, the handbook we be actively
used as a reference, but we'll quickly experience an increase in usage
of new install semantics. Pentoo is one well kept secret, as you know.[2]
zchaos is a titan, imho, and his work deserves accolades as well as exposure
in the greater gentoo community. I certainly appreciate pentoo
more and more every day.
> After reading some accounts in a completely different list I can see a
> lot of the value of just being able to click a few buttons and have
> gentoo running, and then having the luxury of tailoring it later.
> This was the driver to drop the stage1 installs in favor of stage3 in
> the first place.
We can all interpret the past via a variety of lenses. As always your
perspectives have a causal effect on my mind, so go easy with me? I do not
have all of this worked out, but, I am very passionate about this paradigm
shift.....
Bootstrapping, from a micro processor point of view, has a myriad of
semantics, all valid and millions of embedded products use bootstrapping
semantics mostly uniquely created by the coders of those individual
products. Semiconductor companies usually provide the stub code, registers,
details of ram, rom, eeprom mmc, flash, etc etc and the coders write unique
code to package the boot-loader so as to be 'license free'. WE do not have
to go to that level, but surely encouraging and creating a plethora of
pathways to install gentoo is a good thing.
Then folk can think of a variety of ways (catalyst, profile, world_file,
ansible etc etc) of how to put collections of packages and configa onto the
recent installs in an unattended fashion. This will prepare us (gentoo) to
champion the future of VM, Containers and clusters is a very logical and
extensible way. NOBODY is bridging the divide between physical (actual HD)
semantics and those ethereal {vm, container, remote hosted etc) so that it
is one large, but logical endeavour. imho. This is where I believe gentoo
can dominate. Compiling from 100% sources, the gentoo way, is a killer
advantage and gentoo is very well positioned on that. You should read up on
what D.Berk.... wrote some years ago about clusters and look at the who's
who list of research and commercial folks that used gentoo for clusters;
if you have not already done so. I have notice some of those docs
disappearing, but they are all at legacy archives......
Tuning clusters/Clouds is all about managing sources, keeping the
source-trees (gitignore) pristine and keeping the OS pristine. Likewise the
same thing need to happen to the underlying kernel. Like it or not the
Kernel_bloat is at an all time high and that is a separate but parallel
need. Gentoo supporting both OpenRC and systemd is allowing this distro to
morph into something unmatched in the linux world, imho. It's a very good
thing for Gentoo and I believe this will only benefit Gentoo, linux and the
open source communities. Like it or not, Gentoo is a power player. Folks
just try to keep it a secret, commercially. It's gonna explode everywhere,
once it is easy to install. Systemd has dis-lodged many linux users
and that is a wonderfui but time limited opportunity for Gentoo (a window in
time, if you like).
Combined efficiently (virtual and real), will allow the distro to prosper
beyond it's competition. Those non-rolling distros are at a huge
disadvantage on performance, security and maintainability imho. Look at
Suse's recent moves. All we lack is raw speed/simplicity in the installation
semantic(s). imho.
> Still, if I were actually deploying on a cluster I don't think any of
> this is the way I'd probably do it. On a cluster I'd be more
> concerned with integration with a configuration management system.
> I'd be thinking more of things like openstack and coreos for the
> initial install, and then Gentoo is just something that goes on top
> (or in the case of coreos, underneath). It is a bit like sticking
> your filesystem on top of lvm - it just makes things easier down the
> road with almost zero cost.
Those distros that currently offer quickie installs of clusters,
are mostly pathetic at what's needed to run different, tuned or stripped
kernsl underneath. Kernel tuning supremacy for those 'cluster distros'
is mostly not talked about much because that distracts from their 'distro'
goals. Gentoo is a wonderful platform to tune/test clusters optimized
kernels (heaptrack, kernelshark etc etc). We have all of that, and once
those become common topics on gentoo-users (gentoo-expert ?) it's going to
become yet another 'gentoo differentiator' technology.
You are right and you are wrong:: Openstack nor CoreOS are the best approach
for (BS) Big Science, imho. BS needs all resources solving and supporting a
single problem, with as low of latency as possible. Real-time (less than 200
ms) is the goal of all BS. It allows study and reformulation of problem
solution attempts much more efficiently. It (BS) allows you to study the
causal relationships of advance mathematics like never before. Go back a
week or so on gentoo user when those rooooskies came to gentoo user to
implement their new, shiny math toy. Bircough () recognized the needs
(wants and desires) he politely scolded them but put their codes into gentoo
ebuild form. I (unbenounced to the gentoo communities) bounce those codes
and Bircophs rapid response work into the faces of some folks I hassle all
the time. (no gentoo is not alone in having large point sources of angsts
toward me) and they got real quite for a few weeks. I have been pounding on
them to open up what they do for other 'mathematical pathways' to real time
solutions. I have few friend there, but they do listen and change course.
They are now opening up their work to different 'pathways' of mathematics to
real time problems. It's a small world in which we live. Gentoo is
fundamentally important in many realms, too, just do you know.
Compiling native sources on bare metal, with gcc5, RDMA and such is
leading the charge. Virtual, containers and partitioning only add latency
to the result of BS pathways. Why is BS so important to others? Simple.
First there are many, many unique needs, many are "well funded"; so if
gentoo_ness is uptaken so will gentoo centric jobs for coders and admins.
Second if you take the average VM, Container or virtual computer room and
add BS capabilities to it by refocusing say 1/3 to 1/2 of the existing
systems, you will get a much more efficient Cloud/Cluster apparatus from a
given amount of hardware. These BS needs are driving the implementation of
better batch/multi-users/faster solutions for all
computation/sorting/searching needs of the digital world.
BS needs dictate a better usage of raw resources to solve those sets
of problems faster. BS system can be used to tremendously lower
the cost of batch processing, as speed kills. Cray and others have known
this for a long time. But nobody with that expertise wanted to make that
solution cost-effective. Now with clusters, BS innovations, gcc5 and rdma on
bare metal, there is another paradigm shift occurring . BS solutions will
extent to the rank and file capabilities of the average computer rooms,
data centers, cloud and cluster thus generating lots of attention for
Gentoo. This installation situation of Gentoo is the largest impediment to
this reality, and it's just wrong and severely dated.
There is a 'vergence in the force' so get excited and get onboard!
Evolution has turned revolutionary:: keep your processors dry
and aim high......
hth,
James
[1] http://clonezilla.org/
[2] https://code.google.com/p/pentoo/wiki/PentooInstaller
