Thanks for bringing this up for discussion, Keith! One point I'd like us to be clear about for the purposes of this discussion: when we say "move from Centos 7 to another OS", I want us to be clear that Muchos is only "on" CentOS 7 as a default configured image. Muchos *should* be able to run on any similar OS, by merely changing the image it is configured to use. For example, I expect it to work today without any changes on a RHEL7 image, or a RHEL8 or even CentOS Stream 8 image, and would be surprised if it didn't. So, when we talk about "moving", I am interpreting that to mean "what do we want the default image to be?" If we mean something other than selecting the default, then I'm not sure what we're talking about and would appreciate clarification.
Since the intent of Muchos, and the reason it continues to fall under the purview of the Fluo PMC, was for the Fluo developers to have a way to quickly deploy a test cluster of Accumulo and Fluo. While it may be used by some outside that scope, we should not lose sight of that intent. Given that purpose, it becomes clear that Muchos needs to be able to deploy on an operating system that the Accumulo and Fluo developers are actually using to test on to anticipate problems in their own production clusters. So, "[supporting] ... one final OS target platform" doesn't make sense to me if we have developers needing to use it to test on several different OSes because they use different OSes in production. >From Arvind's experience testing Ubuntu, and my own expectations using other Fedora and other Fedora-derived RPM-based distros like RHEL and CentOS, I think it's likely that the biggest incompatibilities preventing use of an arbitrary AMI is going to be the package names, and a few filesystem layout conventions. So, I'm not too worried about choosing one default and users still being able to configure a different one that they need to test on. As long as the differences between major OS families we use are minor, we can all still use the same Muchos to test with, by baking in some conditional logic, or making some aspects more configurable. I largely concur with the list of aspects to consider that Arvind has provided, but would add: - we should provide a reasonable default, but avoid doing things in Muchos that would tightly couple us to a particular image, so that users can easily configure a different image and have reasonable expectations that Muchos will work with little or very minor tweaks - we should focus on OS/OS families actually used by devs in their Accumulo/Fluo testing environments - it should be a GNU/Linux OS, since Accumulo isn't really designed or tested with anything else (sorry BSD users) My preferred default would be Fedora, because it is upstream of RHEL/CentOS/Rocky/Alma/etc., unless one of the RHEL downstream CentOS replacements becomes a de facto standard replacement for CentOS, because I'm likely going to need some kind of EL or upstream-to-EL testing for the enterprise users I support at $dayjob. However, given the ubiquity of Ubuntu among Debian varieties, I think it's reasonable to expect things to work more-or-less out of the box if the user configures a modern Ubuntu image. Related topic: would it make more sense to migrate Muchos away from Ansible, and use something that is maybe a little less hard-coded scripts, and a little more flexible user config? I'm thinking something like https://github.com/hashicorp/terraform#readme ; they have examples with cloud-init https://learn.hashicorp.com/tutorials/terraform/cloud-init ; Terraform is extremely well documented, and it seems to me that a few easily user-editable config templates could replace Muchos and support both EC2 and Azure easily. It's probably a bigger change than when Muchos moved to ansible from the previous scripts, but long-term, it seems better than trying to maintain our own provisioning code that's as flexible. However, I don't have much experience with Terraform. On Wed, Jan 5, 2022 at 6:45 PM Arvind Shyamsundar <arvin...@microsoft.com.invalid> wrote: > > Hi Keith, > Process-wise, I believe it would be simpler and less taxing on effort / > testing / supportability to standardize on one final OS target platform. > Supporting a choice of OS platforms is hard and more effort than rewards. > While selecting the OS, we could consider the following (this is just my > top-of-mind, there may be more aspects to consider): > > - End-of-life / LTS status for that OS. > - Licensing and commercial implications if any. > - Ready and wide availability of image(s) for the cloud providers (Azure, > EC2). > - We should not require the user of Muchos to have to build their own image. > - Those cloud provider image(s) should support cloud-init. > - Those cloud provider image(s) should be updated by the cloud provider (or > partners thereof) on a reasonably regular (quarterly or more frequent?) basis > to try to include as many security patches out-of-the-box. > - The OS should be reasonably "stable" in that it should not increase burden > on the user of Muchos by introducing any regressions (functional / perf) of > its own origin. > - There should be ready availability of OpenJDK binary distribution for that > OS. > ... etc. > > BTW, I did some work in December to evaluate Ubuntu 20.04 as a candidate OS > for the Muchos cluster nodes on Azure. In the proof-of-concept that I did, > the changes to the Muchos code base seemed to be contained to yum packages, > OpenJDK package names, and similar relatively minor changes. The downstream > playbooks for Zookeeper, Hadoop, Accumulo "just worked". > > I hope this helps. > > Arvind. > > -----Original Message----- > From: Keith Turner <ke...@deenlo.com> > Sent: Wednesday, January 5, 2022 2:00 PM > To: fluo-dev <dev@fluo.apache.org> > Subject: [EXTERNAL] [DISCUSS] Move Muchos from Centos 7 to another OS > > We need to move Muchos from Centos 7 to another OS. Not completely sure how > we should proceed. Below is a possible process we could follow to make this > change. > > * We collaborate to create a list of candidate OSes with pros and cons for > each OS, not sure where is best to do this. I don't think a mailing list is > the best place for this collaboration, however it's nice for the > collaboration to be recorded on the mailing list. > * We hold a vote (possibly iterative voting or ranked voting) to try to > reach consensus on one of the candidate OSes we would like Muchos to use. > * After a candidate is selected we can create a branch and start submitting > PRs to move Muchos to that new OS in the branch. When it's functional we can > merge that branch into main. > > Can anyone think of another process we could follow to move Muchos to another > OS? Does anyone object to the process proposed above or have suggestions to > improve it? > > Keith
