On Tue, Feb 2, 2016 at 11:49 AM, Paul Angus <paul.an...@shapeblue.com> wrote:
> Project: Trillian > > We have been working on PoC of a CI environment design which will: > > · Provide fast build or rebuild of environments for testing. > > · Enable multiple independent concurrent builds > > · Be available on-demand through automation or individual > request. > > · Be capable of fully utilising all available hardware > > · Flexible enough to be used to build super-realistic development > environments. > > We intend to contribute and maintain our work within the Apache repos. > However, we are currently building the POC, figuring out the requirements > (and quirks) of the individual pieces, before pushing something concrete > for to the community to review. > > > > We envision that Trillian would cater for a number of use cases: > > 1. CloudStack community integration testing of master against > multiple deployment scenarios (using ASF infra) > > 2. CloudStack community integration testing of PRs against multiple > deployment scenarios (using ASF infra) > > 3. Organisations/individuals running the full suites of tests > available in Marvin against any physical environment they have. > > 4. Organisations/individuals deploying and running the full suites > of tests available in Marvin against virtualised infrastructures which can > be deployed by Marvin. > > As we intend Trillian to test multiple environments concurrently, we use > nested > virtualization on ESXi hosts (our testing has shown that this is the only > hypervisor which can support the nested virtualisation of all other > hypervisors with reasonable performance). We use Ansible to deploy and > configure all aspects of the build as this will greatly lower the barrier > to entry for independent testers. > > We use CloudStack to provision the management server and virtualised > (nested) hosts on the physical hosts. We are creating Ansible playbooks and > roles which can: > > 1. Create guest instances using Rene’s Ansible 2.0 CloudStack > modules - a Marvin VM, a Mgmt Server (CentOS or Ubuntu), any number of > compute hosts (KVM, vSphere or XenServer. Hyper-V later) > > 2. Configure hosts (inc. installing the relevant CloudStack agent > where required) > > 3. Install required ACS packages on management server > > 4. Configure a zone (including adding the compute hosts) via Marvin. > > 5. Run the required Marvin tests. > > 6. Return the results > > We may need to propose enhancements to Marvin in order to sync the > configuration of hosts with the configuration used by Marvin. > > > > Using virtualised test environments, we can have multiple test scenarios > running concurrently. To do this we have found that it is necessary to > create pools or ranges of VLANs and IP addresses and allocate them to > environments. So for any given physical environment which will be used for > testing in, we take the total range(s) of IPs and VLANs available and carve > them into non-overlapping chunks suitable for concurrent use as mgmt, > public and guest networks. These are stored in a MariaDB database. When a > range is being used in a testing environment, that range is marked as > ‘inuse’ in the database. When creating a test environment, Trillian looks > in the database for the next available VLAN range, the next available > public IP range and so on. The returned values are used to populate a > Marvin cfg file which in turn will be used to both build the environment > and when running the Marvin testing. When the virtualised infra is cleaned > up, the database will be updated to reflect that the used ranges are > available again. > > This initiative has only recently been started, and as stated earlier we > are currently figuring out the requirements (and quirks) of the individual > pieces and looking for the most suitable wrapper to glue it all together. > > Also I have found that Marvin requires a little work to make the output > more meaningful/readable (especially in the case of errors and exceptions) > and to make it a little more intelligent about the tests it can/can’t run > based on the chosen infrastructure components. I have also found > unreachable or very slow ISO and template paths hardcoded into Marvin or > individual tests. > > We plan to enhance tests to address these issues and also reduce runtimes > where possible. > > > > NEXT STEPS > > > > We will continue to our work on the PoC and share the results. > > > > We’d like to get some community agreement around the goals and use cases > which we are planning to satisfy before we get too far down the road of our > PoC. So please share your thoughts. > > > Looking forward to hear more, please share it as work continues, there's no reason to keep it secret until "done" :-) This way people might be able to help out, with both developing and testing. -- Erik
