Hi Nux, Right now, yes. Rohit has done a lot of work to nest hypervisors under KVM, but has found shortfalls in it capabilities which has meant we can't do 'everything' ( I don't know the details).
Paul Angus VP Technology , ShapeBlue t: @cloudyangus<tel:@cloudyangus> e: paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com> | w: www.shapeblue.com<http://www.shapeblue.com> -----Original Message----- From: Nux! [mailto:n...@li.nux.ro] Sent: 02 February 2016 11:14 To: d...@cloudstack.apache.org Cc: users@cloudstack.apache.org Subject: Re: [Discuss] Flexible CI/CD - Trillian Hi Paul, This is great news, good job, naming apart. :-) Is ESXi a hard requirement on running Trillian? -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro ----- Original Message ----- > From: "Paul Angus" <paul.an...@shapeblue.com> > To: d...@cloudstack.apache.org > Cc: users@cloudstack.apache.org > Sent: Tuesday, 2 February, 2016 10:49:46 > Subject: [Discuss] Flexible CI/CD - Trillian > Project: Trillian > We have been working on PoC of a CI environment design which will: > > · Provide fast build or rebuild of environments for testing. > > · Enable multiple independent concurrent builds > > · Be available on-demand through automation or individual request. > > · Be capable of fully utilising all available hardware > > · Flexible enough to be used to build super-realistic development > environments. > We intend to contribute and maintain our work within the Apache repos. > However, we are currently building the POC, figuring out the > requirements (and quirks) of the individual pieces, before pushing > something concrete for to the community to review. > > We envision that Trillian would cater for a number of use cases: > > 1. CloudStack community integration testing of master against multiple > deployment scenarios (using ASF infra) > > 2. CloudStack community integration testing of PRs against multiple > deployment scenarios (using ASF infra) > > 3. Organisations/individuals running the full suites of tests available in > Marvin against any physical environment they have. > > 4. Organisations/individuals deploying and running the full suites of > tests available in Marvin against virtualised infrastructures which > can be deployed by Marvin. > As we intend Trillian to test multiple environments concurrently, we > use nested virtualization on ESXi hosts (our testing has shown that > this is the only hypervisor which can support the nested > virtualisation of all other hypervisors with reasonable performance). > We use Ansible to deploy and configure all aspects of the build as > this will greatly lower the barrier to entry for independent testers. > We use CloudStack to provision the management server and virtualised > (nested) hosts on the physical hosts. We are creating Ansible > playbooks and roles which > can: > > 1. Create guest instances using Rene’s Ansible 2.0 CloudStack modules - a > Marvin VM, a Mgmt Server (CentOS or Ubuntu), any number of compute > hosts (KVM, vSphere or XenServer. Hyper-V later) > > 2. Configure hosts (inc. installing the relevant CloudStack agent where > required) > > 3. Install required ACS packages on management server > > 4. Configure a zone (including adding the compute hosts) via Marvin. > > 5. Run the required Marvin tests. > > 6. Return the results > We may need to propose enhancements to Marvin in order to sync the > configuration of hosts with the configuration used by Marvin. > > Using virtualised test environments, we can have multiple test > scenarios running concurrently. To do this we have found that it is > necessary to create pools or ranges of VLANs and IP addresses and > allocate them to environments. So for any given physical environment > which will be used for testing in, we take the total > range(s) of IPs and VLANs available and carve them into > non-overlapping chunks suitable for concurrent use as mgmt, public and > guest networks. These are stored in a MariaDB database. When a range > is being used in a testing environment, that range is marked as > ‘inuse’ in the database. When creating a test environment, Trillian > looks in the database for the next available VLAN range, the next > available public IP range and so on. The returned values are used to > populate a Marvin cfg file which in turn will be used to both build > the environment and when running the Marvin testing. When the > virtualised infra is cleaned up, the database will be updated to reflect that > the used ranges are available again. > This initiative has only recently been started, and as stated earlier > we are currently figuring out the requirements (and quirks) of the > individual pieces and looking for the most suitable wrapper to glue it all > together. > Also I have found that Marvin requires a little work to make the > output more meaningful/readable (especially in the case of errors and > exceptions) and to make it a little more intelligent about the tests > it can/can’t run based on the chosen infrastructure components. I have > also found unreachable or very slow ISO and template paths hardcoded into > Marvin or individual tests. > We plan to enhance tests to address these issues and also reduce > runtimes where possible. > > NEXT STEPS > > We will continue to our work on the PoC and share the results. > > We’d like to get some community agreement around the goals and use > cases which we are planning to satisfy before we get too far down the > road of our PoC. So please share your thoughts. > > > > [ShapeBlue]<http://www.shapeblue.com> > Paul Angus > VP Technology , ShapeBlue > > > d: +44 203 617 0528 | s: +44 203 603 > 0540<tel:+44%20203%20617%200528%20|%20s:%20+44%20203%20603%200540> | > m: +44 7711 418784<tel:+44%207711%20418784> > > e: paul.an...@shapeblue.com | t: > @cloudyangus<mailto:paul.an...@shapeblue.com%20|%20t:%20@cloudyangus> | > w: www.shapeblue.com<http://www.shapeblue.com> > > a: 53 Chandos Place, Covent Garden London WC2N 4HS UK > > > [cid:imagea932fa.png@ce3f6520.47a350c4] > > > Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue > Services India LLP is a company incorporated in India and is operated > under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda > is a company incorporated in Brasil and is operated under license from > Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The > Republic of South Africa and is traded under license from Shape Blue Ltd. > ShapeBlue is a registered trademark. > This email and any attachments to it may be confidential and are > intended solely for the use of the individual to whom it is addressed. > Any views or opinions expressed are solely those of the author and do > not necessarily represent those of Shape Blue Ltd or related > companies. If you are not the intended recipient of this email, you > must neither take any action based upon its contents, nor copy or show > it to anyone. Please contact the sender if you believe you have received this > email in error. > > > > > Find out more about ShapeBlue and our range of CloudStack related services: > IaaS Cloud Design & > Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – > rapid IaaS deployment framework<http://shapeblue.com/csforge/> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | > CloudStack Software > Engineering<http://shapeblue.com/cloudstack-software-engineering/> > CloudStack Infrastructure > Support<http://shapeblue.com/cloudstack-infrastructure-support/> | > CloudStack Bootcamp Training > Courses<http://shapeblue.com/cloudstack-training/> Find out more about ShapeBlue and our range of CloudStack related services: IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
