Is LAVA the right tool for this? Clearly we need something to run tests, but we can use other tools if they are a better fit.
I would rather this was an automated tool that didn't run on developer machines. We should expect most bugs to be visible on developer machines and use an automated tool (on check-in / nightly) to run extra validation. I guess the next question is about hosting of VMs. If we have a physical box, it would make sense to have it in the validation lab. We could rent time on EC2 / Rackspace Cloud / some other cloud compute platform. EC2 is quite good for validation since, as I understand it, you get a fresh instance every time and have to install any extra software you need. There are also Amazon Machine Images going back to the last LTS available for us to use (see https://help.ubuntu.com/community/EC2StartersGuide). That said, creating a few VMs on a box that we own shouldn't be a big deal. I don't know how LAVA will cope with using EC2 machines as the clients. If it wants a permanent connection, that won't work well (since we will want to drop the instance after the end of each run to reset it and save money). -- You received this bug notification because you are a member of linaro- infrastructure-drivers, which is the registrant for Linaro Image Tools. https://bugs.launchpad.net/bugs/888555 Title: Set up l-i-t testing on multiple archs and releases Status in Linaro Image Tools: New Bug description: We need to test l-i-t on all supported releases and architectures. We have seen a number of cases where new features are later found to not work on old releases or older versions of Python. We just had a problem with linaro-hwpack create not working on Oneiric 64 bit, but happily work on 32 bit. For testing new code and especially for release testing we would need to set something up to enable this since a single developer won't have access to all variants of installations. We could perhaps automate something to set up virtual machines on local dev PC:s with possible impact on TDD since tests would take many times longer to run. Perhaps we can use LAVA? For full coverage we'd need to test on all possible combinations of the below components (please add to them as you see fit), but that is probably overkill. Python: 2.6 2.7 Ubuntu: Precise Oneiric Natty Maverick (how far back do we want to ensure support?) Archs: i386 x86_64 To manage notifications about this bug go to: https://bugs.launchpad.net/linaro-image-tools/+bug/888555/+subscriptions _______________________________________________ Mailing list: https://launchpad.net/~linaro-release Post to : [email protected] Unsubscribe : https://launchpad.net/~linaro-release More help : https://help.launchpad.net/ListHelp
