On Fri, 2019-10-25 at 16:04 -0400, Rich Persaud wrote: > There may be lessons in Microsoft's move from diverse hardware to VM- > based testing: > https://www.ghacks.net/2019/09/23/former-microsoft-employee-explains-why-bugs-in-windows-updates-increased/
I don't disagreed, real hardware does have some distinct advantages. We also have to keep in mind that we're testing what the Yocto Project is offering. At some point YP has to trust that the underlying kernels we ship are "ok" and that upstream has covered some of this. Virtual testing works well for the software stack itself which a large part of what YP offers, particularly where we can run upstream's tests for components. Where its struggles is on the variations of hardware that are out there as the article highlights. As such, I think virtual testing for any YP LTS is the realistic option we have right now. > Business requirements for "LTS releases" include security fixes and > non-regression of production systems, i.e. the software supply chain > security topics discussed in [1] above. There were CKI [3] > discussions about sharing test resources (human, machine) for kernel > testing. Some CKI challenges and potential solutions [4] may be > applicable to Yocto LTS testing of packages beyond the Linux kernel. > > VM-based testing could be supplemented with pooled test results from > distributed testing on diverse hardware, across the Yocto contributor > community. Even with VM-based testing, IOMMU PCI passthrough can be > used to give a VM direct access to a PCI device, for device driver > testing. We have already build a mechanism into our test results process for externally contributed test results and we are happy to collect that data although we haven't seen much being contributed yet. I'd love to see more data coming in for any LTS though (or any other release for that matter). Cheers, Richard _______________________________________________ Openembedded-architecture mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
