Hi, I tend to agree about running the same set of test cases to all scenarios. But if the test frameworks are not prepared for a certain technology (like in this case K8), it doesn’t make sense to execute them. It’s a waste of time and we know beforehand that it will fail systematically.
We probably need to treat this case as an isolated scenario and define a different release criteria for it. For Euphrates, we can think of adding support in Functest/Yardstick for containers (VIM abstraction layer or similar) but today we don’t have it. Regards, Jose From: David McBride [mailto:dmcbr...@linuxfoundation.org] Sent: Tuesday, March 21, 2017 19:59 PM To: Narinder Gupta; Fatih Degirmenci Cc: TECH-DISCUSS OPNFV; Heather Kirksey; Tapio Tallgren; Morgan Richomme (morgan.richo...@orange.com); Gaoliang (kubi); Jose Lausuch; Raymond Paik Subject: [release][danube] K8 scenario testing on Joid This is a follow-up to a conversation on IRC (opnfv-release) earlier today. When asked about disabling Functest and Yardstick for the new K8 scenarios on Joid, I agreed. My reasoning was that I didn't want to delay release of new and potentially interesting scenarios. I also know that functest and yardstick testing is not always applicable to every feature or configuration. In addition, I was aware that the K8 scenarios had project-specific testing. My initial thought was that the disabled tests would be documented in the release notes and that the project would be required to contribute tests to functest and yardstick for the following release. Subsequently, after some discussion, I realized that I had made a mistake and that this is something that the test working group should weigh in on, at a minimum. Therefore, please disregard my earlier direction about disabling the tests. The tests are still required for all scenarios until the test working group and/or the TSC indicate otherwise. Let me know if you have any questions. Sorry for the confusion. David -- David McBride Release Manager, OPNFV Mobile: +1.805.276.8018<tel:%2B1.805.276.8018> Email/Google Talk: dmcbr...@linuxfoundation.org<mailto:dmcbr...@linuxfoundation.org> Skype: davidjmcbride1 IRC: dmcbride
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
