Github user nitt10prashant commented on the pull request: https://github.com/apache/cloudstack/pull/960#issuecomment-150501158 @runseb "I personally fail to see the benefit of skipped tests versus a failed test" If i may , skip is a type of failure where you know in advance about failure , since you know about the failure you just Handle them in order to differentiate from a general failure(may be product issue).So its all about categorizing the failure like expected failure and unexpected failure Its about how you see reports If you accept skip as known failure then 100% success on BVT ~ 20% of skipped =80% success with 20% of known failure ,at least not a product issue which you can say without looking at particular script . Without skip you have to analyze each and every failure then you may come up with % of product issue and % of failure due to setup an all "While @nitt10prashant and @pvr9711 you seem to want 100% success on BVT even if it includes some tests that are actually not run." My comment: 1-Not run test cases are the one which are not picked up by runner may be coz of different tags and those test cases will not appear in your report 2-skipped test case are the one which are picked up by runner but due to non availability of configurations or some other reasons could not run completely and they will appear in your report . 3-I want 100% success on BVT without any skip if all the setup config is available .incase they are not i would prefer to skip them in order to find out real failure which does not include any failure because of setup/configuration just by looking at my report .
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---