On 2016-09-08 09:38, John Schwarz wrote: > Hi all, > > The Neutron community has recently started contributing Tempest > scenario tests in the Neutron tree and I'd like to discuss a general > issue we're hitting. Mainly, we're talking about required hardware for > some features and/or VM images that support more features, which makes > it hard (or impossible) to test certain features at the gate > end-to-end. Specifically, tests that require SR-IOV or OVS-DPDK, and > tests relating to VLAN Aware VMs. > > For this reason, we would like to bring up the idea of introducing in > a new concept to the "testing arena" - skip-only tests. These will be > tests that will be merged upstream and skipped unless some > pre-conditions are met (example: "@skip_unless(has_sriov=True)"). The > key idea here though, is that we don't expect upstream gates to add > support for these pre-conditions any time soon. Instead, the tests > will be merged upstream and used on the various downstream gates most > of us have (with specific hardware/deployment that match our > respective interests). > > Another way to look at it is: "I want to contribute tests that require > specific hardware/software/deployments, and I want it to be robust and > shared amongst the community as per the Open Source way, even though > it won't be actually run on the upstream gates". Obviously, we would > continue to investigate ways to "unskip" these kind of tests where > possible. > [...]
This reminds me of this review for a nova-tempest-staging repository: https://review.openstack.org/366091 Joe seems to face similar challenges that you articulate, John, Andreas -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126 __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev