I seem to be unable to convey my point using generalization, so I will give a specific example: I would like to have "update dns server" as an additional network scenario. Currently I could add it to the existing module:
1. tests connectivity 2. re-associate floating ip 3. update dns server In which case, failure to re-associate ip will prevent my test from running, even though these are completely unrelated scenarios, and (IMO) we would like to get feedback on both of them. Another way, is to copy the entire network_basic_ops module, remove "re-associate floating ip" and add "update dns server". For the obvious reasons - this also seems like the wrong way to go. I am looking for an elegant way to share the code of these scenarios. Yair ----- Original Message ----- From: "Salvatore Orlando" <sorla...@nicira.com> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org> Sent: Monday, January 20, 2014 7:22:22 PM Subject: Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases Yair is probably referring to statistically independent tests, or whatever case for which the following is true (P(x) is the probably that a test succeeds): P(4|3|2|1) = P(4|1) * P(3|1) * P(2|1) This might apply to the tests we are adding to network_basic_ops scenario; however it is worth noting that: - in some cases the above relationship does not hold. For instance a public network connectivity test can hardly succeeds if the private connectivity test failed (is that correct? I'm not sure anymore of anything this days!) - Sean correctly pointed out that splitting test will cause repeated activities which will just make the test run longer without any additional benefit. On the other hand, I understand and share the feeling that we are adding too much to the same workflow. Would it make sense to identify a few conceptually independent workflows, identify one or more advanced network scenarios, and keep only internal + public connectivity checks in basic_ops? Salvatore On 20 January 2014 09:23, Jay Pipes < jaypi...@gmail.com > wrote: On Sun, 2014-01-19 at 07:17 -0500, Yair Fried wrote: > OK, > but considering my pending patch (#3 and #4) > what about: > > #1 -> #2 > #1 -> #3 > #1 -> #4 > > instead of > > #1 -> #2 -> #3 -> #4 > > a failure in #2 will prevent #3 and #4 from running even though they are > completely unrelated Seems to me, that the above is a logical fault. If a failure in #2 prevents #3 or #4 from running, then by nature they are related to #2. -jay _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
