test case management may have 3 levels like this: TestSuite — TestArea — TestCase
So as the report, it is better to show the test suite in the summary part and show test area, test case in details Otherwise for one test suite, it will produce multiple reports showing each test area, i think this is not the common way to organize the report BR Leo Wang On Nov 17, 2016, at 06:33, Wenjing Chu <[email protected]<mailto:[email protected]>> wrote: Suggest “TestSuite” -> “TestArea”… for consistency and to reserve Suite for broader concept. Wenjing From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Leo Wang Sent: Tuesday, November 15, 2016 6:27 PM To: Christopher Price <[email protected]<mailto:[email protected]>>; Zhang Yujun <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: [opnfv-tech-discuss] [dovetail] candidate of report content Hi Yujun, Chris I suggest that add a segment for each test case in their definition scripts to explicitly tell this info then the tool can generate the report in a general way BR Leo Wang On Nov 15, 2016, at 17:42, Christopher Price <[email protected]<mailto:[email protected]>> wrote: Agree Yujun, I think we should address that in the naming of the dovetail.test.py files we use. As little translation as possible is best. That I think we need to address as a general activity and have conventions in place for how we handle it. / Chris NB: In addition, I suggest removing any “bulk” operations for now. Let’s first get all our test cases functioning with atomic actions and if there is a need for a “bulk” operation as it is important for our platform then we can add them back. From: <[email protected]<mailto:[email protected]>> on behalf of Yujun Zhang <[email protected]<mailto:[email protected]>> Date: Tuesday, 15 November 2016 at 10:29 To: Leo Wang <[email protected]<mailto:[email protected]>>, TECH-DISCUSS OPNFV <[email protected]<mailto:[email protected]>> Subject: Re: [opnfv-tech-discuss] [dovetail] candidate of report content Is it a log message or final report for end user? If it is targeting end user, a user friendly name for test cases might be prefered, e.g. aggregating the result, short name. - [2/3] objective - [pass] bulk create delete network - [fail] bulk create delete port - [pass] bulk create delete subnet My two cents -- Yujun On Tue, Nov 15, 2016 at 4:34 PM Leo Wang <[email protected]<mailto:[email protected]>> wrote: Hi all With the project moving on, the report of dovetail also needs to evolve with more information to present the status of the test. Here is a candidate report style: ---------------------------------------------------------------------------- Compliance Report Version: 1.0 TestSuite: basic Certify ID: 9b2a4144-21f8-7c2a-987b-8f2c125c2df2 Create Date: 2016-11-01 20:17:21 UTC Upload Date: 2016-11-01 20:27:21 UTC Duration: 514 seconds Status: Pass Rate: 0% (0/1) #passed tests/ total tests Compliance : NO #final result of this compliance Overview: dovetail.ipv6.tc001(2/3) # passed sub tests/ total sub tests -objective: VIM ipv6 operations, to create/delete network, port and subnet - - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network pass - - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port fail - - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet pass Welcome to share your ideas within the mailing list or the wiki page https://wiki.opnfv.org/pages/editpage.action?pageId=8685074 BR Leo Wang _______________________________________________ opnfv-tech-discuss mailing list [email protected]<mailto:[email protected]> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss _______________________________________________ opnfv-tech-discuss mailing list [email protected]<mailto:[email protected]> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
_______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
