Hi, Further to previous discussions titled "your opinion about testing" I'd like to propose a meta data format for the test script files and would obviously welcome peoples input;
Effectively each test in the scripts is represented by a function and a call to run_test, so we have test_function() { ...code } run_test function "Description of the function" I'd like to propose that above every function a here document is placed that contains yaml v1.2 encoded data (yaml.org) with 2 characters for the indent. The block will start with << TEST_METADATA and be terminated with TEST_METADATA. We might want to place it in a comment block but this is not really required. The block will also be wrapped at 80 characters for readability. The compulsory elements to the data will be Name: Name of the function, this ensures pairing between function and comments is not just file relative. Summary: Will often be the description after the run_test but not always as the tense will change Description: A full description of the function, the more information here the better. Components: This is the component described in the commit message (http://wiki.whamcloud.com/display/PUB/Commit+Comments) to make this useful we will need to come up a with a defined set of components that will need to be enforced in the commit message. The format of this entry will be a yaml array. Prerequisites: Pre-requisite tests that must be run before this test can be run. This is again an array which presumes a test may have multiple pre-requisites, but the data should not contain a chain of prerequisites, i.e. if A requires B and B requires C, the pre-requisites of A is B not B & C. TicketIDs: This is an array of ticket numbers that this test explicitly tests. In theory we should aim for the state where every ticket has a test associated with it, and in future we should be able to carry out a gap analysis. As time goes on we may well expand this compulsory list, but this is I believe a sensible starting place. Being part of the source this data will be subject to the same review process as any other change and so we cannot store dynamic data here, such as pass rates etc. Do people think that additional data fields should be permitted on an adhoc basis or should a control list of permitted data elements be kept. I'm tempted to say that adhoc additional fields should be allowed, although this could lead to name clashes if people are not careful. Below is an simple example. ======================================================================= <<TEST_METADATA Name: before_upgrade_create_data Summary: Copies lustre source into a node specific directory and then creates a tarball using that directory Description: This should be called prior to upgrading Lustre and creates a set of data on the Lustre partition which be accessed and checked after the upgrade has taken place. Several methods are using including tar'ing directories so the can later be untar'ed and compared, along with create sha1's of stored data. Component: - lnet - recovery Prerequisites: - before_upgrade_clear_filesystem TicketIDs: - LU-123 - LU-432 TEST_METADATA test_before_upgrade_create_data() { ... } run_test before_upgrade_create_data "Copying lustre source into a directory $IOP_DIR1, creating and then using source to create a tarball" ======================================================================= As I said comments, inputs and thoughts much appreciated Thanks Chris _______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss