Re: [CMake] CTest: Any way around the "one test == one program invocation" assumption?
All in all it'd be much nicer if there was some kind of hook in ctest which could be used to inform it that a number of tests had been run, and let it populate and submit the XML report based on that information. But, if there's any way to do that, it's highly undocumented... It doesn't appear like there is a class for storing the test results before serializing the XML document to a file. Sounds like that's what would have to be written. cmCTestTestHandler::GenerateDartOutput Looks like it may be a place to start? Alternatively, how about making the custom test return a simple XML which had name pass/fail < full test text output> /> name pass/fail /> ... /etc/etc and then add a function into cmCtestHandler to parse the result - in the fixed form you define - back into a standard Dart compatible form which it then submits as usual. A slight modification of ADD_TEST to trigger the new form of output would probably be enough. JB ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTest: Any way around the "one test == one program invocation" assumption?
On Tue, Jun 8, 2010 at 4:32 AM, Chris Hillery wrote: > On Tue, Jun 8, 2010 at 1:00 AM, Biddiscombe, John A. wrote: > >> I'm sure someone has a better answer ... but ... >> >> > I have complete control over the test >> > program and can make it output whatever I want. >> >> Have a look at the XML produced by ctest itself and generate it yourself? >> then submit it and you should see each test separately >> >> (Just thinking out loud) >> > > Yeah, worst case I'll head down that path. It's a hairy path though. For > instance, I'd still definitely want to run "normal" ctest tests as well, > which means the XML would need to be merged in some fashion. Also, it seems > like doing it that way opens you up to possible problems later if ctest or > cdash change the XML format at all. > > All in all it'd be much nicer if there was some kind of hook in ctest which > could be used to inform it that a number of tests had been run, and let it > populate and submit the XML report based on that information. But, if > there's any way to do that, it's highly undocumented... > It doesn't appear like there is a class for storing the test results before serializing the XML document to a file. Sounds like that's what would have to be written. cmCTestTestHandler::GenerateDartOutput Looks like it may be a place to start? -- Philip Lowman ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTest: Any way around the "one test == one program invocation" assumption?
On Tue, Jun 8, 2010 at 1:00 AM, Biddiscombe, John A. wrote: > I'm sure someone has a better answer ... but ... > > > I have complete control over the test > > program and can make it output whatever I want. > > Have a look at the XML produced by ctest itself and generate it yourself? > then submit it and you should see each test separately > > (Just thinking out loud) > Yeah, worst case I'll head down that path. It's a hairy path though. For instance, I'd still definitely want to run "normal" ctest tests as well, which means the XML would need to be merged in some fashion. Also, it seems like doing it that way opens you up to possible problems later if ctest or cdash change the XML format at all. All in all it'd be much nicer if there was some kind of hook in ctest which could be used to inform it that a number of tests had been run, and let it populate and submit the XML report based on that information. But, if there's any way to do that, it's highly undocumented... Ceej aka Chris Hillery ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTest: Any way around the "one test == one program invocation" assumption?
I'm sure someone has a better answer ... but ... > I have complete control over the test > program and can make it output whatever I want. Have a look at the XML produced by ctest itself and generate it yourself? then submit it and you should see each test separately (Just thinking out loud) JB -- John Biddiscombe, email:biddisco @ cscs.ch http://www.cscs.ch/ CSCS, Swiss National Supercomputing Centre | Tel: +41 (91) 610.82.07 Via Cantonale, 6928 Manno, Switzerland | Fax: +41 (91) 610.82.82 ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
