Hi all, I have been wrestling with uncovered files for a while. The current state doesn't satisfy me, so I thought to share my findings and we can hopefully come up with something better. If you want to look at some numbers you can compare the 2 nightly results here: http://my.cdash.org/index.php?project=Qsmtp&date=2012-02-05 [1]
voyager runs with a more or less plain 2.8.7 (default gentoo build), while eto runs current next with the patches described below. I'll describe what happened and what I did now. I got aware of the nice but undocumented CTEST_EXTRA_COVERAGE_GLOB that allows uncovered files to show up on the dashboard. I enabled this and basically got the results you see for voyager. A small hints for those that want to try: if you place your build directory inside the source directory (i.e. the usual "mkdir build && cd build && cmake ..") than things like the compiler test files may show up in the coverage. I added "/CMakeFiles/" to CTEST_CUSTOM_COVERAGE_EXCLUDE to get rid of them. Ok, now I have these uncovered files in the dashboard, but compared to what gcov would report the numbers of uncovered lines are insanely big. This comes because simply _every_ line is counted as uncovered, may it be code, comment, whitespace, #include or whatever. I then added 3ea6be30819e78ddf3d97378a519dc50836049a3 to it, which simply ignored empty lines. This was the best compromise between effort and effect for me: it was only a very few lines of code, it would reduce the numbers by ~10% for my coding style and I had to know nothing about the language used. If you want to test it out you can get that commit from the next branch. David didn't like it, so I had to revert. Instead he told me to just add proper coverage. Well, let's see ;) I then wrote some test cases calling a binary consisting of multiple files. Only a few of them were actually used, the others were uncovered. To my surprise those other files which had been reported to have no coverage and a high number of uncovered lines now were reported as uncovered with 0 lines in CDash. First I thought of a bug in gcov, but by examination of the files that gcov and ctest generated I found out that everything was fine there. The problem here is that ctest does this code when generating Coverage.xml: covSumFile << "\t<File Name=\"" << cmXMLSafe(fileName) << "\" FullPath=\"" << cmXMLSafe( this->CTest->GetShortPathToFile(fullFileName.c_str())) << "\" Covered=\"" << (tested > 0 ? "true":"false") << "\">\n" << "\t\t<LOCTested>" << tested << "</LOCTested>\n" << "\t\t<LOCUnTested>" << untested << "</LOCUnTested>\n" << "\t\t<PercentCoverage>"; So if no lines are covered then it reports 'Covered="false"' which leads CDash to entirely ignoring the line count and not showing a link to the file. In contrast it does that when collecting files using CTEST_EXTRA_COVERAGE_GLOB: covSumFile << "\t<File Name=\"" << cmXMLSafe(fileName) << "\" FullPath=\"" << cmXMLSafe(i->c_str()) << "\" Covered=\"true\">\n" << "\t\t<LOCTested>0</LOCTested>\n" << "\t\t<LOCUnTested>" << untested << "</LOCUnTested>\n" Covered is always set to true here. So the quick and dirty fix I used was to change the one line to << "\" Covered=\"" << (tested+untested > 0 ? "true":"false") << "\">\n" which at the end gave me the results shown by eto in the CDash link above. Ok, so much to the history behind everything, now the questions: was that the right approach? What does 'Covered="false"' actually means? Should we unconditionally send true for gcov files, too? This would at least give consistent results. Is it at the end a CDash bug? Or should CTest be fixed similar to the way I did it for my local build? Opinions? Ideas? Patches? Money? Greetings, Eike 1) You can also see the shiny new CDash version the admins brought to us at the weekend. Nice work, guys!
signature.asc
Description: This is a digitally signed message part.
-- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers