W dniu 2023-03-22 13:46, Daniel Stenberg napisał(a):
On Wed, 22 Mar 2023, Daniel F via curl-library wrote:
One option is to provide coverage data for each platform/test job,
plus one extra created by merging results from all platform. With this
approach you could check if there are any code paths which are not
tested at all by checking merged results, and be able to check
platform-specific coverage if needed.
We have 130+ CI jobs. We could imagine that most of them could get a
test coverage number. How would anyone make sensible use of a hundred
different numbers? Not to mention how it would be difficult to present
the numbers (and the view of what lines that were not executed). Not
easily done I say.
I think about something like lcov output, see examples at link below. It
generates page for every source file, and code lines are marked in blue
or red depending if code in that line was executed or not. This will
give you an 0/1 answer if any test on any platform executed given line
or not. Actual numbers (shown on the left) will not have any useful
meaning for merged results. They may be more useful when checking
results for one particular platform.
https://medium.com/@xianpeng.shen/use-gcov-and-lcov-to-perform-code-coverage-testing-for-c-c-projects-c85708b91c78
Merging results from all these jobs is perhaps even more complicated
since all of them run in their own sandboxes spread over a multitude
of different services.
These jobs could upload files with their coverage data somewhere. At the
end of the day another job would take all files from that day and
generate merged coverage report. lcov allows to merge multiple coverage
files into one.
--
Regards,
Daniel
--
Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
