While I think test coverage is a nice information to have. I would object to using this as a metric to decide whether a PR should be merged. Code-cov act as a mere coverage of APIs, which is a useful aspect, it can be misleading in many cases, especially when such change involves cross-language APIs and automatically generated wrapper. Sometimes the code-cov shows a negative impact on coverage while CI passes, and the giving information was quite misleading if you just look at the PR tabs
I would still trust the reviewer's decision on the pull request merging. Tianqi On Wed, Jun 20, 2018 at 7:14 PM, Marco de Abreu < marco.g.ab...@googlemail.com.invalid> wrote: > Hello, > > I'd like to introduce test coverage metrics of PRs using > https://codecov.io/. > This tool will aggregate coverage reports across multiple runs, platforms > and technologies and gives contributors as well as reviewers a new tool > that allows to improve the quality of a pull request. > > Since we need to gather some data first, I'd like to request merging > https://github.com/apache/incubator-mxnet/pull/11344. This will enable > publishing the coverage data to the service and have no impact on your PRs > - it will just allow me to assess the quality of the service in the > background while I come up with a full integration design. My initial plan > is to start with coverage across Python and C++ and then, with the help of > our community, extend the report across all our supported languages. > > Does anybody object to having us gather this data in the background? > > Best regards, > Marco >
