Re: Code coverage check for PRs

2022-10-05 Thread Gus Heck
One thing codecov gives is a sense of what the coverage was previously without having to go hunt down past builds in jenkins. It is a coverage focused view essentially. It just uses the coverage data the build already calculates. It used to have some nifty graphical visualizations but those seem to

Re: Code coverage check for PRs

2022-10-05 Thread Robert Muir
I would recommend a search for "github actions jacoco" to review what's common out there. If we change 'gradle test' to 'gradle coverage' in our existing PR-test action, the next step is to just not throw away the reports, but make them available. See https://docs.github.com/en/actions/using-workf

Re: Code coverage check for PRs

2022-10-05 Thread Patrick Zhai
Make sense to me, I'll try to look into it! On Tue, Oct 4, 2022, 16:50 Robert Muir wrote: > We already have code coverage integrated into the build. See the > documentation on how to generate the reports: > https://github.com/apache/lucene/blob/main/help/tests.txt > > I think we should stick wit

Re: Code coverage check for PRs

2022-10-04 Thread Robert Muir
We already have code coverage integrated into the build. See the documentation on how to generate the reports: https://github.com/apache/lucene/blob/main/help/tests.txt I think we should stick with jacoco and not some commercial stuff for measuring coverage. Jacoco works great. We just have to put

Re: Code coverage check for PRs

2022-10-04 Thread Patrick Zhai
Hi Robert, thank you for commenting, yeah the functionality I want to add is actually the line by line code coverage stats for the new/changed line that are in the patch so that we don't need to wonder about "whether that line is covered by the test?". But I'm against using the code coverage as any

Re: Code coverage check for PRs

2022-10-04 Thread Robert Muir
btw, you can look at the current reports created by jenkins here: https://ci-builds.apache.org/job/Lucene/job/Lucene-Coverage-main/lastBuild/jacoco/ On Tue, Oct 4, 2022 at 6:51 AM Robert Muir wrote: > > we can run the tests with coverage option and produce coverage graph > from the github actions

Re: Code coverage check for PRs

2022-10-04 Thread Robert Muir
we can run the tests with coverage option and produce coverage graph from the github actions, but need to look at the docs to see where to put it so it will be available. I want us to be careful about the word "check" as I'm adamantly against any such automated check (e.g. coverage > N%) in the lo

Code coverage check for PRs

2022-10-03 Thread Patrick Zhai
Hi folks, I'm not sure whether people have already discussed this but I'm wondering whether we want to add a workflow that pulls out the code coverage whenever a PR was created? It should be easier for both the reviewers and the contributors to figure out what can be improved, or at least figure ou