Hi Mike, hi Evgeny, while I fully support Evgeny's view on the check goals in general, I'm afraid of the support problem:
Check goal as well as report-aggregate causes a lot of traffic on our bug tracker and mailing list. Mostly because people simply don't read documentation or come with questionable requirements (which they often do not understand themselfs -- maybe pushed by someone?). I'm afraid that combining both (while I see valuable use cases) will simply kill our small team with support requests. I would actually love to see all the advanced agregate/check etc. features in a separate project. So we had a chance to focus on the core code coverage engine. Regards, -marc On 2019-06-04 02:16, Evgeny Mandrikov wrote: > Hi Mike, > On Friday, May 24, 2019 at 11:47:57 PM UTC+2, [email protected] > wrote: > >> Hello! >> >> We're using report-aggregate extensively to manage our multi-module Maven >> projects, but also wanted to perform checks project wide. We ended up adding >> a custom check-aggregate goal to an internal fork, and have been using it >> for a while. We'd like to go back to the standard Jacoco, and would be happy >> to put up a PR to add a check-aggregate goal. >> >> I wanted to check in before getting it into a state worthy of a PR and see >> if that's a feature you'd be willing to accept into the codebase. Does that >> seem like a reasonable addition? > > First of all thank you for following correct process about addition of new > feature by first starting a discussion with our little team in mailing list. > And please excuse us for the delay with response. > > Let me be completely transparent and honest with you by expressing my > personal humble opinion about "check" goal in general: > > If I'd be able to scroll time back, then personally would not introduce even > "check" goal and would strongly vote against it. This opinion was formed over > years since its introduction based on my own attempts to use it in various > teams and projects and observations about its usage by others. > > Checks "greater than" and "less then" are weak - frequent failures of these > in continuous integration environment without proper > mentoring/guidance/leadership/reviews leads to the fact that developers > quickly learn how to add enough "easy" pretty useless coverage just to meet > these criterias, while keep skipping "hard" cases. Even seen many cases where > these criterias were leading to a changes in code under tests just to hide > hard cases from these checks (such as usage of Streams/Optionals, change of > design, etc where not needed) and eyes of reviewers. > > And even with proper mentoring/guidance they are weak and important test > cases are frequently missed, drawing these checks pretty much useless. > > Strict check about exact total quickly becomes frequent point of merge > conflicts: 100% coverage is usually not present and very hard to reach, > branches introduce/change uncovered code for good reasons - have a look even > at report for JaCoCo itself ( > https://www.jacoco.org/jacoco/trunk/coverage/index.html ) where we strictly > follow TDD, "accidental" coverage ( unrelated to modified code / tests ) is > hard to find in big projects, and so correct value to resolve merge conflict > is not easy to figure out, thus in eyes of developers this starts to look as > "time waste", leading to frustrations and push-backs. > > BTW aggregate-report is also quite frequently misused - just to bump numbers > up by relying exactly on the "accidental" coverage. > > What IMO works a way better than "check" that fails builds in CI - is > protection of teams from managers who tend to look on raw coverage metric and > proper leadership/work with a team to embrace principles of TDD, including > code coverage measurements outside of CI in IDEs, while keeping their freedom > to commit changes to meet business expectations and deadlines. > > Coming back to initial question, based on the above personally I'm really > unsure about introduction of "check-aggregate" and its following maintaince > of a feature that does not really solve the problem (as it is already the > case for "check") by our small team. I'd rather prefer to have/invest time > into research of a better solutions. > > Having said all this, I'm curious to learn your experience and what you think > after that. And what thinks Marc ;) > > Regards, > Evgeny > > -- > You received this message because you are subscribed to the Google Groups > "JaCoCo and EclEmma Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jacoco/48112df7-f620-4403-a6d6-664e05198531%40googlegroups.com > [1]. Links: ------ [1] https://groups.google.com/d/msgid/jacoco/48112df7-f620-4403-a6d6-664e05198531%40googlegroups.com?utm_medium=email&utm_source=footer -- You received this message because you are subscribed to the Google Groups "JaCoCo and EclEmma Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/194bc920030b3fcc97e2872da6df53cb%40mountainminds.com.
