Hi Evgeny,
First of all, thanks for your detailed response.
Please find my comments below.
On Tuesday, August 29, 2017 at 9:38:35 PM UTC+1, Evgeny Mandrikov wrote:
>
> Hi,
>
> IMO term "coverage of static fields" is quite vague. And so not clear what
> and why you want to measure?
>
In this example I want to measure coverage per test case as if I was
running every single
test case on a new JVM. (I'm aware that running each test case on a new JVM
would be
insanely expensive, that's the reason I'm exploring other solutions)
The motivation behind is, suppose that *test3* from my example actually
triggers a bug.
Assuming the current coverage per test case from my example, it won't make
any sense
to report failing test case *test3* to a developer because it does not
cover anything.
>From here and given what is explained below any tricks with ClassLoaders
> sound crazy.
>
Yes, I do agree with you. Any trick with any classloader would be indeed
crazy and most
likely won't work for all cases (as for example the one you kindly
described below).
> JaCoCo does not record access to fields. It records execution of code that
> initializes fields. Code to initialize static fields is executed when class
> is initialized. So behavior that you observe is correct one, because class
> is initialized only once.
>
Any suggestion on how can I make coverage per test case 100% accurate if
only the first
test case (that makes the class-under-test be loaded and initialised) would
record the
execution of code that initialise static fields? I.e., how can I address
the fact that coverage
reported for *test3* (from my example) is 0, when it should be 3 lines?
> JaCoCo doesn't introduce any new ClassLoaders into application and doesn't
> do any tricks with existing ones.
>
Good to know, thanks.
>
> If two ClassLoaders load class (not return already loaded class), then at
> runtime these two classes are different, with all the consequences.
> Including two initializations of two classes, memory allocation in JVM to
> represent each, two instrumentations, etc. Not counting that this might be
> not expected by your application and tests and hence they might work
> incorrectly - e.g. usage of instance of one class as an argument to the
> equals method of another class will give you false even if
> getClass().getName() is the same, classes were loaded from the same class
> file on disk and have the same code:
>
> class Foo {
> public boolean equals(Object o) {
> if (o == null || getClass() != o.getClass()) {
> return false;
> }
> ...
> }
> }
>
>
Thanks for the detailed example.
> As far as JaCoCo concerned: if at runtime you have two different classes
> that are actually the same code loaded by different class loaders, JaCoCo
> knows that their code is the same thanks to
> http://www.jacoco.org/jacoco/trunk/doc/classids.html So recorded coverage
> will be stored in a single place, even if there will be two
> instrumentations.
>
I'm aware of it.
--
Best,
Jose
>
> Hope this helps.
>
>
> Evgeny
>
> On Tuesday, August 29, 2017 at 11:55:52 AM UTC+2, José Carlos de Campos
> wrote:
>>
>> Hi,
>>
>> I found the methods dump(true)
>> <http://www.jacoco.org/jacoco/trunk/doc/api/org/jacoco/agent/rt/IAgent.html#dump(boolean)>
>>
>> or reset()
>> <http://www.jacoco.org/jacoco/trunk/doc/api/org/jacoco/agent/rt/IAgent.html#reset()>
>>
>> of IAgent
>> <http://www.jacoco.org/jacoco/trunk/doc/api/org/jacoco/agent/rt/IAgent.html>
>> class very useful when we want
>> to get, for example, periodically coverage. However, I also found that
>> both methods
>> do not work I was expecting, special for classes with static fields.
>>
>> Let's suppose we have the following class:
>>
>> 1. public class PublicStaticFields {
>>
>> 2. public static String className =
>> PublicStaticFields.class.getCanonicalName();
>>
>> public static String s;
>>
>> static {
>> 5. s = "SsS";
>> 6. } }
>>
>>
>> and the following test class which achieves 100% line coverage.
>>
>> import static org.junit.Assert.assertEquals;import static
>> org.junit.Assert.assertNotNull;
>> import org.junit.Test;
>> public class TestPublicStaticFields {
>>
>> @Test
>> public void test1() {
>> assertEquals(PublicStaticFields.class.getCanonicalName(),
>> PublicStaticFields.className);
>> assertEquals("SsS", PublicStaticFields.s);
>> }
>>
>> @Test
>> public void test2() {
>> // empty
>> }
>>
>> @Test
>> public void test3() {
>> assertEquals(PublicStaticFields.class.getCanonicalName(),
>> PublicStaticFields.className);
>> assertEquals("SsS", PublicStaticFields.s);
>> }
>>
>> @Test
>> public void test4() {
>> assertNotNull(new PublicStaticFields());
>> }}
>>
>>
>> Now, suppose we want some periodically coverage. To keep the example
>> simple, let's
>> assume periodically as per test case. I.e., every time a test case
>> finishes, coverage
>> is collected and dumped. (something similar to what
>> sonar-jacoco-listeners
>> <https://github.com/SonarSource/sonar-java/blob/master/sonar-jacoco-listeners/src/main/java/org/sonar/java/jacoco/JacocoController.java>
>> does)
>>
>> For this example we should get
>>
>> - 3 lines covered by test1 (lines 2, 5, and 6)
>> - 0 lines covered by test2
>> - 3 lines covered by test3 (lines 2, 5, and 6)
>> - 4 lines covered by test4 (lines 1, 2, 5, 6)
>>
>> however we get:
>>
>> - 3 lines covered by test1 (lines 2, 5, and 6) : As expected
>> - 0 lines covered by test2 : As expected
>> - 0 lines covered by test3 : Lines 2, 5, and 6 should have been
>> covered
>> - 1 line covered by test4 (line 1) : Lines 2, 5, and 6 should have
>> been covered as well
>>
>> I do not think there is any bug in methods dump or reset of class IAgent,
>> I just found it a bit
>> odd that every time we dump/reset coverage we lost coverage of static
>> fields.
>>
>> As far I know, the only option to get coverage data of static fields per
>> test case is by loading
>> all classes under test with a custom classloader. And I don't think
>> JaCoCo implements its
>> own classloader, so it is not surprising that we lose track of coverage
>> of static fields. Please
>> correct me if JaCoCo actually implements is own classloader.
>>
>> Any thoughts on this? Would a JaCoCo's custom classloader be able to
>> address the above
>> scenario? What would the drawbacks be? As far I understood, classes get
>> instrumented when
>> they are loaded, assuming they will be reloaded several times, would that
>> mean they would
>> have to be instrumented several times as well? (I wonder how much would
>> that affect JaCoCo's
>> runtime).
>>
>>
>> --
>> Thanks in advance,
>> Jose
>>
>
--
You received this message because you are subscribed to the Google Groups
"JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jacoco/c187a344-4415-4951-8db9-a58b8211b3b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
