On Sat, 26 Jan 2019 at 01:46, Joel Sherrill <j...@rtems.org> wrote: > > > On Fri, Jan 25, 2019 at 12:50 PM Vijay Kumar Banerjee < > vijaykumar9...@gmail.com> wrote: > >> >> >> On Fri, 25 Jan 2019 at 23:39, Joel Sherrill <j...@rtems.org> wrote: >> >>> Sorry for the duplicate email. I didn't use the right email address and >>> the response did not go to devel@ >>> >>> On Fri, Jan 25, 2019 at 8:43 AM Jiri Gaisler <j...@gaisler.se> wrote: >>> >>>> Sorry for the noise, but I have a better patch that actually solves the >>>> root problem. I ran the full testsuite on sis and covoar against libscore.a >>>> like this: >>>> >>>> $ sparc-rtems5-ar -rcs sparc-rtems5/c/leon3/cpukit/score/libscore.a >>>> sparc-rtems5/c/leon3/cpukit/score/src/*.o >>>> >>>> $ rtems-test --rtems-bsp=leon3-sis-cov sparc-rtems5/c/leon3/testsuites >>>> >>>> $ /opt/rtems/5/share/rtems/tester/bin/covoar -v -f TSIM -S >>>> /opt/rtems/5/share/rtems/tester/rtems/testing/coverage/score-symbols.ini -O >>>> coverage -E >>>> /opt/rtems/5/share/rtems/tester/rtems/testing/coverage/Explanations.txt -p >>>> RTEMS-5 sparc-rtems5/c/leon3/testsuites/*/*.exe >>>> >>>> summary.txt: >>>> >>>> Bytes Analyzed : 39376 Bytes Not Executed : 2864 >>>> Percentage Executed : 92.73 Percentage Not Executed : 7.273 Uncovered >>>> ranges found : 108 Total branches found : 860 Uncovered branches >>>> found : 200 61 branches always taken 139 branches never taken >>>> >>>> All symbol sizes and ranges are now correctly reported/calculated in >>>> the reports. There are two remaining issues to be solved: >>>> >>>> 1. File names are printed as 'unknown' in reports (minor problem but >>>> useful) >>>> >>> Originally we got the info via the binutils command line tools. I think >>> the >>> filename came from addr2line. When we designed it originally, Ian Lance >>> Taylor said parsing the tool output was more easier, reliable, and more >>> stable than tracking any elf reading library. >>> >>> But it was slower than accessing the data directly via ELF and DWARF. >>> Since Chris added this support for the dynamic loader, it was natural to >>> begin to replace that code. Unfortunately, there are some warts left from >>> the transition. The name may be one of them. >>> >>> We also think the beginning and end of a method are not marked properly. >>> >>> And when something is inlined, it isn't considered part of the main >>> method. >>> The original method considered assembly from inlined code part of the >>> caller. It is now reported as part of the callee. We have identified a >>> needed >>> change to (1) account for the inlined assembly as part of the caller and >>> (2) add additional tracking of the coverage of inlined methods. >>> >>> The immediate need is to get the reporting so it accurately annoates >>> the generated assembly from a method and anything inlined into it. >>> This should let us do DO-178 Level A style coverage -- we are >>> sure we use ever instruction and take/not take every branch. >>> >>> Hello, >> This is awesome to have the sis support in covoar, the very next thing >> that >> comes to my mind is adding the '-f' option in the rtems-test coverage >> script, >> this will enable us to get the HTML reports. >> >> The issues noted here, can make a very good project for GSoC and I was >> hoping to work on them, but there is a good amount of complexity >> involved, >> and I am confused where to get started. :) >> > > This should be another thread but here's a short answer > > + Verify Jiri didn't break reporting from qemu testing > > Reporting from qemu works properly with the patch. Checked with rtems-test
> Then I recall a few odd problems: > > + method entry and exit not being marked as executed even though > we know it was. > + inlined code not being marked because it wasn't reported as > "in the method" > > If all reports look trustworthy, then look into additionally tracking > coverage of inlined methods but track it back to their source. > > >> >> If there is unreachable code or branches where both paths are not >>> exercised, we need tests to exercise them OR the code is useless >>> and can be removed. Either way, we win. >>> >>>> 2. We should add RISC-V support to covoar ... >>>> >>> >>> I am sure you have seen the Target* files. They are not large. Some is >>> trivial. Other parts are to recognize nops and branch instruction >>> variants. >>> >>> Is the workflow something like this ... >> 1. write Target_ARCH.cc >> 2. Add it to the TargetFactory >> 3. Add ini file in the tools . >> >> I have a question. Is there some reason that the files in covoar/ don't >> have a >> copyright notice? >> > > Sloppiness. :( > > The original code was written entirely by OAR. We could probably use git > blame to augment that. > > >> >> On a RISC architecture, I wouldn't expect a lot of variation in nops. >>> The x86 has an awful set that range in size. :( >>> >>> Let me know if you agree to the patch .. >>>> >>> >>> The patch looks good to me. >>> >>>> _______________________________________________ >>>> devel mailing list >>>> devel@rtems.org >>>> http://lists.rtems.org/mailman/listinfo/devel >>> >>>
_______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel