On 30 May 2018 at 22:55, Cillian O'Donnell <cpodonne...@gmail.com> wrote:
> > > On Wed, 30 May 2018, 18:15 Vijay Kumar Banerjee, <vijaykumar9...@gmail.com> > wrote: > >> >> >> On Wed, 30 May 2018, 22:42 Cillian O'Donnell, <cpodonne...@gmail.com> >> wrote: >> >>> >>> >>> On Wed, 30 May 2018, 16:58 Gedare Bloom, <ged...@rtems.org> wrote: >>> >>>> On Wed, May 30, 2018 at 11:32 AM, Vijay Kumar Banerjee >>>> <vijaykumar9...@gmail.com> wrote: >>>> > On 30 May 2018 at 20:18, Gedare Bloom <ged...@rtems.org> wrote: >>>> >> >>>> >> Hello Vijay, >>>> >> >>>> >> Do you expect/need an answer to something in here? >>>> >> >>>> >> gedare >>>> >> >>>> > Hello, >>>> > >>>> > I wanted to know if there were any plans on how covoar >>>> > should store the reports when running for multisets. >>>> > Earlier it used to be done by the coverage script, >>>> > after the recent changes covoar can process multi sets. >>>> > >>>> > I think, covoar should store the reports into separate directories >>>> > for each set . eg. score/ , rtems/ . As the coverage can already read >>>> > from separate directories. >>>> >>> >>> Sorry I thought all questions had been answered here. I think you have >>> the right idea. Each set should be a sub-directory of coverage directory. >>> >>> By the way I tested your changes and everything seems fine. Still have >>> to do a review of coverage.py to see how close we are to merging. >>> >> I have squashed everything and sent a patch to devel@. This will make it >> easy to go through all the changes. Please have a look. >> > > Ah just seen it, will take a look. It'll be good to get Chris' thoughts > too. We'll have something more definitive about how close it is to merging. > Sure. > > >>>> > Any advice on how should it be approached ? >>>> >>>> It would help me if you could draw/write a diagram of what the >>>> filesystem tree might look like with separate directories, and what >>>> will go in each subdirectory. >>>> >>>> I don't have enough context to give any useful advice on this question. >>>> >>>> -Gedare >>>> >>>> >> >>>> >> On Wed, May 30, 2018 at 10:29 AM, Vijay Kumar Banerjee >>>> >> <vijaykumar9...@gmail.com> wrote: >>>> >> > On 30 May 2018 at 00:54, Joel Sherrill <j...@rtems.org> wrote: >>>> >> >> >>>> >> >> >>>> >> >> >>>> >> >> On Tue, May 29, 2018 at 11:27 AM, Vijay Kumar Banerjee >>>> >> >> <vijaykumar9...@gmail.com> wrote: >>>> >> >>> >>>> >> >>> >>>> >> >>> >>>> >> >>> On Tue, 29 May 2018, 20:10 Joel Sherrill, <j...@rtems.org> >>>> wrote: >>>> >> >>>> >>>> >> >>>> >>>> >> >>>> >>>> >> >>>> On Mon, May 28, 2018 at 11:08 PM, Chris Johns <chr...@rtems.org >>>> > >>>> >> >>>> wrote: >>>> >> >>>>> >>>> >> >>>>> On 29/5/18 4:26 am, Joel Sherrill wrote: >>>> >> >>>>> > On Mon, May 28, 2018 at 5:43 AM, Vijay Kumar Banerjee >>>> >> >>>>> > <vijaykumar9...@gmail.com >>>> >> >>>>> > <mailto:vijaykumar9...@gmail.com>> wrote: >>>> >> >>>>> > >>>> >> >>>>> > Hello, >>>> >> >>>>> > >>>> >> >>>>> > The coverage reports are now showing results. >>>> >> >>>>> > The current branch that holds all the changes is >>>> >> >>>>> > the cov-tester-support branch in my forked repo >>>> >> >>>>> > >>>> >> >>>>> > >>>> >> >>>>> > https://github.com/thelunatic/rtems-tools/tree/cov-tester- >>>> support >>>> >> >>>>> > >>>> >> >>>>> > >>>> >> >>>>> > <https://github.com/thelunatic/rtems-tools/tree/ >>>> cov-tester-support> >>>> >> >>>>> > >>>> >> >>>>> > (Please have a look into the code and test it.) >>>> >> >>>>> > >>>> >> >>>>> > It is close to merging (hopefully). These are the issues >>>> >> >>>>> > that would need a fix before it can be merged : >>>> >> >>>>> > >>>> >> >>>>> > 1. I have added some #FIXME in the code (have a look) >>>> >> >>>>> > in coverage script. I have set the value of the >>>> targe to >>>> >> >>>>> > be >>>> >> >>>>> > sparc-rtems5, which makes it limited to sparc-rtems5 >>>> only. >>>> >> >>>>> > We >>>> >> >>>>> > can >>>> >> >>>>> > include the target in the bsp ini file, That would >>>> >> >>>>> > be a quick fix for this. >>>> >> >>>>> > >>>> >> >>>>> > >>>> >> >>>>> > Yes. This needs to be fixed. >>>> >> >>>>> > >>>> >> >>>>> > My hack to add 4 in ObjdumpProcessor.cc needs to be addressed >>>> >> >>>>> > also. >>>> >> >>>>> > I am thinking of adding a method to Target_*.cc to ask for >>>> the >>>> >> >>>>> > size >>>> >> >>>>> > of an >>>> >> >>>>> > instruction. >>>> >> >>>>> > Then pass it the last instruction. That way we can throw on >>>> other >>>> >> >>>>> > architectures for >>>> >> >>>>> > now. If Chris solves this with his changes before we try >>>> another >>>> >> >>>>> > architecture, >>>> >> >>>>> > great. >>>> >> >>>>> > If not, it will be easy to fix. >>>> >> >>>>> >>>> >> >>>>> What is the overall requirement? >>>> >> >>>> >>>> >> >>>> >>>> >> >>>> To know the ending address of the function. >>>> >> >>>> >>>> >> >>>> Technically there are three pieces of information: >>>> >> >>>> >>>> >> >>>> + start address >>>> >> >>>> + end address >>>> >> >>>> + size >>>> >> >>>> >>>> >> >>>> If you know two of those, you can compute the third. >>>> >> >>>> >>>> >> >>>> I don't care if this comes from DWARF, ELF, or parsing the >>>> >> >>>> disassembly. >>>> >> >>>> It just needs to be reliable. >>>> >> >>>> >>>> >> >>>> And.. I am not proposing my solution as permanent. Just to keep >>>> us >>>> >> >>>> moving. You want to change to an internal disassembler (which >>>> >> >>>> would also need to have the source interspersed) and DWARF. So >>>> >> >>>> this code would go away then. >>>> >> >>>> >>>> >> >>>>> >>>> >> >>>>> >>>> >> >>>>> What defines the function and so size are attempting to get >>>> coverage >>>> >> >>>>> of? What if >>>> >> >>>>> that function calls an inline function and that function is >>>> inlined? >>>> >> >>>>> What if >>>> >> >>>>> that inlined function calls another inlined function? >>>> >> >>>> >>>> >> >>>> >>>> >> >>>> Then it is all inlined. It is done consistently now. I have >>>> never >>>> >> >>>> seen a >>>> >> >>>> case >>>> >> >>>> where two instances of a method differed as the assembly level. >>>> [1] >>>> >> >>>> >>>> >> >>>> The actual body of the inlined method is evaluated at each >>>> expansion >>>> >> >>>> point. >>>> >> >>>> That is why a few years ago, I pushed hard to reduce the >>>> complexity >>>> >> >>>> of >>>> >> >>>> inline methods because we got test patch explosion when an >>>> inlined >>>> >> >>>> method >>>> >> >>>> included complex conditionals. >>>> >> >>>> >>>> >> >>>> Similarly, I think it would be helpful to coverage and >>>> verification >>>> >> >>>> efforts to >>>> >> >>>> follow the **shudder** MISRA rule which want you to write simple >>>> >> >>>> conditional >>>> >> >>>> expressions rather than compound ones. I have taken to writing >>>> code >>>> >> >>>> this >>>> >> >>>> way as much as possible. But haven't pushed it as a coding rule. >>>> >> >>>> >>>> >> >>>> if (a) { >>>> >> >>>> if (b) { >>>> >> >>>> do x; >>>> >> >>>> } >>>> >> >>>> } >>>> >> >>>> >>>> >> >>>> Versus >>>> >> >>>> if (a && b) { >>>> >> >>>> do x; >>>> >> >>>> } >>>> >> >>>> >>>> >> >>>> The reason is that the first is easier to analyse coverage on. >>>> >> >>>> >>>> >> >>>> [1] We both expect LTO could change this. >>>> >> >>>> >>>> >> >>>> [2] ESA did specifically mention this one though. Also in >>>> general >>>> >> >>>> terms, >>>> >> >>>> an RTEMS Project response to MISRA rules. Which ones we follow, >>>> >> >>>> reject, etc.. But I refuse to adopt hard rules which can't be >>>> >> >>>> enforced >>>> >> >>>> by free open source tools. >>>> >> >>>> >>>> >> >>>>> >>>> >> >>>>> >>>> >> >>>>> The DWARF data provides details about the PC low and PC high >>>> of what >>>> >> >>>>> is >>>> >> >>>>> called >>>> >> >>>>> concrete functions and then it provides the details about >>>> inlines. >>>> >> >>>> >>>> >> >>>> >>>> >> >>>> We don't (currently) report on the inlines as independent >>>> methods. >>>> >> >>>> >>>> >> >>>>> >>>> >> >>>>> >>>> >> >>>>> > >>>> >> >>>>> > 2. coverage used to feed ini file for each symbol to >>>> covoar >>>> >> >>>>> > in a loop and store the result in a separate >>>> directory >>>> >> >>>>> > for each symbol . Which is needed no more needed as >>>> >> >>>>> > covoar can now process multi sets from a >>>> >> >>>>> > single ini file. I think the results from covoar >>>> should >>>> >> >>>>> > be >>>> >> >>>>> > store in a separate directory for each symbol >>>> >> >>>>> > example :- score/ >>>> >> >>>>> > >>>> >> >>>>> > >>>> >> >>>>> > A bit of history will help here. Originally covoar was run >>>> against >>>> >> >>>>> > a >>>> >> >>>>> > single set of >>>> >> >>>>> > code by the scripting framework. We would do coverage on >>>> either >>>> >> >>>>> > "core >>>> >> >>>>> > parts" >>>> >> >>>>> > or "developmental" (e.g. nearly all non-networked code). The >>>> >> >>>>> > optimization was >>>> >> >>>>> > either at -O2 or -Os. So there were four coverage variants. >>>> >> >>>>> > >>>> >> >>>>> > Turned out that when we added something to "all", the >>>> percentage >>>> >> >>>>> > would drop >>>> >> >>>>> > and reflect badly on the rest of the code. I remember adding >>>> the >>>> >> >>>>> > dosfs and >>>> >> >>>>> > the coverage dropped almost 20 percent. >>>> >> >>>>> > >>>> >> >>>>> > This led to the idea that we should report on a per >>>> >> >>>>> > directory/subsystem basis. >>>> >> >>>>> > The score, rtems, posix, sapi, and libcsupport should have >>>> high >>>> >> >>>>> > coverage now >>>> >> >>>>> > and the reports should reflect that independent of whether >>>> the >>>> >> >>>>> > dosfs >>>> >> >>>>> > needs a >>>> >> >>>>> > lot more tests. >>>> >> >>>>> > >>>> >> >>>>> > Before each directory/subsystem required a completely >>>> separate run >>>> >> >>>>> > of >>>> >> >>>>> > covoar. >>>> >> >>>>> > If we are headed to a solution where one analysis run of >>>> covoar >>>> >> >>>>> > generates different >>>> >> >>>>> > reports, that should speed up the processing time a lot! >>>> >> >>>>> > >>>> >> >>>>> > The other issue is what should the top directory look >>>> like/contain >>>> >> >>>>> > when a single >>>> >> >>>>> > run produces multiple subsystem reports. Seems like we would >>>> need >>>> >> >>>>> > at >>>> >> >>>>> > least a >>>> >> >>>>> > top level html and text file. >>>> >> >>>>> > >>>> >> >>>>> > >>>> >> >>>>> > >>>> >> >>>>> > 3. currently we are using the leon3-qemu-cov as the bsp. >>>> >> >>>>> > Are we going to have two ini file for each bsp ? ( >>>> one >>>> >> >>>>> > without coverage >>>> >> >>>>> > and one with coverage support) >>>> >> >>>>> > >>>> >> >>>>> > Earlier the approach was to include a section 'coverage' >>>> >> >>>>> > to the bsp ini to put the values we needed for coverage. >>>> >> >>>>> > I think that approach would be more "convenient" for the >>>> user. >>>> >> >>>>> > >>>> >> >>>>> > >>>> >> >>>>> > This was something Chris suggested. I think it was to avoid >>>> adding >>>> >> >>>>> > to >>>> >> >>>>> > the bsp ini file until the code was closer to merging. >>>> >> >>>>> > >>>> >> >>>>> > Chris.. what do you think? Long term, a section would be >>>> nice. >>>> >> >>>>> > >>>> >> >>>>> >>>> >> >>>>> Sorry I cannot remember without looking it up and I am >>>> currently >>>> >> >>>>> buried >>>> >> >>>>> in >>>> >> >>>>> family issues. >>>> >> >>>> >>>> >> >>>> >>>> >> >>>> OK. Having the Python scripting loop over the sets or covoar >>>> looping >>>> >> >>>> over them >>>> >> >>>> is an open issue. Historically, looping over desired symbol >>>> sets was >>>> >> >>>> outside >>>> >> >>>> covoar. So looping inside covoar may take some work. >>>> >> >>> >>>> >> >>> >>>> >> >>> covoar can already loop over the >>>> >> >>> sets it seems, which is implemented >>>> >> >>> in DesiredSymbols. But it stores all the >>>> >> >>> reports generated from into the same directory. >>>> >> >> >>>> >> >> >>>> >> >> If there is an index that makes its possible to navigate through >>>> the >>>> >> >> different "subsystems", then that's the key thing. You don't want >>>> >> >> to think score is poorly covered due to dosfs. >>>> >> >> >>>> >> >>> >>>> >> >>> >>>> >> >>> P.S:Sorry for the previous mail with no message >>>> >> >>>> >>>> >> >>>> >>>> >> >>>> --joel >>>> >> >>>>> >>>> >> >>>>> >>>> >> >>>>> Chris >>>> >> >>>> >>>> >> >>>> >>>> >> >> >>>> >> > >>>> >> > >>>> >> > _______________________________________________ >>>> >> > devel mailing list >>>> >> > devel@rtems.org >>>> >> > http://lists.rtems.org/mailman/listinfo/devel >>>> > >>>> > >>>> _______________________________________________ >>>> devel mailing list >>>> devel@rtems.org >>>> http://lists.rtems.org/mailman/listinfo/devel >>>> >>>
_______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
