Re: [yocto] Discussion: Package testing

2012-06-21 Thread Björn Stenberg
Burton, Ross wrote: The problem with that is that you can only test the external interfaces, not the inner state (black box vs white box). i.e. DBus has test cases for each C file inside that C file, so it can exercise both the external API and internal state. I haven't examined dbus in

Re: [yocto] Discussion: Package testing

2012-06-19 Thread Burton, Ross
On 18 June 2012 16:04, Björn Stenberg b...@enea.com wrote: Typically the tests don't mix production and test code, since the purpose is to test the production code. Rather the tests are isolated in separate files that we relatively easily can build and install separately from the base

Re: [yocto] Discussion: Package testing

2012-06-18 Thread Richard Purdie
On Thu, 2012-06-14 at 15:26 +0200, Björn Stenberg wrote: Many source packages include their own package test suites. We are looking at running these tests on target in a structured way, and I would like a discussion about how to best fit this into the Yocto framework. The actions that need

Re: [yocto] Discussion: Package testing

2012-06-18 Thread Fredrik Hugosson
On 06/18/2012 01:43 PM, Richard Purdie wrote: On Thu, 2012-06-14 at 15:26 +0200, Björn Stenberg wrote: Many source packages include their own package test suites. We are looking at running these tests on target in a structured way, and I would like a discussion about how to best fit this into

Re: [yocto] Discussion: Package testing

2012-06-18 Thread Björn Stenberg
Fredrik Hugosson wrote: I think you both might be missing some cases here. The problem is that 'make check' does not have a standardized meaning. Even if there was a standardized meaning to make check, there are lots of packages that use completely differents systems for their building and

Re: [yocto] Discussion: Package testing

2012-06-18 Thread Björn Stenberg
Richard Purdie wrote: c) (-ptest) Install all test files to /opt/ptest/${PN}-${PV} (for example). Make a package ptest-runner that has a script /opt/ptest/run-all-tests to iterate over all installed tests and run them. This is our suggestion. Can we suitably extract the tests out of the

Re: [yocto] Discussion: Package testing

2012-06-15 Thread Björn Stenberg
Frans Meulenbroeks wrote: a run-all-tests.sh script is somewhat problematic as different systems have different packages installed so different tests. The idea is that the top level run-all script looks in the /opt/ptest directory and runs all package tests that are installed there. It has no

[yocto] Discussion: Package testing

2012-06-14 Thread Björn Stenberg
Hi. Many source packages include their own package test suites. We are looking at running these tests on target in a structured way, and I would like a discussion about how to best fit this into the Yocto framework. The actions that need to be performed for a package test are roughly: 1)

Re: [yocto] Discussion: Package testing

2012-06-14 Thread Frans Meulenbroeks
Bjorn, these are some interesting ideas. Some feedback below. This is from the perspective of asomeone who develops for embedded systems boards 2012/6/14 Björn Stenberg b...@enea.com Hi. Many source packages include their own package test suites. We are looking at running these tests on