Re: [OE-core] ptest with BBCLASSEXTEND
On Mon, 2019-06-03 at 11:10 +0800, Changqing Li wrote: > Based on this list, except lib*perl and packages that are dealing > with by others, > > I picked some for your reference to evaluate the workload. If more > info is needed, > > feel free to tell me. If it helps, I reran the minimal image comparison and found the following issues: Recipes where we have problems (differences) in minimal images: bash | 76 | 5 | 0 | 163 diffutils| 19 | 0 | 2 | 1 e2fsprogs| 0 | 0 | 0 | 111 elfutils | 194 | 0 | 9 | 32 gettext | 354 | 21 | 37| 32 gzip | 19 | 0 | 3 | 3 kbd | 5 | 2 | 0 | 0 m4 | 145 | 7 | 18| 5 openssh | 15 | 1 | 1 | 145 parted | 41 | 1 | 22| 5 perl | 2413| 18 | 245 | 861 python | 301 | 6 | 41| 327 python3 | 30277 | 2 | 1043 | 1012 Others: acl tests are timing out? gawk shows failures in sdk image but minimal: gawk | 407 | 24 | 0 | 20 gstreamer shows two fewer failures in minimal image: gstreamer1.0 | 104 | 1 | 0 | 104 (I'm using https://autobuilder.yocto.io/pub/non-release/20190601-2/testresults/testresult-report.txt for comparison) There are also some open ptest bugs in bugzilla which I didn't duplicate here. The above list is much smaller than the original particularly now we have the perl module issues resolved. Cheers, Richard > diffutils depend on perl > elfutils: mingli have work on it and send patch to oe-core mail > list > gettext: depend on certain locales for some tests > >depend on autoconf for some test > gzip: have tests depend on diff, while diff in busybox > incompatible with POSIX >have tests depend on grep, while grep in busybox > don't > support context options >have tests depend on perl > kbd: have tests depend on readlink, busybox's readlink don't > support -e option > m4: one test depend on perl, some test skip since depend on > certain locale/codeset, need to check one by one > > -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] ptest with BBCLASSEXTEND
On 5/22/19 6:31 AM, Richard Purdie wrote: On Thu, 2019-05-16 at 23:28 +0100, richard.pur...@linuxfoundation.org wrote: Just to update from my perspective: quilt and lib*-perl all look like missing perl module dependencies. The place those dependencies are missing may be for the ptest or may be the actual PN package itself, depending on where the module dependency is. I've sent out a further patch series which resolves some of the issues. Some quilt tests now pass, some still fail. The perl modules fail in different ways to before, there are clearly more missing dependencies to be added. A quick summary of roughly the recipes still with problems (not 100% accurately filtered): == PTest Result Summary == -- Recipe | Passed | Failed| Skipped | Time(s) -- bash | 41 | 3 | 0 | 107 T bluez5 | 23 | 2 | 0 | 5 dbus-test| 0 | 0 | 0 | 91 T diffutils| 19 | 0 | 2 | 1 e2fsprogs| 0 | 0 | 0 | 108 T elfutils | 172 | 24| 7 | 19 gettext | 354 | 21| 37| 33 glib-2.0 | 245 | 0 | 0 | 367 gstreamer1.0 | 103 | 1 | 0 | 103 gzip | 19 | 0 | 3 | 3 kbd | 5 | 2 | 0 | 0 libconvert-asn1-perl | 0 | 22| 0 | 0 liberror-perl| 2 | 13| 0 | 0 libpcre | 3 | 0 | 0 | 2 libtest-needs-perl | 0 | 2 | 0 | 0 libtimedate-perl | 3 | 2 | 0 | 0 liburi-perl | 0 | 39| 2 | 0 libxml-namespacesupport-perl | 0 | 1 | 0 | 0 libxml-parser-perl | 0 | 15| 0 | 0 libxml-perl | 0 | 6 | 0 | 0 libxml-sax-base-perl | 2 | 17| 0 | 1 libxml-sax-perl | 0 | 14| 0 | 0 libxml-simple-perl | 0 | 14| 0 | 0 lttng-tools | 3397| 626 | 1 | 350 T m4 | 144 | 8 | 18| 5 nettle | 97 | 0 | 2 | 8 openssh | 15 | 1 | 1 | 138 openssl | 146 | 1 | 8 | 87 opkg | 0 | 0 | 0 | 9 parted | 41 | 1 | 22| 4 perl | 2423| 19| 238 | 727 python | 301 | 6 | 41| 325 python3 | 30277 | 2 | 1043 | 1005 quilt| 10 | 47| 0 | 24 tcl | 149 | 0 | 0 | 177 valgrind | 149 | 1 | 9 | 552 -- Cheers, Richard Randy, Based on this list, except lib*perl and packages that are dealing with by others, I picked some for your reference to evaluate the workload. If more info is needed, feel free to tell me. diffutils depend on perl elfutils: mingli have work on it and send patch to oe-core mail list gettext: depend on certain locales for some tests depend on autoconf for some test gzip: have tests depend on diff, while diff in busybox incompatible with POSIX have tests depend on grep, while grep in busybox don't support context options have tests depend on perl kbd: have tests depend on readlink, busybox's readlink don't support -e option m4: one test depend on perl, some test skip since depend on certain locale/codeset, need to check one by one -- BRs Sandy(Li Changqing) -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org
Re: [OE-core] ptest with BBCLASSEXTEND
On Wed, 2019-05-22 at 18:28 -0400, Jonathan Rajotte-Julien wrote: > Hi, > > > I think Jonathan Rajotte might be able to help us with lttng-tools. > > Finally had some time for this. > > See [1][2]. > > [2] does bring the matter of forcing lttng-ust support of lttng-tools > when ptest > is deployed. "lttng-ust" is already set in PACKAGECONFIG by default > but is there any > way to "enforce" the lttng-ust config & dependency when lttng-tools- > ptest is used? > > [1] > http://lists.openembedded.org/pipermail/openembedded-core/2019-May/282733.html > [2] > http://lists.openembedded.org/pipermail/openembedded-core/2019-May/282734.html Thanks for looking at this. It sounds like we need some runtime dependencies as well as the PACKAGECONFIG option. I think the lttng- tool ust option should be changed: -PACKAGECONFIG[lttng-ust] = "--with-lttng-ust, --without-lttng-ust, lttng-ust" +PACKAGECONFIG[lttng-ust] = "--with-lttng-ust, --without-lttng-ust, lttng-ust,lttng-ust" i.e. force the dependency at runtime as well as buildtime. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] ptest with BBCLASSEXTEND
Hi, > I think Jonathan Rajotte might be able to help us with lttng-tools. Finally had some time for this. See [1][2]. [2] does bring the matter of forcing lttng-ust support of lttng-tools when ptest is deployed. "lttng-ust" is already set in PACKAGECONFIG by default but is there any way to "enforce" the lttng-ust config & dependency when lttng-tools-ptest is used? [1] http://lists.openembedded.org/pipermail/openembedded-core/2019-May/282733.html [2] http://lists.openembedded.org/pipermail/openembedded-core/2019-May/282734.html Cheers -- Jonathan Rajotte-Julien EfficiOS -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] ptest with BBCLASSEXTEND
On Thu, 2019-05-16 at 23:28 +0100, richard.pur...@linuxfoundation.org wrote: > Just to update from my perspective: > > quilt and lib*-perl all look like missing perl module dependencies. > The place those dependencies are missing may be for the ptest or may > be the actual PN package itself, depending on where the module > dependency is. I've sent out a further patch series which resolves some of the issues. Some quilt tests now pass, some still fail. The perl modules fail in different ways to before, there are clearly more missing dependencies to be added. A quick summary of roughly the recipes still with problems (not 100% accurately filtered): == PTest Result Summary == -- Recipe | Passed | Failed| Skipped | Time(s) -- bash | 41 | 3 | 0 | 107 T bluez5 | 23 | 2 | 0 | 5 dbus-test| 0 | 0 | 0 | 91 T diffutils| 19 | 0 | 2 | 1 e2fsprogs| 0 | 0 | 0 | 108 T elfutils | 172 | 24| 7 | 19 gettext | 354 | 21| 37| 33 glib-2.0 | 245 | 0 | 0 | 367 gstreamer1.0 | 103 | 1 | 0 | 103 gzip | 19 | 0 | 3 | 3 kbd | 5 | 2 | 0 | 0 libconvert-asn1-perl | 0 | 22| 0 | 0 liberror-perl| 2 | 13| 0 | 0 libpcre | 3 | 0 | 0 | 2 libtest-needs-perl | 0 | 2 | 0 | 0 libtimedate-perl | 3 | 2 | 0 | 0 liburi-perl | 0 | 39| 2 | 0 libxml-namespacesupport-perl | 0 | 1 | 0 | 0 libxml-parser-perl | 0 | 15| 0 | 0 libxml-perl | 0 | 6 | 0 | 0 libxml-sax-base-perl | 2 | 17| 0 | 1 libxml-sax-perl | 0 | 14| 0 | 0 libxml-simple-perl | 0 | 14| 0 | 0 lttng-tools | 3397| 626 | 1 | 350 T m4 | 144 | 8 | 18| 5 nettle | 97 | 0 | 2 | 8 openssh | 15 | 1 | 1 | 138 openssl | 146 | 1 | 8 | 87 opkg | 0 | 0 | 0 | 9 parted | 41 | 1 | 22| 4 perl | 2423| 19| 238 | 727 python | 301 | 6 | 41| 325 python3 | 30277 | 2 | 1043 | 1005 quilt| 10 | 47| 0 | 24 tcl | 149 | 0 | 0 | 177 valgrind | 149 | 1 | 9 | 552 -- Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] ptest with BBCLASSEXTEND
> Its not fast to parse this creates about 59 variants of the image. Its > (ab)using the mcextend class to an image and add the ptest to that > image for each ptest in the PTESTS_SLOW and PTESTS_FAST variables. It > also adds openssh since we need an ssh client in the images for > testimage to work. I also had to add extra space and memory to the > image config so all the tests could work. > > You can get a list of the target images with: > For the non-initiated, you will need to have the following in your local.conf for this to work. DISTRO_FEATURES_append = " ptest" That said, I should be able to take a look at the problem related to lttng-tools. Thanks -- Jonathan Rajotte-Julien EfficiOS -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] ptest with BBCLASSEXTEND
On 5/17/19 5:13 AM, Randy MacLeod wrote: On 5/16/19 8:36 AM, Richard Purdie wrote: On Thu, 2019-05-16 at 11:54 +0100, Richard Purdie wrote: https://autobuilder.yocto.io/pub/non-release/20190514-14/testresults/testresult-report.txt A couple of patches (on the list) gets us to the results below which is better. The perl module failures and quilt are from missing perl module dependencies. -- Recipe | Passed | Failed | Skipped | Time(s) -- acl | 3 | 0 | 0 | 0 apr | 44 | 0 | 0 | 58 apr-util | 19 | 0 | 0 | 8 attr | 0 | 0 | 0 | 0 bash | 36 | 8 | 0 | 108 T bluez5 | 23 | 2 | 0 | 5 busybox | 528 | 4 | 76 | 4 bzip2 | 6 | 0 | 0 | 1 dbus-test | 0 | 0 | 0 | 91 T diffstat | 439 | 0 | 0 | 3 diffutils | 19 | 0 | 2 | 2 e2fsprogs | 0 | 0 | 0 | 108 T elfutils | 172 | 24 | 7 | 20 ethtool | 2 | 0 | 0 | 1 flex | 114 | 0 | 0 | 2 gawk | 342 | 0 | 0 | 2 gdbm | 24 | 0 | 0 | 2 gdk-pixbuf | 20 | 0 | 0 | 39 gettext | 354 | 21 | 37 | 33 glib-2.0 | 242 | 3 | 0 | 347 gstreamer1.0 | 103 | 1 | 0 | 104 gzip | 19 | 0 | 3 | 4 json-glib | 14 | 0 | 0 | 0 kbd | 5 | 2 | 0 | 1 libconvert-asn1-perl | 0 | 22 | 0 | 0 liberror-perl | 0 | 15 | 0 | 0 libevent | 1 | 0 | 0 | 86 libpcre | 0 | 0 | 0 | 1 libtest-needs-perl | 0 | 2 | 0 | 0 libtimedate-perl | 3 | 2 | 0 | 0 liburi-perl | 0 | 39 | 2 | 0 libusb1 | 4 | 0 | 0 | 6 libxml-namespacesupport-perl | 0 | 1 | 0 | 0 libxml-parser-perl | 0 | 15 | 0 | 0 libxml-perl | 0 | 6 | 0 | 0 libxml-sax-base-perl | 0 | 19 | 0 | 0 libxml-sax-perl | 0 | 14 | 0 | 0 libxml-simple-perl | 0 | 14 | 0 | 0 libxml2 | 134 | 0 | 0 | 17 lttng-tools | 3397 | 626 | 1 | 350 T lzo | 5 | 0 | 0 | 2 m4 | 144 | 8 | 18 | 5 nettle | 97 | 0 | 2 | 9 openssh | 0 | 0 | 1 | 1 openssl | 146 | 1 | 8 | 91 opkg | 0 | 0 | 0 | 11 pango | 12 | 0 | 0 | 1 parted | 41 | 1 | 22 | 5 perl | 2423 | 19 | 238 | 713 python | 301 | 6 | 41 | 325 python3 | 30277 | 2 | 1043 | 990 quilt | 0 | 57 | 0 | 0 sed | 86 | 0 | 0 | 4 slang | 97 | 0 | 0 | 22 strace | 466 | 18 | 147 | 176 tcl | 149 | 0 | 0 | 179 util-linux | 336 | 2 | 80 | 41 valgrind | 149 | 1 | 9 | 544 zlib | 1 | 0 | 0 | 1 Cheers, Richard During the weekly YP bug review
Re: [OE-core] ptest with BBCLASSEXTEND
On Thu, 2019-05-16 at 17:13 -0400, Randy MacLeod wrote: > On 5/16/19 8:36 AM, Richard Purdie wrote: > > On Thu, 2019-05-16 at 11:54 +0100, Richard Purdie wrote: > > > https://autobuilder.yocto.io/pub/non-release/20190514-14/testresults/testresult-report.txt > > > > A couple of patches (on the list) gets us to the results below which is > > better. The perl module failures and quilt are from missing perl module > > dependencies. > > > > -- > > Recipe | Passed | Failed| Skipped | Time(s) > > -- > > acl | 3 | 0 | 0 | 0 > > apr | 44 | 0 | 0 | 58 > > apr-util | 19 | 0 | 0 | 8 > > attr | 0 | 0 | 0 | 0 > > bash | 36 | 8 | 0 | 108 T > > bluez5 | 23 | 2 | 0 | 5 > > busybox | 528 | 4 | 76| 4 > > bzip2| 6 | 0 | 0 | 1 > > dbus-test| 0 | 0 | 0 | 91 T > > diffstat | 439 | 0 | 0 | 3 > > diffutils| 19 | 0 | 2 | 2 > > e2fsprogs| 0 | 0 | 0 | 108 T > > elfutils | 172 | 24| 7 | 20 > > ethtool | 2 | 0 | 0 | 1 > > flex | 114 | 0 | 0 | 2 > > gawk | 342 | 0 | 0 | 2 > > gdbm | 24 | 0 | 0 | 2 > > gdk-pixbuf | 20 | 0 | 0 | 39 > > gettext | 354 | 21| 37| 33 > > glib-2.0 | 242 | 3 | 0 | 347 > > gstreamer1.0 | 103 | 1 | 0 | 104 > > gzip | 19 | 0 | 3 | 4 > > json-glib| 14 | 0 | 0 | 0 > > kbd | 5 | 2 | 0 | 1 > > libconvert-asn1-perl | 0 | 22| 0 | 0 > > liberror-perl| 0 | 15| 0 | 0 > > libevent | 1 | 0 | 0 | 86 > > libpcre | 0 | 0 | 0 | 1 > > libtest-needs-perl | 0 | 2 | 0 | 0 > > libtimedate-perl | 3 | 2 | 0 | 0 > > liburi-perl | 0 | 39| 2 | 0 > > libusb1 | 4 | 0 | 0 | 6 > > libxml-namespacesupport-perl | 0 | 1 | 0 | 0 > > libxml-parser-perl | 0 | 15| 0 | 0 > > libxml-perl | 0 | 6 | 0 | 0 > > libxml-sax-base-perl | 0 | 19| 0 | 0 > > libxml-sax-perl | 0 | 14| 0 | 0 > > libxml-simple-perl | 0 | 14| 0 | 0 > > libxml2 | 134 | 0 | 0 | 17 > > lttng-tools | 3397| 626 | 1 | 350 T > > lzo | 5 | 0 | 0 | 2 > > m4 | 144 | 8 | 18| 5 > > nettle | 97 | 0 | 2 | 9 > > openssh | 0 | 0 | 1 | 1 > > openssl | 146 | 1 | 8 | 91 > > opkg | 0 | 0 | 0 | 11 > > pango| 12 | 0 | 0 | 1 > > parted | 41 | 1 | 22| 5 > > perl | 2423| 19| 238 | 713 > > python | 301 | 6 | 41| 325 > > python3 | 30277 | 2 | 1043 | 990 > > quilt| 0 | 57| 0 | 0 > > sed | 86 | 0 | 0 | 4 > > slang| 97 | 0 | 0 | 22 > > strace | 466 | 18| 147 | 176 > > tcl | 149 | 0 | 0 | 179 > >
Re: [OE-core] ptest with BBCLASSEXTEND
On 5/16/19 8:36 AM, Richard Purdie wrote: On Thu, 2019-05-16 at 11:54 +0100, Richard Purdie wrote: https://autobuilder.yocto.io/pub/non-release/20190514-14/testresults/testresult-report.txt A couple of patches (on the list) gets us to the results below which is better. The perl module failures and quilt are from missing perl module dependencies. -- Recipe | Passed | Failed| Skipped | Time(s) -- acl | 3 | 0 | 0 | 0 apr | 44 | 0 | 0 | 58 apr-util | 19 | 0 | 0 | 8 attr | 0 | 0 | 0 | 0 bash | 36 | 8 | 0 | 108 T bluez5 | 23 | 2 | 0 | 5 busybox | 528 | 4 | 76| 4 bzip2| 6 | 0 | 0 | 1 dbus-test| 0 | 0 | 0 | 91 T diffstat | 439 | 0 | 0 | 3 diffutils| 19 | 0 | 2 | 2 e2fsprogs| 0 | 0 | 0 | 108 T elfutils | 172 | 24| 7 | 20 ethtool | 2 | 0 | 0 | 1 flex | 114 | 0 | 0 | 2 gawk | 342 | 0 | 0 | 2 gdbm | 24 | 0 | 0 | 2 gdk-pixbuf | 20 | 0 | 0 | 39 gettext | 354 | 21| 37| 33 glib-2.0 | 242 | 3 | 0 | 347 gstreamer1.0 | 103 | 1 | 0 | 104 gzip | 19 | 0 | 3 | 4 json-glib| 14 | 0 | 0 | 0 kbd | 5 | 2 | 0 | 1 libconvert-asn1-perl | 0 | 22| 0 | 0 liberror-perl| 0 | 15| 0 | 0 libevent | 1 | 0 | 0 | 86 libpcre | 0 | 0 | 0 | 1 libtest-needs-perl | 0 | 2 | 0 | 0 libtimedate-perl | 3 | 2 | 0 | 0 liburi-perl | 0 | 39| 2 | 0 libusb1 | 4 | 0 | 0 | 6 libxml-namespacesupport-perl | 0 | 1 | 0 | 0 libxml-parser-perl | 0 | 15| 0 | 0 libxml-perl | 0 | 6 | 0 | 0 libxml-sax-base-perl | 0 | 19| 0 | 0 libxml-sax-perl | 0 | 14| 0 | 0 libxml-simple-perl | 0 | 14| 0 | 0 libxml2 | 134 | 0 | 0 | 17 lttng-tools | 3397| 626 | 1 | 350 T lzo | 5 | 0 | 0 | 2 m4 | 144 | 8 | 18| 5 nettle | 97 | 0 | 2 | 9 openssh | 0 | 0 | 1 | 1 openssl | 146 | 1 | 8 | 91 opkg | 0 | 0 | 0 | 11 pango| 12 | 0 | 0 | 1 parted | 41 | 1 | 22| 5 perl | 2423| 19| 238 | 713 python | 301 | 6 | 41| 325 python3 | 30277 | 2 | 1043 | 990 quilt| 0 | 57| 0 | 0 sed | 86 | 0 | 0 | 4 slang| 97 | 0 | 0 | 22 strace | 466 | 18| 147 | 176 tcl | 149 | 0 | 0 | 179 util-linux | 336 | 2 | 80| 41 valgrind | 149 | 1 | 9 | 544 zlib | 1 | 0 | 0 | 1 Cheers, Richard During the weekly YP bug review meeting, we discussed creating a defect per package for each
Re: [OE-core] ptest with BBCLASSEXTEND
On Thu, 2019-05-16 at 11:54 +0100, Richard Purdie wrote: > https://autobuilder.yocto.io/pub/non-release/20190514-14/testresults/testresult-report.txt A couple of patches (on the list) gets us to the results below which is better. The perl module failures and quilt are from missing perl module dependencies. -- Recipe | Passed | Failed| Skipped | Time(s) -- acl | 3 | 0 | 0 | 0 apr | 44 | 0 | 0 | 58 apr-util | 19 | 0 | 0 | 8 attr | 0 | 0 | 0 | 0 bash | 36 | 8 | 0 | 108 T bluez5 | 23 | 2 | 0 | 5 busybox | 528 | 4 | 76| 4 bzip2| 6 | 0 | 0 | 1 dbus-test| 0 | 0 | 0 | 91 T diffstat | 439 | 0 | 0 | 3 diffutils| 19 | 0 | 2 | 2 e2fsprogs| 0 | 0 | 0 | 108 T elfutils | 172 | 24| 7 | 20 ethtool | 2 | 0 | 0 | 1 flex | 114 | 0 | 0 | 2 gawk | 342 | 0 | 0 | 2 gdbm | 24 | 0 | 0 | 2 gdk-pixbuf | 20 | 0 | 0 | 39 gettext | 354 | 21| 37| 33 glib-2.0 | 242 | 3 | 0 | 347 gstreamer1.0 | 103 | 1 | 0 | 104 gzip | 19 | 0 | 3 | 4 json-glib| 14 | 0 | 0 | 0 kbd | 5 | 2 | 0 | 1 libconvert-asn1-perl | 0 | 22| 0 | 0 liberror-perl| 0 | 15| 0 | 0 libevent | 1 | 0 | 0 | 86 libpcre | 0 | 0 | 0 | 1 libtest-needs-perl | 0 | 2 | 0 | 0 libtimedate-perl | 3 | 2 | 0 | 0 liburi-perl | 0 | 39| 2 | 0 libusb1 | 4 | 0 | 0 | 6 libxml-namespacesupport-perl | 0 | 1 | 0 | 0 libxml-parser-perl | 0 | 15| 0 | 0 libxml-perl | 0 | 6 | 0 | 0 libxml-sax-base-perl | 0 | 19| 0 | 0 libxml-sax-perl | 0 | 14| 0 | 0 libxml-simple-perl | 0 | 14| 0 | 0 libxml2 | 134 | 0 | 0 | 17 lttng-tools | 3397| 626 | 1 | 350 T lzo | 5 | 0 | 0 | 2 m4 | 144 | 8 | 18| 5 nettle | 97 | 0 | 2 | 9 openssh | 0 | 0 | 1 | 1 openssl | 146 | 1 | 8 | 91 opkg | 0 | 0 | 0 | 11 pango| 12 | 0 | 0 | 1 parted | 41 | 1 | 22| 5 perl | 2423| 19| 238 | 713 python | 301 | 6 | 41| 325 python3 | 30277 | 2 | 1043 | 990 quilt| 0 | 57| 0 | 0 sed | 86 | 0 | 0 | 4 slang| 97 | 0 | 0 | 22 strace | 466 | 18| 147 | 176 tcl | 149 | 0 | 0 | 179 util-linux | 336 | 2 | 80| 41 valgrind | 149 | 1 | 9 | 544 zlib | 1 | 0 | 0 | 1 Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org
[OE-core] ptest with BBCLASSEXTEND
We have a need to check the dependencies of our ptest packages. The
best way to do that is to take a minimal image and run them under that
in isolation. That would be a bit tedious and boring.
It got me thinking about creative ways to do that and I thought I'd
share the result.
I created an image recipe:
$ cat p-t-image.bb
require recipes-core/images/core-image-minimal.bb
require conf/distro/include/ptest-packagelists.inc
PTESTS = "${PTESTS_SLOW} ${PTESTS_FAST}"
ALL_PTESTS = "${@' '.join(['p-t-image-'+x for x in
d.getVar('PTESTS').split()])}"
IMAGE_INSTALL_append = " ${MCNAME} openssh"
BBCLASSEXTEND = "${@' '.join(['mcextend:'+x for x in
d.getVar('PTESTS').split()])}"
IMAGE_OVERHEAD_FACTOR = "1.0"
IMAGE_ROOTFS_EXTRA_SPACE = "1324288"
# ptests need more memory than standard to avoid the OOM killer
QB_MEM = "-m 1024"
Its not fast to parse this creates about 59 variants of the image. Its
(ab)using the mcextend class to an image and add the ptest to that
image for each ptest in the PTESTS_SLOW and PTESTS_FAST variables. It
also adds openssh since we need an ssh client in the images for
testimage to work. I also had to add extra space and memory to the
image config so all the tests could work.
You can get a list of the target images with:
bitbake p-t-image-elfutils-ptest -e | grep ALL_PTESTS=
then run:
bitbake
and then
bitbake -c testimage
which batch build and then runs all the images in parallel.
The results also get aggregated automatically into the json file so if
you delete it before starting, the json file is equavalient from a
ptest perspective to if they'd all been run in the same image.
The results I got from this are:
--
Recipe | Passed | Failed| Skipped | Time(s)
--
acl | 1 | 1 | 0 | 0
apr | 0 | 0 | 0 | -
apr-util | 4 | 0 | 0 | -
attr | 0 | 0 | 0 | 0
bash | 36 | 8 | 0 | 107 T
bluez5 | 23 | 2 | 0 | 5
busybox | 528 | 4 | 76| 4
bzip2| 0 | 0 | 0 | 1
dbus-test| 0 | 0 | 0 | 91 T
diffstat | 439 | 0 | 0 | 4
diffutils| 19 | 0 | 2 | 2
e2fsprogs| 0 | 0 | 0 | 78
elfutils | 172 | 24| 7 | 21
ethtool | 2 | 0 | 0 | 0
flex | 0 | 0 | 0 | 1
gawk | 342 | 0 | 0 | 2
gdbm | 24 | 0 | 0 | 1
gdk-pixbuf | 20 | 0 | 0 | 39
gettext | 0 | 0 | 0 | 1
glib-2.0 | 242 | 3 | 0 | 344
gstreamer1.0 | 103 | 1 | 0 | 105
gzip | 0 | 0 | 0 | 1
json-glib| 14 | 0 | 0 | 1
kbd | 5 | 2 | 0 | 0
libconvert-asn1-perl | 0 | 22| 0 | 0
liberror-perl| 0 | 15| 0 | 1
libevent | 1 | 0 | 0 | 86
libpcre | 0 | 0 | 0 | 0
libtest-needs-perl | 0 | 2 | 0 | 0
libtimedate-perl | 3 | 2 | 0 | 0
liburi-perl | 0 | 39| 2 | 0
libusb1 | 4 | 0 | 0 | 6
libxml-namespacesupport-perl | 0 | 1 | 0 | 0
libxml-parser-perl | 0 | 15| 0 | 0
libxml-perl | 0 | 6 | 0 | 0
libxml-sax-base-perl | 0 | 19| 0 | 1
libxml-sax-perl | 0 | 14| 0 | 0
libxml-simple-perl | 0 | 14| 0 | 0
libxml2 | 134 | 0 | 0 | 16
lttng-tools | 3397| 626 | 1 | 348 T
lzo | 5 | 0 | 0 | 2
m4 | 0 | 0 | 0 | 1
nettle
