Hi, Nico, On Fri, May 22, 2020 at 2:38 PM Nico Kadel-Garcia <nka...@gmail.com> wrote: > > On Thu, May 21, 2020 at 1:04 PM Richard Shaw <hobbes1...@gmail.com> wrote: > > > > On Thu, May 21, 2020 at 11:56 AM Aleksandra Fedorova <al...@bookwar.info> > > wrote: > >> > >> On Thu, May 21, 2020 at 6:30 PM Richard Shaw <hobbes1...@gmail.com> wrote: > >> > > >> > On Thu, May 21, 2020 at 10:37 AM Przemo Firszt <prz...@firszt.eu> wrote: > >> >> > >> >> "FreeCAD -t 0" performs approx 470 tests. No GUI required. Example > >> >> output starts here: > >> >> https://travis-ci.org/github/FreeCAD/FreeCAD/jobs/689681966#L9103 > >> > > >> > > >> > May need some work to get working from inside mock: > >> > > >> > # ./FreeCAD -t 0 > >> > FreeCAD 0.18, Libs: 0.18RUnknown > >> > © Juergen Riegel, Werner Mayer, Yorik van Havre 2001-2019 > >> > ##### #### ### #### > >> > # # # # # # > >> > # ## #### #### # # # # # > >> > #### # # # # # # # ##### # # > >> > # # #### #### # # # # # > >> > # # # # # # # # # ## ## ## > >> > # # #### #### ### # # #### ## ## ## > >> > > >> > Aborted (core dumped) > >> > >> Please don't run it inside mock. > > Mock testing is extremely useful for testing in a completely defined > environment, for standalone laptop environments with no active > network, and also for testing docker and small VM compatibility. If > tests need to be segregated for entirely local, or isolated testing, > then perhaps the tests can be segregated with some "local" option? > > Since a mistyped "%build" step can do "rm -rf /" as the build user, > it's just a lot safer for everyonr in the build environments to use > the mock or other chroot cages or docker images.
I used the wrong term here. Running tests in a mock environment is ok, if it fits the use case. What I tried to highlight was running tests as a part of the %check section of the RPM spec file. This means running tests in a very specific mock environment of the koji builder during the build of the binary rpm. The downsides of this approach are: 1) you don't test the installed package, you test a binary you have just built You can not check this way that configuration files for that binary are put into the right places 2) this environment is "dirty" as it has all build files in the working directory and environment has all build requirements installed These build files are not going to be packaged and won't appear on the target environment, but they could cause side effects on the test run. 3) it couples testing with builds. You can not test package you already have. You can not retrigger test with the same package if the test failure was caused by infra issue or the issue with the dependency. You can not reuse test from one package to test the compose or another package. 4) it creates load on koji builders When we run integration tests via Fedora CI we use cloud infrastructure outside of Koji. We can run heavier tests and retrigger them as much as needed. -- Aleksandra Fedorova bookwar _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
